A. Smyntyna / Code
ENRU

fine-tune

Continuing to train a released model on your own smaller dataset so it specializes. Costs a rounding error of what the original training cost, which is why one open-weights release turns into hundreds of variants for coding, documents, roleplay or a single company's tone.

Most fine-tunes now use an adapter, a small set of extra parameters trained on top of frozen weights, so the result is a few hundred megabytes rather than a new copy of the model.

It changes behavior far more reliably than it adds knowledge. If the aim is for the model to know your documents, retrieval usually beats fine-tuning and costs nothing to update.

Read next