A. Smyntyna / Code
ENRU

tool calling

The model writing a request for a command instead of an answer. The harness puts a list of available tools in the prompt, the model replies with text shaped like a call ("run bash with ls src"), and the harness runs it and pastes the output back as context. The model never executes anything itself.

Nothing in the model changes to make this work. It is still predicting the next token. What changed is that somebody wrote a program which reads the output, notices a tool-shaped block in it, and acts.

That is also the whole of the security story. A local model cannot touch a file, open a socket or delete anything. The harness can, on your behalf, and the permissions it runs under are the ones you gave it.

Reliability here is a training property rather than a size property. A model has to have been fine-tuned to hold the exact output format for many turns in a row, which is why small models from a newer generation often call tools more dependably than larger ones from an older generation.

Read next