Ollama
A local model runner built on llama.cpp, driven from the terminal. `ollama run gemma4` downloads the weights and starts a chat. It also serves an OpenAI-compatible API on port 11434, which is what lets other software on your machine talk to the model without knowing anything about how it is loaded.
Ollama's contribution was making the download and the run one command. It picks a quantization for you, keeps models in one place, and unloads them when they go idle.
The tradeoff against a GUI runner is control. You get fewer knobs, and the defaults it picks are conservative, which is usually right and occasionally means you are running a smaller quantization than your machine can hold.