just-in-time loading
Letting the runner load a model only when a request arrives and drop it again after an idle timeout, instead of holding the weights in memory all day. You trade a few seconds of load time per request for the gigabytes back in between. LM Studio does this per model, with the timeout as a setting.
The load itself is a memory-mapped read off the SSD, so it costs roughly the file size divided by your disk's read speed. A 20 GB model on a drive doing 3 GB a second is about seven seconds before anything starts computing.
Whether it is worth it depends entirely on rhythm. If requests arrive more often than the timeout, the model never unloads and you have paid nothing for the setting. If you ask it something twice a day, keeping 20 GB parked costs you the memory for 24 hours to save fourteen seconds.
The failure mode is holding several models at once. Each one loaded on demand stays resident until its own timer runs out, and a machine can end up with three sets of weights it never asked to keep.