distributed inference
Running one model across several machines, because it does not fit on any of them. The memory pools and the model loads. The speed usually does not pool: exo reports 1.8x on two devices and 3.2x on four when it can shard the layers, and nothing at all when it can only chain them.
The tool most people reach for is exo. It finds the other machines on its own, decides how to split the model based on what it sees, and serves an OpenAI-compatible endpoint, so anything already pointed at a local model works against a cluster with no change.
The cost is in the physical layer. exo's fast path is RDMA over Thunderbolt 5, which needs macOS 26.2 or later, every machine cabled directly to every other machine, and matching OS versions down to the beta number. Its Linux build is CPU-only, so a pile of NVIDIA cards is not what it clusters today.
The thing to be clear about before spending money: pooling memory buys you the ability to load the model. Whether it also runs at a useful speed depends entirely on which of the two splits it ends up using.