edge device

Hardware that runs a model locally, out at the edge of the network, rather than calling a datacenter: a phone, a laptop, a small server in a closet. The point is that inference happens on the device itself, so the data never leaves. A 1-bit build of a 27B model, 3.9 GB on disk, runs on an iPhone 17 Pro Max at around 11 tokens per second.