pipeline parallelism
Splitting a model by layer across machines: the first machine holds layers 1 to 20, the next holds 21 to 40, and a token walks the chain. It is what makes an oversized model fit, and it makes nothing faster, because only one machine is computing at any moment.
Four machines each reading a quarter of the weights, one after another, read the same total bytes as one machine reading all of them. The bandwidth does not add up. What you bought is capacity.
This is the honest answer to "does clustering make it faster". On its own, no. Tensor parallelism is the split that recovers speed, and it needs an interconnect fast enough to move activations between every machine on every layer.