Pipeline parallelism (PP)
Splitting the model by layers across GPUs/nodes, assembly-line style; less communication, so it can cross nodes.
Splitting the model by layers across GPUs/nodes, assembly-line style; less communication, so it can cross nodes. Risk: pipeline "bubbles." (vLLM: --pipeline-parallel-size.)