Straggler Problem
When one GPU is slower than others (older hardware, thermal throttling, interference), it becomes the bottleneck.
When one GPU is slower than others (older hardware, thermal throttling, interference), it becomes the bottleneck. The entire ring must wait for the straggler to finish each round. One slow GPU reduces effective speedup from P× to less.