Synchronisation Barrier

A point where all P GPUs pause and wait for the slowest GPU to finish. In Ring Attention, barriers occur between rounds (after each GPU completes blockwise attention). Synchronisation overhead is typically <1% with fast networks.