Latency Hiding

Making communication latency disappear by overlapping it with computation. If computation time ≥ communication time, the communication cost is "hidden" in the wall-clock time. Ring Attention's key advantage.