Pipeline bubble
Idle gaps in pipeline parallelism; reduced by keeping many micro-batches in flight.
Idle gaps in pipeline parallelism; reduced by keeping many micro-batches in flight.
Idle gaps in pipeline parallelism; reduced by keeping many micro-batches in flight.
Idle gaps in pipeline parallelism; reduced by keeping many micro-batches in flight.