Optimizer (Adam / AdamW / 8-bit Adam)
The algorithm that applies gradients to weights, with adaptive step sizes and momentum.
The algorithm that applies gradients to weights, with adaptive step sizes and momentum. Adam stores extra state per weight. (M03)