Softmax

A function that turns a vector of raw scores into a probability

A function that turns a vector of raw scores into a probability

The function that converts raw alignment scores into attention weights. softmax(eᵢ) = exp(eᵢ) / Σⱼ exp(eⱼ). Guarantees all outputs are positive and sum to 1. See the Softmax Function tutorial.

A function that converts a vector of scores into a probability distribution (non-negative numbers summing to 1). Each element represents the probability of a token being the next one to generate.

A function that converts raw scores into probabilities: softmax(x_i) = exp(x_i) / Σ_k exp(x_k). Used in attention to ensure weights sum to 1.