Alignment model
The small neural network inside the attention mechanism that scores how well a decoder state matches each encoder hidden state.
The small neural network inside the attention mechanism that scores how well a decoder state matches each encoder hidden state. In Bahdanau's formulation: vₐᵀ tanh(Wₐ sₜ₋₁ + Uₐ hᵢ). Outputs a scalar alignment score for each source position. Learned end-to-end during training.