Activation Function
A non-linear function applied to a neuron's weighted sum before passing the result to the next layer.
A non-linear function applied to a neuron's weighted sum before passing the result to the next layer. Without activation functions, a multi-layer network collapses to a single linear function (no matter how many layers). Common choices: sigmoid (used in 1986 paper), tanh, ReLU (standard today).