Q, K, V, O (Query, Key, Value, Output projections)
The four weight matrices in attention; the most common place to attach LoRA adapters.
The four weight matrices in attention; the most common place to attach LoRA adapters. (M01, M07)
The four weight matrices in attention; the most common place to attach LoRA adapters.
The four weight matrices in attention; the most common place to attach LoRA adapters. (M01, M07)