Module 06 — PEFT: Parameter-Efficient Fine-Tuning

Goal: Understand the family of techniques that made fine-tuning accessible to everyone, the problem they all solve, and how they differ — so that when you go deep on LoRA next module, you see it as the standout member of a well-understood family, not a magic trick. This is the conceptual bridge between “SFT in principle” and “fine-tuning huge models on one GPU in practice.”

1. The problem PEFT exists to solve

From Module 03 and Module 05 you know the painful arithmetic of full fine-tuning: to update all of a model’s weights you must store, for every parameter, the weight, its gradient, and the optimizer’s bookkeeping — roughly 4× the model size in memory just for the math, plus the activations flowing through. A 7B model balloons to dozens of gigabytes; a 70B model needs a cluster.

Worse, full fine-tuning produces a whole new copy of the model (tens of GB) for every task. Want the model to do five different jobs? Five full copies. That’s expensive to store, slow to switch, and impractical to serve.

PEFT asks a simple, powerful question:

What if we freeze the giant pretrained model and train only a tiny number of new parameters — leaving 99%+ of the model untouched?

If that worked, fine-tuning would suddenly fit on a single consumer GPU, each task’s “patch” would be a few megabytes instead of gigabytes, and you could keep dozens of them. It does work — astonishingly well — and that’s the PEFT revolution.

2. The core insight: you don’t need to move much

Why can training less than 1% of a model rival training all of it? The intuition:

A pretrained model already contains enormous capability. To specialize it (a style, a task, a format), you don’t need to rebuild that capability — you only need to steer it. Steering turns out to require far less “movement” in weight space than building did. Researchers found that the update a model needs for a new task is low-dimensional — it lives in a much smaller space than the full pile of billions of weights. PEFT methods exploit exactly that: they give the model just enough new, trainable capacity to steer, and nothing more.

Analogy: Pretraining built a fully-equipped ship. Full fine-tuning rebuilds the whole ship for each voyage. PEFT just adds a small, trainable rudder. The ship already knows how to sail; you only need to point it.

This is also why PEFT resists overfitting (Module 03, §7) and forgets less (Module 05, §7): with few trainable parameters and the original model frozen, there’s far less capacity to memorize noise or drift away from the base model’s broad competence.

3. The PEFT family tree

PEFT is an umbrella over several approaches. You should be able to recognize all of these and explain the big two. They differ mainly in where they add the trainable parameters.

PEFT (freeze the base model; train a tiny new set)
│
├── Additive methods — ADD new trainable pieces
│   ├── Adapters — small bottleneck layers inserted between existing layers
│   ├── Prompt-based — prepend trainable "virtual tokens" to the input
│   │   ├── Prompt Tuning (soft prompts)
│   │   ├── Prefix Tuning (trainable vectors at every layer)
│   │   └── P-Tuning (v1/v2)
│   └── (others)
│
├── Reparameterization methods — REPRESENT the weight update compactly
│   ├── LoRA  ★ the dominant method (Module 07)
│   ├── QLoRA — LoRA on a quantized base (Module 08)
│   └── LoRA variants — DoRA, AdaLoRA, rsLoRA, LoHa/LoKr, PiSSA, ...
│
└── Selective methods — train only a CHOSEN SUBSET of existing weights
    ├── BitFit — train only the bias terms
    └── (layer-freezing, train only top N layers, etc.)

Let’s walk the branches, briefly, so the landscape is clear.

4. Additive method 1: Adapters

The original PEFT idea (2019). Insert small new neural-network modules — adapters — between the existing (frozen) layers of the Transformer. Each adapter is a tiny “bottleneck”: it squeezes the representation down to a small size, transforms it, and expands it back. You train only these adapters.

Pro: Works; modular (swap adapters per task).
Con: Because adapters are inserted in the computation path, they add a little inference latency every time you run the model — the signal has to pass through the extra modules. This drawback is precisely what LoRA was designed to remove (Module 07 explains how LoRA adds zero inference cost after merging). Still, “adapter” remains a useful general term, and you’ll hear LoRA’s patches loosely called “adapters.”

5. Additive method 2: Prompt-based (soft prompts, prefix tuning)

A clever, different angle. Instead of touching the model’s layers at all, you prepend a handful of trainable vectors to the input — “virtual tokens” or soft prompts. They don’t correspond to real words; they’re free parameters the model learns to interpret as a powerful, compressed instruction.

Prompt Tuning: add trainable vectors only at the input embedding.
Prefix Tuning: add trainable vectors at every layer (more capacity, more effect).
P-Tuning (v1/v2): related variants with their own tricks.

Think of it as learning the perfect prompt as numbers rather than words — a “soft” prompt the model tunes for itself.

Pro: Extremely few parameters (sometimes just thousands); great for very lightweight task-switching; the base model is utterly untouched.
Con: Generally less powerful than LoRA for substantial behavior change, can be finicky to train, and consumes some of your context window. Important to know, less often your first choice today.

6. Selective method: BitFit and friends

The simplest idea: don’t add anything — just unfreeze a small, chosen subset of the existing weights and train only those. BitFit trains only the bias terms (a tiny fraction of parameters). Others freeze all but the top few layers.

Pro: Dead simple, no new architecture, minimal memory.
Con: Limited capacity; usually underperforms LoRA on meaningful tasks. Good for intuition and very mild adaptation; rarely the production choice.

7. Reparameterization: LoRA, the one that won

The branch that became the default. The idea (full detail in Module 07): instead of changing a big weight matrix W directly, freeze W and learn a small, low-rank “update” matrix that, when added to W, produces the new behavior. Because that update is represented as the product of two skinny matrices, it has tiny trainable size — yet it can be merged back into W after training so there is no extra inference cost at all.

LoRA hit the sweet spot that the other methods each missed on one axis:

Method	Trainable params	Inference overhead	Adaptation power	Ease
Full fine-tuning	100% (huge)	none	highest	costly
Adapters	small	some latency	good	medium
Prompt/Prefix tuning	tiny	small (uses context)	moderate	finicky
BitFit (selective)	tiny	none	limited	easy
LoRA	small	none (after merge)	strong	easy

That last row is why ~everyone reaches for LoRA first. It’s powerful enough for real tasks, costs nothing extra at inference, produces tiny portable files, and is well-supported everywhere.

8. The shared PEFT superpowers (why the whole family matters)

Beyond saving memory, all reparameterization/additive PEFT methods share advantages that matter enormously in practice — keep these in mind, because they shape real production systems:

Tiny, portable artifacts. A LoRA “adapter” is often a few MB to tens of MB, versus a full model’s many GB. You can email it, version it in git, store hundreds of them.
Multi-task serving from one base. Load the big base model once in memory and hot-swap small adapters per request — serve dozens of specialized “models” at roughly the cost of one. (Module 14 covers multi-LoRA serving.)
Cheaper, faster experiments. Less memory and fewer trainable parameters mean you can iterate on a single GPU in minutes/hours instead of renting a cluster.
Less forgetting, less overfitting. The frozen base preserves general ability (Module 05, §7); few trainable params resist memorizing noise (Module 03, §7).
Composability. You can sometimes combine or stack adapters, or merge one in and train another on top.

These aren’t minor conveniences — they’re the reason fine-tuning went from “big-lab activity” to “anyone with a laptop and a Colab notebook.” PEFT democratized the field.

9. How to choose within PEFT (a quick guide)

For nearly all practical work today:

Default to LoRA (Module 07). It’s the best general trade-off and the most supported.
Use QLoRA (Module 08) when the model is too big to fit on your GPU in normal precision — i.e., most of the time on a single consumer card. QLoRA = LoRA + a quantized (compressed) base model.
Consider LoRA variants (DoRA, rsLoRA, etc., Module 07 §10) only when you’ve mastered plain LoRA and want to chase the last few percent.
Reach for prompt/prefix tuning when you need extreme parameter frugality or ultra-cheap task switching and the adaptation is mild.
Use BitFit/selective mainly for learning or very light touch-ups.
Use full fine-tuning only when you have the hardware and evidence (from evaluation) that PEFT genuinely isn’t enough — which is rarer than beginners assume.

The honest expert summary: “In 2024–2026, if someone says ‘we fine-tuned a model,’ they almost always mean LoRA or QLoRA.” The rest of this family is essential context and occasionally the right tool — but LoRA is where you’ll live. So let’s go deep on it.

Module 06 checklist

I can state the problem PEFT solves (memory + a model-copy per task).
I can explain why training <1% of a model can work (low-dimensional updates).
I can name the three PEFT branches (additive, reparameterization, selective) with an example of each.
I can explain adapters and their inference-latency drawback.
I can explain soft/prefix prompts as “learning the prompt as numbers.”
I can list the shared PEFT superpowers (tiny artifacts, multi-task serving, less forgetting).
I understand why LoRA became the default and when to pick alternatives.

➡️ Next: Module 07 — LoRA In Depth