Module 01 — What Is a Language Model?

Goal: Understand what’s actually inside a language model and how it learned to be smart, so that everything later (“we update the weights,” “we add a LoRA adapter to the attention layers”) means something concrete. We go slowly and use no fine-tuning here. This is the foundation the whole course stands on.

1. The core idea: a next-word prediction machine

Strip away all the hype and a language model does exactly one thing:

Given some text, it predicts what comes next.

That’s it. You give it “The capital of France is” and it predicts “Paris.” You give it the first half of an email and it predicts the second half. Everything impressive — answering questions, writing code, holding a conversation — emerges from doing this one thing extremely well, over and over.

Here’s the part that surprises people: the model doesn’t predict a single next word with certainty. It produces a probability for every possible next word. For “The capital of France is” it might assign:

Paris    →  0.91
the      →  0.02
located  →  0.015
a        →  0.01
...      →  (tiny probabilities spread over tens of thousands of options)

Then a sampling step picks one (usually the high-probability ones), appends it to the text, and the whole process repeats to generate the next word, and the next. This loop — predict, pick, append, repeat — is called autoregressive generation. “Autoregressive” just means “it feeds its own output back in as input.”

Hold onto this: fine-tuning is the act of changing those probabilities. When we teach a model to write in your company’s voice, we are nudging it so that, given your prompts, the words you want get higher probabilities.

2. Tokens: how text becomes numbers

Models don’t see letters or words. They see tokens — numbers. Before any text reaches the model, a component called the tokenizer chops it into tokens and maps each to an integer ID.

A token is usually a word-piece, not a whole word. Common words are one token; rarer words get split. For example:

"fine-tuning is fun"  →  ["fine", "-", "tuning", " is", " fun"]  →  [3506, 12, 28286, 374, 2523]

(The exact split depends on the tokenizer.) A useful rule of thumb in English: 1 token ≈ 0.75 words, or about 4 characters. So 1,000 tokens is roughly 750 words.

Why you must care about tokens

Tokens are the currency of everything:

Cost is measured in tokens (APIs charge per token).
Context length — the maximum amount of text a model can consider at once — is measured in tokens (e.g., “8k context” = 8,192 tokens).
Training data size is measured in tokens.
Subtle bugs (a model that “can’t spell” or “can’t count letters”) often come from tokenization, because the model never saw individual letters — it saw chunks.

Key insight for later: Different models have different tokenizers. When you fine-tune, you must use the exact tokenizer that came with your base model. Mixing them is a classic, baffling beginner bug.

3. Embeddings: turning tokens into meaning

A token ID like 28286 is just an arbitrary label — it carries no meaning by itself. The first thing the model does is convert each token ID into a list of numbers called an embedding (also called a vector).

Think of an embedding as the token’s coordinates in a giant “meaning space.” In this space, tokens with similar meanings sit close together. The vectors for “king” and “queen” are near each other; “king” and “banana” are far apart. The famous example: the direction from “man” to “woman” is roughly the same as from “king” to “queen” — the geometry of the space actually encodes relationships.

These embeddings are typically a few thousand numbers long (e.g., 4,096 dimensions for a mid-size model). They are learned during training — nobody hand-writes them. The model figures out, from reading billions of words, which words belong near which.

The collection of all these embeddings is one of the model’s big tables of weights (more on weights in a moment). And yes — fine-tuning can adjust embeddings too, though usually we leave them mostly alone.

4. Weights and parameters: where the “knowledge” lives

A model is, concretely, a huge pile of numbers called weights (or parameters — same thing). When you hear “Llama-3 8B,” the 8B = 8 billion parameters. Those 8 billion numbers are the model. Everything it knows — grammar, facts, reasoning patterns, style — is encoded in the specific values of those numbers.

How are the weights used? The model is organized as a stack of mathematical operations. The dominant operation is matrix multiplication: the incoming numbers (from the embeddings) get multiplied by big grids of weights, producing new numbers, which get multiplied by the next grid, and so on, layer after layer.

You don’t need to be able to do matrix multiplication by hand. You need exactly one intuition:

A weight matrix is a transformation. It takes a list of numbers representing “what we understand so far” and reshapes it into a richer representation. Stacking many such transformations lets simple inputs become sophisticated understanding.

This is the heart of fine-tuning. Training — and fine-tuning — is nothing more than gradually adjusting the values of these weight numbers so the model’s predictions get better. Full fine-tuning adjusts all of them (expensive). LoRA, which you’ll learn to love, adjusts a tiny clever subset (cheap). Now you can see why that’s even possible: it’s all just numbers in matrices, and you can choose which numbers to nudge.

5. The Transformer: the architecture that made it all work

Modern LLMs use an architecture called the Transformer (introduced in 2017 in a paper memorably titled “Attention Is All You Need”). You don’t need to implement one, but understanding its two key pieces makes later choices (“apply LoRA to the attention projections”) obvious instead of mysterious.

A Transformer is a tall stack of identical layers (also called blocks). A model might have 32 such layers. Each layer has two main sub-parts:

5a. Attention — “which other words should I pay attention to?”

When the model processes the word “it” in “The dog chased the ball until it stopped,” it needs to figure out what “it” refers to. Attention is the mechanism that lets every word look at every other word and decide which ones are relevant, then pull in information from them.

Mechanically, attention is computed using three sets of weight matrices, traditionally named Query (Q), Key (K), and Value (V), plus an Output (O) projection. You’ll meet these names again the moment you touch LoRA, because the Q, K, V, and O projection matrices are the most common place we attach LoRA adapters. Remember them.

The intuition:

The Query is “what am I looking for?”
The Key is “what do I have to offer?”
Each word’s Query is compared against every word’s Key to decide relevance.
The Value is the actual information that gets pulled in, weighted by that relevance.

5b. The MLP / Feed-Forward Network — “think harder about each word”

After attention mixes information between words, each word’s representation is passed through a small two-layer network (the MLP, or feed-forward network, FFN) that does additional processing. This is where a lot of the model’s raw “knowledge storage” is believed to live, and these layers contain the majority of the model’s parameters. Advanced fine-tuning sometimes targets these too.

Putting a layer together

   input to the layer
        │
        ▼
   ┌─────────────┐
   │  Attention  │   ← words look at each other (Q, K, V, O matrices)
   └──────┬──────┘
        │  (+ a "residual" shortcut that adds the input back in)
        ▼
   ┌─────────────┐
   │  MLP / FFN  │   ← each word gets extra processing
   └──────┬──────┘
        │  (+ another residual shortcut)
        ▼
   output of the layer → becomes input to the next layer

Stack ~32 of these, add the embedding table at the bottom and a final layer that turns the last representation into next-token probabilities at the top, and you have a modern LLM. That final step uses an “unembedding” matrix (often called the LM head) to project back out to a probability for every token in the vocabulary.

You now know enough architecture for everything in this course. When a later module says “we inject low-rank matrices into the attention projections of every layer,” you can picture exactly where that happens and why it would change the model’s behavior.

6. Pretraining: how the model got smart in the first place

Where do those 8 billion well-tuned weights come from? A phase called pretraining, done once by the model’s creators (Meta, Mistral, Google, Alibaba, etc.) at enormous cost (often millions of dollars and weeks on thousands of GPUs).

The recipe is conceptually simple:

Gather a colossal amount of text — much of the public internet, books, code, etc. (trillions of tokens).
Repeatedly show the model a chunk of text with the next token hidden.
Have it predict that next token.
Measure how wrong it was (the loss — covered in Module 03).
Nudge every weight a tiny bit to be less wrong next time.
Repeat trillions of times.

That’s it. By doing nothing but “predict the next token” across the whole internet, the model is forced to learn grammar, facts, reasoning, translation, coding, and more — because all of those help predict text. This is called self-supervised learning: no human labels are needed, because the “right answer” is just the actual next word in the text.

The output of pretraining is a base model (also called a foundation model or pretrained model). It is broadly knowledgeable but raw — it just continues text. It doesn’t naturally “answer questions” or “follow instructions.” Ask a pure base model “What is the capital of France?” and it might continue with “What is the capital of Germany? What is the capital of Spain?” — because it saw lists like that on the internet. It’s completing text, not helping you.

7. From base model to assistant: the post-training pipeline

To turn a raw base model into the helpful chatbot you’re used to, creators add post-training, which is itself made of fine-tuning steps — the very techniques this course teaches:

Instruction tuning / SFT — show the model thousands of instruction → good response examples so it learns to follow instructions and answer questions. (Module 05.)
Preference tuning (RLHF / DPO) — show the model pairs of responses labeled “this one is better” so it learns taste: to be helpful, harmless, and honest. (Module 12.)

The result is an instruct model (e.g., “Llama-3-8B-Instruct”) — the chat-ready version. You’ll often start your fine-tuning from one of these.

Crucial distinction you’ll use constantly:

A base / pretrained model just continues text. Good starting point if you want maximal flexibility or are teaching a very different format.

An instruct / chat model already follows instructions. Usually the better starting point for most practical fine-tuning, because you build on existing helpfulness instead of recreating it.

Choosing which to start from is a real decision you’ll make in later modules. Now you understand what the choice actually means.

8. A grounding analogy to carry forward

Think of the base model as a brilliant new graduate who has read almost everything ever written but has never had a job. They know an astonishing amount but don’t know your company, your customers, or how you like things done.

Prompting is leaving them a detailed sticky note for each task.
RAG is giving them a reference binder to look things up in while they work.
Fine-tuning is actually training them on the job until your way of doing things becomes second nature.

That analogy is the entire subject of the next module, where we decide when on-the-job training is the right call — and when it would be a waste.

Module 01 checklist

I can explain “next-token prediction” and “autoregressive generation” plainly.
I know what a token is and why token count matters (cost, context, data size).
I can say what an embedding is (coordinates in meaning space).
I understand that weights/parameters are the model, and training adjusts them.
I can name the two parts of a Transformer layer (attention with Q/K/V/O, and the MLP).
I can explain pretraining and the difference between a base and an instruct model.

➡️ Next: Module 02 — Why Fine-Tune?