Transformer

Appears in 1 paper · 2 tutorials

A neural network architecture based on self-attention, introduced in "Attention Is All You Need" (2017).

As used in Paper 20 — Gemini: A Family of Highly Capable Multimodal Models →

A neural network architecture based on self-attention, introduced in "Attention Is All You Need" (2017). Processes sequences in parallel and has largely replaced RNNs/LSTMs.

As used in Fine-Tuning & Model Customization →

The neural-network architecture behind modern LLMs, a stack of layers each containing attention and an MLP. (M01)

As used in LLM Infrastructure →

The architecture nearly all modern LLMs use: a tall stack of identical layers.

Paper 20 — Gemini: A Family of Highly Capable Multimodal Models → Fine-Tuning & Model Customization → LLM Infrastructure →