Scaling Laws / Emergent Capabilities
The observation that larger models have qualitatively different capabilities (reasoning, instruction-following) that smaller models lack.
The observation that larger models have qualitatively different capabilities (reasoning, instruction-following) that smaller models lack. This paper shows that alignment (smaller + aligned) can beat scale (larger + unaligned), suggesting alignment is as important as scale.