Jamba
An LLM from AI21 Labs (2024) that alternates Mamba and Attention blocks in its architecture.
An LLM from AI21 Labs (2024) that alternates Mamba and Attention blocks in its architecture. Hybrid model combining the strengths of both: Mamba's efficiency on long sequences + Attention's expressiveness. First commercial LLM using Mamba-style blocks.