An alternative to the Transformer architecture that processes sequences with linear O(n) complexity instead of quadratic O(n^2) attention, enabling efficient handling of very long sequences. Mamba introduced selective state spaces where the model dynamically filters information based on content. Hybrid architectures like Jamba combine SSM efficiency with attention's retrieval strength.
Start with the shortest useful explanation before going deeper.
An alternative to the Transformer architecture that processes sequences with linear O(n) complexity instead of quadratic O(n^2) attention, enabling efficient handling of very long sequences. Mamba introduced selective state spaces where the model dynamically filters information based on content. Hybrid architectures like Jamba combine SSM efficiency with attention's retrieval strength.
Use the quick analogy first so the term is easier to reason about when you meet it in code, docs, or prompts.
Think of it as a piece of the context or inference stack behind agentic and LLM-powered Solana products.
Place the term inside its Solana layer so the definition is easier to reason about.
LLMs, RAG, embeddings, inference, and agent-facing primitives.
Turn the term from vocabulary into something operational for product and engineering work.
This term unlocks adjacent concepts quickly, so it works best when you treat it as a junction instead of an isolated definition.
Use this compact block when you want to give an agent or assistant grounded context without dumping the entire page.
State Space Model (Mamba) (state-space-model) Category: AI / ML Definition: An alternative to the Transformer architecture that processes sequences with linear O(n) complexity instead of quadratic O(n^2) attention, enabling efficient handling of very long sequences. Mamba introduced selective state spaces where the model dynamically filters information based on content. Hybrid architectures like Jamba combine SSM efficiency with attention's retrieval strength. Aliases: SSM, Mamba Related: Transformer, Mixture of Experts (MoE), Attention Mechanism
Use glossary context, relationships, mental models, and builder paths to get structured answers instead of generic chat output.
Optional: paste Anchor, Solana, or Rust code so the Copilot can map primitives back to glossary terms.
The Copilot will answer using the current term, related concepts, mental models, and the surrounding glossary graph.
These branches show which concepts this term touches directly and what sits one layer beyond them.
The neural network architecture underlying modern LLMs, introduced in 'Attention Is All You Need' (2017). Transformers use self-attention mechanisms to process input sequences in parallel (unlike recurrent networks). Key components: multi-head attention, positional encoding, feedforward layers, and layer normalization. Variants include encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5).
A neural network architecture that routes each input to a subset of specialized 'expert' sub-networks rather than activating all parameters, dramatically improving efficiency. Only a fraction of total parameters are active per token (e.g., DeepSeek-V3 has 671B total but ~37B active). MoE enables training much larger models at manageable compute costs. Used in production models like Mixtral, Jamba, and DeepSeek-V3.
A neural network component that allows models to weigh the relevance of different parts of the input when producing output. Self-attention computes query-key-value dot products across all positions, enabling each token to 'attend' to every other token. Multi-head attention runs multiple attention functions in parallel. Attention is O(n²) in sequence length, driving context window research.
These are the next concepts worth opening if you want this term to make more sense inside a real Solana workflow.
These entries are easy to mix up when you are reading quickly, prompting an LLM, or onboarding into a new layer of Solana.
A class of LLMs trained with reinforcement learning to generate step-by-step internal chain-of-thought before producing a final answer, enabling stronger performance on complex math, coding, and logic tasks. Pioneered by OpenAI's o1 (September 2024) and followed by o3, DeepSeek-R1, and Claude's extended thinking mode. Unlike standard LLMs that answer directly, reasoning models produce a variable-length internal CoT, allowing controllable compute at inference time.
A generative AI architecture that creates images, video, or audio by learning to reverse a noise-adding process—starting from pure noise and iteratively denoising to produce coherent output. Diffusion models power leading image generators (Stable Diffusion, DALL-E 3, Midjourney) and video generators (Sora). Key variants include latent diffusion (operating in compressed space) and diffusion transformers (DiT).
A large AI model trained on broad data that can be adapted for many downstream tasks. Foundation models (GPT-4, Claude, Llama 3, Gemini) are pre-trained on internet-scale text/code and can be fine-tuned, prompted, or used via APIs for specific applications. The term emphasizes that one base model serves as the foundation for diverse use cases rather than training task-specific models.
Glossary entries become useful when they are connected. These links are the shortest path to adjacent ideas.
The neural network architecture underlying modern LLMs, introduced in 'Attention Is All You Need' (2017). Transformers use self-attention mechanisms to process input sequences in parallel (unlike recurrent networks). Key components: multi-head attention, positional encoding, feedforward layers, and layer normalization. Variants include encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5).
A neural network architecture that routes each input to a subset of specialized 'expert' sub-networks rather than activating all parameters, dramatically improving efficiency. Only a fraction of total parameters are active per token (e.g., DeepSeek-V3 has 671B total but ~37B active). MoE enables training much larger models at manageable compute costs. Used in production models like Mixtral, Jamba, and DeepSeek-V3.
A neural network component that allows models to weigh the relevance of different parts of the input when producing output. Self-attention computes query-key-value dot products across all positions, enabling each token to 'attend' to every other token. Multi-head attention runs multiple attention functions in parallel. Attention is O(n²) in sequence length, driving context window research.
These entries live beside the current term and help the page feel like part of a larger knowledge graph instead of a dead end.
A neural network trained on vast text corpora to understand and generate human language. LLMs (GPT-4, Claude, Llama, Gemini) use transformer architectures with billions of parameters. They power chatbots, code generation, summarization, and reasoning tasks. In blockchain development, LLMs assist with smart contract writing, audit review, documentation, and code explanation.
The neural network architecture underlying modern LLMs, introduced in 'Attention Is All You Need' (2017). Transformers use self-attention mechanisms to process input sequences in parallel (unlike recurrent networks). Key components: multi-head attention, positional encoding, feedforward layers, and layer normalization. Variants include encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5).
A neural network component that allows models to weigh the relevance of different parts of the input when producing output. Self-attention computes query-key-value dot products across all positions, enabling each token to 'attend' to every other token. Multi-head attention runs multiple attention functions in parallel. Attention is O(n²) in sequence length, driving context window research.
A large AI model trained on broad data that can be adapted for many downstream tasks. Foundation models (GPT-4, Claude, Llama 3, Gemini) are pre-trained on internet-scale text/code and can be fine-tuned, prompted, or used via APIs for specific applications. The term emphasizes that one base model serves as the foundation for diverse use cases rather than training task-specific models.