Entity · technique
state space model
techniqueactive
state-space-model-ce3a98b7·5 events·first seen Aliases: state space model, State-Space Model (SSM), State Space Model (SSM), state-space models
Co-occurring entities
MambaHugging FaceWhen Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models?CaMBRAINSelf-Supervised LearningElectroencephalography (EEG)TransformersKey-Value CacheSleep Consolidation MechanismMulti-hop Graph RetrievalCellular AutomataFast WeightsFalcon MambaTechnology Innovation InstituteMamba2Bamba
More like this (12)
An Exact Instrument for State Usage in Selective State-Space Models, and the Input-Driven Migration It Revealsphysical world modelESM (Evolutionary Scale Modeling)species distribution modelingEnergy-Based Modelsworld modelVPT Modellatent dynamical systemsstochastic-deterministic boundary (SDB)Sequential Monte Carlo (SMC)Diffusion ModelsModel Spec
Recent events (5)
Formal theory of when tool use increases expressive power of finite-precision recurrent models
A new arXiv paper provides an exact, architecture-level characterization of when external tool access increases the computational expressivity of finite-precision recurrent sequence models, including SSMs. The key dichotomy: finite-state tools add essentially nothing (absorbable at logarithmic bit cost), while a single infinite-state tool (a read/write tape) makes the system Turing complete with only O(log|Q| + log|Γ|) bits of controller state. The paper further shows this construction is realized by a natural one-layer finite-precision selective affine SSM, with selectivity identified as essential.
CaMBRAIN: Real-time, Continuous EEG Inference with Causal State Space Models
CaMBRAIN is a Mamba-based causal state space model designed for real-time, continuous inference on variable-length EEG signals, addressing quadratic scaling limitations of attention-based models. It introduces a multi-stage self-supervised training pipeline for long-range memory retention and achieves state-of-the-art results across three EEG datasets with over 10x throughput improvement.
Language Models Need Sleep: Periodic Context Consolidation via Fast Weights and SSM Blocks
This paper proposes a sleep-like consolidation mechanism for transformer-based LLMs to address the quadratic scaling of attention with context length. During 'sleep' phases, the model performs N offline recurrent passes over accumulated context, updating fast weights in state-space model (SSM) blocks via a learned local rule, then clears the KV cache. The approach is evaluated on synthetic tasks (cellular automata, multi-hop graph retrieval) and math reasoning, where standard transformers and SSM-attention hybrids fail, with performance scaling with sleep duration N.
Falcon Mamba: First Strong Attention-Free 7B Model
Technology Innovation Institute (TII) releases Falcon Mamba, a 7B parameter state space model (SSM) based on the Mamba architecture, announced as the first attention-free model at this scale to match or exceed transformer-based models on standard benchmarks. The model is hosted on Hugging Face and represents a significant milestone for SSM-based architectures competing with transformers. This release advances the case for pure SSM models as viable alternatives to attention-based LLMs at the 7B scale.
Bamba: Inference-Efficient Hybrid Mamba2 Model
Hugging Face published a blog post introducing Bamba, a hybrid architecture combining Mamba2 state-space layers with attention layers, designed for inference efficiency. The model targets reduced KV-cache memory and improved throughput compared to pure transformer architectures. The post covers architecture details, training approach, and benchmarking results positioning Bamba as a practical alternative for deployment-constrained settings.