Entity · technique

Cross-Layer Sparse Attention

techniqueactivecross-layer-sparse-attention-9a2a86d7·1 events·first seen Jun 5, 2026

Aliases: Cross-Layer Sparse Attention

Co-occurring entities

CLSA YOCO Cross-Layer Sparse Attention with Shared Routing

More like this (12)

Cross-Layer Sparse Attention with Shared Routing Block Sparse Attention MiniMax Sparse Attention sparse attention ProbSparse Attention Graph Convolutional Attention Graph Attention Network Spectral Attention DeepSeek Sparse Attention Multi-head Latent Attention (MLA)cross-attention Adaptive Depth Sparse Framework

Recent events (1)

6arXiv · cs.CL·Jun 5, 2026·source ↗

CLSA: Cross-Layer Sparse Attention with Shared Routing for Efficient Long-Context Inference

Researchers propose Cross-Layer Sparse Attention (CLSA), a method that builds on KV-sharing architectures (like YOCO) to share both the KV cache and the routing index across decoder layers. A single indexer computes token-level top-k selection once and reuses it across layers, reducing routing overhead while preserving fine-grained selectivity. Experiments on short- and long-context benchmarks show up to 7.6x decoding speedup and 17.1x overall throughput improvement at 128K context, addressing pre-filling, KV-cache storage, and decoding bottlenecks simultaneously.

Long Context Evolution Inference Economics CLSA YOCO Cross-Layer Sparse Attention with Shared Routing +1 more