paper
Compressed Computation is (probably) not Computation in Superposition
paperactiveprovisional
compressed-computation-is-probably-not-computation-in-superposition-cdbb9677·1 events·first seen 2d agoAliases: Compressed Computation is (probably) not Computation in Superposition
Co-occurring entities
More like this (12)
Braun et al. 2025 Compressed Computationthought compressionSuperposition-Inspired Signal-to-Noise Accountcomputational imagingSuperposition Model (neural networks)SKIM (SKIll coMpression)superpositioncontext compactiongradient compressioninference-time computeEnd-to-End Context Compression at ScaleNNCF (Neural Network Compression Framework)
Recent events (1)
Paper argues Compressed Computation toy model is not computation in superposition
A new arXiv preprint challenges the Compressed Computation (CC) toy model introduced by Braun et al. (2025), which appeared to compute 100 ReLU functions using only 50 neurons. The authors show that apparent performance gains arise from unintended input mixing via a noisy residual stream rather than genuine superposition, with learned neuron directions concentrating in the subspace of the top 50 eigenvalues of the mixing matrix. A semi-non-negative matrix factorization baseline derived purely from the mixing matrix reproduces the qualitative loss profile, supporting the conclusion that CC is not a valid toy model of computation in superposition.