technique
Superposition-Inspired Signal-to-Noise Account
techniqueactive
superposition-inspired-signal-to-noise-account-fe0973ad·1 events·first seen 29d agoAliases: Superposition-Inspired Signal-to-Noise Account
Co-occurring entities
More like this (12)
Superposition Model (neural networks)superpositionCompressed Computation is (probably) not Computation in SuperpositionNeural Super SamplingArtificial Analysis Big Bench AudioSemantic-Acoustic Equilibriumsignal-to-noise ratio (SNR)Robust Dual-Signal FusionCompositional Residual (eps*)Audio Interaction ModelConditional Scale EntropyOperadic consistency: a label-free signal for compositional reasoning failures in LLMs
Recent events (1)
Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency
This paper establishes a quantitative scaling law linking LLM factual recall to both model parameter count and topic frequency in training data, evaluated across 38 models on 8,900+ scholarly references. Recall quality follows a sigmoid function in the log-linear combination of these two variables, explaining 60% of variance across 16 dense models from four families and 74-94% within individual families. The authors propose a superposition-inspired mechanism where recall is gated by a signal-to-noise ratio: concept frequency provides signal and model capacity sets the noise floor. This provides a predictive framework for understanding and anticipating LLM confabulation patterns.