paper
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics
paperactiveprovisional
language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics-68b57b7a·1 events·first seen 14d agoAliases: Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics
More like this (12)
Arithmetic Pedagogy for Language ModelsReasoning Language ModelsLanguage Modeling LossScaling Laws for Neural Language ModelsDecomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness1B-scale language modelsLanguage Model Finetuning7B language modelLanguage Models are Few-Shot LearnersUncertainty QuantificationGeometry of Semantic Space: Comparative Study of Discrete and Continuous ModelsThe Value Axis: Language Models Encode Whether They're on the Right Track
Recent events (1)
LMs compare quantities via number and unit heuristics rather than exact conversion
A new arXiv paper investigates how language models handle quantity comparisons involving measurement units (e.g., 110 cm vs. 1.2 m) across multiple unit systems. The authors find that accuracy degrades near comparison boundaries and that LM behavior is explained by linear surrogate models using numerical-difference and unit-scale-difference cues. Causal interventions confirm these heuristics are mechanistically operative, suggesting LMs do not perform exact unit conversion but instead rely on a bag of numerical and unit-scale shortcuts.