p-true--ecb0bdd6·1 events·first seen Aliases: P(True)
A new arXiv paper identifies that LLM abstention involves two distinct axes — whether a model would answer correctly, and whether a question is even answerable (e.g., rests on a false premise) — and shows these cannot be collapsed into a single confidence score. Across five instruction-tuned models (2B–14B parameters), standard confidence signals like P(IK) and P(True) are nearly blind to false-premise questions, while hidden-state probes achieve 0.69–0.77 AUROC on the same task. The authors propose a two-axis calibrated policy that roughly triples challenge precision and certifies dual coverage budgets at 0.75 correct-answer coverage versus 0.31 for single-threshold baselines. The blind spot does not shrink with scale, making this a structural finding rather than a capability gap.