Bradley-Terry-Davidson

HypoArena benchmark evaluates LLMs on open-ended hypothesis discovery before conclusions are known

Researchers introduce Prospective Hypothesis Discovery (PHD), a new capability framing that tests whether LLMs can autonomously generate grounded, testable hypotheses from inconclusive evidence rather than answering pre-specified questions. The benchmark HypoArena includes 988 cases across six scientific and analytical domains, constructed via a novel Retrospective Context Regression pipeline that strips conclusions from expert documents. Evaluation of 15 frontier LLMs using a Bradley-Terry-Davidson arena framework reveals clear capability stratification and model-dependent effects of structured analytical prompting, with some models regressing under structured guidance. The work positions hypothesis formulation as a distinct, underexplored evaluation target for LLMs.