RRBench

RRBench: evaluating open-weight LLMs on longitudinal data preparation tasks in governance-restricted settings

Researchers from UCL introduce RRBench, an open-source benchmark and evaluation framework for assessing open-weight LLMs on data preparation tasks in longitudinal population studies, where cloud-based models are prohibited by data governance rules. The benchmark uses ground-truth cleaning scripts from six sweeps of a British cohort study, covering tasks like category harmonization and multi-wave merging, with automated evaluation of LLM-generated R code. Current 31–35B parameter open-weight models achieve up to 87.9% average task completion on consumer-grade hardware, suggesting a viable path for AI-assisted data preparation in privacy-sensitive research contexts.