Entity · paper

Rubrics on Trial

paperactiveprovisionalrubrics-on-trial-b557b662·1 events·first seen 3d ago

Aliases: Rubrics on Trial

More like this (12)

Rubric Reward RubricsTree When Rubrics Change: Cross-Rubric Generalization for Critical Thinking Essay Scoring DeepRubric Rubric-Conditioned Self-Distillation Preference-Aware Rubric Learning rubric-based rewards rubric-based reward shaping EDIT: Evidence-Diagnosed Intervention Training for Rule-Faithful LLM Grading Judgement-of-Learning When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA

Recent events (1)

5arXiv · cs.CL·3d ago·source ↗

Rubrics on Trial: Self-evolving evaluation rubrics from a single query via synthetic pairwise evidence

A new arXiv preprint introduces 'Rubrics on Trial,' a query-only framework that automatically constructs and validates evaluation rubrics for LLMs without external annotations or model training. The system evolves rubric sets from scratch using synthetic rubric-conditioned response pairs, screening out non-discriminative, over-specific, or style-only rubrics. Evaluated across five preference benchmark suites, it achieves best average accuracy and leads on six of seven evaluation sets, addressing a key bottleneck in scalable LLM evaluation.

Evaluation and Benchmarking Alignment and RLHF Rubrics on Trial