4arXiv cs.CL (Computation and Language)·
RLAES: Reinforcement learning framework for joint essay scoring and feedback generation with rubric rewards
Researchers propose RLAES, a unified LLM framework that jointly optimizes automated essay scoring (AES) and feedback generation via reinforcement learning. The system introduces Rubric-based Feedback Evaluation (RFE) with 166 fine-grained binary rubric items and an LLM-as-judge, plus Adaptive Gated Feedback Optimization (AGFO) and Adjacent Contrastive Reasoning (ACR) for score calibration. On the ASAP benchmark, RLAES-AGFO achieves QWK=0.803 among LLM-based methods while maintaining feedback quality comparable to GPT-5.5. Code and datasets are publicly released.
Related guides (3)
Related events (8)
AMARIS: Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
AMARIS introduces a persistent evaluation memory system to improve rubric-based reward shaping in LLM fine-tuning via reinforcement learning. Unlike prior adaptive rubric methods that discard evaluation diagnostics after each step, AMARIS accumulates step-level summaries and retrieves relevant historical context via both static (recent steps) and dynamic (semantic similarity) retrieval to inform rubric updates. The system runs asynchronously alongside the RL training loop with approximately 5% time overhead. Experiments across closed and open-ended domains show consistent improvements over baselines, with ablations confirming that combining both retrieval modes yields the strongest results.
PARL: Preference-Aware Rubric Learning for Personalized LLM Evaluation
This paper introduces PARL (Preference-Aware Rubric Learning), a framework that reframes personalized LLM evaluation as a learning problem rather than static judgment. PARL induces preference-aware evaluation rubrics from raw user interaction histories and uses a discriminative reinforcement learning objective to contrast user-authored responses against model outputs, capturing user-specific decision boundaries. Experiments on personalized text generation tasks show PARL produces high-fidelity rubrics that generalize across users and tasks, outperforming existing LLM-as-a-judge and automatic metric approaches.
Cross-rubric generalization for automated essay scoring using LLM fine-tuning with trait-based representations
A new arXiv paper introduces a framework for automated essay scoring (AES) that generalizes to previously unseen scoring rubrics, rather than just unseen prompts. The approach uses rubric-agnostic intermediate representations called 'traits' combined with target-essay supervision, achieving a 5.0% macro F1 improvement over a baseline in the hardest generalization setting. A fine-tuned Llama-based model outperforms GPT-5-mini prompting by 2.1% macro F1 and trails GPT-5 by only 1.9%, demonstrating that structured intermediate representations improve rubric generalization.
Rubrics on Trial: Self-evolving evaluation rubrics from a single query via synthetic pairwise evidence
A new arXiv preprint introduces 'Rubrics on Trial,' a query-only framework that automatically constructs and validates evaluation rubrics for LLMs without external annotations or model training. The system evolves rubric sets from scratch using synthetic rubric-conditioned response pairs, screening out non-discriminative, over-specific, or style-only rubrics. Evaluated across five preference benchmark suites, it achieves best average accuracy and leads on six of seven evaluation sets, addressing a key bottleneck in scalable LLM evaluation.
PsyScore: Psychometrically-aware framework integrating IRT scoring with ZPD-scaffolded LLM feedback for essay assessment
PsyScore is a new framework for Automated Essay Scoring (AES) that unifies diagnostic assessment and instructional feedback through a shared latent ability representation. It combines a neural Item Response Theory scorer (based on the Graded Partial Credit Model) with a multi-agent LLM feedback generator conditioned on estimated student proficiency, operationalizing Vygotsky's Zone of Proximal Development. Experiments on the ASAP++ dataset show competitive scoring performance alongside more pedagogically aligned feedback. The work addresses a gap between psychometric rigor and LLM-based adaptive instruction.
Experiential Learning proposes LLM-as-a-Coach to replace scalar RL rewards with textual feedback distillation
Researchers introduce Experiential Learning (EL), a post-training method that replaces scalar RL rewards with rich textual feedback from an LLM-as-a-Coach, which distills per-response assessments into transferable experiential knowledge used to condition a teacher model and update the policy via context distillation. Evaluated across two policy families with both self-feedback and proprietary model feedback, EL consistently outperforms rubric-based RL on open-ended tasks and shows better out-of-distribution generalization while mitigating reward hacking. The work targets a known weakness of RLHF-style training on non-verifiable tasks where scalar rewards lose fine-grained preference signal.
ExpRL: RL-based mid-training using human QA data as reward scaffolds for LLM reasoning
ExpRL proposes an automated approach to LLM mid-training that replaces manually curated reasoning traces with large corpora of human-written QA data used as reward scaffolds rather than imitation targets. Reference solutions are hidden from the policy and used only to construct problem-specific grading rubrics, enabling dense process-level rewards that reinforce partial progress and intermediate reasoning steps. On challenging math reasoning benchmarks, ExpRL outperforms SFT, sparse-reward GRPO, and self-distillation as an RL initialization strategy, with additional mixed-domain experiments suggesting broader applicability.
QUBRIC: Co-designing queries and rubrics for RL beyond verifiable rewards
QUBRIC is a framework that jointly optimizes queries and rubrics for reinforcement learning in settings where rewards are not strictly verifiable. The approach uses teacher-derived key points to rewrite open-ended queries into evaluable scenarios, applies contrastive rubric generation to capture teacher-policy gaps, and filters for learnability before GRPO training. Trained only on instruction-following data, QUBRIC achieves a +5.5 point gain on ArenaHard over an SFT baseline and transfers to legal, moral, and narrative reasoning benchmarks (+6.3 points average), suggesting rubric-based RL can complement RLVR in non-verifiable domains.