Entity · paper

Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape Pre-, Intra-, and Post-CoT Calibration

paperactiveprovisional

post-training-shifts-confidence-a-three-stage-analysis-of-how-sft-rl-and-opd-shape-pre-intra-and-post-cot-calibration-85e72837

·1 events·first seen 3d ago

Aliases: Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape Pre-, Intra-, and Post-CoT Calibration

Co-occurring entities

EIT-NLP PosConf

More like this (12)

Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision post-training alignment Uncertainty Calibration RL Post-Training Builds Compositional Reasoning Strategies Consistency Training Can Entrench Misalignment Human-AI Teaming Through the Lens of Calibration Toward Calibrated Mixture-of-Experts Under Distribution Shift Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal Expected Calibration Error FORCE: Efficient VLA Reinforcement Fine-Tuning via Value-Calibrated Warm-up and Self-Distillation model calibration Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

Recent events (1)

5arXiv · cs.CL·3d ago·source ↗

PosConf: Position-aware confidence calibration across SFT, RL, and OPD post-training stages

A new arXiv paper introduces a three-stage calibration framework analyzing how supervised fine-tuning (SFT), reinforcement learning (RL), and on-policy distillation (OPD) shape model confidence before, during, and after chain-of-thought reasoning. The authors find that each post-training method produces distinct calibration profiles at different reasoning stages, and that RL confidence becomes informative only after a path-commitment phase while OPD confidence degrades later. They propose PosConf, a position-aware confidence strategy that selectively uses confidence from reliable relative-position intervals, improving RL answer aggregation by 6.1 points over majority voting and OPD early stopping by up to 4.3 points.

Evaluation and Benchmarking Alignment and RLHF EIT-NLP Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape Pre-, Intra-, and Post-CoT Calibration PosConf