Entity · paper

Understanding Reasoning from Pretraining to Post-Training

paperactiveprovisionalunderstanding-reasoning-from-pretraining-to-post-training-6a14215b·1 events·first seen 45h ago

Aliases: Understanding Reasoning from Pretraining to Post-Training

More like this (12)

RL Post-Training Builds Compositional Reasoning Strategies Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning Reasoning Language Models Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers Long-context Reasoning Benchmarks Self-Supervised Pretraining Reasoning Enhancement temporally ordered pre-training Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks Large Reasoning Models

Recent events (1)

6arXiv · cs.CL·45h ago·source ↗

Controlled study reveals how pretraining shapes RL post-training returns for LLM reasoning

A new arXiv preprint uses chess as a controlled testbed to study the relationship between pretraining and RL post-training across the full LLM training pipeline, spanning models from 5M to 1B parameters. Key findings: post-RL performance is well-predicted by pretraining loss, RL reward curve slope improves approximately linearly with pretraining tokens, and RL behaves differently on easy versus hard problems (amplifying already-preferred moves vs. surfacing near-absent correct moves). The predictive pattern transfers to a math-domain 1B model, suggesting the findings generalize beyond the chess testbed.

Evaluation and Benchmarking Alignment and RLHF Understanding Reasoning from Pretraining to Post-Training