math-23e16352·4 events·first seen Aliases: MATH
Researchers investigate whether LLMs can distill solution traces into reusable natural-language abstractions (strategies, reminders) and apply them to future problems, analogous to human experiential learning. Using the MATH training set, abstractions are extracted by either a stronger teacher model or the LLMs themselves into a retrievable library, then applied via inference-time retrieval or RL with abstraction-augmented prompts. Both modes improve performance on mathematical and logical reasoning benchmarks, with self-extracted abstractions matching teacher-extracted ones. The framework generalizes across datasets and models, suggesting a promising direction for continual self-improvement in LLMs.
PPL-Factory is a data selection framework for LLM fine-tuning that combines task-aware perplexity scoring with budget-aware selection criteria, distinguishing between language modeling and reasoning task objectives. Experiments on GSM8K show the method outperforms state-of-the-art data selection baselines using only 1% of training data, and with 10% of data exceeds full-data fine-tuning by 0.9 points on GSM8K and 4.8 points on MATH. The approach addresses a known limitation of existing perplexity-based methods that score entire sequences without accounting for task-specific learning objectives.
Researchers introduce MAST (Mechanism-Aligned Selective Targeting), a method for selectively unlearning capabilities induced by reinforcement learning from verifiable rewards (RLVR) in language models while minimizing collateral damage to retained knowledge. The approach ranks attention-projection tensors by off-principal energy and gradient coupling to identify a targeted subset for update, rather than applying full-parameter gradient ascent. Evaluated on Qwen2.5-Math-1.5B and Qwen3-1.7B-Base, MAST achieves statistically significant forgetting on target MATH problems while preserving GSM8K performance, whereas full-parameter unlearning collapses retained capabilities. The method generalizes across seeds and unlearning objectives (NPO/SimNPO).
DeepSeek has released DeepSeek-R1-Lite-Preview, a reasoning-focused model claiming o1-preview-level performance on AIME and MATH benchmarks. The model features a transparent, real-time chain-of-thought process and demonstrates inference scaling behavior where longer reasoning chains yield better results. DeepSeek has indicated that open-source model weights and a full API are forthcoming. The model is currently accessible via chat.deepseek.com.