A new arXiv preprint derives Rademacher complexity-based generalization bounds for learning to predict intermediate iterates of projected gradient descent solvers applied to box-constrained quadratic programs. The authors propose a k-neighborhood data collection strategy that augments converged-solution datasets with intermediate solver states, increasing training data without additional solver runs. The work connects to GLENS, a data-efficient global search method, and frames the approach within the Dynamic Data-Driven Application Systems (DDDAS) paradigm for tightening data-model-optimization loops.
A new arXiv preprint presents a geometric analysis of the memorization-to-generalization delay (grokking) in neural networks, attributing it to radial inflation of hidden representations under cross-entropy optimization. The authors formalize a radial-angular decomposition of activation-space dynamics and derive three testable propositions about the effects of penalizing radial growth. Empirically, a single-hyperparameter norm penalty that constrains activations to a sqrt(d)-radius hypersphere accelerates grokking up to 6x on modular arithmetic tasks across MLPs and Transformers, and halves training steps for a 10M-parameter nanoGPT on 3-digit addition.
This paper extends Pedro Domingos' 2020 first-order path-kernel interpolation formula for gradient-descent-trained models to second-order forms. The authors derive curvature-weighted correction terms for standard SGD, an additional sampling-induced component coupling prediction curvature with mini-batch gradient noise covariance, and an extension to SGD with momentum. A concentration estimate for the terminal prediction is also established, quantifying fluctuation around the expected second-order representation.
A new arXiv paper introduces MARC, a system combining graph-neural diffusion denoising, computer-algebra energy descent, and symbolic checking to solve continuous algebraic constraint systems. The paper's central finding is a careful ablation: when matched against random multi-start under the same refinement budget, learned diffusion proposals only narrowly outperform random search, and only in high-dimensional uncoupled regimes. Across eight real-world systems from robotics, positioning, and algebra, classical multi-start solved all instances, none falling in the learning-favorable regime. The work provides a principled regime map for when learned proposals actually help, serving as a cautionary calibration for the broader trend of applying learned heuristics to combinatorial/constraint problems.
A new arXiv preprint derives explicit characterizations of generalization error as a joint function of model width, sample count, and regularization in a quadratic two-layer network with structured data. The analysis reveals a phase diagram with distinct scaling regimes governed by data-dependent power laws tied to the spectral structure of the target function. The work extends scaling law theory beyond fixed-feature or infinite-width regimes by operating in a finite-sample, feature-learning setting, and characterizes interpolation threshold transitions.
A new arXiv preprint introduces KGRL (Knowledge- and Gradient-Guided Reinforcement Learning), a neuro-symbolic algorithm for Parametrized Action Markov Decision Processes (PAMDPs) where each decision involves both a symbolic action and continuous numerical parameters. KGRL uses a Datalog knowledge base to prune infeasible actions and constrain parameter spaces, then applies gradient-based refinement to estimate optimal parameters during training and deployment. The approach improves sample efficiency and episodic return over state-of-the-art PAMDP baselines, and provides local procedural explanations for its decisions.
A new arXiv preprint introduces AdaPrefix-GRPO, a method that addresses GRPO's failure to learn from problems where no rollout succeeds by prepending correct solution prefixes and dynamically adjusting prefix length to maintain ~50% success rate per problem throughout training. The prefix assistance is gradually withdrawn so the final model solves problems unaided. On hard math benchmarks, the method more than doubles GRPO accuracy for a 0.6B model (2.1x) and achieves 1.7x improvement on AIME, while halving trace length, with larger gains on smaller models. The implementation requires only data preparation changes and a loss mask, leaving the trainer unchanged.
Researchers propose Adaptive Data Scheduling (ADS), a dual-level framework that replaces uniform sampling in RL post-training with adaptive distribution over semantic clusters and policy-boundary sample selection. Evaluated across three LLMs and seven reasoning benchmarks, ADS improves average accuracy by 5.2% over GRPO and generalizes across RL objectives. The method addresses a structural limitation in standard RL post-training pipelines by accounting for semantic data structure and evolving policy capability during training.
A new arXiv preprint introduces negative-shifted gradient descent as a method for mixed-sign spectral regularization in overparameterized linear regression, escaping structural limitations of the negative-ridge endpoint. The authors identify a Marchenko-Pastur barrier in a Gaussian spike-plus-flat model and prove that early-stopped paths improve on all admissible endpoints by a polynomial factor in risk under explicit conditions. The main theorem handles general high-effective-rank tails and recovers all head scales simultaneously, with technical control via localized Duhamel integrals and a finite-grid hold-out inequality for validation-selected algorithms.