Entity · model
Qwen 2.5-7B
modelactive
qwen-2-5-7b-4c046071·7 events·first seen Aliases: Qwen 2.5-7B, Qwen 2.5, Qwen-2.5-7B, Qwen 2.5 7B
Co-occurring entities
Llama-3.1-8BLlama 3.2LCBGRPODMC-OptimCWM 32BReinforcement Learning for Code OptimizationRECAPTrain the Model, Not the Reader: Decodability Supervision for Verifiable Activation ExplanationsPythia-410MMaRAGemma 2 9BMaLoRAMuSiQue2WikiMultiHopQARULER QA-2Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe CascadeTextCraftTraitFactoryORBIT
More like this (12)
Recent events (7)
Reinforcement learning for code optimization via calibrated timing rewards and adapted GRPO
A new arXiv preprint addresses the challenge of extending RL-based code generation to code optimization, where execution time is added to the reward signal. The authors identify three failure modes (measurement noise, reward sparsity, GRPO instability) and propose a three-stage solution: a new benchmark (DMC-Optim) with calibrated timing, a composed correctness-plus-speed reward with offline simulator, and adapted GRPO training. On Qwen 2.5 7B and CWM 32B, strict top-50% pass@1 improves from 18.0% to 31.3% and 30.7% to 50.4% respectively, with larger gains at stricter percentiles, while preserving correctness scores.
RECAP: Decodability supervision for verifiable neural activation explanations
A new arXiv paper identifies a fundamental flaw in natural-language autoencoder-based explanation methods for neural activations: high reconstruction scores do not certify individual factual claims, and the standard training recipe produces co-adapted private codes (false wordings that reconstruct well) in all tested runs. The authors introduce two audit protocols and RECAP (Readable Encodings via Co-trained Auxiliary Predictors), which trains linear probe heads alongside the target model to keep designated content independently decodable. On RECAP-trained models, fresh verbalizers state designated content truthfully and private codes vanish, with probes achieving AUC 0.96 for distinguishing true from false claims—and remaining robust (AUC 0.95) against adversarial explanation edits that collapse control probes to chance. The work has direct implications for mechanistic interpretability and AI safety, as it challenges a widely-used faithfulness test and proposes a more verifiable alternative.
MaLoRA and MaRA: Selective state-space adapters improve multi-hop reasoning over LoRA
A new arXiv preprint proposes two adapter families — MaLoRA (token-level dynamic scaling via Mamba recurrence) and MaRA (context-level segment retrieval via cross-segment state tracking) — as improvements over standard LoRA for language model reasoning. Evaluated on three frozen backbones (Qwen-2.5-7B, Llama-3.1-8B, Gemma-2-9B) and two multi-hop QA benchmarks (MuSiQue, 2WikiMultihopQA), the methods yield average gains of +6.8 F1 (+10.5% relative) over LoRA, with up to +18.2% relative improvement on the hardest configuration. Token-level gains also transfer to RULER QA-2 under length-stress conditions.
Early abort cascade for LLM agents predicts episode failure from hidden activations, saving ~47% inference compute
A new arXiv paper introduces a method to detect doomed LLM agent episodes early by probing internal hidden-state activations, rather than waiting for observable failure. The approach uses a cascade of calibrated per-round gates with recall budgets, guaranteeing that eventually-successful episodes survive at a user-specified rate. On TextCraft with Qwen-2.5-7B and Llama-3.2-3B, the cascade saves 37–47% of inference compute at a 90% recall target, outperforming behavior-only baselines by roughly 2x. The work provides both a practical deployment mechanism and theoretical guidance on sample complexity for certifying high recall targets.
ORBIT: Training-free multi-attribute behavioral steering via orthogonal subspace rotation
Researchers introduce ORBIT (Orthogonal Rotation-Based Intervention Technique), a training-free activation steering method that simultaneously controls multiple behavioral attributes in language models. The approach constructs a joint subspace from per-attribute steering planes via SVD and applies a single norm-preserving rotation, avoiding the norm imbalance and directional cancellation problems of naive vector summation. The authors also release TraitFactory, a new multi-attribute behavioral benchmark, and evaluate across Llama-3.2-3B, Qwen-2.5-7B, and Llama-3.1-8B. ORBIT outperforms existing training-free baselines on multi-attribute steering while better preserving output coherence.
Geopolitical Bias in LLMs Originates in Post-Training, Not Pre-Training Data
A study testing seven open-weight LLM pairs (base vs. chat models) across seven labs finds that geopolitical bias is introduced during post-training rather than inherited from pre-training data. Six of seven labs showed post-training shifts favoring the developer's home country or region, with Alibaba's Qwen 2.5 showing the most extreme shift (18x increase in China-favourability log-odds). The effect is also language-dependent: Mistral becomes pro-France only under French prompting. The authors argue this implicates alignment and RLHF processes as active shapers of geopolitical perspective, calling for greater transparency and auditing of post-training pipelines.
DiSP: A Sample-and-Judge Framework for Efficient In-Context Learning Demonstration Selection
DiSP reframes ICL demonstration selection as a prediction problem rather than a search problem, arguing it is cheaper to judge whether a query-context pair will succeed than to find an optimal context. The framework stratifies queries by difficulty using a lightweight router, trains level-specific judges, and applies stop-on-acceptance judging under an explicit budget. Evaluated on five classification datasets with Llama 3-8B and Qwen 2.5-7B, DiSP improves over strong learned selection baselines by up to 3.4% accuracy while achieving up to 23x wall-clock speedup.