4arXiv cs.CL (Computation and Language)·
VDAR-Router: Difficulty-aware retrieval-based LLM routing framework
Researchers introduce VDAR-Router, a training-free LLM routing framework that routes queries to appropriate models by first generating an explicit difficulty analysis of each query, then retrieving historical examples with similar difficulty profiles to estimate model suitability. The approach addresses a gap in existing routing methods that rely on surface semantics or embedding similarity without accounting for query difficulty. Experiments on three datasets show improved cost-performance trade-offs over existing baselines.
Related guides (2)
Related events (8)
PRP: Proactive Routing Paradigm for efficient visual reasoning via early difficulty estimation
Researchers introduce PRP (Proactive Routing Paradigm), a method for routing multimodal queries to either a small draft model or a large target model before generating a full output, based on jointly estimated difficulty signals. The approach uses Draft Rating Learning to give the draft model an internal confidence estimator and Joint Rating Learning to predict target model competence, enabling fine-grained instance-level routing. Experiments across multimodal reasoning benchmarks show inference acceleration without accuracy loss. The work addresses a gap in speculative/cooperative inference for vision-language models, where existing language-model routing methods fail.
Reroute: Training-free recoverable visual token routing for vision-language models
A new arXiv preprint proposes Reroute, a training-free plug-in that replaces the standard rank-and-remove visual token pruning paradigm in VLMs with a recoverable routing mechanism. Instead of permanently discarding low-ranked tokens, Reroute defers them to re-enter the candidate pool at later decoder stages, addressing the problem that token importance shifts across decoder depth. Evaluated on LLaVA-1.5 and Qwen backbones augmented with FastV, PDrop, and Nüwa pruning methods, Reroute improves grounding performance under aggressive token reduction without sacrificing general VQA accuracy. The approach preserves the theoretical compute and KV-cache budget of the underlying pruning method.
RL-trained LLMs learn retriever-specific query formulation strategies for RAG
A new arXiv paper presents the first systematic study of using reinforcement learning to teach LLMs to adapt query formulation strategies to different retrieval backends. The authors find that different retrievers have surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), making cross-retriever strategy transfer ineffective. They introduce a branching-based rollout technique to stabilize training over multi-step retrieval trajectories and show gains from retriever-specific human guidance and model scaling.
RASER: Recoverability-Aware Selective Escalation Router for Multi-Hop Question Answering
RASER introduces a family of lightweight routers that decide whether to escalate retrieval complexity for multi-hop QA without making additional LLM calls. Built on top of one-shot RAG using six derived features, RASER-2 and RASER-3 route queries to progressively more expensive retrieval strategies (PRUNE, IRCoT) only when needed. Across six LLMs and three benchmarks, the routers match SOTA F1 while consuming only 41-49% of the tokens required by always-escalating baselines.
TRACE-ROUTER: Task-level LLM routing for agentic workflows using contextual bandits
TRACE-ROUTER is a new routing framework that addresses a fundamental mismatch in enterprise LLM deployment: existing per-call routers cannot correctly attribute feedback to individual routing decisions in long-horizon agentic workflows. The system assigns each task to a single model at admission using a contextual bandit and updates its policy using the task's terminal reward, jointly optimizing accuracy and latency. On tau2-Bench, it outperforms latency-matched interpolation between individual models by 7-8 accuracy points; on Terminal-Bench it achieves 7.1 higher accuracy points than the strongest single-model baseline with 36% lower latency.
DIRECT: Adaptive test-time compute routing for embodied VLM planners
Researchers introduce DIRECT, a routing framework that dynamically allocates test-time compute for Vision-Language Models acting as embodied planners, using multimodal scene context to decide per-prompt how much compute to spend. Experiments on VLABench and RoboMME benchmarks show that different scaling axes (chain-of-thought depth, model size, memory history) yield qualitatively distinct gains, and that naive uniform scaling is wasteful. On a physical Franka arm, DIRECT matches or exceeds a stronger model's success rate at up to 65% lower average latency, improving the success-cost Pareto frontier.
AdaDSF: Adaptive Depth Sparse Framework for inference-efficient LLMs without retraining
Researchers introduce AdaDSF, a method that converts pre-trained LLMs into depth-sparse models without full retraining by using cosine similarity between layer inputs and outputs to identify redundant computation. A lightweight router selects informative tokens per layer, and a feature-preserving alignment objective maintains output quality. Evaluated on GPT-NeoX and Qwen2.5, AdaDSF reduces inference FLOPs while outperforming baselines including MoD, D-LLM, and DLO under comparable sparsity budgets.
Dual-adapter routing system improves knowledge editing precision in LLMs
A new arXiv paper introduces a route-specialized dual-adapter architecture for knowledge editing in LLMs, separating the concerns of writing edits (edit adapter) and suppressing them when irrelevant (locality adapter). A relevance router gates which adapter is applied, addressing the locality problem in memory-assisted editing. Evaluated on CounterFact, zsRE, and MQuAKE benchmarks using Llama-3.1-8B-Instruct and Qwen3-8B, the method achieves best-in-class probability-preference accuracy across all three datasets. Ablations show the gain comes from the architectural separation rather than increased parameter capacity.