4arXiv cs.CL (Computation and Language)·
DIRECT framework combines DPO and controlled decoding for efficient sequence labeling with LLMs
DIRECT is a proposed framework for sequence labeling that applies Direct Preference Optimization (DPO) after supervised fine-tuning to improve task alignment, then uses a controlled decoding process that enforces fixed output formats and restricts predictions to candidate label sets. A template-filling mechanism with KV Cache reuse reduces redundant computation by generating only label tokens. Experiments across eight datasets show improvements in both accuracy and inference efficiency over existing LLM-based sequence labeling methods.
Related guides (3)
Related events (8)
Preference Tuning LLMs with Direct Preference Optimization Methods
A Hugging Face blog post surveys Direct Preference Optimization (DPO) and related preference tuning methods for aligning large language models. The post covers the landscape of DPO variants and their practical application via the TRL library. It serves as a technical reference for practitioners implementing RLHF alternatives.
DIRECT: Adaptive test-time compute routing for embodied VLM planners
Researchers introduce DIRECT, a routing framework that dynamically allocates test-time compute for Vision-Language Models acting as embodied planners, using multimodal scene context to decide per-prompt how much compute to spend. Experiments on VLABench and RoboMME benchmarks show that different scaling axes (chain-of-thought depth, model size, memory history) yield qualitatively distinct gains, and that naive uniform scaling is wasteful. On a physical Franka arm, DIRECT matches or exceeds a stronger model's success rate at up to 65% lower average latency, improving the success-cost Pareto frontier.
AGDO: Attention-guided denoising and optimization framework improves diffusion language model reasoning
Researchers propose AGDO, a framework that replaces random masking in diffusion large language models (dLLMs) with attention-guided denoising order and token weighting during fine-tuning and reinforcement learning. The work is motivated by an empirical finding that tokens with stronger attention to unmasked context are more stable and critical for reasoning. Experiments on math and coding benchmarks show AGDO outperforms existing post-training methods for dLLMs, advancing the case for attention-aware training in parallel-decoding language models.
Preference Optimization for Vision Language Models
This Hugging Face blog post covers the application of Direct Preference Optimization (DPO) to vision-language models (VLMs). It likely discusses how preference learning techniques originally developed for text-only LLMs can be adapted to multimodal settings. The post addresses training methodology for aligning VLMs with human preferences across both visual and textual modalities.
Fine-tune Llama 2 with DPO
This Hugging Face blog post provides a practical guide to fine-tuning Llama 2 using Direct Preference Optimization (DPO) via the TRL library. It covers the alignment technique that bypasses the need for a separate reward model compared to RLHF, walking through dataset preparation, training configuration, and implementation details. The post targets practitioners looking to apply preference-based alignment to open-weights models.
Distill to Detect (D2D): surfacing stealth preferential biases in LLMs via cartridge distillation
Researchers introduce Distill to Detect (D2D), a method for exposing hidden preferential biases in language models by distilling the distributional shift between a suspected model and its base into a KV-cache prefix adapter (cartridge), amplifying the bias signal into generated text. The work addresses a fundamental asymmetry in bias detection: stealth biases introduced anywhere in a model's supply chain can remain invisible to text-based inspection, weight analysis, or representation probing. D2D is grounded in a theoretical framework based on Fisher-weighted projection of logit distribution shifts and is validated across multiple bias types. The technique offers a practical auditing tool for detecting hidden behaviors in deployed LLMs.
DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Speech Translation with SpeechLLMs
The paper proposes Decoder-Only Attention (DOA), a training-free streaming policy for simultaneous speech-to-text translation (SimulST) that works with off-the-shelf decoder-only Speech LLMs. DOA derives proxy alignment signals from self-attention rather than cross-attention, enabling long-form simultaneous translation without retraining. Experiments on Phi4-Multimodal and Qwen3-Omni demonstrate low-latency performance approaching offline decoding quality, validating that decoder self-attention contains sufficient alignment information for streaming decisions.
ASRD: Training-free anchor-guided revocable decoding for diffusion LLMs improves accuracy and throughput
A new arXiv preprint introduces ASRD (Anchor Supervised Revocable Decoding), a training-free framework for improving decoding quality in diffusion large language models. The method addresses error propagation and local error reinforcement in revocable decoding by separating trusted 'anchor tokens' (identified via temporal consistency) from uncertain candidates, then applying anchor-guided generation and anchor-perturbed verification. Experiments on math and coding benchmarks show up to 6.4% accuracy improvement and 7.2× inference throughput gains over remasking baselines.