3arXiv cs.CL (Computation and Language)·
Novel training criterion reduces shortcut reliance in L2 spoken English automated scoring systems
A new arXiv paper introduces a training criterion designed to reduce shortcut reliance in automated language proficiency assessment systems, targeting both audio-based and ASR-text-based scoring pipelines. The authors demonstrate that transformer-based auto-markers over-rely on exploitable features relative to human raters, creating malpractice opportunities for test-takers. The proposed criterion reduces this over-reliance, bringing model behavior closer to human reference correlations.
Related guides (1)
Related events (8)
Audit reveals protocol-level shortcuts in large audio-language model judges for speech evaluation
Researchers audit 'protocol-level shortcuts' in large audio-language models (LALMs) used as automatic judges for speech evaluation, testing across three deployment protocols: feature-blueprint judging, reference-conditioned judging, and pairwise A/B comparison. Across six judges and four attributes, several LALMs are found to rely on shortcuts rather than actual audio content — for example, incorrect specialist labels collapse emotion accuracy to 0.10 or below for five judges, and Qwen3-Omni-Thinking shows position bias in A/B comparisons. The findings indicate that high human-agreement scores can overstate judge validity, and that each model-protocol pair requires its own shortcut probe to be trustworthy.
Systematic study reveals effectiveness-fluency trade-offs in LLM conditioning methods
A new arXiv paper systematically evaluates a range of LLM conditioning methods across both concept injection and removal scenarios, finding that efficient steering methods often degrade fluency significantly. A key finding is that activation steering is substantially less effective on instruction-tuned models than on base models, a previously overlooked interaction. Simple prompting and supervised fine-tuning work for concept injection but not removal, and cheap textual metrics are found to correlate well with expensive LLM-as-judge evaluations.
Multi-stage explainability framework translates transformer speech models into clinical cognitive impairment narratives
A new arXiv preprint proposes a framework for making transformer-based speech cognitive impairment detection clinically interpretable by combining SHAP token attribution, linguistic feature analysis, and a four-stage LLM reasoning pipeline using LLaMA-3.1-70B-Instruct. The system is built on the SpeechCARE-Adaptive Gating Network multimodal model (F1=72.11% on NIA PREPARE) and maps outputs to four cognitive-linguistic dimensions. Physician evaluation on 70 samples showed strong alignment with clinical profiles and a System Usability Scale score of 82/100, suggesting practical clinical workflow integration potential.
Cross-rubric generalization for automated essay scoring using LLM fine-tuning with trait-based representations
A new arXiv paper introduces a framework for automated essay scoring (AES) that generalizes to previously unseen scoring rubrics, rather than just unseen prompts. The approach uses rubric-agnostic intermediate representations called 'traits' combined with target-essay supervision, achieving a 5.0% macro F1 improvement over a baseline in the hardest generalization setting. A fine-tuned Llama-based model outperforms GPT-5-mini prompting by 2.1% macro F1 and trails GPT-5 by only 1.9%, demonstrating that structured intermediate representations improve rubric generalization.
Continual learning approach for disfluency-aware ASR with explicit disfluency tokens
A new arXiv preprint addresses the challenge of transcribing disfluent speech (hesitations, repetitions, fillers) in ASR systems, which typically omit such markers causing information loss. The authors introduce explicit disfluency tokens into a pretrained ASR model and apply continual learning to adapt across datasets with varying disfluency distributions while mitigating catastrophic forgetting. The work identifies a trade-off between disfluency marker learning and general ASR performance, and finds a consistent cross-attention head mechanism shared across continual learning methods.
Energy-based transformers as unified predictors of reading difficulty in computational psycholinguistics
A new arXiv preprint introduces energy-based transformer measures as predictors of human reading difficulty, evaluated across three reading-time corpora (Natural Stories, UCL eye-tracking, UCL self-paced reading). The energy measure outperforms surprisal alone and appears to subsume both surprisal and attention entropy effects, suggesting it could serve as a single unified predictor. The work connects transformer language models to Hopfield networks and dense associative memory literature, marking the first application of energy-based transformer measures in computational psycholinguistics.
CTC oracle gap anatomy: acoustic scoring saturates, linguistic MBR decoding recovers WER
A new arXiv paper systematically diagnoses why CTC-internal N-best rescoring fails to improve over greedy decoding on LibriSpeech, showing that blank-path proliferation causes a 53% degradation in rank correlation between CTC scores and WER as beam size grows. The authors demonstrate that the bottleneck is linguistic rather than acoustic: MBR decoding with RoBERTa pseudo-log-likelihood achieves 9% relative WER reduction on LibriSpeech test-other and generalizes across two architectures and three domains. The paper also analyzes MWER sequence-level fine-tuning failure at near-converged checkpoints, attributing collapse to a vanishingly small training oracle gap.
Rubric-Conditioned Self-Distillation: structured feedback for reasoning model post-training
A new arXiv preprint proposes Rubric-Conditioned Self-Distillation (RCSD), a post-training framework that replaces scalar reward signals and noisy chain-of-thought annotations with structured rubrics for fine-grained credit assignment. The method conditions a teacher model on criterion-level rubrics to provide token-level guidance on the student's own sampled trajectories, avoiding reliance on a single reference rationale. Evaluated on science reasoning benchmarks, RCSD outperforms GRPO by 1.0 points and OPSD by 0.9 points on average.