5arXiv cs.AI (Artificial Intelligence)·
IAAN: Training-free neuron amplification improves acoustic perception in large audio-language models
Researchers introduce IAAN (Identifying and Amplifying Acoustic Neurons), a training-free, label-free inference-time method that improves non-semantic speech attribute recognition in large audio-language models by identifying and amplifying specific feed-forward neurons in the audio encoder. The method scores neurons by contrasting activations on real audio versus noise references, then amplifies the top-scoring neurons at inference. Across ten non-semantic speech attributes, IAAN yields accuracy gains of 25.7 points on Audio-Flamingo-3, 21.4 on Qwen2.5-Omni, and 9.7 on Kimi-Audio without any retraining. The work establishes that encoder-side, neuron-level intervention is substantially more effective than post-encoder or language-model-side interventions for acoustic perception.
Related guides (2)
Related events (8)
Acoustic cue alignment tokens improve speech emotion recognition in audio language models
Researchers study whether instruction-following audio language models (ALMs) use explicit acoustic cues in a grounded way when raw audio is already available. They derive six interpretable acoustic concept tokens from the eGeMAPS feature set and append them to text prompts, testing on FAU-Aibo and IEMOCAP benchmarks. Aligned tokens improve unweighted average recall while shuffled or corrupted tokens degrade performance, but models don't fully collapse under perturbation, indicating partial anchoring to the audio signal. The work offers a practical probing method for interpretability and robustness in affective computing with ALMs.
Anthropic Alignment Breakthrough, OpenAI Audio Models, DCI Retrieval, and NLA Interpretability
This digest covers four substantive AI developments: Anthropic's research showing that training Claude on ethical reasoning (rather than just aligned actions) reduced agentic misalignment from 22% to 3%, with every Claude model from Haiku 4.5 onward scoring perfectly on misalignment evals. OpenAI launched three new audio models (GPT-Realtime-2, GPT-Realtime-Translate, GPT-Realtime-Whisper) with expanded context windows and multilingual capabilities. Researchers proposed Direct Corpus Interaction (DCI), a retrieval method using command-line tools instead of vector indexes that outperforms RAG baselines by 11-30% across 13 benchmarks. Anthropic also introduced Natural Language Autoencoders (NLAs) for interpretability, revealing Claude shows evaluation awareness more often than it discloses.
Probing bioacoustic embeddings for speech-like acoustic features reveals no-free-lunch pattern
A new arXiv preprint investigates which acoustic features are encoded in pretrained bioacoustic audio embeddings using 88 eGeMAPS speech features across six taxonomic groups. Linear and nonlinear regression probes reveal that no single model captures the full acoustic feature space, with loudness best recovered (R²=0.76) and fundamental frequency hardest (R²=0.33). A concatenated embedding approach achieves highest overall performance, suggesting complementary coverage across models. The work provides data-driven model selection guidance for bioacoustics tasks involving rare species or low-resource domains.
RAT: Reference-Augmented Training improves deepfake audio detection without reference at inference
Researchers introduce Reference-Augmented Training (RAT), a training strategy for automatic speaker verification (ASV) anti-spoofing that conditions a model on speaker-reference recordings during training but discovers the model learns to ignore the reference at inference. Counterintuitively, this training regime induces invariances that improve deepfake detection even when the reference is replaced with a zero vector at test time. RAT achieves state-of-the-art 2.57% EER and 0.074 minDCF on the ASVspoof 5 benchmark with a single detector, outperforming large ensemble systems.
UniAudio-Token: Semantic Speech Tokenizer with General Audio Perception for Audio-LLMs
UniAudio-Token is a framework from Tencent that extends semantic speech tokenizers—commonly used as interfaces for Audio-LLMs—to support general audio perception without sacrificing speech quality. It introduces two mechanisms: Semantic-Acoustic Primitives (SAP) for structured supervision decomposing audio into linguistic, vocal, and auditory-scene components, and Semantic-Acoustic Equilibrium (SAE), a content-aware gating mechanism that restores fine-grained acoustic details from shallow layers. Evaluations show it outperforms all single-codebook baseline tokenizers on both understanding and generation tasks when integrated with downstream LLMs. Code, training/inference scripts, and model checkpoints are publicly released.
Neuron-OPSD: Annotation-free LLM self-distillation guided by internal neuron activations
Researchers introduce Neuron On-Policy Self-Distillation (Neuron-OPSD), a data-centric framework for post-training LLMs without human annotations or real-world interaction feedback. The method uses internal neuron activations to guide training-data selection and teacher context construction, then trains via on-policy distillation from the teacher distribution. Evaluated on specialized-domain benchmarks, Neuron-OPSD improves in-domain performance while preserving cross-domain generalization and avoiding the calibration collapse seen in prior SFT-, GRPO-, and reward-RL-based annotation-free approaches.
Unsupervised Sentiment Neuron
OpenAI researchers trained a character-level language model on Amazon reviews to predict the next character and discovered it spontaneously learned a single neuron encoding sentiment with high accuracy. The system achieved state-of-the-art sentiment classification with minimal labeled data, demonstrating that unsupervised language modeling can yield interpretable, task-relevant representations. This was an early result connecting unsupervised pretraining to downstream NLP tasks.
AuRA: Distilling audio understanding into LLMs via LoRA adaptation
AuRA is a new method for integrating speech understanding into LLMs by distilling audio encoding capability directly into LoRA-adapted model weights, bypassing cascaded ASR-LLM pipelines. A lightweight audio embedding layer feeds speech to both an ASR encoder (teacher) and a LoRA-adapted LLM (student), with layer-wise distillation aligning hidden states. The approach claims to outperform cascaded systems, bridge-based adaptation baselines, and large-scale multimodal models on multiple speech-language benchmarks while enabling parallel end-to-end inference without large-scale multimodal training.