DramaSR-LRM: Reasoning LLM with multimodal tool-use for speaker recognition in TV dramas
Researchers introduce DramaSR-532K, a large-scale benchmark of 532K annotated dialogue lines across 900+ characters from long-form TV dramas, targeting multimodal speaker recognition. They also propose DramaSR-LRM, a system built on a large reasoning model that uses multimodal tool-use to aggregate auditory, linguistic, and visual cues for speaker attribution. The approach significantly outperforms baselines, especially on short utterances where acoustic biometrics alone are unreliable. Data and code are to be publicly released.
Related guides (2)
Related events (8)
RL-based alignment improves interactivity in full-duplex spoken dialogue models
Researchers propose a post-training alignment method using reinforcement learning to improve interactivity in full-duplex spoken dialogue models, which can listen and speak simultaneously. The method addresses four canonical axes of interactivity—pause handling, turn-taking, backchanneling, and user interruption—each with axis-specific reward functions, plus an LLM-based reward to prevent semantic degradation. The approach is applied to two open-source models, Moshi and PersonaPlex, showing consistent improvements in both offline and real-time multi-turn evaluation.
AIR: Adaptive Interleaved Reasoning with Code in Multimodal LLMs via Reinforcement Learning
Researchers propose AIR, a system that trains multimodal large language models to adaptively interleave reasoning with code execution for numerical computation tasks, going beyond prior work that focused only on visual operations. The approach combines a two-stage cold-start data pipeline, RL dataset filtering, and a group-constrained reward function for tool-invocation decisions. Experiments show a 6.1 percentage point average improvement on evaluation benchmarks, with interleaved reasoning samples gaining 9.9 pp and tool-use success exceeding 95%.
AudioDER: Deduplication-enhanced reasoning dataset for post-training large audio-language models
Researchers introduce AudioDER, a ~191k-sample post-training dataset for Large Audio-Language Models (LALMs) built via an acoustic similarity-based deduplication pipeline to reduce redundancy and improve corpus diversity. Each sample pairs an audio clip with a multiple-choice question, answer candidates, a caption, and a chain-of-thought rationale generated by Qwen3-30B. Post-training Qwen2-Audio-7B-Instruct on AudioDER yields consistent gains on audio reasoning benchmarks including MMAU-mini, MMSU, and MMAR. The work addresses a data quality gap in audio-language training rather than proposing a new model architecture.
WhoSaidIt: Human-LLM Collaborative Annotation for Multilingual Speaker-Attribute Classification
This paper proposes a human-LLM collaborative re-annotation framework for stabilizing noisy multilingual speaker-attribute labels under resource constraints. LLMs surface recurring annotation rationales through iterative expert interaction, combined with disagreement-focused sampling for targeted re-annotation. The resulting WhoSaidIt dataset covers nine speaker-attribute labels across multiple languages. Benchmarking of recent LLMs reveals substantial cross-lingual annotation divergence and highlights both capabilities and limitations of LLMs in this classification task.
ModeratorLM: Role-conditioned turn-taking for multi-party voice agents with 40%+ precision gains
Researchers introduce ModeratorLM, a voice agent system that conditions turn-taking behavior on an explicitly assigned conversational role in multi-party settings, built on a streaming speech LLM. A reasoning-augmented variant adds chain-of-thought over conversational context. Evaluated on real-world meeting data and the new RolePlayConv synthetic dataset, the system achieves over 40% improvement in turn-taking precision and 70% in recall while reducing false-positive interruptions versus non-role-conditioned baselines.
Multimodal NLP pipeline for insurance fraud detection at FNOL using synthetic dialogue and audio
A new arXiv preprint introduces a synthetic multimodal framework for insurance fraud detection at the First Notice of Loss (FNOL) stage, combining ASR, speaker diarisation, NER, regex extraction, LLM-RAG retrieval, and speaker embeddings into a rule-based risk scoring system. The framework generates synthetic agent-customer dialogue transcripts and two-speaker audio to address the scarcity of multimodal fraud datasets. Component-level evaluations show stability and transfer potential, offering a reproducible baseline for multimodal fraud detection research.
Survey: Human-View Video Understanding with MLLMs — Watch, Remember, Reason Framework
A new arXiv survey paper proposes a unified 'human-view' framework for analyzing multimodal LLM-based video understanding, organized around three functional abilities: watching (perception), remembering (memory), and reasoning. The authors introduce a formulation characterizing video understanding systems by perceptual representations, memory states, reasoning traces, and predictions, then survey methods, datasets, and benchmarks across these dimensions. The work covers challenges including spatio-temporal perception, long-video processing, streaming understanding, and faithful reasoning, with application domains spanning egocentric, sports, medical, and narrative video.
OpenMedReason: Large-scale multimodal medical reasoning corpus with 450K instances for clinical VLM training
Researchers introduce OpenMedReason, a 450K-instance open multimodal medical reasoning corpus with reasoning traces derived from human-authored biomedical literature rather than synthetic chains of thought. The dataset covers diverse medical imaging modalities and is paired with OpenMedReason-Bench, a held-out benchmark evaluating LVLMs on perception, medical knowledge, and rationale axes. Training with OpenMedReason yields a 20% average VQA accuracy improvement over base models and achieves performance within 4.2% of leading comparable-scale medical VLMs. Both the dataset and code are publicly released.