Entity · paper

Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs

paperactiveprovisionalhierarchical-acoustic-semantic-modeling-modality-separation-and-semantic-coherence-for-full-duplex-slms-f2926182·1 events·first seen 5d ago

Aliases: Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs

Co-occurring entities

Lychee-FD FullDuplexBench 1.5

More like this (12)

Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models Multimodal Large Language Models Cross-Modal Masking for Robust Silent Speech Synthesis Using sEMG and Lipreading Local Modality Substitution Leveraging Audio-LLMs to Filter Speech-to-Speech Training Data Interleaved Speech Language Models Latently Work In Text Language Modeling Loss Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates Operadic consistency: a label-free signal for compositional reasoning failures in LLMs Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models

Recent events (1)

5arXiv · cs.CL·5d ago·source ↗

Lychee-FD: Hierarchical parameter separation tackles modality interference in full-duplex spoken language models

Researchers introduce Lychee-FD, a native end-to-end full-duplex Spoken Language Model (SLM) framework that addresses modality interference between acoustic and semantic processing. The paper identifies gradient conflicts arising from shared deep parameter spaces as the root cause of performance degradation in full-duplex SLMs, and proposes a hierarchical parameter separation strategy with a dedicated semantic alignment channel. Experiments show +7.4% improvement on Spoken QA and +28.5% on FullDuplexBench 1.5 without inference efficiency loss.

Evaluation and Benchmarking Multimodal Progress Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs Lychee-FD FullDuplexBench 1.5