A new arXiv paper investigates cross-lingual transfer learning from Sinhala to improve automatic speech recognition for Dhivehi, a severely low-resource language of the Maldives. Across 17 experiments spanning five transfer paradigms, the best system—continual pre-training on Sinhala followed by Dhivehi fine-tuning with KenLM—achieves 12.89% WER, outperforming the Dhivehi-only baseline by 13.50% WER. A Turkish control experiment confirms that linguistic relatedness, not just additional data, drives the improvement.
A new arXiv preprint compares human listeners against three off-the-shelf ASR systems (Whisper-large-V3, Google Chirp 3, and Omnilingual) on recognizing continuous Dutch speech from a single speaker with severe dysarthria. Both humans and ASR systems exceeded 70% WER on average, confirming the extreme difficulty of dysarthric speech recognition. Fine-tuning on dysarthric speech substantially reduced WER, with personalized models outperforming human listeners, though WER remained above 23%. The study highlights the need for personalized ASR approaches for dysarthric speakers.
A new arXiv paper conducts a broad empirical study of cross-lingual transfer in few-shot in-context learning (ICL), spanning seven tasks, six models, and a typologically diverse set of languages. The study finds that conventional heuristics from supervised fine-tuning — such as relying on linguistic similarity or data availability — do not consistently transfer to the ICL regime. The authors also analyze language confusion as a key obstacle in generative cross-lingual ICL and propose alternative heuristics for source language selection.
Researchers identify that English-centric byte-level tokenizers cause catastrophic autoregressive collapse when lightweight ASR models like Moonshine are applied to morphologically rich languages like Bengali. They propose a vocabulary transplantation pipeline that swaps in BanglaBERT's WordPiece vocabulary, reducing token fertility from 9.16 to 1.30 and cutting autoregressive sequence length by 85.8%. The modified model achieves 21.54% WER and an RTF of 0.0053 on the 882-hour Lipi-Ghor dataset, offering a reproducible blueprint for cross-script adaptation of compact ASR models without retraining from scratch.
Researchers propose a pipeline that uses LLMs to generate scenario-level dialogues and TTS to synthesize multi-speaker audio, creating simulated conversational training data for ASR systems. Evaluated on the Hungarian BEA-Dialogue benchmark, a model trained on 67 hours of real plus 636 hours of synthetic data outperforms a zero-shot model trained on 2,700 hours of real Hungarian speech. The study tests five LLM families under multiple budget and mixing configurations using a FastConformer-Large backbone, finding that generator choice and data composition significantly affect gains.
Researchers introduce BayLing-Duplex, a speech language model that achieves native full-duplex interaction — simultaneous listening and speaking — using a single autoregressive LLM with no auxiliary VAD or turn-taking module. Built by fine-tuning GLM-4-Voice on 400K samples plus a lightweight DPO stage, it reaches 92% turn-taking success and 100% interruption success on InstructS2S-Eval, and improves speech-response quality substantially over Moshi. The approach adds only special tokens to the standard vocabulary, making it portable across LLM architectures without architectural changes.
Researchers propose an RL-based training approach for translating extremely low-resource or unseen languages by rewarding models for extracting and applying in-context linguistic knowledge (e.g., grammar books) rather than memorizing specific languages. Using chrF as a surface-level reward signal, RL-trained models outperform both in-context learning and supervised fine-tuning on completely unseen languages at test time. The work extends outcome-based RL beyond math and coding reasoning tasks, suggesting broader applicability to language learning from context.
Researchers propose a post-training alignment method using reinforcement learning to improve interactivity in full-duplex spoken dialogue models, which can listen and speak simultaneously. The method addresses four canonical axes of interactivity—pause handling, turn-taking, backchanneling, and user interruption—each with axis-specific reward functions, plus an LLM-based reward to prevent semantic degradation. The approach is applied to two open-source models, Moshi and PersonaPlex, showing consistent improvements in both offline and real-time multi-turn evaluation.
Researchers introduce Lychee-FD, a native end-to-end full-duplex Spoken Language Model (SLM) framework that addresses modality interference between acoustic and semantic processing. The paper identifies gradient conflicts arising from shared deep parameter spaces as the root cause of performance degradation in full-duplex SLMs, and proposes a hierarchical parameter separation strategy with a dedicated semantic alignment channel. Experiments show +7.4% improvement on Spoken QA and +28.5% on FullDuplexBench 1.5 without inference efficiency loss.