Researchers introduce SAMPA, a Whisper large-v3 fine-tuned model for automatic prosodic boundary segmentation in Brazilian Portuguese speech. The model is trained on the NURC-SP dataset and achieves F1=0.731 on held-out test data and F1=0.796 on the out-of-distribution MuPe-Diversidades dataset. The work addresses a gap in NLP tooling for Brazilian Portuguese, where prior approaches relied on rule-based or traditional ML methods rather than modern deep learning.
OpenAI introduced Whisper, an open-source automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. The model demonstrates strong robustness to accents, background noise, and technical language, approaching human-level accuracy in English transcription. Whisper supports transcription in multiple languages as well as translation to English, and the weights and inference code were released publicly.
Meta has released SAM Audio, a unified multimodal audio separation model that accepts text, visual, and temporal span prompts to isolate sounds from complex audio mixtures. The system is powered by Perception Encoder Audiovisual (PE-AV), an extension of Meta's open-source Perception Encoder released earlier in 2025, and uses a flow-matching diffusion transformer architecture. Alongside the model, Meta is releasing SAM Audio-Bench (the first in-the-wild audio separation benchmark) and SAM Audio Judge (an automatic evaluation model for audio separation). All components are available today via the Segment Anything Playground.
Researchers propose a masked multimodal speech synthesis framework that jointly trains on surface electromyography (sEMG) and video-based lipreading signals using modality masking to improve robustness to sensor failure or degradation. In multispeaker settings, the approach reduces word error rate by up to 14 absolute percentage points over the strongest unimodal baseline. Masking strategies outperform degradation-specific data augmentation for handling missing modalities, with phone-level analysis revealing complementary contributions across vowels and consonant groups.
This Hugging Face blog post provides a practical guide for fine-tuning OpenAI's Whisper model for multilingual automatic speech recognition using the Transformers library. It covers dataset preparation, training configuration, and evaluation using the Word Error Rate metric. The post targets practitioners seeking to adapt Whisper to low-resource or domain-specific languages.
Researchers adapted Whisper to a single dysarthric speaker using up to 100.8 hours of read speech and user corrections collected via a mobile app, reducing word error rate from a high baseline to 9.7%. Fine-tuning outperformed LoRA adaptation and the Qwen3-ASR foundation model in this personalized setting. The study demonstrates that speaker-specific fine-tuning of foundation ASR models can reach practical deployment quality for dysarthric users.
Researchers present a speech-based evaluation system for the German Syndrom-Kurz-Test dementia screening battery, combining transcript-derived scores with Whisper embeddings to reduce transcription scoring errors. The system also approximates expert overall ratings even when motor (nonverbal) subtests are omitted, addressing a key accessibility limitation of speech-only assessment. Models show strong correlation with expert ratings and effective discrimination between cognitive status groups.
Researchers propose WordVoice, a framework enabling explicit, decoupled word-level control over acoustic attributes (duration, boundary, energy, pitch, tone) in LLM-based text-to-speech systems. The work introduces a bound-token mechanism for acoustic planning within the LLM and a fine-grained modulation module to align discrete tokens with continuous waveforms. Accompanying the model is WordVoice-5A, a 4.7k-hour bilingual annotated dataset with five-dimensional word-level labels. The system targets high-precision applications such as audiobook narration and video dubbing where implicit TTS generation falls short.
ActiveSAM is a training-free, zero-shot inference framework that wraps Segment Anything Model 3 (SAM 3) to perform open-vocabulary semantic segmentation more efficiently. It estimates an image-conditioned active class subset at low resolution before running full-resolution decoding only on retained classes, using bucketed prompt multiplexing and margin-aware background calibration. Across eight benchmarks, it outperforms the prior state-of-the-art SegEarth-OV3 by ~1.4 mIoU on average while running up to 5.5x faster on large-vocabulary datasets, with strong robustness to image corruption relevant to autonomous driving and embodied AI.