Entity · paper

Connecting Speech to Words through Images

paperactiveconnecting-speech-to-words-through-images-84687ed7·1 events·first seen Jun 16, 2026

Aliases: Connecting Speech to Words through Images

More like this (12)

CapSpeech-TTS Thinking-with-Images How Do Instructions Shape Speech? Cross-Attention Attribution for Style-Captioned Text-to-Speech Speech-to-Speech speech-to-avatar systems Hearsay: Vision-Language Medical Diagnoses Without an Image Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns huggingface/speech-to-speech Interleaved Speech Language Models Latently Work In Text Artificial Analysis Text to Image image-semantic guided poetry detection vision-language grounding

Recent events (1)

4arXiv · cs.CL·Jun 16, 2026·source ↗

Visually grounded method connects spoken words to written text without textual supervision

Researchers present a method for learning mappings between written words and spoken audio using only images and spoken captions, with no explicit text supervision. The approach uses image captioning to build a written vocabulary, then applies unsupervised word discovery to align spoken utterances to those words. The system outperforms a neural baseline on spoken word retrieval and keyword spotting tasks in English, with implications for low-resource language processing.

Evaluation and Benchmarking Multimodal Progress Connecting Speech to Words through Images