4arXiv cs.LG (Machine Learning)·
SCENT framework uses language guidance to bridge vision and olfactory sensor representations
Researchers introduce SCENT, a multimodal learning framework that uses Vision-Language Models to generate semantic scene descriptors as a bridge between electronic-nose (e-nose) signals and visual/textual representations. A smell encoder is trained to map e-nose signals into a shared embedding space, with a language-guided latent decomposition separating object-specific odors from ambient environmental contributions. Evaluated on the New York Smells dataset, SCENT achieves state-of-the-art cross-modal retrieval on smell-to-image and smell-to-text tasks, outperforming vision-only baselines. The work extends multimodal learning into olfaction, a largely unexplored sensory modality.
Related guides (1)
Related events (8)
Cross-modal masking framework improves silent speech synthesis from sEMG and lipreading
Researchers propose a masked multimodal speech synthesis framework that jointly trains on surface electromyography (sEMG) and video-based lipreading signals using modality masking to improve robustness to sensor failure or degradation. In multispeaker settings, the approach reduces word error rate by up to 14 absolute percentage points over the strongest unimodal baseline. Masking strategies outperform degradation-specific data augmentation for handling missing modalities, with phone-level analysis revealing complementary contributions across vowels and consonant groups.
A Dive into Vision-Language Models
This Hugging Face blog post provides a technical overview of vision-language model (VLM) pretraining approaches, covering architectures and training strategies used to align visual and textual representations. It surveys key models and techniques in the multimodal learning space as of early 2023. The post serves as an educational reference for practitioners working with or building VLMs.
Visual Salamandra: Pushing the Boundaries of Multimodal Understanding
BSC-LT (Barcelona Supercomputing Center Language Technologies) has released Visual Salamandra, a 7B multimodal model announced via Hugging Face blog. The post describes a vision-language model building on the Salamandra language model family. As a tier-2 source with an empty body, specific capability details and benchmark results are not available from this item alone.
smolagents Now Supports Vision-Language Models
Hugging Face has added vision-language model (VLM) support to its smolagents framework, enabling agents to process and reason over visual inputs alongside text. This update extends the agentic tooling ecosystem to multimodal workflows. The announcement comes from the Hugging Face blog, which serves as the primary communication channel for the smolagents project.
Vision Language Models (Better, faster, stronger)
A Hugging Face blog post surveys the state of vision-language models (VLMs) in 2025, covering advances in architecture, training, efficiency, and deployment. The post reviews progress across major open and closed VLMs, highlighting trends in multimodal capability, speed improvements, and practical deployment patterns. As a tier-2 commentary piece, it synthesizes the current landscape rather than announcing new research.
Vision Language Models Explained
A Hugging Face blog post providing a technical overview of vision language models (VLMs), covering their architecture, training approaches, and capabilities. The post serves as an educational resource explaining how VLMs combine visual and language understanding. As a tier-2 commentary piece, it synthesizes existing knowledge rather than presenting new research findings.
LoMo: Local Modality Substitution for Deeper Vision-Language Fusion
This paper identifies a 'carrier sensitivity' problem in Vision-Language Models (VLMs), where replacing textual queries with rendered-image equivalents causes significant performance degradation due to asymmetric roles of text and images in training data. The authors propose Local Modality Substitution (LoMo), a data curation paradigm that reformulates single-modality prompts into interleaved multimodal sequences by dynamically rendering text spans as images, enforcing cross-modal representational invariance. Evaluated across 13 multimodal benchmarks, LoMo improves over standard supervised fine-tuning by 2.67 points on LLaVA-OneVision-1.5-8B and 2.82 points on Qwen3.5-9B. The approach is architecture-agnostic and lightweight, requiring no changes to model architecture.
Semantic Browsing: structured diversity for text-to-image generation via text-level variation
Researchers introduce Semantic Browsing, a method for generating structured, navigable galleries of images where each variation corresponds to a meaningful semantic decision rather than stochastic noise. The approach decouples semantic decision-making from pixel generation by inducing diversity at the text level, using a Vision Language Model in an agentic workflow to enumerate interpretable axes of variation. This addresses the well-known mode collapse problem in text-to-image models, where strict prompt adherence reduces output diversity to a single visual interpretation.