5arXiv cs.AI (Artificial Intelligence)·
Symbal: Automated detection of systematic misalignments in MLLM-generated image captions
Researchers from Stanford introduce Symbal, a dual-stage framework using off-the-shelf foundation models to detect recurring, visually-correlated errors in multimodal LLM-generated captions. They also release SymbalBench, a benchmark of 1.7 million image-text pairs across 420 vision-language datasets covering natural and medical imaging domains. Symbal correctly identifies systematic misalignments in 63.8% of datasets, nearly 4x better than the closest baseline, and works without access to the underlying MLLM. The work addresses a practical data-quality auditing problem relevant to anyone building or evaluating vision-language datasets.
Related guides (2)
Related events (8)
Gazer: Training-free semantic correction for autoregressive visual models using MLLM feedback
Researchers introduce Gazer, a training-free framework that integrates multimodal large language model feedback into the sampling loop of autoregressive visual models (AVMs) to correct semantic errors during generation. The system operates in two stages: Reflective Diagnosis identifies semantic errors in intermediate generation states, and Semantic Correction rewinds and adjusts the generation trajectory to better match the target prompt. Experiments on compositional image and video benchmarks show improved semantic alignment and compositional accuracy across multiple AVMs without additional training. The work addresses a known weakness of next-scale prediction AVMs, where semantic errors accumulate across discrete generation scales.
CSB dataset benchmarks a decade of VLM progress on complex social scene understanding
Researchers introduce the Complex Social Behavior (CSB) dataset of 100 images depicting complex social interactions, used to evaluate nine vision-language models spanning 2017–2025 against human descriptions and a gold standard. MLLMs have largely closed the accuracy gap with top-ranked human descriptions and nearly eliminated most error types (object detection, recognition, hallucination, scene understanding), with spatial dependence errors being the notable remaining failure mode. The study also finds that MLLMs have eliminated the accuracy gap between simple MS-COCO scenes and complex social scenes, a gap that pre-MLLM models struggled with significantly.
Training framework reduces calibration error 60%+ in Medical VQA multimodal LLMs
A new arXiv preprint proposes a finetuning framework to improve verbalized uncertainty calibration in multimodal LLMs applied to Medical Visual Question Answering. The composite loss function combines Brier-style calibration, anchor regularization, contrastive image-text alignment, and KL-based stabilization, evaluated on MedGemma 4B IT and Qwen2-VL 7B Instruct across three medical VQA benchmarks. The method reduces calibration error by 60% or more and improves discrimination by 26% or more while preserving predictive accuracy, outperforming prompting-, sampling-, and training-based baselines.
Social Gaze Consistency as a Semantic Cue for AI-Generated Image Detection
This paper introduces Social Gaze Consistency (SGC), a high-level semantic detection axis based on the mutual coherence of gaze direction, head-eye alignment, and pupil placement between interacting individuals in images. The authors construct a controlled diagnostic dataset with region-specific gaze perturbations and a Block-Compositional Caption Supervision scheme to train detectors without generator-fingerprint memorization shortcuts. Cross-architecture validation shows +3.7 pp improvement on the COCOAI Interaction subset when applied to FakeVLM, with gains transferring from a single inpainter (FLUX.1-Fill) to multi-generator suites. The work argues that diffusion models share a spectral weakness in periocular structure, making gaze coherence a robust, backbone-agnostic detection signal orthogonal to existing low-level artifact methods.
MAGIC: Multimodal Alignment & Grounding-aware Instruction Coreset for Vision-Language Models
MAGIC is a training-free coreset selection method for multimodal instruction tuning that uses three intrinsic signals—Multimodal Gain, Bridging Relevance, and Skill-Neuron Signatures—to identify compact, behaviorally faithful training subsets without backpropagation. The method operates in a three-stage pipeline: filtering low-gain examples, ranking by a quality objective, and bucket-wise budget allocation over neuron signatures. On LLaVA-665K and Vision-Flan datasets with 20% data budgets, MAGIC matches or slightly exceeds full fine-tuning performance (100.3% and 101.6% relative) while reducing wall-clock training time by 73.7%. Results transfer to LLaVA-1.5-7B and -13B target models.
Multilingual word-level forced alignment using MMS and learned dynamic programming outperforms MFA
Researchers present a forced alignment system combining Meta's Massively Multilingual Speech (MMS) model with a self-supervised phoneme boundary detector (UnSupSeg) and a learned dynamic programming decoder. Trained on TIMIT and Buckeye, the system outperforms Montreal Forced Aligner and MMS-based alignment on both datasets and generalizes to unseen languages (Dutch, German, Hebrew) without additional training. The approach claims potential to scale to 1100+ languages supported by MMS, making it relevant for low-resource speech processing pipelines.
Visual Salamandra: Pushing the Boundaries of Multimodal Understanding
BSC-LT (Barcelona Supercomputing Center Language Technologies) has released Visual Salamandra, a 7B multimodal model announced via Hugging Face blog. The post describes a vision-language model building on the Salamandra language model family. As a tier-2 source with an empty body, specific capability details and benchmark results are not available from this item alone.
Moment-Video: Benchmark Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
Moment-Video is a new benchmark of 1,000 human-verified video-QA pairs designed to evaluate how well video multimodal large language models (MLLMs) handle brief, localized visual events that may span only a few frames. The benchmark covers 7 domains and 25 subcategories across four task types: Temporal Occurrence, Temporal Counting, Action Description, and Temporal Reasoning. Evaluation of 33 proprietary and open-source models reveals severe deficiencies: the best model (Seed-2.0-Pro) achieves only 39.6% accuracy, while most open-source models score below 25%. Diagnostic analyses show that denser frame sampling helps but does not resolve the bottleneck, pointing to fundamental limitations in how current video MLLMs represent and preserve transient visual evidence.