5arXiv cs.LG (Machine Learning)·
TPIPS: Text-prompted image perceptual similarity metric conditioned on semantic aspects
Researchers introduce TPIPS (Text-Prompted Image Perceptual Similarity), a new perceptual metric that conditions visual similarity judgments on user-specified semantic aspects (e.g., shape vs. color) rather than collapsing them into a single scalar. The work includes a large-scale dataset of human similarity judgments over image triplets annotated across multiple free-form aspects, and benchmarks frontier VLMs against human consensus, finding a significant performance gap. A fine-tuned VLM trained on this data achieves closer alignment with human perception and enables downstream applications in text-guided retrieval, compositional search, and generative model evaluation.
Related guides (2)
Related events (8)
Imaginative Perception Tokens improve spatial reasoning in vision-language models
Researchers introduce Imaginative Perception Tokens (IPT), intermediate perceptual representations that externalize what a VLM would perceive from alternative spatial viewpoints, enabling reasoning about unobserved spatial structure. The approach is evaluated on three new tasks—Perspective Taking, Path Tracing, and Multiview Counting—using ~20K examples built on the BAGEL backbone. IPT supervision consistently outperforms textual chain-of-thought training for spatial tasks, with the authors finding that forcing spatial computation through language can degrade performance, suggesting a modality mismatch. The work provides both a practical supervision technique and a diagnostic finding about the limits of language-mediated spatial reasoning.
Real Images, Worse Judgments: Evaluating VLMs on Concreteness and Imagery
This paper evaluates whether vision-language models (VLMs) benefit from real image context when making lexical judgments about word concreteness and imagery. The authors find that real-image contexts frequently hurt alignment with human ratings, especially when visual evidence is least relevant to the word being judged. Probing and canonical correlation analysis reveal that real images cause representational shifts and increased sensitivity to spurious visual cues. Instructing models to focus on text-only content at inference time partially mitigates this degradation.
PGT: Procedurally Generated Tasks for Improving Visual Grounding in MLLMs
This paper introduces Procedurally Generated Tasks (PGT), a data-driven framework that overlays geometric primitives on images to create dense supervision signals for fine-grained visual grounding in multimodal large language models. PGT serves both as a training augmentation method and a diagnostic tool to isolate perception failures from semantic priors. Instruction tuning on LLaVA-v1.5-Instruct augmented with PGT data yields gains of up to +20% on the What'sUp benchmark and +13.3% on CV-Bench-2D. The results suggest that spatial reasoning deficits in MLLMs stem primarily from inadequate supervision rather than architectural or resolution constraints.
Multi-domain benchmark for detecting AI-generated text-rich images from GPT-Image-2
Researchers introduce a new benchmark of 8,602 images across six categories (commercial posters, infographics, academic posters, receipts, tables, UI screenshots) specifically for detecting AI-generated text-rich images produced by OpenAI's GPT-Image-2. Five zero-shot detectors are evaluated, revealing highly domain-dependent performance and severe sensitivity to JPEG compression even in the strongest conventional detector. A multimodal VLM is also explored as a detector, showing promise but limitations on structured formats. The work highlights a gap in existing benchmarks that focus on object-centric rather than text-layout-centric images.
Domain-generalized framework for pixel-level image tampering detection in VLM-generated content
Researchers propose a training framework for detecting pixel-level image tampering in content generated by modern vision-language models such as ChatGPT, Gemini, and Qwen-Image. The approach combines balanced minibatch sampling and a late-injection strategy to improve out-of-distribution robustness without overfitting to limited new-domain data. The framework outperforms the prior state-of-the-art PIXAR method by 26.1% and 26.8% relative improvement in gIoU and cIoU respectively, evaluated on OOD VLMs including GPT-Images-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5. The work is directly motivated by the growing difficulty of detecting AI-generated image manipulations as generation quality improves.
Semantic Browsing: structured diversity for text-to-image generation via text-level variation
Researchers introduce Semantic Browsing, a method for generating structured, navigable galleries of images where each variation corresponds to a meaningful semantic decision rather than stochastic noise. The approach decouples semantic decision-making from pixel generation by inducing diversity at the text level, using a Vision Language Model in an agentic workflow to enumerate interpretable axes of variation. This addresses the well-known mode collapse problem in text-to-image models, where strict prompt adherence reduces output diversity to a single visual interpretation.
Social Gaze Consistency as a Semantic Cue for AI-Generated Image Detection
This paper introduces Social Gaze Consistency (SGC), a high-level semantic detection axis based on the mutual coherence of gaze direction, head-eye alignment, and pupil placement between interacting individuals in images. The authors construct a controlled diagnostic dataset with region-specific gaze perturbations and a Block-Compositional Caption Supervision scheme to train detectors without generator-fingerprint memorization shortcuts. Cross-architecture validation shows +3.7 pp improvement on the COCOAI Interaction subset when applied to FakeVLM, with gains transferring from a single inpainter (FLUX.1-Fill) to multi-generator suites. The work argues that diffusion models share a spectral weakness in periocular structure, making gaze coherence a robust, backbone-agnostic detection signal orthogonal to existing low-level artifact methods.
Pair-In, Pair-Out (PIPO): Unified Latent Compression and Multi-Token Prediction for Efficient LLM Inference
PIPO is a new inference efficiency framework that unifies input-side latent compression with output-side multi-token prediction (MTP) by treating them as mirror operations: a compressor folds two input tokens into one latent, while an MTP head unfolds one hidden state into an additional output token. To avoid the expensive verifier pass typically required by speculative decoding, PIPO trains a lightweight confidence head using On-Policy Distillation (OPD), which naturally aligns with rejection-sampling criteria. Experiments on Qwen3.5-4B and 9B backbones across AIME 2025, GPQA-Diamond, LiveCodeBench v6, and LongBench v2 show up to 2.64× first-token-latency speedup and +7.15 pass@4 improvement over regular decoding.