paper
Gaze Heads: How VLMs Look at What They Describe
paperactiveprovisional
gaze-heads-how-vlms-look-at-what-they-describe-f3ab77a7·1 events·first seen 2d agoAliases: Gaze Heads: How VLMs Look at What They Describe
Co-occurring entities
More like this (12)
Gaze Heads: How VLMs Look at What They DescribeVisualMemLabVLA: Grounding Vision-Language-Action Models in Scientific LaboratoriesVision-Language Modelsgender bias in VLMsvisual language modelFakeVLMSocial Gaze ConsistencyThe Lipreading Gap: Do VSR Models Perceive Visual Speech Like Human Lipreaders?GLM-4-VoiceVision-Language-Action modelsSmolVLM
Recent events (1)
Gaze Heads: Attention heads in VLMs that track and control image region description
Researchers identify a small set of attention heads in vision-language model backbones, called 'gaze heads', whose attention patterns track the image region currently being described. Using comic strips as a controlled testbed, they show that intervening on the top-100 gaze heads (fewer than 9% of all heads) can steer the model to describe any chosen region at 83.1% accuracy, without retraining. The mechanism generalizes across model sizes from 2B to 32B parameters and to natural images (COCO), establishing a practical inference-time control lever for multimodal models via mechanistic analysis.