Entity · paper

Towards Robustness against Typographic Attack with Training-free Concept Localization

paperactiveprovisionaltowards-robustness-against-typographic-attack-with-training-free-concept-localization-7ce49987·1 events·first seen Jul 3, 2026

Aliases: Towards Robustness against Typographic Attack with Training-free Concept Localization

Co-occurring entities

ViT (Vision Transformer)RIO-Bench CLIP

More like this (12)

How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models Training-Free Semantic Correction for Autoregressive Visual Models TILDE: TILt-based Distributional Erasure for Concept Unlearning Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures Reference-Augmented Training Achieving Precise Text-To-Cypher Via Grounded Knowledge Graph Data Generation Language Modeling Loss Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation Graph-Regularized Agentic Context Evolution Unintended Effects of Geographic Conditioning in Large Language Models Robust Classification

Recent events (1)

5arXiv · cs.CL·Jul 3, 2026·source ↗

Training-free mechanistic defense against typographic attacks on CLIP-based vision encoders

Researchers propose a training-free method to defend CLIP-based vision encoders against typographic attacks, where irrelevant text embedded in images biases visual representations toward lexical rather than semantic meaning. The approach uses sampling-based mechanistic interpretability to identify specific Vision Transformer attention heads responsible for encoding lexical information, then applies targeted circuit-level interventions to suppress this behavior. Without any retraining, the method outperforms both supervised and training-free baselines on object classification and improves Visual Question Answering accuracy under typographic attack conditions on RIO-Bench across several state-of-the-art LVLMs.

Evaluation and Benchmarking AI Safety Research ViT (Vision Transformer)Towards Robustness against Typographic Attack with Training-free Concept Localization RIO-Bench +2 more