Entity · technique

Visual Attribution Distillation

techniqueactiveprovisionalvisual-attribution-distillation-6f4fce97·1 events·first seen 34h ago

Aliases: Visual Attribution Distillation

More like this (12)

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels Visual Contrastive Self-Distillation In-context Vector Distillation Model Distillation on-policy distillation Parallel Decoding Distillation Weak-to-Strong Generalization via Direct On-Policy Distillation knowledge distillation Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation On-Policy Distillation (OPD)Visual Question Answering Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation

Recent events (1)

4arXiv · cs.CL·34h ago·source ↗

VAD: Counterfactual target reconstruction for multimodal on-policy distillation

Researchers introduce Visual Attribution Distillation (VAD), a counterfactual algorithm for multimodal knowledge distillation that isolates visually attributable corrections from teacher signals during on-policy training. VAD evaluates a privileged-view teacher with and without visual evidence to estimate a signed proxy for visual evidence direction, then reconstructs a student-anchored supervision target from the visually aligned component. Evaluated across six fine-grained visual benchmarks at 4B and 9B parameter scales, VAD outperforms direct privileged-view distillation and visual-advantage weighting baselines. The work addresses the source-mixing problem in multimodal distillation, where visual signals are entangled with linguistic priors and teacher-specific artifacts.

Alignment and RLHF Multimodal Progress Visual Attribution Distillation