Decisive-Token Supervision

DIVE: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation

DIVE is a frozen-backbone distillation framework that addresses a fundamental limitation in token-level in-context vector distillation: uniform cross-entropy supervision treats all output tokens equally, but long-form outputs like medical reports are dominated by low-information template tokens while diagnostically critical tokens receive insufficient gradient signal. The method introduces decisive-token supervision (upweighting pathology-related tokens and EOS events) and state-conditioned dynamic steering (hidden-state-dependent adapters replacing fixed residuals) to correct supervision imbalance and autoregressive drift. Evaluated on MIMIC-CXR and CheXpert Plus with two medical VLM backbones, DIVE achieves best BLEU-4, ROUGE-L, and RadGraph F1 across all dataset-backbone combinations while remaining competitive on CheXbert F1.