How Transparent is DiffusionGemma?

Interpretability study of DiffusionGemma reveals novel diffusion-specific reasoning phenomena

Researchers investigate the reasoning transparency of DiffusionGemma, a diffusion-based language model, decomposing transparency into variable and algorithmic components. They show that mapping information through an interpretable token bottleneck reduces DiffusionGemma's opaque serial depth from 28.6X to just 1.1X that of autoregressive Gemma 4, with no performance loss. Interpretability case studies uncover diffusion-specific phenomena including non-chronological reasoning, token smearing, and intermediate-context reasoning. Monitorability tests find DiffusionGemma comparable to Gemma 4, suggesting diffusion LMs are not inherently less amenable to safety oversight.