DiGress

Graph Convolutional Attention: Spectral perspective improves graph denoising and diffusion

A new arXiv preprint introduces Spectral Attention and its practical realization, Graph Convolutional Attention (GCA), motivated by a theoretical analysis showing that linear attention is suboptimal for graph denoising tasks. The authors prove that standard linear attention can only learn an average spectral filter over the training distribution, while GCA provably outperforms it by a margin tied to spectral diversity. Empirically, GCA improves graph denoising and diffusion on synthetic and real datasets, and in the DiGress graph diffusion model it matches standard graph-transformer performance without expensive structural feature computation, enabling faster inference when combined with PEARL positional encodings.