Supercharging Graph Transformers with Advective Diffusion
Qitian Wu, Chenxiao Yang, Kaipeng Zeng, Michael M. Bronstein
Abstract
The capability of generalization is a cornerstone for the success of modern learning systems. For non-Euclidean data, e.g., graphs, that particularly involves topological structures, one important aspect neglected by prior studies is how machine learning models generalize under topological shifts. This paper proposes AdvDIFFormer, a physics-inspired graph Transformer model designed to address this challenge. The model is derived from advective diffusion equations which describe a class of continuous message passing process with observed and latent topological structures. We show that AdvDIFFormer has provable capability for controlling generalization error with topological shifts, which in contrast cannot be guaranteed by graph diffusion models, i.e., the generalization of common graph neural networks in continuous space. Empirically, the model demonstrates superiority in various predictive tasks across information networks, molecular screening and protein interactions
BibTeX
@inproceedings{
wu2025supercharging,
title={Supercharging Graph Transformers with Advective Diffusion},
author={Qitian Wu and Chenxiao Yang and Kaipeng Zeng and Michael M. Bronstein},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=MaOYl3P84E}
}