Deep Ensembles for Graphs with Higher-order Dependencies
Steven Krieg, William Burgis, Patrick Soga, Nitesh Chawla
Abstract
Graph neural networks (GNNs) continue to achieve state-of-the-art performance on many graph learning tasks, but rely on the assumption that a given graph is a sufficient approximation of the true neighborhood structure. In the presence of higher-order sequential dependencies, we show that the tendency of traditional graph representations to underfit each node's neighborhood causes existing GNNs to generalize poorly. To address this, we propose a novel Deep Graph Ensemble (DGE), which captures neighborhood variance by training an ensemble of GNNs on different neighborhood subspaces of the same node within a higher-order network structure. We show that DGE consistently outperforms existing GNNs on semisupervised and supervised tasks on six real-world data sets with known higher-order dependencies, even under a similar parameter budget. We demonstrate that learning diverse and accurate base classifiers is central to DGE's success, and discuss the implications of these findings for future work on GNNs.
BibTeX
@inproceedings{
krieg2023deep,
title={Deep Ensembles for Graphs with Higher-order Dependencies},
author={Steven Krieg and William Burgis and Patrick Soga and Nitesh Chawla},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=hZftxQGJ4Re}
}