HFLR: Optimizing GNN Training via High-Fixed-Low-Resampling

Abstract

Training graph neural networks (GNNs) on large-scale graphs is challenging due to neighbor explosion problem. To alleviate this, various sampling methods have been proposed. However, they still suffer from several issues like sparse relationships between layers of computation graphs or edge information loss during sampling. Different from them, we propose a novel sampling strategy named HFLR, which uses a small subset of total nodes for training. The basic principle is that not all nodes contribute to accuracy improvement. Specifically, in each epoch, we sample a small number of nodes for computing loss and updating parameters, where some are fixed for all epochs, while others are resampled in each epoch. To guarantee an unbiased estimation of training loss, we further present normalization techniques. Extensive experiments on six large-scale graphs demonstrate our method achieves comparable F<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> scores with 1.7x-3.3x speedups over other sampling-based algorithms.