GFMAE: Self-Supervised GNN-Free Masked Autoencoders

Abstract

Generative self-supervised learning, represented by graph autoencoders (GAEs), has begun to exhibit significant potential in addressing graph tasks. However, GAEs often rely on Graph Neural Networks (GNNs) for encoding and decoding, this can pose a computation challenge due to the inherent complexities of the aggregation mechanism in GNNs. Furthermore, the bipartite structure of GAEs introduces additional computational burdens. In contrast, Multi-Layer Perceptrons (MLPs) have no graph dependency and can train much faster than GNNs. Motivated by this, in this work, we introduce a simple yet effective alternative: the GNN-Free Masked AutoEncoder (GFMAE), which employs MLPs rather than GNNs to serve as the backbone model to speed up training. Additionally, we devise comprehensive decoding strategies to compensate for the inability of MLPs in characterizing the graph. Our comprehensive experiments conducted on eight datasets demonstrate that GFMAE achieves performance comparable to GNNs while also enhancing the training efficiency of generative models with GNNs as the backbone.