AAAI 2025technical0 citations
NAAM: Node-Aware Attention Mechanism for Distilling GNNs-to-MLP (Student Abstract)
Itsuki Nakayama, Makoto Onizuka
Abstract
Recently, researchers have focused on methods that not only distill knowledge from a Graph Neural Network (GNN) into a Multi-Layer Perceptron (MLP) but also leverage multiple teacher GNNs. However, existing methods assign a single attention weight to each teacher GNN. We propose a NodeAware Attention Mechanism (NAAM) that flexibly adjusts the attention weight for each node to leverage multiple GNNs fully. Experimental results show that NAAM outperforms existing GNN-to-MLP methods. our source code is available at: https://github.com/NakayamaItsuki/NAAM.
BibTeX
@article{Nakayama_Onizuka_2025, title={NAAM: Node-Aware Attention Mechanism for Distilling GNNs-to-MLP (Student Abstract)}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/35278}, DOI={10.1609/aaai.v39i28.35278}, abstractNote={Recently, researchers have focused on methods that not only distill knowledge from a Graph Neural Network (GNN) into a Multi-Layer Perceptron (MLP) but also leverage multiple teacher GNNs. However, existing methods assign a single attention weight to each teacher GNN. We propose a NodeAware Attention Mechanism (NAAM) that flexibly adjusts the attention weight for each node to leverage multiple GNNs fully. Experimental results show that NAAM outperforms existing GNN-to-MLP methods. our source code is available at: https://github.com/NakayamaItsuki/NAAM.}, number={28}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Nakayama, Itsuki and Onizuka, Makoto}, year={2025}, month={Apr.}, pages={29433-29435} }