A Unified Spatiotemporal Frequency Graph Neural Network for fMRI-based Brain Functional Connectivity Analysis

Abstract

Analyzing functional connectivity patterns from resting-state functional magnetic resonance imaging (fMRI) requires unraveling its interrelations across spatial, temporal, and frequency domains. To comprehensively analyze four-dimensional (4D) fMRI data, we propose the Spatiotemporal Frequency Graph Neural Network (STFreqGNN), which processes dynamic heterogeneous graphs across spatial, temporal, and frequency domains using a transformer-style architecture. To reduce the complexity of multi-domain analysis with small sample sizes for fMRI datasets and ensure domainspecific interpretability, we introduce two structure-informed modules in the spatial and temporal domains to improve knowledge aggregation within each domain. Specifically, the plugin GNNs transmit information within the static homogeneous brain region graphs, and recurrent blocks aggregate features from heterogeneous nodes defined across different temporal windows. Additionally, we design cross-domain masked self-attention blocks to prevent attention captured by irrelevant or redundant token pairs, finally enabling efficient disease-specific feature learning. Experimental results on both public and in-house datasets suggest that the proposed method is not only superior to several state-of-the-art methods on fMRI-based classification but also preserves interpretation ability in all these domains.