AIConfPaper
← Search

Xiaowen Dong

42 accepted papers

2026

Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin

Enrique Queipo-de-Llano, Alvaro Arroyo, Federico Barbero, Xiaowen Dong, Michael M. Bronstein, Yann LeCun +1
ICLR 2026poster

Attention sinks and compression valleys have attracted significant attention as two puzzling phenomena in large language models, but have been studied in isolation. In this work, we present a surprising connection between attention sinks and compression valleys, tracing both to the formation of mass…

Cited by 0Source ↗Scholar ↗
2026

How Out-of-Distribution Detection Learning Theory Enhances Transformer: Learnability and Reliability

Yijin Zhou, Yutang Ge, Wenyuan XIE, Linqian Zeng, Xiaowen Dong, Yu Guang Wang
ICML 2026poster

Transformers excel in natural language processing and computer vision tasks. However, they still face challenges in generalizing to Out-of-Distribution (OOD) datasets, i.e. data whose distribution differs from that seen during training. OOD detection aims to distinguish outliers while preserving in-…

Cited by 0Source ↗Scholar ↗
2026

Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement

Huidong Liang, Haitz Sáez de Ocáriz Borde, Baskaran Sripathmanathan, Michael M. Bronstein, Xiaowen Dong
ICLR 2026poster

Long-range dependencies are critical for effective graph representation learning, yet most existing datasets focus on small graphs tailored to inductive tasks, offering limited insight into long-range interactions. Current evaluations primarily compare models employing global attention (e.g., graph…

Cited by 0Source ↗code ↗Scholar ↗
2026

gLSTM: Mitigating Over-Squashing by Increasing Storage Capacity

Hugh Blayney, Alvaro Arroyo, Xiaowen Dong, Michael M. Bronstein
ICLR 2026poster

Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wide variety of applications, they are known to suffer from over-squashing, where information from a large receptive field…

Cited by 0Source ↗code ↗Scholar ↗
2025

Bundle Neural Network for message diffusion on graphs

Jacob Bamberger, Federico Barbero, Xiaowen Dong, Michael M. Bronstein
ICLR 2025spotlight

The dominant paradigm for learning on graphs is message passing. Despite being a strong inductive bias, the local message passing mechanism faces challenges such as over-smoothing, over-squashing, and limited expressivity. To address these issues, we introduce Bundle Neural Networks (BuNNs), a novel…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

Heterogeneous Graph Structure Learning through the Lens of Data-generating Processes

Keyue Jiang, Bohan Tang, Xiaowen Dong, Laura Toni
AISTATS 2025poster

Inferring the graph structure from observed data is a key task in graph machine learning to capture the intrinsic relationship between data entities. While significant advancements have been made in learning the structure of homogeneous graphs, many real-world graphs exhibit heterogeneous patterns w…

Cited by 0Source ↗Scholar ↗
2025

Neural Spacetimes for DAG Representation Learning

Haitz Sáez de Ocáriz Borde, Anastasis Kratsios, Marc T. Law, Xiaowen Dong, Michael M. Bronstein
ICLR 2025poster

We propose a class of trainable deep learning-based geometries called Neural SpaceTimes (NSTs), which can universally represent nodes in weighted Directed Acyclic Graphs (DAGs) as events in a spacetime manifold. While most works in the literature focus on undirected graph representation learning or…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

On Measuring Long-Range Interactions in Graph Neural Networks

Jacob Bamberger, Benjamin Gutteridge, Scott le Roux, Michael M. Bronstein, Xiaowen Dong
ICML 2025poster

Long-range graph tasks --- those dependent on interactions between `distant' nodes --- are an open problem in graph neural network research. Real-world benchmark tasks, especially the Long Range Graph Benchmark, have become popular for validating the long-range capability of proposed architectures.…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning

Alvaro Arroyo, Alessio Gravina, Benjamin Gutteridge, Federico Barbero, Claudio Gallicchio, Xiaowen Dong +2
NeurIPS 2025poster

Graph Neural Networks (GNNs) are models that leverage the graph structure to transmit information between nodes, typically through the message-passing operation. While widely successful, this approach is well-known to suffer from representational collapse as the number of layers increases and insens…

Cited by 0Source ↗Scholar ↗
2025

On the Stability of Graph Convolutional Neural Networks: A Probabilistic Perspective

Ning Zhang, Henry Kenlay, Li Zhang, Mihai Cucuringu, Xiaowen Dong
NeurIPS 2025poster

Graph convolutional neural networks (GCNNs) have emerged as powerful tools for analyzing graph-structured data, achieving remarkable success across diverse applications. However, the theoretical understanding of the stability of these models, i.e., their sensitivity to small changes in the graph str…

Cited by 0Source ↗Scholar ↗
2025

Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks

Ali Hariri, Alvaro Arroyo, Alessio Gravina, Moshe Eliasof, Carola-Bibiane Schönlieb, Davide Bacciu +3
NeurIPS 2025spotlight

ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dep…

Cited by 0Source ↗Scholar ↗
2025

Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation

Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Tian Jin +3
ACL 2025long

Post-training is essential for enabling large language models (LLMs) to follow human instructions. However, its effectiveness depends on high-quality instruction data, which is challenging to obtain in the real world due to privacy concerns, data scarcity, and high annotation costs. To fill this gap…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Training-Free Message Passing for Learning on Hypergraphs

Bohan Tang, Zexi Liu, Keyue Jiang, Siheng Chen, Xiaowen Dong
ICLR 2025poster

Hypergraphs are crucial for modelling higher-order interactions in real-world data. Hypergraph neural networks (HNNs) effectively utilise these structures by message passing to generate informative node features for various downstream tasks like node classification. However, the message passing modu…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

A Characterization Theorem for Equivariant Networks with Point-wise Activations

Marco Pacini, Xiaowen Dong, Bruno Lepri, Gabriele Santin
ICLR 2024poster

Equivariant neural networks have shown improved performance, expressiveness and sample complexity on symmetrical domains. But for some specific symmetries, representations, and choice of coordinates, the most common point-wise activations, such as ReLU, are not equivariant, hence they cannot be emp…

Cited by 1Source ↗PDF ↗Scholar ↗
2024

Bayesian Optimization of Functions over Node Subsets in Graphs

Huidong Liang, Xingchen Wan, Xiaowen Dong
NeurIPS 2024poster

We address the problem of optimizing over functions defined on node subsets in a graph. The optimization of such functions is often a non-trivial task given their combinatorial, black-box and expensive-to-evaluate nature. Although various algorithms have been introduced in the literature, most are…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2024

Hypergraph Transformer for Semi-Supervised Classification

Zexi Liu, Bohan Tang, Ziyuan Ye, Xiaowen Dong, Siheng Chen, Yanfeng Wang
ICASSP 2024accepted

Hypergraphs play a pivotal role in the modelling of data featuring higher-order relations involving more than two entities. Hypergraph neural networks emerge as a powerful tool for processing hypergraph-structured data, delivering remarkable performance across various tasks, e.g., hypergraph node cl…

Cited by 0Source ↗Scholar ↗
2024

Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching

Fernando Moreno-Pino, Alvaro Arroyo, Harrison Waldon, Xiaowen Dong, Alvaro Cartea
NeurIPS 2024poster

Time-series data in real-world settings typically exhibit long-range dependencies and are observed at non-uniform intervals. In these settings, traditional sequence-based recurrent models struggle. To overcome this, researchers often replace recurrent models with Neural ODE-based architectures to ac…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

STEntConv: Predicting Disagreement between Reddit Users with Stance Detection and a Signed Graph Convolutional Network

Isabelle Lorge, Li Zhang, Xiaowen Dong, Janet Pierrehumbert
COLING 2024main

The rise of social media platforms has led to an increase in polarised online discussions, especially on political and socio-cultural topics such as elections and climate change. We propose a simple and entirely novel unsupervised method to better predict whether the authors of two posts agree or di…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2023

Bayesian Optimisation of Functions on Graphs

Xingchen Wan, Pierre Osselin, Henry Kenlay, Binxin Ru, Michael A Osborne, Xiaowen Dong
NeurIPS 2023poster

The increasing availability of graph-structured data motivates the task of optimising over functions defined on the node set of graphs. Traditional graph search algorithms can be applied in this case, but they may be sample-inefficient and do not make use of information about the function values; on…

Cited by 5Source ↗PDF ↗Scholar ↗
2023

DRew: Dynamically Rewired Message Passing with Delay

Benjamin Gutteridge, Xiaowen Dong, Michael M. Bronstein, Francesco Di Giovanni
ICML 2023poster

Message passing neural networks (MPNNs) have been shown to suffer from the phenomenon of over-squashing that causes poor performance for tasks relying on long-range interactions. This can be largely attributed to message passing only occurring locally, over a node's immediate neighbours. Rewiring ap…

Cited by 87Source ↗PDF ↗code ↗Scholar ↗
2023

Graph classification Gaussian processes via spectral features

Felix L. Opolka, Yin-Cong Zhi, Pietro Liò, Xiaowen Dong
UAI 2023poster

Graph classification aims to categorise graphs based on their structure and node attributes. In this work, we propose to tackle this task using tools from graph signal processing by deriving spectral features, which we then use to design two variants of Gaussian process models for graph classificati…

Cited by 5Source ↗PDF ↗Scholar ↗
2023

Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization

Haitz Sáez de Ocáriz Borde, Alvaro Arroyo, Ismael Morales López, Ingmar Posner, Xiaowen Dong
NeurIPS 2023poster

Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curv…

Cited by 10Source ↗PDF ↗Scholar ↗
2023

Structure-aware robustness certificates for graph classification

Pierre Osselin, Henry Kenlay, Xiaowen Dong
UAI 2023poster

Certifying the robustness of a graph-based machine learning model poses a critical challenge for safety. Current robustness certificates for graph classifiers guarantee output invariance with respect to the total number of node pair flips (edge addition or edge deletion), which amounts to an {$l_{0}…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2022

Adaptive Gaussian Processes on Graphs via Spectral Graph Wavelets

Felix Opolka, Yin-Cong Zhi, Pietro Lió, Xiaowen Dong
AISTATS 2022poster

Graph-based models require aggregating information in the graph from neighbourhoods of different sizes. In particular, when the data exhibit varying levels of smoothness on the graph, a multi-scale approach is required to capture the relevant information. In this work, we propose a Gaussian process…

Cited by 26Source ↗PDF ↗Scholar ↗
2022

Learning to Infer Structures of Network Games

Emanuele Rossi, Federico Monti, Yan Leng, Michael Bronstein, Xiaowen Dong
ICML 2022spotlight

Strategic interactions between a group of individuals or organisations can be modelled as games played on networks, where a player’s payoff depends not only on their actions but also on those of their neighbours. Inferring the network structure from observed game outcomes (equilibrium actions) is an…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2022

Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity

Valentin Hofmann, Xiaowen Dong, Janet Pierrehumbert, Hinrich Schuetze
NAACL 2022findings

The increasing polarization of online political discourse calls for computational tools that automatically detect and monitor ideological divides in social media. We introduce a minimally supervised method that leverages the network structure of online discussion forums, specifically Reddit, to dete…

Cited by 13Source ↗PDF ↗code ↗Scholar ↗
2022

Understanding over-squashing and bottlenecks on graphs via curvature

Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, Michael M. Bronstein
ICLR 2022oral

Most graph neural networks (GNNs) use the message passing paradigm, in which node features are propagated on the input graph. Recent works pointed to the distortion of information flowing from distant nodes as a factor limiting the efficiency of message passing for tasks relying on long-distance int…

Cited by 584Source ↗PDF ↗code ↗Scholar ↗
2021

Adversarial Attacks on Graph Classifiers via Bayesian Optimisation

Xingchen Wan, Henry Kenlay, Binxin Ru, Arno Blaas, Michael Osborne, Xiaowen Dong
NeurIPS 2021poster

Graph neural networks, a popular class of models effective in a wide range of graph-based learning tasks, have been shown to be vulnerable to adversarial attacks. While the majority of the literature focuses on such vulnerability in node-level classification tasks, little effort has been dedicated t…

Cited by 34Source ↗PDF ↗code ↗Scholar ↗
2021

Beltrami Flow and Neural Diffusion on Graphs

Benjamin Paul Chamberlain, James Rowbottom, Davide Eynard, Francesco Di Giovanni, Xiaowen Dong, Michael M. Bronstein
NeurIPS 2021poster

We propose a novel class of graph neural networks based on the discretized Beltrami flow, a non-Euclidean diffusion PDE. In our model, node features are supplemented with positional encodings derived from the graph topology and jointly evolved by the Beltrami flow, producing simultaneously contin…

Cited by 95Source ↗PDF ↗code ↗Scholar ↗
2021

Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels

Binxin Ru, Xingchen Wan, Xiaowen Dong, Michael Osborne
ICLR 2021poster

Current neural architecture search (NAS) strategies focus only on finding a single, good, architecture. They offer little insight into why a specific network is performing well, or how we should modify the architecture if we want further improvements. We propose a Bayesian optimisation (BO) approach…

Cited by 140Source ↗PDF ↗Scholar ↗
2021

On The Stability of Graph Convolutional Neural Networks Under Edge Rewiring

Henry Kenlay, Dorina Thanou, Xiaowen Dong
ICASSP 2021accepted

Graph neural networks are experiencing a surge of popularity within the machine learning community due to their ability to adapt to nonEuclidean domains and instil inductive biases. Despite this, their stability, i.e., their robustness to small perturbations in the input, is not yet well understood.…

Cited by 0Source ↗Scholar ↗
2015

Laplacian matrix learning for smooth graph signal representation

Xiaowen Dong, Dorina Thanou, Pascal Frossard, Pierre Vandergheynst
ICASSP 2015accepted

The construction of a meaningful graph plays a crucial role in the emerging field of signal processing on graphs. In this paper, we address the problem of learning graph Laplacians, which is similar to learning graph topologies, such that the input data form graph signals with smooth variations on t…

Cited by 0Source ↗Scholar ↗