Aladin Virmaux — accepted AI-conference papers

2024

SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention

Romain Ilbert, Ambroise Odonnat, Vasilii Feofanov, Aladin Virmaux, Giuseppe Paolo, Themis Palpanas +1

ICML 2024oral

Transformer-based architectures achieved breakthrough performance in natural language processing and computer vision, yet they remain inferior to simpler linear baselines in multivariate long-term forecasting. To better understand this phenomenon, we start by studying a toy linear forecasting proble…

Cited by 17 Source ↗PDF ↗code ↗Scholar ↗

2023

Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumption

Vasilii Feofanov, Malik Tiomoko, Aladin Virmaux

ICML 2023poster

We propose a theoretical framework to analyze semi-supervised classification under the low density separation assumption in a high-dimensional regime. In particular, we introduce QLDS, a linear classification model, where the low density separation assumption is implemented via quadratic margin maxi…

Cited by 8 Source ↗PDF ↗Scholar ↗

2022

Deciphering Lasso-based Classification Through a Large Dimensional Analysis of the Iterative Soft-Thresholding Algorithm

Malik Tiomoko, Ekkehard Schnoor, Mohamed El Amine Seddik, Igor Colin, Aladin Virmaux

ICML 2022spotlight

This paper proposes a theoretical analysis of a Lasso-based classification algorithm. Leveraging on a realistic regime where the dimension of the data $p$ and their number $n$ are of the same order of magnitude, the theoretical classification error is derived as a function of the data statistics. As…

Cited by 4 Source ↗PDF ↗Scholar ↗

2021

Ego-Based Entropy Measures for Structural Representations on Graphs

George Dasoulas, Giannis Nikolentzos, Kevin Scaman, Aladin Virmaux, Michalis Vazirgiannis

ICASSP 2021accepted

Machine learning on graph-structured data has attracted high research interest due to the emergence of Graph Neural Networks (GNNs). Most of the proposed GNNs are based on the node homophily, i.e neighboring nodes share similar characteristics. However, in many complex networks, nodes that lie to di…

Cited by 0 Source ↗Scholar ↗

2021

Lipschitz normalization for self-attention layers with application to graph neural networks

George Dasoulas, Kevin Scaman, Aladin Virmaux

ICML 2021spotlight

Attention based neural networks are state of the art in a large range of applications. However, their performance tends to degrade when the number of layers increases. In this work, we show that enforcing Lipschitz continuity by normalizing the attention scores can significantly improve the performa…

Cited by 52 Source ↗PDF ↗code ↗Scholar ↗

2020

Coloring Graph Neural Networks for Node Disambiguation

George Dasoulas, Ludovic Dos Santos, Kevin Scaman, Aladin Virmaux

IJCAI 2020poster

In this paper, we show that a simple coloring scheme can improve, both theoretically and empirically, the expressive power of Message Passing Neural Networks (MPNNs). More specifically, we introduce a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambigua…

Cited by 0 Source ↗PDF ↗Scholar ↗

2018

Lipschitz regularity of deep neural networks: analysis and efficient estimation

Aladin Virmaux, Kevin Scaman

NeurIPS 2018poster

Deep neural networks are notorious for being sensitive to small well-chosen perturbations, and estimating the regularity of such architectures is of utmost importance for safe and robust practical applications. In this paper, we investigate one of the key characteristics to assess the regularity of…

Cited by 676 Source ↗PDF ↗code ↗Scholar ↗