AIConfPaper
← Search

Simone Scardapane

12 accepted papers

2026

MASS: MoErging through Adaptive Subspace Selection

Donato Crisostomi, Alessandro Zirilli, Antonio Andrea Gargiulo, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri +2
ICLR 2026poster

Model merging has recently emerged as a lightweight alternative to ensembling, combining multiple fine-tuned models into a single set of parameters with no additional training overhead. Yet, existing merging methods fall short of matching the full accuracy of separately fine-tuned endpoints. We pres…

Cited by 0Source ↗code ↗Scholar ↗
2026

Universal Properties of Activation Sparsity in Modern Large Language Models

Filip Szatkowski, Patryk Będkowski, Alessio Devoto, Jan Dubiński, Pasquale Minervini, Mikołaj Piórczyński +2
ICLR 2026poster

Activation sparsity is an intriguing property of deep neural networks that has been extensively studied in ReLU-based models, due to its advantages for efficiency, robustness, and interpretability. However, methods relying on exact zero activations do not directly apply to modern Large Language Mod…

Cited by 0Source ↗Scholar ↗
2025

Adaptive Computation Modules: Granular Conditional Computation for Efficient Inference

Bartosz Wójcik, Alessio Devoto, Karol Pustelnik, Pasquale Minervini, Simone Scardapane
AAAI 2025technical

While transformer models have been highly successful, they are computationally inefficient. We observe that for each layer, the full width of the layer may be needed only for a small subset of tokens inside a batch and that the "effective" width needed to process a token can vary from layer to layer…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2025

Task Singular Vectors: Reducing Task Interference in Model Merging

Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, Emanuele Rodolà
CVPR 2025poster

Task Arithmetic has emerged as a simple yet effective method to merge models without additional training. However, by treating entire networks as flat parameter vectors, it overlooks key structural information and is susceptible to task interference. In this paper, we study task vectors at the layer…

Cited by 11Source ↗PDF ↗code ↗Scholar ↗
2024

A Simple and Effective L_2 Norm-Based Strategy for KV Cache Compression

Alessio Devoto, Yu Zhao, Simone Scardapane, Pasquale Minervini
EMNLP 2024main

The deployment of large language models (LLMs) is often hindered by the extensive memory requirements of the Key-Value (KV) cache, especially as context lengths increase. Existing approaches to reduce the KV cache size involve either fine-tuning the model to learn a compression strategy or leveragin…

Cited by 24Source ↗PDF ↗code ↗Scholar ↗
2024

Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion

Filip Szatkowski, Bartosz Wójcik, Mikołaj Piórczyński, Simone Scardapane
NeurIPS 2024poster

Transformer models can face practical limitations due to their high computational requirements. At the same time, such models exhibit significant activation sparsity, which can be leveraged to reduce the inference cost by converting parts of the network into equivalent Mixture-of-Experts (MoE) layer…

Cited by 3Source ↗PDF ↗Scholar ↗
2024

From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module

Claudio Battiloro, Indro Spinelli, Lev Telyatnikov, Michael M. Bronstein, Simone Scardapane, Paolo Di Lorenzo
ICLR 2024poster

Latent Graph Inference (LGI) relaxed the reliance of Graph Neural Networks (GNNs) on a given graph topology by dynamically learning it. However, most of LGI methods assume to have a (noisy, incomplete, improvable, ...) input graph to rewire and can solely learn regular graph topologies. In the wake…

Cited by 20Source ↗PDF ↗Scholar ↗
2024

Position: Topological Deep Learning is the New Frontier for Relational Learning

Theodore Papamarkou, Tolga Birdal, Michael M. Bronstein, Gunnar E. Carlsson, Justin Curry, Yue Gao +16
ICML 2024poster

Topological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper posits that TDL is the new frontier for relational learning. TDL may complement graph representation learning and geometric deep learning by incorporat…

Cited by 40Source ↗PDF ↗Scholar ↗
2023

EGG-GAE: scalable graph neural networks for tabular data imputation

Lev Telyatnikov, Simone Scardapane
AISTATS 2023poster

Missing data imputation (MDI) is crucial when dealing with tabular datasets across various domains. Autoencoders can be trained to reconstruct missing values, and graph autoencoders (GAE) can additionally consider similar patterns in the dataset when imputing new values for a given instance. However…

Cited by 19Source ↗PDF ↗code ↗Scholar ↗
2020

Differentiable Branching In Deep Networks for Fast Inference

Simone Scardapane, Danilo Comminiello, Michele Scarpiniti, Enzo Baccarelli, Aurelio Uncini
ICASSP 2020accepted

In this paper, we consider the design of deep neural networks augmented with multiple auxiliary classifiers departing from the main (backbone) network. These classifiers can be used to perform early-exit from the network at various layers, making them convenient for energy-constrained applications s…

Cited by 0Source ↗Scholar ↗
2019

Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events

Danilo Comminiello, Marco Lella, Simone Scardapane, Aurelio Uncini
ICASSP 2019accepted

Learning from data in the quaternion domain enables us to exploit internal dependencies of 4D signals and treating them as a single entity. One of the models that perfectly suits with quaternion-valued data processing is represented by 3D acoustic signals in their spherical harmonics decomposition.…

Cited by 0Source ↗Scholar ↗
2019

Widely Linear Kernels for Complex-valued Kernel Activation Functions

Simone Scardapane, Steven Van Vaerenbergh, Danilo Comminiello, Aurelio Uncini
ICASSP 2019accepted

Complex-valued neural networks (CVNNs) have been shown to be powerful nonlinear approximators when the input data can be properly modeled in the complex domain. One of the major challenges in scaling up CVNNs in practice is the design of complex activation functions. Recently, we proposed a novel fr…

Cited by 0Source ↗Scholar ↗