AIConfPaper
← Search

Thalaiyasingam Ajanthan

24 accepted papers

2026

Factored Gossip DiLoCo: Reducing Blocking Communication within DiLoCo

Chamin Hewa Koneputugodage, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Hadi Mohaghegh Dolatabadi, Shamane Siriwardhana, Gil Avraham +4
ICML 2026poster

To make large-scale distributed training practical outside high-bandwidth datacenters, we must reduce blocking, high-volume synchronization. While DiLoCo communicates infrequently, its outer synchronization remains bandwidth-heavy and brittle to stragglers and transient failures. We relax exact sync…

Cited by 0Source ↗Scholar ↗
2026

Taming Curvature: Architecture Warm-up for Stable Transformer Training

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Chamin P Hewa Koneputugodage, Gil Avraham, Violetta Shevchenko +3
ICLR 2026poster

Training billion-parameter Transformers is often brittle, with transient loss spikes and divergence that waste compute. Even though the recently developed Edge of Stability (EoS) theory provides a powerful tool to understand and control the stability of optimization methods via the (preconditioned)…

Cited by 0Source ↗Scholar ↗
2025

Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval

Ziwei Wang, Sameera Ramasinghe, Chenchen Xu, Julien Monteil, Loris Bazzani, Thalaiyasingam Ajanthan
ICCV 2025poster

Structuring latent representations in a hierarchical manner enables models to learn patterns at multiple levels of abstraction. However, most prevalent image understanding models focus on visual similarity, and learning visual hierarchies is relatively unexplored. In this work, for the first time, w…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Gil Avraham, Violetta Shevchenko, Yan Zuo +2
NeurIPS 2025poster

Pretraining language models with extended context windows enhances their ability to leverage rich information during generation. Existing methods split input sequences into chunks, broadcast them across multiple devices, and compute attention block by block which incurs significant communication ove…

Cited by 0Source ↗Scholar ↗
2025

Nesterov Method for Asynchronous Pipeline Parallel Optimization

Thalaiyasingam Ajanthan, Sameera Ramasinghe, Yan Zuo, Gil Avraham, Alexander Long
ICML 2025poster

Pipeline Parallelism (PP) enables large neural network training on small, interconnected devices by splitting the model into multiple stages. To maximize pipeline utilization, asynchronous optimization is appealing as it offers 100% pipeline utilization by construction. However, it is inherently cha…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings

Evan Markou, Thalaiyasingam Ajanthan, Stephen Gould
NeurIPS 2025spotlight

Many high-dimensional optimisation problems exhibit rich geometric structures in their set of minimisers, often forming smooth manifolds due to over-parametrisation or symmetries. When this structure is known, at least locally, it can be exploited through reduction mappings that reparametrise part o…

Cited by 0Source ↗Scholar ↗
2025

Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, Alexander Long
NeurIPS 2025poster

Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging due to communication bottlenecks. While existing compression techniques are effective in data-parallel, they do not extend to model parallelism. Unlike data-pa…

Cited by 0Source ↗Scholar ↗
2025

Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization

Alexander Long, Chamin P Hewa Koneputugodage, Thalaiyasingam Ajanthan, Yan Zuo, Gil Avraham, Violetta Shevchenko +2
NeurIPS 2025poster

We consider a decentralized setup in which the participants collaboratively train and serve a large neural network, and where each participant only processes a subset of the model. In this setup, we explore the possibility of unmaterializable weights, where a full weight set is never available to a…

Cited by 0Source ↗Scholar ↗
2024

Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame

Evan Markou, Thalaiyasingam Ajanthan, Stephen Gould
NeurIPS 2024poster

Neural Collapse (NC) is a recently observed phenomenon in neural networks that characterises the solution space of the final classifier layer when trained until zero training loss. Specifically, NC suggests that the final classifier layer converges to a Simplex Equiangular Tight Frame (ETF), which m…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2023

Semi-Supervised Semantic Segmentation under Label Noise via Diverse Learning Groups

Peixia Li, Pulak Purkait, Thalaiyasingam Ajanthan, Majid Abdolshah, Ravi Garg, Hisham Husain +4
ICCV 2023poster

Semi-supervised semantic segmentation methods use a small amount of clean pixel-level annotations to guide the interpretation of a larger quantity of unlabelled image data. The challenges of providing pixel-accurate annotations at scale mean that the labels are typically noisy, and this contaminates…

Cited by 14PDF ↗Scholar ↗
2022

Improved Gradient-Based Adversarial Attacks for Quantized Networks

Kartik Gupta, Thalaiyasingam Ajanthan
AAAI 2022technical

Neural network quantization has become increasingly popular due to efficient memory consumption and faster computation resulting from bitwise operations on the quantized networks. Even though they exhibit excellent generalization capabilities, their robustness properties are not well-understood. In…

Cited by 22Source ↗PDF ↗code ↗Scholar ↗
2022

Retrieval Augmented Classification for Long-Tail Visual Recognition

Alexander Long, Wei Yin, Thalaiyasingam Ajanthan, Vu Nguyen, Pulak Purkait, Ravi Garg +3
CVPR 2022poster

We introduce Retrieval Augmented Classification (RAC), a generic approach to augmenting standard image classification pipelines with an explicit retrieval module. RAC consists of a standard base image encoder fused with a parallel retrieval branch that queries a non-parametric external memory of pre…

Cited by 127PDF ↗Scholar ↗
2021

Calibration of Neural Networks using Splines

Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu, Richard Hartley
ICLR 2021poster

Calibrating neural networks is of utmost importance when employing them in safety-critical applications where the downstream decision making depends on the predicted probabilities. Measuring calibration error amounts to comparing two empirical distributions. In this work, we introduce a binning-free…

Cited by 153Source ↗PDF ↗code ↗Scholar ↗
2021

Mirror Descent View for Neural Network Quantization

Thalaiyasingam Ajanthan, Kartik Gupta, Philip Torr, Richad Hartley, Puneet Dokania
AISTATS 2021poster

Quantizing large Neural Networks (NN) while maintaining the performance is highly desirable for resource-limited devices due to reduced memory and time complexity. It is usually formulated as a constrained optimization problem and optimized via a modified version of gradient descent. In this work, b…

Cited by 33Source ↗PDF ↗code ↗Scholar ↗
2021

Understanding the effects of data parallelism and sparsity on neural network training

Namhoon Lee, Thalaiyasingam Ajanthan, Philip Torr, Martin Jaggi
ICLR 2021poster

We study two factors in neural network training: data parallelism and sparsity; here, data parallelism means processing training data in parallel using distributed systems (or equivalently increasing batch size), so that training can be accelerated; for sparsity, we refer to pruning parameters in a…

Cited by 9Source ↗PDF ↗Scholar ↗
2020

A Signal Propagation Perspective for Pruning Neural Networks at Initialization

Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, Philip H. S. Torr
ICLR 2020spotlight

Network pruning is a promising avenue for compressing deep neural networks. A typical approach to pruning starts by training a model and then removing redundant parameters while minimizing the impact on what is learned. Alternatively, a recent approach shows that pruning can be done at initializatio…

Cited by 198Source ↗code ↗Scholar ↗
2020

Pairwise Similarity Knowledge Transfer for Weakly Supervised Object Localization

Amir Rahimi, Amirreza Shaban, Thalaiyasingam Ajanthan, Richard Hartley, Byron Boots
ECCV 2020poster

Weakly Supervised Object Localization (WSOL) methods only require image level labels as opposed to expensive bounding box annotations required by fully supervised algorithms. We study the problem of learning localization model on target classes with weakly supervised image labels, helped by a fully…

Cited by 16Source ↗PDF ↗code ↗Scholar ↗
2019

Learning to Adapt for Stereo

Alessio Tonioni, Oscar Rahnama, Thomas Joy, Luigi Di Stefano, Thalaiyasingam Ajanthan, Philip H.S. Torr
CVPR 2019poster

Real world applications of stereo depth estimation require models that are robust to dynamic variations in the environment. Even though deep learning based stereo methods are successful, they often fail to generalize to unseen variations in the environment, making them less suitable for practical ap…

Cited by 93PDF ↗code ↗Scholar ↗
2019

Proximal Mean-Field for Neural Network Quantization

Thalaiyasingam Ajanthan, Puneet K. Dokania, Richard Hartley, Philip H. S. Torr
ICCV 2019poster

Compressing large Neural Networks (NN) by quantizing the parameters, while maintaining the performance is highly desirable due to reduced memory and time complexity. In this work, we cast NN quantization as a discrete labelling problem, and by examining relaxations, we design an efficient iterative…

Cited by 24PDF ↗code ↗Scholar ↗
2019

SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY

Namhoon Lee, Thalaiyasingam Ajanthan, Philip Torr
ICLR 2019poster

Pruning large neural networks while maintaining their performance is often desirable due to the reduced space and time complexity. In existing methods, pruning is done within an iterative optimization procedure with either heuristically designed pruning schedules or additional hyperparameters, under…

Cited by 1524Source ↗PDF ↗code ↗Scholar ↗
2018

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, Philip H. S. Torr
ECCV 2018poster

Incremental learning (IL) has received a lot of attention recently, however, the literature lacks a precise problem definition, proper evaluation settings, and metrics tailored specifically for the IL problem. One of the main objectives of this work is to fill these gaps so as to provide a common gr…

Cited by 1482Source ↗PDF ↗code ↗Scholar ↗
2017

Efficient Linear Programming for Dense CRFs

Thalaiyasingam Ajanthan, Alban Desmaison, Rudy Bunel, Mathieu Salzmann, Philip H. S. Torr, M. Pawan Kumar
CVPR 2017poster

The fully connected conditional random field (CRF) with Gaussian pairwise potentials has proven popular and effective for multi-class semantic segmentation. While the energy of a dense CRF can be minimized accurately using a linear programming (LP) relaxation, the state-of-the-art algorithm is too s…

Cited by 21PDF ↗Scholar ↗
2016

Memory Efficient Max Flow for Multi-Label Submodular MRFs

Thalaiyasingam Ajanthan, Richard Hartley, Mathieu Salzmann
CVPR 2016poster

Multi-label submodular Markov Random Fields (MRFs) have been shown to be solvable using max-flow based on an encoding of the labels proposed by Ishikawa, in which each variable X_i is represented by l nodes (where l is the number of labels) arranged in a column. However, this method in general requi…

Cited by 9PDF ↗code ↗Scholar ↗
2015

Iteratively Reweighted Graph Cut for Multi-Label MRFs With Non-Convex Priors

Thalaiyasingam Ajanthan, Richard Hartley, Mathieu Salzmann, Hongdong Li
CVPR 2015poster

While widely acknowledged as highly effective in computer vision, multi-label MRFs with non-convex priors are difficult to optimize. To tackle this, we introduce an algorithm that iteratively approximates the original energy with an appropriately weighted surrogate energy that is easier to minimize.…

Cited by 14Source ↗PDF ↗Scholar ↗