AIConfPaper
← Search

Gil Avraham

13 accepted papers

2026

Factored Gossip DiLoCo: Reducing Blocking Communication within DiLoCo

Chamin Hewa Koneputugodage, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Hadi Mohaghegh Dolatabadi, Shamane Siriwardhana, Gil Avraham +4
ICML 2026poster

To make large-scale distributed training practical outside high-bandwidth datacenters, we must reduce blocking, high-volume synchronization. While DiLoCo communicates infrequently, its outer synchronization remains bandwidth-heavy and brittle to stragglers and transient failures. We relax exact sync…

Cited by 0Source ↗Scholar ↗
2026

Taming Curvature: Architecture Warm-up for Stable Transformer Training

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Chamin P Hewa Koneputugodage, Gil Avraham, Violetta Shevchenko +3
ICLR 2026poster

Training billion-parameter Transformers is often brittle, with transient loss spikes and divergence that waste compute. Even though the recently developed Edge of Stability (EoS) theory provides a powerful tool to understand and control the stability of optimization methods via the (preconditioned)…

Cited by 0Source ↗Scholar ↗
2025

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Gil Avraham, Violetta Shevchenko, Yan Zuo +2
NeurIPS 2025poster

Pretraining language models with extended context windows enhances their ability to leverage rich information during generation. Existing methods split input sequences into chunks, broadcast them across multiple devices, and compute attention block by block which incurs significant communication ove…

Cited by 0Source ↗Scholar ↗
2025

Nesterov Method for Asynchronous Pipeline Parallel Optimization

Thalaiyasingam Ajanthan, Sameera Ramasinghe, Yan Zuo, Gil Avraham, Alexander Long
ICML 2025poster

Pipeline Parallelism (PP) enables large neural network training on small, interconnected devices by splitting the model into multiple stages. To maximize pipeline utilization, asynchronous optimization is appealing as it offers 100% pipeline utilization by construction. However, it is inherently cha…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, Alexander Long
NeurIPS 2025poster

Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging due to communication bottlenecks. While existing compression techniques are effective in data-parallel, they do not extend to model parallelism. Unlike data-pa…

Cited by 0Source ↗Scholar ↗
2025

Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization

Alexander Long, Chamin P Hewa Koneputugodage, Thalaiyasingam Ajanthan, Yan Zuo, Gil Avraham, Violetta Shevchenko +2
NeurIPS 2025poster

We consider a decentralized setup in which the participants collaboratively train and serve a large neural network, and where each participant only processes a subset of the model. In this setup, we explore the possibility of unmaterializable weights, where a full weight set is never available to a…

Cited by 0Source ↗Scholar ↗
2024

Accept the Modality Gap: An Exploration in the Hyperbolic Space

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Ajanthan Thalaiyasingam
CVPR 2024highlight

Recent advancements in machine learning have spotlighted the potential of hyperbolic spaces as they effectively learn hierarchical feature representations. While there has been progress in leveraging hyperbolic spaces in single-modality contexts its exploration in multimodal settings remains under e…

Cited by 10Source ↗PDF ↗Scholar ↗
2024

BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Anton van den Hengel
AAAI 2024technical

Inferring the 3D structure of a non-rigid dynamic scene from a single moving camera is an under-constrained problem. Inspired by the remarkable progress of neural radiance fields (NeRFs) in photo-realistic novel view synthesis of static scenes, it has also been extended to dynamic settings. Such me…

Cited by 7Source ↗PDF ↗Scholar ↗
2024

Improving the Convergence of Dynamic NeRFs via Optimal Transport

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Hisham Husain, Anton van den Hengel
ICLR 2024poster

Synthesizing novel views for dynamic scenes from a collection of RGB inputs poses significant challenges due to the inherent under-constrained nature of the problem. To mitigate this ill-posedness, practitioners in the field of neural radiance fields (NeRF) often resort to the adoption of intricate…

Cited by 1Source ↗PDF ↗Scholar ↗
2024

ViewFusion: Towards Multi-View Consistency via Interpolated Denoising

Xianghui Yang, Yan Zuo, Sameera Ramasinghe, Loris Bazzani, Gil Avraham, Anton van den Hengel
CVPR 2024poster

Novel-view synthesis through diffusion models has demonstrated remarkable potential for generating diverse and high-quality images. Yet the independent process of image generation in these prevailing methods leads to challenges in maintaining multiple-view consistency. To address this we introduce V…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2022

Learning Instance and Task-Aware Dynamic Kernels for Few-Shot Learning

Rongkai Ma, Pengfei Fang, Gil Avraham, Yan Zuo, Tianyu Zhu, Tom Drummond +1
ECCV 2022poster

"Learning and generalizing to novel concepts with few samples (Few-Shot Learning) is still an essential challenge to real-world applications. A principle way of achieving few-shot learning is to realize a model that can rapidly adapt to the context of a given task. Dynamic networks have been shown c…

Cited by 16Source ↗PDF ↗code ↗Scholar ↗
2019

EMPNet: Neural Localisation and Mapping Using Embedded Memory Points

Gil Avraham, Yan Zuo, Thanuja Dharmasiri, Tom Drummond
ICCV 2019poster

Continuously estimating an agent's state space and a representation of its surroundings has proven vital towards full autonomy. A shared common ground among systems which successfully achieve this feat is the integration of previously encountered observations into the current state being estimated.…

Cited by 24PDF ↗Scholar ↗