AIConfPaper
← Search

Sameera Ramasinghe

30 accepted papers

2026

Factored Gossip DiLoCo: Reducing Blocking Communication within DiLoCo

Chamin Hewa Koneputugodage, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Hadi Mohaghegh Dolatabadi, Shamane Siriwardhana, Gil Avraham +4
ICML 2026poster

To make large-scale distributed training practical outside high-bandwidth datacenters, we must reduce blocking, high-volume synchronization. While DiLoCo communicates infrequently, its outer synchronization remains bandwidth-heavy and brittle to stragglers and transient failures. We relax exact sync…

Cited by 0Source ↗Scholar ↗
2026

LumiNet: Perception-Driven Knowledge Distillation via Statistical Logit Calibration

MD ISMAIL HOSSAIN, M M Lutfe Elahi, Sameera Ramasinghe, Ali Cheraghian, Fuad Rahman, Nabeel Mohammed +1
ICML 2026poster

In the knowledge distillation literature, feature-based methods have dominated due to their ability to effectively tap into extensive teacher models. In contrast, logit-based approaches, which aim to distill `dark knowledge' from teachers, typically exhibit inferior performance compared to feature-b…

Cited by 0Source ↗Scholar ↗
2026

Taming Curvature: Architecture Warm-up for Stable Transformer Training

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Chamin P Hewa Koneputugodage, Gil Avraham, Violetta Shevchenko +3
ICLR 2026poster

Training billion-parameter Transformers is often brittle, with transient loss spikes and divergence that waste compute. Even though the recently developed Edge of Stability (EoS) theory provides a powerful tool to understand and control the stability of optimization methods via the (preconditioned)…

Cited by 0Source ↗Scholar ↗
2025

Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval

Ziwei Wang, Sameera Ramasinghe, Chenchen Xu, Julien Monteil, Loris Bazzani, Thalaiyasingam Ajanthan
ICCV 2025poster

Structuring latent representations in a hierarchical manner enables models to learn patterns at multiple levels of abstraction. However, most prevalent image understanding models focus on visual similarity, and learning visual hierarchies is relatively unexplored. In this work, for the first time, w…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Hadi Mohaghegh Dolatabadi, Gil Avraham, Violetta Shevchenko, Yan Zuo +2
NeurIPS 2025poster

Pretraining language models with extended context windows enhances their ability to leverage rich information during generation. Existing methods split input sequences into chunks, broadcast them across multiple devices, and compute attention block by block which incurs significant communication ove…

Cited by 0Source ↗Scholar ↗
2025

Nesterov Method for Asynchronous Pipeline Parallel Optimization

Thalaiyasingam Ajanthan, Sameera Ramasinghe, Yan Zuo, Gil Avraham, Alexander Long
ICML 2025poster

Pipeline Parallelism (PP) enables large neural network training on small, interconnected devices by splitting the model into multiple stages. To maximize pipeline utilization, asynchronous optimization is appealing as it offers 100% pipeline utilization by construction. However, it is inherently cha…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism

Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, Alexander Long
NeurIPS 2025poster

Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging due to communication bottlenecks. While existing compression techniques are effective in data-parallel, they do not extend to model parallelism. Unlike data-pa…

Cited by 0Source ↗Scholar ↗
2025

Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization

Alexander Long, Chamin P Hewa Koneputugodage, Thalaiyasingam Ajanthan, Yan Zuo, Gil Avraham, Violetta Shevchenko +2
NeurIPS 2025poster

We consider a decentralized setup in which the participants collaboratively train and serve a large neural network, and where each participant only processes a subset of the model. In this setup, we explore the possibility of unmaterializable weights, where a full weight set is never available to a…

Cited by 0Source ↗Scholar ↗
2025

VI^3NR: Variance Informed Initialization for Implicit Neural Representations

Chamin Hewa Koneputugodage, Yizhak Ben-Shabat, Sameera Ramasinghe, Stephen Gould
CVPR 2025poster

Implicit Neural Representations (INRs) are a versatile and powerful tool for encoding various forms of data, including images, videos, sound, and 3D shapes. A critical factor in the success of INRs is the initialization of the network, which can significantly impact the convergence and accuracy of t…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

A sampling theory perspective on activations for implicit neural representations

Hemanth Saratchandran, Sameera Ramasinghe, Violetta Shevchenko, Alexander Long, Simon Lucey
ICML 2024poster

Implicit Neural Representations (INRs) have gained popularity for encoding signals as compact, differentiable entities. While commonly using techniques like Fourier positional encodings or non-traditional activation functions (e.g., Gaussian, sinusoid, or wavelets) to capture high-frequency content,…

Cited by 6Source ↗PDF ↗Scholar ↗
2024

Accept the Modality Gap: An Exploration in the Hyperbolic Space

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Ajanthan Thalaiyasingam
CVPR 2024highlight

Recent advancements in machine learning have spotlighted the potential of hyperbolic spaces as they effectively learn hierarchical feature representations. While there has been progress in leveraging hyperbolic spaces in single-modality contexts its exploration in multimodal settings remains under e…

Cited by 10Source ↗PDF ↗Scholar ↗
2024

BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Anton van den Hengel
AAAI 2024technical

Inferring the 3D structure of a non-rigid dynamic scene from a single moving camera is an under-constrained problem. Inspired by the remarkable progress of neural radiance fields (NeRFs) in photo-realistic novel view synthesis of static scenes, it has also been extended to dynamic settings. Such me…

Cited by 7Source ↗PDF ↗Scholar ↗
2024

Backpropagation-free Network for 3D Test-time Adaptation

Yanshuo Wang, Ali Cheraghian, Zeeshan Hayder, Jie Hong, Sameera Ramasinghe, Shafin Rahman +4
CVPR 2024poster

Real-world systems often encounter new data over time which leads to experiencing target domain shifts. Existing Test-Time Adaptation (TTA) methods tend to apply computationally heavy and memory-intensive backpropagation-based approaches to handle this. Here we propose a novel method that uses a bac…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2024

From Activation to Initialization: Scaling Insights for Optimizing Neural Fields

Hemanth Saratchandran, Sameera Ramasinghe, Simon Lucey
CVPR 2024highlight

In the realm of computer vision Neural Fields have gained prominence as a contemporary tool harnessing neural networks for signal representation. Despite the remarkable progress in adapting these networks to solve a variety of problems the field still lacks a comprehensive theoretical framework. Thi…

Cited by 2Source ↗PDF ↗Scholar ↗
2024

Improving the Convergence of Dynamic NeRFs via Optimal Transport

Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, Hisham Husain, Anton van den Hengel
ICLR 2024poster

Synthesizing novel views for dynamic scenes from a collection of RGB inputs poses significant challenges due to the inherent under-constrained nature of the problem. To mitigate this ill-posedness, practitioners in the field of neural radiance fields (NeRF) often resort to the adoption of intricate…

Cited by 1Source ↗PDF ↗Scholar ↗
2024

Neural Experts: Mixture of Experts for Implicit Neural Representations

Yizhak Ben-Shabat, Chamin P Hewa Koneputugodage, Sameera Ramasinghe, Stephen Gould
NeurIPS 2024poster

Implicit neural representations (INRs) have proven effective in various tasks including image, shape, audio, and video reconstruction. These INRs typically learn the implicit field from sampled input points. This is often done using a single network for the entire domain, imposing many global constr…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

ViewFusion: Towards Multi-View Consistency via Interpolated Denoising

Xianghui Yang, Yan Zuo, Sameera Ramasinghe, Loris Bazzani, Gil Avraham, Anton van den Hengel
CVPR 2024poster

Novel-view synthesis through diffusion models has demonstrated remarkable potential for generating diverse and high-quality images. Yet the independent process of image generation in these prevailing methods leads to challenges in maintaining multiple-view consistency. To address this we introduce V…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2023

Curvature-Aware Training for Coordinate Networks

Hemanth Saratchandran, Shin-Fang Chng, Sameera Ramasinghe, Lachlan MacDonald, Simon Lucey
ICCV 2023poster

Coordinate networks are widely used in computer vision due to their ability to represent signals as compressed, continuous entities. However, training these networks with first-order optimizers can be slow, hindering their use in real-time applications. Recent works have opted for shallow voxel-base…

Cited by 8PDF ↗code ↗Scholar ↗
2023

How much does Initialization Affect Generalization?

Sameera Ramasinghe, Lachlan Ewen MacDonald, Moshiur Farazi, Hemanth Saratchandran, Simon Lucey
ICML 2023poster

Characterizing the remarkable generalization properties of over-parameterized neural networks remains an open problem. A growing body of recent literature shows that the bias of stochastic gradient descent (SGD) and architecture choice implicitly leads to better generalization. In this paper, we sho…

Cited by 10Source ↗PDF ↗Scholar ↗
2022

Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs

Sameera Ramasinghe, Simon Lucey
ECCV 2022poster

"Coordinate-MLPs are emerging as an effective tool for modeling multidimensional continuous signals, overcoming many drawbacks associated with discrete grid-based approximations. However, coordinate-MLPs with ReLU activations, in their rudimentary form, demonstrate poor performance in representing s…

Cited by 140Source ↗PDF ↗code ↗Scholar ↗
2022

Few-Shot Class-Incremental Learning for 3D Point Cloud Objects

Townim Chowdhury, Ali Cheraghian, Sameera Ramasinghe, Sahar Ahmadi, Morteza Saberi, Shafin Rahman
ECCV 2022poster

"Few-shot class-incremental learning (FSCIL) aims to incrementally fine-tune a model trained on base classes for a novel set of classes using a few examples without forgetting the previous training. Recent efforts of FSCIL addresses this problem primarily on 2D image data. However, due to the advanc…

Cited by 21Source ↗PDF ↗code ↗Scholar ↗
2022

Gaussian Activated Neural Radiance Fields for High Fidelity Reconstruction & Pose Estimation

Shin-Fang Chng, Sameera Ramasinghe, Jamie Sherrah, Simon Lucey
ECCV 2022poster

"Despite Neural Radiance Fields (NeRF) showing compelling results in photorealistic novel views synthesis of real-world scenes, most existing approaches require accurate prior camera poses. Although approaches for jointly recovering the radiance field and camera pose exist, they rely on a cumbersome…

Cited by 133Source ↗PDF ↗Scholar ↗
2022

Trading Positional Complexity vs Deepness in Coordinate Networks

Jianqiao Zheng, Sameera Ramasinghe, Xueqian Li, Simon Lucey
ECCV 2022poster

"It is well noted that coordinate-based MLPs benefit---in terms of preserving high-frequency information---through the encoding of coordinate positions as an array of Fourier features. Hitherto, the rationale for the effectiveness of these \emph{positional encodings} has been mainly studied through…

Cited by 24Source ↗PDF ↗code ↗Scholar ↗
2021

Conditional Generative Modeling via Learning the Latent Space

Sameera Ramasinghe, Kanchana Nisal Ranasinghe, Salman Khan, Nick Barnes, Stephen Gould
ICLR 2021poster

Although deep learning has achieved appealing results on several machine learning tasks, most of the models are deterministic at inference, limiting their application to single-modal settings. We propose a novel general-purpose framework for conditional generation in multimodal spaces, that uses lat…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2021

Rethinking conditional GAN training: An approach using geometrically structured latent manifolds

Sameera Ramasinghe, Moshiur R Farazi, Salman Khan, Nick Barnes, Stephen Gould
NeurIPS 2021poster

Conditional GANs (cGAN), in their rudimentary form, suffer from critical drawbacks such as the lack of diversity in generated outputs and distortion between the latent and output manifolds. Although efforts have been made to improve results, they can suffer from unpleasant side-effects such as the…

Cited by 11Source ↗PDF ↗code ↗Scholar ↗
2021

Synthesized Feature Based Few-Shot Class-Incremental Learning on a Mixture of Subspaces

Ali Cheraghian, Shafin Rahman, Sameera Ramasinghe, Pengfei Fang, Christian Simon, Lars Petersson +1
ICCV 2021poster

Few-shot class incremental learning (FSCIL) aims to incrementally add sets of novel classes to a well-trained base model in multiple training sessions with the restriction that only a few novel instances are available per class. While learning novel classes, FSCIL methods gradually forget base (old)…

Cited by 86PDF ↗Scholar ↗
2020

Spectral-GANs for High-Resolution 3D Point-cloud Generation

Sameera Ramasinghe, Salman Khan, Nick Barnes, Stephen Gould
IROS 2020poster

Point-clouds are a popular choice for robotics and computer vision tasks due to their accurate shape description and direct acquisition from range-scanners. This demands the ability to synthesize and reconstruct high-quality point-clouds. Current deep generative models for 3D data generally work on…

Cited by 42Source ↗code ↗Scholar ↗