Bamdev Mishra — accepted AI-conference papers

2026

Minibatch selection for Language Models via Partition Matroid Constrained Gradient Matching

Prayas Agrawal, Prateek Chanda, Ishita Khatri, Ganesh Ramakrishnan, Bamdev Mishra, Pratik Kumar Jawanpuria

ICML 2026poster

Training Large Language Models (LLMs) on heterogeneous datasets requires optimizing domain representations to balance convergence speed and domain coverage. While recent methods reduce computational overhead by selecting high-quality data subsets, they typically perform selection independently per d…

Cited by 0 Source ↗Scholar ↗

2026

Riemannian Federated Learning via Averaging Gradient Streams

Zhenwei Huang, Wen Huang, Pratik Jawanpuria, Bamdev Mishra

ICLR 2026poster

Federated learning (FL) as a distributed learning paradigm has a significant advantage in addressing large-scale machine learning tasks. In the Euclidean setting, FL algorithms have been extensively studied with both theoretical and empirical success. However, there exist few works that investigate…

Cited by 0 Source ↗Scholar ↗

2025

A Riemannian Approach to Ground Metric Learning for Optimal Transport

Pratik Jawanpuria, Dai Shi, Bamdev Mishra, Junbin Gao

ICASSP 2025accepted

Optimal transport (OT) theory has attracted much attention in machine learning and signal processing applications. OT defines a notion of distance between probability distributions of source and target data points. A crucial factor that influences OT-based distances is the ground metric of the embed…

Cited by 0 Source ↗Scholar ↗

2024

A Framework for Bilevel Optimization on Riemannian Manifolds

Andi Han, Bamdev Mishra, Pratik Jawanpuria, Akiko Takeda

NeurIPS 2024poster

Bilevel optimization has gained prominence in various applications. In this study, we introduce a framework for solving bilevel optimization problems, where the variables in both the lower and upper levels are constrained on Riemannian manifolds. We present several hypergradient estimation strategie…

Cited by 8 Source ↗PDF ↗code ↗Scholar ↗

2024

Riemannian coordinate descent algorithms on matrix manifolds

Andi Han, Pratik Jawanpuria, Bamdev Mishra

ICML 2024poster

Many machine learning applications are naturally formulated as optimization problems on Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the feasibility of the variables while moving along a descent direction on the manifold. This results in updating all the variable…

Cited by 4 Source ↗PDF ↗Scholar ↗

2024

SLTrain: a sparse plus low rank approach for parameter and memory efficient pretraining

Andi Han, Jiaxiang Li, Wei Huang, Mingyi Hong, Akiko Takeda, Pratik Jawanpuria +1

NeurIPS 2024poster

Large language models (LLMs) have shown impressive capabilities across various tasks. However, training LLMs from scratch requires significant computational power and extensive memory capacity. Recent studies have explored low-rank structures on weights for efficient fine-tuning in terms of paramete…

Cited by 15 Source ↗PDF ↗code ↗Scholar ↗

2024

Submodular framework for structured-sparse optimal transport

Piyushi Manupriya, Pratik Jawanpuria, Karthik S. Gurumoorthy, SakethaNath Jagarlapudi, Bamdev Mishra

ICML 2024poster

Unbalanced optimal transport (UOT) has recently gained much attention due to its flexible framework for handling un-normalized measures and its robustness properties. In this work, we explore learning (structured) sparse transport plans in the UOT setting, i.e., transport plans have an upper bound o…

Cited by 2 Source ↗PDF ↗code ↗Scholar ↗

2023

Riemannian Accelerated Gradient Methods via Extrapolation

Andi Han, Bamdev Mishra, Pratik Jawanpuria, Junbin Gao

AISTATS 2023poster

In this paper, we propose a convergence acceleration scheme for general Riemannian optimization problems by extrapolating iterates on manifolds. We show that when the iterates are generated from the Riemannian gradient descent method, the scheme achieves the optimal convergence rate asymptotically a…

Cited by 10 Source ↗PDF ↗Scholar ↗

2021

On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry

Andi Han, Bamdev Mishra, Pratik Jawanpuria, Junbin Gao

NeurIPS 2021poster

In this paper, we comparatively analyze the Bures-Wasserstein (BW) geometry with the popular Affine-Invariant (AI) geometry for Riemannian optimization on the symmetric positive definite (SPD) matrix manifold. Our study begins with an observation that the BW metric has a linear dependence on SPD mat…

Cited by 53 Source ↗PDF ↗code ↗Scholar ↗

2019

Riemannian adaptive stochastic gradient algorithms on matrix manifolds

Hiroyuki Kasai, Pratik Jawanpuria, Bamdev Mishra

ICML 2019oral

Adaptive stochastic gradient algorithms in the Euclidean space have attracted much attention lately. Such explorations on Riemannian manifolds, on the other hand, are relatively new, limited, and challenging. This is because of the intrinsic non-linear structure of the underlying manifold and the ab…

Cited by 70 Source ↗PDF ↗code ↗Scholar ↗

2018

A Dual Framework for Low-rank Tensor Completion

Madhav Nimishakavi, Pratik Kumar Jawanpuria, Bamdev Mishra

NeurIPS 2018poster

One of the popular approaches for low-rank tensor completion is to use the latent trace norm regularization. However, most existing works in this direction learn a sparse combination of tensors. In this work, we fill this gap by proposing a variant of the latent trace norm that helps in learning a n…

Cited by 44 Source ↗PDF ↗code ↗Scholar ↗

2018

A Unified Framework for Structured Low-rank Matrix Learning

Pratik Jawanpuria, Bamdev Mishra

ICML 2018oral

We consider the problem of learning a low-rank matrix, constrained to lie in a linear subspace, and introduce a novel factorization for modeling such matrices. A salient feature of the proposed factorization scheme is it decouples the low-rank and the structural constraints onto separate factors. We…

Cited by 31 Source ↗PDF ↗Scholar ↗

2018

Inexact trust-region algorithms on Riemannian manifolds

Hiroyuki Kasai, Bamdev Mishra

NeurIPS 2018poster

We consider an inexact variant of the popular Riemannian trust-region algorithm for structured big-data minimization problems. The proposed algorithm approximates the gradient and the Hessian in addition to the solution of a trust-region sub-problem. Addressing large-scale finite-sum problems, we sp…

Cited by 27 Source ↗PDF ↗code ↗Scholar ↗

2018

Riemannian Stochastic Recursive Gradient Algorithm

Hiroyuki Kasai, Hiroyuki Sato, Bamdev Mishra

ICML 2018accepted

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of re…

Cited by 60 Source ↗PDF ↗code ↗Scholar ↗

2018

Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis

Hiroyuki Kasai, Hiroyuki Sato, Bamdev Mishra

AISTATS 2018poster

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and…

Cited by 0 Source ↗PDF ↗Scholar ↗

2016

Low-rank tensor completion: a Riemannian manifold preconditioning approach

Hiroyuki Kasai, Bamdev Mishra

ICML 2016poster

We propose a novel Riemannian manifold preconditioning approach for the tensor completion problem with rank constraint. A novel Riemannian metric or inner product is proposed that exploits the least-squares structure of the cost function and takes into account the structured symmetry that exists in…

Cited by 153 Source ↗PDF ↗Scholar ↗