AIConfPaper
← Search

Krishnateja Killamsetty

14 accepted papers

2026

Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning

Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han, Abhishek Bhandwaldar, Prateek Chanda, Kai Xu +5
ICLR 2026poster

Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. Existing parameter-efficient methods often limit model expressivity or introduce new parameters per task, creating scalab…

Cited by 0Source ↗code ↗Scholar ↗
2025

DELIFT: Data Efficient Language model Instruction Fine-Tuning

Ishika Agarwal, Krishnateja Killamsetty, Lucian Popa, Marina Danilevsky
ICLR 2025poster

Fine-tuning large language models (LLMs) is crucial for task specialization but often becomes resource-intensive due to redundant or uninformative data. Existing data selection methods typically rely either on computationally expensive gradient-based metrics or static embeddings that fail to adapt d…

Cited by 3Source ↗PDF ↗code ↗Scholar ↗
2025

Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs

Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang, Krishnateja Killamsetty, Shivchander Sudalairaj, Wenlong Zhao +7
ICLR 2025poster

The rise of large language models (LLMs) has created a significant disparity: industrial research labs with their computational resources, expert teams, and advanced infrastructures, can effectively fine-tune LLMs, while individual developers and small organizations face barriers due to limited reso…

Cited by 3Source ↗PDF ↗code ↗Scholar ↗
2024

SCoRe: Submodular Combinatorial Representation Learning

Anay Majee, Suraj Nandkishor Kothawade, Krishnateja Killamsetty, Rishabh K Iyer
ICML 2024poster

In this paper we introduce the **SCoRe** (**S**ubmodular **Co**mbinatorial **Re**presentation Learning) framework, a novel approach in representation learning that addresses inter-class bias and intra-class variance. SCoRe provides a new combinatorial viewpoint to representation learning, by introdu…

Cited by 3Source ↗PDF ↗Scholar ↗
2023

INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models

H S V N S Kowndinya Renduchintala, Krishnateja Killamsetty, Sumit Bhatia, Milan Aggarwal, Ganesh Ramakrishnan, Rishabh K Iyer +1
EMNLP 2023long findings

A salient characteristic of pre-trained language models (PTLMs) is a remarkable improvement in their generalization capability and emergence of new capabilities with increasing model capacity and pre-training dataset size. Consequently, we are witnessing the development of enormous models pushing th…

Cited by 0Source ↗code ↗Scholar ↗
2022

A Nested Bi-level Optimization Framework for Robust Few Shot Learning

Krishnateja Killamsetty, Changbin Li, Chen Zhao, Feng Chen, Rishabh Iyer
AAAI 2022technical

Model-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal.Hence, it fails to address the domain shift between base and novel classes in few-shot learning. In this work, we propose a novel…

Cited by 16Source ↗PDF ↗code ↗Scholar ↗
2022

AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning

Krishnateja Killamsetty, Guttu Sai Abhishek, Aakriti Lnu, Ganesh Ramakrishnan, Alexandre V. Evfimievski, Lucian Popa +1
NeurIPS 2022accept

Deep neural networks have seen great success in recent years; however, training a deep model is often challenging as its performance heavily depends on the hyper-parameters used. In addition, finding the optimal hyper-parameter configuration, even with state-of-the-art (SOTA) hyper-parameter optimiz…

Cited by 23Source ↗PDF ↗code ↗Scholar ↗
2022

GCR: Gradient Coreset Based Replay Buffer Selection for Continual Learning

Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, Pradeep Shenoy
CVPR 2022poster

Continual learning (CL) aims to develop techniques by which a single model adapts to an increasing number of tasks encountered sequentially, thereby potentially leveraging learnings across tasks in a resource-efficient manner. A major challenge for CL systems is catastrophic forgetting, where earlie…

Cited by 157PDF ↗Scholar ↗
2022

Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming

Ayush Maheshwari, Krishnateja Killamsetty, Ganesh Ramakrishnan, Rishabh Iyer, Marina Danilevsky, Lucian Popa
ACL 2022findings

A critical bottleneck in supervised machine learning is the need for large amounts of labeled data which is expensive and time-consuming to obtain. Although a small amount of labeled data cannot be used to train a model, it can be used effectively for the generation of humaninterpretable labeling fu…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2022

ORIENT: Submodular Mutual Information Measures for Data Subset Selection under Distribution Shift

Athresh Karanam, Krishnateja Killamsetty, Harsha Kokel, Rishabh K Iyer
NeurIPS 2022accept

Real-world machine-learning applications require robust models that generalize well to distribution shift settings, which is typical in real-world situations. Domain adaptation techniques aim to address this issue of distribution shift by minimizing the disparities between domains to ensure that the…

Cited by 14Source ↗PDF ↗Scholar ↗
2021

GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning

Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, Rishabh Iyer
AAAI 2021technical

Large scale machine learning and deep models are extremely data-hungry. Unfortunately, obtaining large amounts of labeled data is expensive, and training state-of-the-art models (with hyperparameter tuning) requires significant computing resources and time. Secondly, real-world data is noisy and imb…

Cited by 247Source ↗PDF ↗code ↗Scholar ↗
2021

GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training

Krishnateja Killamsetty, Durga S, Ganesh Ramakrishnan, Abir De, Rishabh Iyer
ICML 2021spotlight

The great success of modern machine learning models on large datasets is contingent on extensive computational resources with high financial and environmental costs. One way to address this is by extracting subsets that generalize on par with the full data. In this work, we propose a general framewo…

Cited by 267Source ↗PDF ↗code ↗Scholar ↗
2021

RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning

Krishnateja Killamsetty, Xujiang Zhao, Feng Chen, Rishabh K Iyer
NeurIPS 2021poster

Semi-supervised learning (SSL) algorithms have had great success in recent years in limited labeled data regimes. However, the current state-of-the-art SSL algorithms are computationally expensive and entail significant compute time and energy requirements. This can prove to be a huge limitation for…

Cited by 98Source ↗PDF ↗code ↗Scholar ↗
2021

SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios

Suraj Nandkishor Kothawade, Nathan Alexander Beck, Krishnateja Killamsetty, Rishabh K Iyer
NeurIPS 2021poster

Active learning has proven to be useful for minimizing labeling costs by selecting the most informative samples. However, existing active learning methods do not work well in realistic scenarios such as imbalance or rare classes,out-of-distribution data in the unlabeled set, and re…

Cited by 115Source ↗PDF ↗code ↗Scholar ↗