← Search

Wasim Huleihel

8 accepted papers

2025

AdaRankGrad: Adaptive Gradient Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning

ICLR 2025poster

Training and fine-tuning large language models (LLMs) come with challenges related to memory and computational requirements due to the increasing size of the model weights and the optimizer states. To tackle these challenges, various techniques have been developed, such as low-rank adaptation (LoRA)…

Cited by 3SourcePDFScholar
2024

Statistical and Computational Limits of Detecting and Recovering Hidden Submatrices

ICASSP 2024accepted

We study the problems of detection and recovery of hidden submatrices with elevated means inside a large Gaussian random matrix. We consider two different structures for the planted submatrices. In the first model, the planted matrices are disjoint, and their row and column indices can be arbitrary.…

Cited by 0SourceScholar