Harsh Bihany

LoRMA: Low-Rank Multiplicative Adaptation for LLMs

Large Language Models have shown remarkable capabilities in the NLP domain. Their effectiveness can mainly be attributed to their ability to adapt to an array of downstream tasks. However, generally, full fine-tuning is a computationally expensive job. To mitigate this, many techniques have been dev…