Arian Raje

Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning

Large Language Models (LLMs) have yet to effectively leverage the vast amounts of edge-device data, and Federated Learning (FL) offers a promising paradigm to collaboratively fine-tune LLMs without transferring private edge data to the cloud. To operate within the computational and communication con…