2024
LinguaLinked: Distributed Large Language Model Inference on Mobile Devices
ACL 2024system demonstrations
Deploying Large Language Models (LLMs) locally on mobile devices presents a significant challenge due to their extensive memory requirements. In this paper, we introduce LinguaLinked, a system for decentralized, distributed LLM inference on mobile devices. LinguaLinked enables collaborative executio…