Benyamin Jamialahmadi

Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models

Deploying large language models (LLMs) in real-world applications is often hindered by strict computational and latency constraints. While dynamic inference offers the flexibility to adjust model behavior based on varying resource budgets, existing methods are frequently limited by hardware ineffici