Yushan Su

PruMUX: Augmenting Data Multiplexing with Model Compression

As language models increase in size by the day, methods for efficient inference are critical to leveraging their capabilities for various applications. Prior work has investigated techniques like model pruning, knowledge distillation, and data multiplexing to increase model throughput without sacrif…