2026
ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference
ICML 2026poster
Multi-device inference can reduce Transformer latency by parallelizing computation. However, existing methods require high inter-device bandwidth, making them impractical for bandwidth-constrained environments. We present ASTRA, a communication-efficient framework that integrates sequence parallelis…