2025
FPE2M2: Approaching Lossless and Efficient Quantization with Native Floating Point
ACL 2025finding
Auto-regressive decoding is a memory-bound job, meaning decoding inference performance is limited by the bandwidth rather than the computational capabilities of the GPU. Weight-only quantization is a promising method to address the memory-bound limitations. Previous studies have followed one of two…