Paul N. Whatmough

STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization

Quantization is the key method for reducing inference latency, power and memory footprint of generative AI models. However, accuracy often degrades sharply when activations are at low bit widths. Recent work suggests that invertible linear transformations (e.g. rotations) can aid quantization, by r…