2026
TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies
CVPR 2026
Native FP8 support in modern hardware is essential for training large Transformers, but is severely hindered by extreme activation outliers. Existing solutions either rely on complex mixed-precision engineering or invasive architectural modifications. This paper fundamentally challenges the conventi