ICML 2025poster0 citations
PARQ: Piecewise-Affine Regularized Quantization
Lisa Jin, Jianhao Ma, Zechun Liu, Andrey Gromov, Aaron Defazio, Lin Xiao
Abstract
We develop a novel optimization method for quantization-aware training (QAT). Specifically, we show that convex, piecewise-affine regularization (PAR) can effectively induce neural network weights to cluster towards discrete values. We minimize PAR-regularized loss functions using an aggregate proximal stochastic gradient method (AProx) and prove that it enjoys last-iterate convergence. Our approach provides an interpretation of the straight-through estimator (STE), a widely used heuristic for QAT, as the asymptotic form of PARQ. We conduct experiments to demonstrate that PARQ obtains competitive performance on convolution- and transformer-based vision tasks.
quantization-aware trainingproximal gradient methodconvex regularizationstochastic gradient methodlast-iterate convergence
BibTeX
@inproceedings{
jin2025parq,
title={{PARQ}: Piecewise-Affine Regularized Quantization},
author={Lisa Jin and Jianhao Ma and Zechun Liu and Andrey Gromov and Aaron Defazio and Lin Xiao},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=8PCxOlwbIn}
}