NianHui Guo*

Enhancing Optimization Robustness in 1-bit Neural Networks through Stochastic Sign Descent

"Binary Neural Networks (BNNs) offer a promising avenue toward achieving efficient deep-learning models but are hindered by the inherent challenge of aligning noisy floating-point gradients with binary parameters. To address this, we introduce Diode, a groundbreaking optimizer designed explicitly fo…