Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products
Xuyang Zhang, Xi Zhang, Liang Chen, Hao Shi, Qingshan Guo
Abstract
Recent theoretical advances reveal that the Hadamard product induces nonlinear representations and implicit high-dimensional mappings for the field of deep learning, yet their practical deployment in efficient vision models remains underdeveloped. To address this gap, we introduce the Adaptive Cross-Hadamard (ACH) module, a novel operator that embeds learnability through differentiable discrete sampling and dynamic softsign normalization. This enables parameter-free feature reuse while stabilizing gradient propagation. Integrated into Hadaptive-Net (Hadamard Adaptive Network) via neural architecture search, our approach achieves unprecedented efficiency. Comprehensive experiments demonstrate state-of-the-art accuracy/speed trade-offs on image classification task, establishing Hadamard operations as fundamental building blocks for efficient vision models.
BibTeX
@inproceedings{
zhang2026expressive,
title={Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products},
author={Xuyang Zhang and Xi Zhang and Liang Chen and Hao Shi and Qingshan Guo},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=eQmoST3UMN}
}