Polarity Is All You Need to Learn and Transfer Faster
Qingyang Wang, Michael Alan Powell, Eric W Bridgeford, Ali Geisa, Joshua T Vogelstein
Abstract
Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities are **adequately** set ***a priori***, then networks learn with less time and data. We also explicitly illustrate situations in which *a priori* setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.
BibTeX
@inproceedings{icml2023_polarityisallyou,
title = {Polarity Is All You Need to Learn and Transfer Faster},
author = {Qingyang Wang and Michael Alan Powell and Eric W Bridgeford and Ali Geisa and Joshua T Vogelstein},
booktitle = {ICML 2023},
year = {2023}
}