2022
A Differential Entropy Estimator for Training Neural Networks
ICML 2022spotlight
Mutual Information (MI) has been widely used as a loss regularizer for training neural networks. This has been particularly effective when learn disentangled or compressed representations of high dimensional data. However, differential entropy (DE), another fundamental measure of information, has no…