Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems
Atsushi Nitanda, Kazusato Oko, Denny Wu, Nobuhito Takenouchi, Taiji Suzuki
Abstract
The entropic fictitious play (EFP) is a recently proposed algorithm that minimizes the sum of a convex functional and entropy in the space of measures --- such an objective naturally arises in the optimization of a two-layer neural network in the mean-field regime. In this work, we provide a concise primal-dual analysis of EFP in the setting where the learning problem exhibits a finite-sum structure. We establish quantitative global convergence guarantees for both the continuous-time and discrete-time dynamics based on properties of a proximal Gibbs measure introduced in Nitanda et al. (2022). Furthermore, our primal-dual framework entails a memory-efficient particle-based implementation of the EFP update, and also suggests a connection to gradient boosting methods. We illustrate the efficiency of our novel implementation in experiments including neural network optimization and image synthesis.
BibTeX
@inproceedings{icml2023_primalanddualana,
title = {Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems},
author = {Atsushi Nitanda and Kazusato Oko and Denny Wu and Nobuhito Takenouchi and Taiji Suzuki},
booktitle = {ICML 2023},
year = {2023}
}