Latent-Guided Cooperative Energy-Based Models

Abstract

Energy-based models (EBMs) provide a flexible framework for generative models with strong distribution modeling capabilities. Nevertheless, their broader adoption has been limited by the difficulty of stable and efficient training. In this paper, we propose a unified and efficient latent-guided cooperative EBM that leverages informative target latent variables to guide the joint energy in capturing both data distribution and semantic structure, along with a cooperative generator designed for effective MCMC initialization. Our joint space optimization only requires MCMC sampling in the data space, and allows the energy to learn semantic data–latent relationships directly from real data. Experiments show our method improves generation quality and training stability with fewer resources, and performs effectively across multiple downstream tasks.