Adversarial Random Forests for Density Estimation and Generative Modeling
David S. Watson, Kristin Blesch, Jan Kapar, Marvin N. Wright
Abstract
We propose methods for density estimation and data synthesis using a novel form of unsupervised random forests. Inspired by generative adversarial networks, we implement a recursive procedure in which trees gradually learn structural properties of the data through alternating rounds of generation and discrimination. The method is provably consistent under minimal assumptions. Unlike classic tree-based alternatives, our approach provides smooth (un)conditional densities and allows for fully synthetic data generation. We achieve comparable or superior performance to state-of-the-art probabilistic circuits and deep learning models on various tabular data benchmarks while executing about two orders of magnitude faster on average. An accompanying $R$ package, $arf$, is available on $CRAN$.
BibTeX
@InProceedings{pmlr-v206-watson23a,
title = {Adversarial Random Forests for Density Estimation and Generative Modeling},
author = {Watson, David S. and Blesch, Kristin and Kapar, Jan and Wright, Marvin N.},
booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics},
pages = {5357--5375},
year = {2023},
editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem},
volume = {206},
series = {Proceedings of Machine Learning Research},
month = {25--27 Apr},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v206/watson23a/watson23a.pdf},
url = {https://proceedings.mlr.press/v206/watson23a.html},
abstract = {We propose methods for density estimation and data synthesis using a novel form of unsupervised random forests. Inspired by generative adversarial networks, we implement a recursive procedure in which trees gradually learn structural properties of the data through alternating rounds of generation and discrimination. The method is provably consistent under minimal assumptions. Unlike classic tree-based alternatives, our approach provides smooth (un)conditional densities and allows for fully synthetic data generation. We achieve comparable or superior performance to state-of-the-art probabilistic circuits and deep learning models on various tabular data benchmarks while executing about two orders of magnitude faster on average. An accompanying $R$ package, $arf$, is available on $CRAN$.}
}