DynaDojo: An Extensible Benchmarking Platform for Scalable Dynamical System Identification
Logan Mondal Bhamidipaty, Tommy Bruzzese, Caryn Tran, Rami Ratl Mrad, Max Kanwal
Abstract
Modeling complex dynamical systems poses significant challenges, with traditional methods struggling to work across a variety of systems and scale to high-dimensional dynamics. In response, we present DynaDojo, a novel benchmarking platform designed for data-driven dynamical system identification. DynaDojo enables comprehensive evaluation of how an algorithm's performance scales across three key dimensions: (1) the number of training samples provided, (2) the complexity of the dynamical system being modeled, and (3) the training samples required to achieve a target error threshold. Furthermore, DynaDojo enables studying out-of-distribution generalization (by providing multiple test conditions for each system) and active learning (by supporting closed-loop control). Through its user-friendly and easily extensible API, DynaDojo accommodates a wide range of user-defined $\texttt{Algorithms}$, $\texttt{Systems}$, and $\texttt{Challenges}$ (scaling metrics). The platform also prioritizes resource-efficient training for running on a cluster. To showcase its utility, in DynaDojo $\texttt{0.9}$, we include implementations of 7 baseline algorithms and 20 dynamical systems, along with many demo notebooks. This work aspires to make DynaDojo a unifying benchmarking platform for system identification, paralleling the role of OpenAI’s Gym in reinforcement learning.
BibTeX
@inproceedings{
bhamidipaty2023dynadojo,
title={DynaDojo: An Extensible Platform for Benchmarking Scaling in Dynamical System Identification},
author={Logan Mondal Bhamidipaty and Tommy Bruzzese and Caryn Tran and Rami Ratl Mrad and Max Kanwal},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2023},
url={https://openreview.net/forum?id=pTSNoBTk8E}
}