Designing Observation and Action Models for Efficient Reinforcement Learning with LLMs
Deunsol Yoon, Sunghoon Hong, Whiyoung Jung, Junseok Park, Geon-Hyeong Kim, Woohyung Lim, Soonyoung Lee, Seung Hwan Kim
Abstract
Large Language Models (LLMs) have emerged as powerful tools for semantic reasoning, enabling the formalization of tasks that traditionally relied on manual human intuition. This capability extends to environment design in Reinforcement Learning (RL). While prior research predominantly focuses on reward design, the design of observation and action spaces remains relatively underexplored. We propose LOAM, a framework leveraging LLMs to construct refined agent spaces from raw environments. To mitigate the computational burden of identifying the best candidate model from stochastic LLM outputs, LOAM incorporates a continuous racing mechanism that dynamically allocates resources to prioritize the most promising configurations without additional training overhead. Empirical evaluations on HumanoidBench and Isaac Lab demonstrate that LOAM consistently outperforms handcrafted baselines in both learning speed and asymptotic performance.
BibTeX
@inproceedings{
yoon2026designing,
title={Designing Observation and Action Models for Efficient Reinforcement Learning with {LLM}s},
author={Deunsol Yoon and Sunghoon Hong and Whiyoung Jung and Junseok Park and Geon-Hyeong Kim and Woohyung Lim and Soonyoung Lee and Seung Hwan Kim and Kanghoon Lee},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=OtZCoLMXDC}
}