Efficient Hierarchical Reinforcement Learning with Dynamic Kolmogorov–Arnold Network for Long-Horizon Robotic Manipulation

Abstract

Long-horizon robotic manipulation remains a critical challenge in robotics. Hierarchical reinforcement learning offers a promising solution, but often suffers from an imbalance dilemma: simplifying skill learning increases the complexity of planning, thereby expanding the solution space and computational burden of planning. To tackle this challenge, we propose a Hierarchical Reinforcement Learning framework with Dynamic Kolmogorov-Arnold Network (DyKAN) based Actor Critic, named HIKER. Firstly, HIKER innovates with a dual-chain design that decomposes the complex task into two intersecting sub-chains, reducing the optimization conflict across subtasks and alleviating the burden on the planning model. Secondly, we develop DyKAN, a scalable neural network for both actor and critic in the skill model of HIKER. DyKAN adaptively adjusts grids and basis functions while preserving learned knowledge, enabling efficient learning of complex manipulation skills. Furthermore, to optimize DyKAN's performance, we design a per-layer update module that uses Dynamic Tanh (DyT) and low-rank decomposition to ensure stable, low-cost updates during training. Finally, experiments on long-horizon robotic manipulation tasks demonstrate that HIKER significantly improves efficiency and robustness, yielding higher-quality skill models and achieving a 10.9% increase in task success rate under the high noise condition. Further insights are available on the website: https://sites.google.com/view/hikerdykan.