Learning Three-Dimensional Bin Packing with Adjustable-Order Semi-Online Setting

Abstract

The online setting brings greater flexibility and practicality to the three-dimensional bin packing problem (3DBPP) but at the cost of algorithm performance. Existing methods mitigate the performance impact by introducing semionline settings with look-ahead or buffer zones. However, these methods either fail to fundamentally alter the packing order or reduce packing efficiency. This paper proposes a novel semionline setting that allows for the observation of multiple items and the selection of one for packing, thereby adjusting the packing order without reducing packing efficiency. We do work for solving the semi-online packing problem via reinforcement learning which faces two real-world challenges: (1) a variable and difficult-to-predict number of observed items, and (2) the obstruction of robotic arm movement by already packed items. On the one hand, we design a policy network capable of adapting to variable item quantities. On the other hand, we introduce a guided bottom-up packing reward function to free up space for robotic arm motion. We show that our method outperforms the baselines in terms of space utilization with the condition of observing at least two items. Further experiments demonstrate the functionality of our reward function, which can guide a virtual robot to complete packing tasks.