Occupancy-belief Planning of Plant Manipulation for Staking

Abstract

While agricultural robotics has made great strides in recent years, manipulation of plants for tasks such as staking and harvesting remains highly challenging due to the high variability in dynamics and deformable nature of plants. To address the challenges created by dynamics uncertainty, we develop a system applying an occupancy-belief planning concept to plant manipulation for staking. We first train a dynamics model that predicts a per-pixel probability that the plant occupies the corresponding slice in space after a drag action using a large set of simulators. This model is then used to plan a manipulation action that maximizes the probability areas swept by the stake tying tool’s operating region are occupied by the plant, and minimize the probability areas swept by the non-operating side regions of the tool are occupied. We demonstrate our method both in simulation and with zero-shot sim-to-real transfer to a physical implementation. We show that adding consideration of belief through use of occupancy-belief allows our method to outperform both the visual foresight type approaches it is based on and other baselines and ablations, especially in the real-world case.