27 accepted papers
Automating tasks in outdoor agricultural fields poses significant challenges due to environmental variability, unstructured terrain, and diverse crop characteristics. We present a robotic system that leverages imitation learning for autonomous pepper harvesting designed to operate in these complex s
Interactive sensors are an important component of robotic systems but often require manual replacement due to wear and tear. Automating this process can enhance system autonomy and facilitate long-term deployment. We developed an autonomous sensor exchange and calibration system for an agriculture c
In cluttered environments where visual sensors encounter heavy occlusion, such as in agricultural settings, tactile signals can provide crucial spatial information for the robot to locate rigid objects and maneuver around them. We introduce SonicBoom, a holistic hardware and learning pipeline that e
Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between syn-thetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the
Autonomous navigation is crucial for various robotics applications in agriculture. However, many existing methods depend on RTK-GPS devices, which can be susceptible to loss of radio signal or intermittent reception of corrections from the internet. Consequently, research has increasingly focused on
In this paper, we present a transformer-based method to spatio-temporally associate apple fruitlets in stereo-images collected on different days and from different camera poses. State-of-the-art association methods in agriculture are dedicated towards matching larger crops using either high-resoluti
In this paper, we present a next-best-view planning approach to autonomously size apple fruitlets. State-of-the-art viewpoint planners in agriculture are designed to size large and more sparsely populated fruit. They rely on lower resolution maps and sizing methods that do not generalize to smaller…
Monitoring crop nutrients can aid farmers in optimizing fertilizer use. Many existing robots rely on vision-based phenotyping, however, which can only indirectly estimate nutrient deficiencies once crops have undergone visible color changes. We present a contact-based phenotyping robot platform that
There is growing interest in automating agricultural tasks that require intricate and precise interaction with specialty crops, such as trees and vines. However, developing robotic solutions for crop manipulation remains a difficult challenge due to complexities involved in modeling their deformable…
In this paper, we present a method for creating high-quality 3D models of sorghum panicles for phenotyping in breeding experiments. This is achieved with a novel reconstruction approach that uses seeds as semantic landmarks in both 2D and 3D. To evaluate the performance, we develop a new metric for…
Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabiliti…
In this work, we present a method to extract the skeleton of a self-occluded tree canopy by estimating the unobserved structures of the tree. A tree skeleton compactly describes the topological structure and contains useful information such as branch geometry, positions and hierarchy. This can be cr…
Object detection and semantic segmentation are two of the most widely adopted deep learning algorithms in agricultural applications. One of the major sources of variability in image quality acquired outdoors for such tasks is changing lighting conditions that can alter the appearance of the objects…
We outline a neural network-based pipeline for perception, control and planning of a 7 DoF robot for tasks that involve reaching into a dormant grapevine canopy. The proposed system consists of a 6 DoF industrial robot arm and a linear slider that can actuate on an entire grape vine. Our approach us…
Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images. Transferring knowledge from other auxiliary tasks is a powerful tool for improving the learning efficiency. However, the usage…
Simultaneous Localization and Mapping (SLAM) is a fundamental task to mobile and aerial robotics. LiDAR based systems have proven to be superior compared to vision based systems due to its accuracy and robustness. In spite of its superiority, pure LiDAR based systems fail in certain degenerate cases…
The need for fast and precise measurements of plant attributes makes robotic solutions an ideal replacement for labor-intensive phenotyping processes. In this work we present a deep learning-based high throughput, online pipeline for in-situ sorghum stalk detection and grasping. We use a variation o…
Perception system performance can vary dramatically with contextual factors such as environmental geometry, appearance, and other phenomena. In this work we present a theoretical framework for understanding the role of context in perception and discuss three approaches for predicting probabilistic p…
Many modern perception systems require human engineers to tune parameters in order to adapt to various environments and applications. This incurs a large startup cost when deploying a robotic system by relying on human expertise and ground truth instrumentation. To alleviate this, we propose a techn
Modern perception systems are notoriously complex, featuring dozens of interacting parameters that must be tuned to achieve good performance. Conventional tuning approaches require expensive ground truth, while heuristic methods are difficult to generalize. In this work, we propose an introspective…
We proposed a multimodal end-to-end policy based on deep reinforcement learning (DRL) that leverages sensor fusion to reduced performance drops in noisy environment from 50% to 10% compared with the baseline and makes the policy functional even in the face of partial sensor failure by using a novel
This paper describes an algorithm for visually detecting crop stalks in-situ. Field-based crop stalk detection is a challenging computer vision problem due to occlusion by leaves, similarity of color and texture between stalks and surrounding foliage, and high stalk density within rows. Detecting st…
The established processes for measuring physiological and morphological traits (phenotypes) of crops in outdoor test plots are labor intensive and error-prone. Low-cost, reliable, field-based robotic phenotyping will enable geneticists to more easily map genotypes to phenotypes, which in turn will i…
The popularity of cameras for perception is enabled in part by powerful intrinsic calibration routines, commonly requiring a user to manually collect images of a known calibration target. The manual nature of this process produces training data that is unevenly spread in the camera relative pose spa
Assembly of large structures requires large fixtures, often referred to as monuments. Their cost and massive size limit flexibility and scalability of the manufacturing process. Numerous small mobile robots can replace these large structures and, therefore, replicate the efficiency of the assembly l…
The ballbot is a human sized, dynamically stable mobile robot that balances on a single, spherical wheel. The current framework for navigation and control makes the assumption that the robot is operating on a level surface without any center-of-mass offset; however, in practice, such a dynamically s…
The ubiquitous additive Gaussian noise model is favored in statistical modeling applications for its flexibility and ease of use. Often noise is assumed to be well-represented by a constant covariance, while in reality error characteristics may change predictably. We present an efficient parametric…