64 accepted papers
Multitask learning poses significant challenges due to the highly multimodal and diverse nature of robot action distributions. However, effectively fitting policies to these complex task distributions is often difficult, and existing monolithic models often underfit the action distribution and lack
Estimating simulation-ready scenes from real-world observations is crucial for downstream planning and policy learning tasks. Regretfully, existing methods struggle in cluttered environments, often exhibiting prohibitive computational cost, poor robustness, and restricted generality when scaling to …
This paper addresses the challenge of autonomous excavation of challenging terrains, in particular those that are prone to jamming and inter-particle adhesion when tackled by a standard penetrate-drag-scoop motion pattern. Inspired by human excavation strategies, our approach incorporates oscillator
Social robot navigation algorithms are often demonstrated in overly simplified scenarios, prohibiting the extraction of practical insights about their relevance to real-world domains. Our key insight is that an understanding of the inherent complexity of a social robot navigation scenario could help
High-resolution visuotactile sensors provide detailed contact information that is promising to infer the physical properties of objects in contact. This paper introduces a novel technique for high-resolution stiffness estimation of heterogeneous deformable objects using the Punyo bubble sensor. We d
Searching for objects in cluttered environments requires selecting efficient viewpoints and manipulation actions to remove occlusions and reduce uncertainty in object locations, shapes, and categories. In this work, we address the problem of manipulation-enhanced semantic mapping, where a robot has…
Metric-semantic 3D mapping is the process of creating class-labeled 3D maps by fusing the information from images captured by a moving camera. The memory usage required by standard solutions grows linearly with the number of semantic classes being considered, which can pose a bottleneck in large and
Modeling the dynamics of deformable objects is challenging due to their diverse physical properties and the difficulty of estimating states from limited visual information. We address these challenges with a neural dynamics framework that combines object particles and spatial grids in a hybrid repre…
Fruit monitoring plays an important role in crop management, and rising global fruit consumption combined with labor shortages necessitates automated monitoring with robots. However, occlusions from plant foliage often hinder accurate shape and pose estimation. Therefore, we propose an active fruit
Soft-bubble tactile sensors have the potential to capture dense contact and force information across a large contact surface. However, it is difficult to extract contact forces directly from observing the bubble surface because local contacts change the global surface shape significantly due to memb…
Predictive models are a crucial component of many robotic systems. Yet, constructing accurate predictive models for a variety of deformable objects, especially those with unknown physical properties, remains a significant challenge. This paper introduces AdaptiGraph, a learning-based dynamics modeli…
This paper presents a novel experience-based technique, called EHGO, for sample-efficient adaptive control of nonlinear systems in the presence of dynamical modeling errors. The starting point for EHGO is a database seeded with many trajectories optimized under a reference estimate of real system dy…
"Text-to-image diffusion models have shown remarkable success in synthesizing photo-realistic images. Apart from creative applications, can we use such models to synthesize samples that aid the few-shot training of discriminative models? In this work, we propose AlignDiff, a general framework for sy…
Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable w
Soft robots have desirable qualities for use in underwater environments thanks to their inherent compliance and lack of need for exposed hardware. Nevertheless, these advantages come at the cost of considerable control challenges. Data-driven model predictive control (MPC) is an approach that has sh
Multi-agent path finding is a computationally challenging problem that is relevant to many areas in robotics. Experience-based planning methods have been shown to significantly reduce the planning time of this problem, but the type of problem in which experience can be used has so far been limited t…
Teleoperated avatar robots allow people to transport their manipulation skills to environments that may be difficult or dangerous to work in. Current systems are able to give operators direct control of many components of the robot to immerse them in the remote environment, but operators still strug…
Semantic 3D mapping, the process of fusing depth and image segmentation information between multiple views to build 3D maps annotated with object classes in real-time, is a recent topic of interest. This paper highlights the fusion overconfidence problem, in which conventional mapping methods assign…
Estimating visual-tactile models of deformable objects is challenging because vision suffers from occlusion, while touch data is sparse and noisy. We propose a novel data-efficient method for dense heterogeneous model estimation by leveraging experience from diverse training objects. The method is…
This paper introduces a method for learning the force response of heterogeneous, deformable objects directly from robot sensor data without prior knowledge. The method estimates an object's force response given robot force or torque measurements using a novel volumetric stiffness field representatio…
Autonomous lander missions on extraterrestrial bodies will need to sample granular material while coping with domain shift, no matter how well a sampling strategy is tuned on Earth. This paper proposes an adaptive scooping strategy that uses deep Gaussian process method trained with meta-learning to…
Complex dexterous manipulations require switching between prehensile and non-prehensile grasps, and sliding and pivoting the object against the environment. This paper presents a manipulation planner that is able to reason about diverse changes of contacts to discover such plans. It implements a hyb…
This letter presents the first implementation of autonomous robotic auscultation of heart and lung sounds. To select auscultation locations that generate high-quality sounds, a Bayesian Optimization (BO) formulation leverages visual anatomical cues to predict where high-quality sounds might be locat
This paper presents an approach to measure the ability of a shared control system to track user input while simultaneously ensuring safety. These CPI metrics, based on reachability theory, are used to measure the Conservativeness (C), Permissiveness (P) and the amount of Intervention (I) applied to
Deep anterior lamellar keratoplasty (DALK) is a technique for cornea transplantation which is associated with reduced patient morbidity. DALK has been explored as a potential application of robot microsurgery because the small scales, fine control requirements, and difficulty of visualization make i
This paper presents a novel iterative closest points (ICP) variant, non-penetration iterative closest points (NPICP), which prevents interpenetration in 6DOF pose optimization and/or joint optimization of multiple object poses. This capability is particularly advantageous in cluttered scenarios, whe…
Robots have to take highly responsive real-time actions, driven by complex decisions involving a pipeline of sensing, perception, planning, and reaction tasks. These tasks must be scheduled on resource-constrained devices such that the performance goals and the requirements of the application are me…
Disinfection robots have applications in promoting public health and reducing hospital acquired infections and have drawn considerable interest due to the COVID-19 pan-demic. To disinfect a room quickly, motion planning can be used to plan robot disinfection trajectories on a reconstructed 3D map of…
Shared control can assist a human tele-operator in performing tasks on a remote robot, but also adds complexity in the user interface to allow the user to select the mode of assistance. This letter presents an expert action recommender framework that learns what actions are helpful to accomplish a t
Model predictive control (MPC) is a powerful feedback technique that is often used in data-driven robotics. The performance of data-driven MPC depends on the accuracy of the model, which often requires careful tuning. Furthermore, specifying the task with an objective function and synthesizing a fee…
We present a bilevel, contact-implicit trajectory optimization (TO) formulation that searches for robot trajectories with learned soft contact models. On the lower-level, contact forces are solved via a quadratic program (QP) with the maximum dissipation principle (MDP), based on which the dynamics…
We present a planner for large-scale (un)labeled object sorting tasks, which uses two types of manipulation actions: overhead grasping and planar pushing. The grasping action offers completeness guarantee under mild assumptions, and the planar pushing is an acceleration strategy that moves multiple…
This paper proposes a hybrid planning framework that generates complex dynamic motion plans for jumping legged robots to traverse challenging terrains. By employing a motion primitive, the original problem is decoupled as path planning followed by a trajectory optimization (TO) module that handles d…
We present non-convex maximal dissipation principle (NMDP), a time integration scheme for articulated bodies with simultaneous contacts. Our scheme resolves contact forces via the maximal dissipation principle (MDP). Whereas prior MDP solvers assume linearized dynamics and integrate using the forwar…
Robot design is a time-consuming process involving repeated experiments in a variety of environments to optimize multiple, possibly conflicting performance metrics. Moreover, the optimal robot performance for a given design depends on how the robot adapts its behavior to its environment. We propose…
UV radiation has been used as a disinfection strategy to deactivate a wide range of pathogens, but existing irradiation strategies do not ensure sufficient exposure of all environmental surfaces and/or require long disinfection times. We present a near-optimal coverage planner for mobile UV disinfec…
This paper presents a novel computational model to address the problem that contact is an infinite phenomena involving continuous regions of interaction. The problem is cast as a semi-infinite program with complementarity constraints (SIPCC). Rather than pre-discretize contacting surfaces into a fin…
Time-optimal trajectories for dynamic robotic vehicles are difficult to compute even for state-of-the-art nonlinear programming (NLP) solvers, due to nonlinearity and bang-bang control structure. This paper presents a bilevel optimization framework that addresses these problems by decomposing the sp…
This paper presents a semi-empirical method for simulating contact with elastically deformable objects whose force response is learned using entirely data-driven models. A point-based surface representation and an inhomogeneous, nonlinear force response model are learned from a robotic arm acquiring…
Robotic automation has the potential to assist human surgeons in performing suturing tasks in microsurgery, and in order to do so a robot must be able to guide a needle with sub-millimeter precision through soft tissue. This paper presents a robotic suturing system that uses 3D optical coherence tom…
In this paper, we propose a semi-empirical approach for simulating robot locomotion on granular media. We first develop a contact model based on the stick-slip behavior between rigid objects and granular grains, which is then learned through running extensive experiments. The contact model represent…
Optical coherence tomography (OCT) has found great success in ophthalmology where it plays a key role in screening and diagnostics. Clinical ophthalmic OCT systems are typically deployed as tabletop instruments that require chinrest stabilization and trained ophthalmic photographers to operate. Thes…
This paper proposes a machine learning method to predict the solutions of related nonlinear optimal control problems given some parametric input, such as the initial state. The map between problem parameters to optimal solutions is called the problem-optimum map, and is often discontinuous due to no…
This paper proposes a technique for 3D object scanning via in-hand manipulation, in which an object reoriented in front of a video camera with multiple grasps and regrasps. In-hand object tracking is a significant challenge under fast movement, rapid appearance changes, and occlusions. This paper pr…
This paper formulates two variants of packing problems in which the set of items is known but the arrival order is unknown. The goal is to certify that the items can be packed in a given container, and/or to optimize the size or cost of a container so that that the items are guaranteed to be packabl…
Recent progress in the field of robotic manipulation has generated interest in fully automatic object packing in warehouses. This paper proposes a formulation of the packing problem that is tailored to the automated warehousing domain. Besides minimizing waste space inside a container, the problem r…
This paper presents a general framework to find time-optimal trajectories for dynamic vehicles like drones and autonomous cars. Hindered by its nonlinear objective and complex constraints, this problem is hard even for state-of the-art nonlinear programming (NLP) solvers. The proposed framework addr…
This paper presents a Bayesian approach for active tactile exploration of a planar shape in the presence of both localization and shape uncertainty. The goal is to dock the robot's end-effector against the shape -- reaching a point of contact that resists a desired load -- with as few probing actio…
Nonlinear optimal control problems are challenging to solve efficiently due to non-convexity. This paper introduces a trajectory optimization approach that achieves realtime performance by combining machine learning to predict optimal trajectories with refinement by quadratic optimization. First, a…
Deep anterior lamellar keratoplasty (DALK) is a promising technique for corneal transplantation that avoids the chronic immunosuppression comorbidities and graft rejection risk associated with penetrating keratoplasty (PKP), the standard procedure. In DALK, surgeons must insert a needle 90% through…
In this paper, we present a real-time falling robot stabilization system for a humanoid robot in which the robot can prevent falling using hand contact with walls and other surfaces in the environment. Instead of ignoring or avoiding interaction with environmental obstacles, our system uses obstacle…
In order to conduct many desirable functions, service robots will need to actuate buttons and switches that are designed for humans. This paper presents the design of a robot named SwitchIt that is small, relatively inexpensive, easily mounted on a mobile robot, and actuates buttons reliably. Its op…
Walking and climbing robots need to plan longterm routes on both horizontal and vertical terrain, but onboard sensors take images from vantage points that provide strongly foreshortened images that cause the appearance of terrain features to vary greatly by distance and viewing angle. This paper pre…
Nonlinear optimal control problems are challenging to solve due to the prevalence of local minima that prevent convergence and/or optimality. This paper describes nearest-neighbors optimal control (NNOC), a data-driven framework for nonlinear optimal control using indirect methods. It determines ini…
The addition of manipulation capabilities to telepresence robots holds the promise of enabling remote humans to perform tele-labor, hands-on training, and collaborative manipulation, but the use of a robot as a mediator to humanhuman physical interaction is not yet well understood. This paper studie…
During outbreaks of contagious diseases, healthcare workers are at high risk for infection due to routine interaction with patients, handling of contaminated materials, and challenges associated with safely removing protective gear. This poses an opportunity for the use of remote-controlled robots t…
Differential dynamic programming (DDP) is a widely used trajectory optimization technique that addresses nonlinear optimal control problems, and can readily handle nonlinear cost functions. However, it does not handle either state or control constraints. This paper presents a novel formulation of DD…
A novel method is presented for efficiently testing the stability of an object under gravity and contact forces, that accommodates empirical determination of the set of admissible forces exerted at contacting pairs of surfaces. These admissible force volumes may exhibit a wide variety of geometries…
Convolutional neural networks (CNN) are a deep learning technique that has achieved state-of-the-art prediction performance in computer vision and robotics, but assume the input data can be formatted as an image or video (e.g. predicting a robot grasping location given RGB-D image input). This paper…
Intraoperative 3D imaging has great potential for enhancing surgical visualization. This is especially so in ophthalmic surgery where live volumetric imaging from optical coherence tomography systems recently incorporated into surgical microscopes has freed surgeons from the otherwise universal top-…
Despite increasing popularity of compliant and underactuated hands, few tools are available for modeling them. Thus, we propose a simulation technique to predict the success of a compliant gripper grasping irregular objects, which could be used in mechanism design as well as grasp planning. The simu
Asymptotically-optimal sampling-based motion planners, like RRT*, perform vast amounts of collision checking, and are hence rather slow to converge in complex problems where collision checking is relatively expensive. This paper presents two novel motion planners, Lazy-PRM* and Lazy-RRG*, that elimi…
Optimization is often difficult to apply to robots due to the presence of significant errors in the optimization model, which may cause constraints to be violated during execution on a real robot. This work presents a method to optimize trajectories with large modeling errors using a combination of…