47 accepted papers
During complex object manipulation, manipulator systems often face the configuration disconnectivity problem due to closed-chain constraints. Although regrasping can be adopted to guarantee connectivity, it introduces additional issues such as impact and efficiency. Therefore, regrasping numbers sho
In this work, we present FRTree planner, a novel robot navigation framework that leverages a tree structure of free regions, specifically designed for navigation in cluttered and unknown environments with narrow passages. The framework continuously incorporates real-time perceptive information to id
3D reconstruction is a key technology for recovering three-dimensional scenes from RGB-D data. Although mature solutions exist for 3D reconstruction based on RGB-D images, indoor scene reconstruction still faces challenges in balancing memory consumption, geometric representation accuracy, camera po
Tangram assembly, the art of human intelligence and manipulation dexterity, is a new challenge for robotics and reveals the limitations of state-of-the-arts. Here, we describe our initial exploration and highlight key problems in reasoning, planning, and manipulation for robotic tangram assembly. We
Despite the significant success of imitation learning in robotic manipulation, its application to bimanual tasks remains highly challenging. Existing approaches mainly learn a policy to predict a distant next-best end-effector pose (NBP) and then compute the corresponding joint rotation angles for m…
In this work, we propose a trajectory optimization approach for robot navigation in cluttered 3D environments. We represent the robot's geometry as a semialgebraic set defined by polynomial inequalities such that robots with general shapes can be suitably characterized. We exploit the collision-free
As automation technologies advance, the need for compact and multi-modal sensors in robotic applications is growing. To address this demand, we introduce CompdVision, a novel sensor that employs a compound-eye imaging system to combine near-field 3D visual and tactile sensing within a compact form f…
Detecting moving events produced by moving objects is a crucial task in the realms of autonomous driving and mobile robots. Moving objects have the potential to create ghost artifacts in mapped environments and pose risks to autonomous navigation. LiDAR serves as a vital sensor for autonomous system…
"The advancement of diffusion models has pushed the boundary of text-to-3D object generation. While it is straightforward to composite objects into a scene with reasonable geometry, it is nontrivial to texture such a scene perfectly due to style inconsistency and occlusions between objects. To tackl…
In-hand manipulation is challenging in robotics due to the intricate contact dynamics and high degrees of control freedom. Precise manipulation with high accuracy often requires tactile perception, which adds further complexity to the system. Despite the challenges in perception and control, the rol…
Large-scale 3D mapping is an important task for robotics and autonomous driving. However, mobile robots and autonomous vehicles with limited hardware resources may face issues with large memory consumption. It is challenging to achieve a balance between mapping quality and memory consumption. To add
3D point cloud maps are widely used in robotic tasks like localization and planning. However, dynamic objects, such as cars and pedestrians, can introduce ghost artifacts during the map generation process, leading to reduced map quality and hindering normal robot navigation. Online dynamic object re
This letter introduces ERRA, an embodied learning architecture that enables robots to jointly obtain three fundamental capabilities (reasoning, planning, and interaction) for solving long-horizon language-conditioned manipulation tasks. ERRA is based on tightly-coupled probabilistic inferences at tw
This paper tackles the task of singulating and grasping paper-like deformable objects. We refer to such tasks as paper-flipping. In contrast to manipulating deformable objects that lack compression strength (such as shirts and ropes), minor variations in the physical properties of the paper-like def…
It is essential to accurately identify gait phases when active exoskeleton devices assist with the lower limbs. This work focuses on IMU-based phase detection for stair ambulation. In order to enhance the detection sensitivity of phase transition, this work utilises the LSTM-CRF hybrid model. Four I
Humans excel in grasping objects through diverse and robust policies, many of which are so probabilistically rare that exploration-based learning methods hardly observe and learn. Inspired by the human learning process, we propose a method to extract and exploit latent intents from demonstrations, a
We study six-dimensional (6D) perceptive peg-in-hole problem for practical connector insertion task in this paper. To enable the manipulator system to handle different types of pegs in complex environment, we develop a perceptive robotic assembly system that utilizes an in-hand RGB-D camera for peg-…
With the growing interest in vision-based tactile sensors, various types of sensors that utilize digital imaging are being developed. Among them, a group of sensors captures the tactile flow resulted from the contact deformation using the optical flow algorithm from computer vision and achieves full
Tactile sensing is an essential perception for robots to complete dexterous tasks. As a promising tactile sensing technique, vision-based tactile sensors have been developed to improve robot performance in manipulation and grasping. Here we propose a new design of a vision-based tactile sensor, DelT
"Classical LIDAR semantic segmentation is not robust for real-world applications, e.g., autonomous driving, since it is closed-set and static. The closed-set network is only able to output labels of trained classes, even for objects never seen before, while a static network cannot update its knowled…
In this letter, we proposea novel vision-based grasp system for closed-loop 6-degrees of freedom grasping of unknown objects in cluttered environments. The key factor in our system is that we make the most of a geometry-aware scene representation based on a truncated signed distance function (TSDF)
Perceiving the dynamical environment both visually and tactilely is crucial for the survival of animals, and therefore, is considered of importance in robotics research. Recently, there has been an increasing interest in vision-based tactile sensors due to their high sensing resolution and robustnes…
Robotic grippers with visuotactile sensors have access to rich tactile information for grasping tasks but encounter difficulty in partially encompassing large objects with sufficient grip force. While hierarchical gecko-inspired adhesives are a potential technique for bridging performance gaps, they
Tactile sensing on human feet is crucial for motion control, however, has not been explored in robotic counterparts. This work is dedicated to endowing tactile sensing to legged robot’s feet and showing that a single-legged robot can be stabilized with only tactile sensing signals from its foot. We…
Classical close-set semantic segmentation networks have limited ability to detect out-of-distribution (OOD) objects, which is important for safety-critical applications such as autonomous driving. Incrementally learning these OOD objects with few annotations is an ideal way to enlarge the knowledge…
Recent studies on vision-based tactile sensing have shown promising results on the perception of contact information, which could improve the performance of dexterous manipulations. However, 3-dimensional contact deformation tracking at a higher resolution is desired and remains a challenge for visi
Self-supervised learning for depth estimation possesses several advantages over supervised learning. The benefits of no need for ground-truth depth, online fine-tuning, and better generalization with unlimited data attract researchers to seek self-supervised solutions. In this work, we propose a new…
Monitoring the state of contact is essential for robotic devices, especially grippers that implement geckoinspired adhesives where intimate contact is crucial for a firm attachment. However, due to the lack of deformable sensors, few have demonstrated tactile sensing for gecko grippers. We present V…
The supercoiled polymer (SCP) actuator is a novel artificial muscle, which is manufactured by twisting and coiling polymer fibers. This new artificial muscle is soft, low-cost and shows good linearity. Being utilized as an actuator, the artificial muscle could generate significant mechanical power i…
We design a multiscopic vision system that utilizes a low-cost monocular RGB camera to acquire accurate depth estimation. Unlike multi-view stereo with images captured at unconstrained camera poses, the proposed system controls the motion of a camera to capture a sequence of images in horizontally o
In this work, we address a planar non-prehensile sorting task. Here, a robot needs to push many densely packed objects belonging to different classes into a configuration where these classes are clearly separated from each other. To achieve this, we propose to employ Monte Carlo tree search equipped…
The widespread use of fluidic actuation for soft robots creates a high demand for soft pumps and compressors. However, current off-the-shelf pumps are usually rigid, noisy, and cumbersome. As a result, it is hard to integrate most commercial pumps into soft robotic systems, which restricts the auton…
Structures with origami design enable objects to transform into various three-dimensional shapes. Traditionally origami structures are designed with zero-thickness flat paper sheets. However, the thickness and intersection of origami facets are non-negligible in most cases, uniquely when integrating…
It is critical to predict the motion of surrounding vehicles for self-driving planning, especially in a socially compliant and flexible way. However, future prediction is challenging due to the interaction and uncertainty in driving behaviors. We propose planning-informed trajectory prediction (PiP)…
Self-supervised learning for visual object tracking possesses valuable advantages compared to supervised learning, such as the non-necessity of laborious human annotations and online training. In this work, we exploit an end-to-end Siamese network in a cycle-consistent self-supervised framework for…
The properties and applications of auxetics have been widely explored in the past years. Through proper utilization of auxetic structures, designs with unprecedented mechanical and structural behaviors can be produced. Taking advantage of this, we present the development of novel and low-cost 3D str…
This article describes an innovative design of variable stiffness soft actuator, which can potentially be utilized for manipulation and locomotion of soft robots. The new actuator is a combination of two types of actuations: soft pneumatic actuation and muscle-like supercoiled polymer (SCP) actuatio…
Retrieving rich contact information from robotic tactile sensing has been a challenging, yet significant task for the effective perception of object properties that the robot interacts with. This letter is dedicated to developing an algorithm to estimate contact force and torque for vision-based tac
In essence, successful grasp boils down to correct responses to multiple contact events between fingertips and objects. In most scenarios, tactile sensing is adequate to distinguish contact events. Due to the nature of high dimensionality of tactile information, classifying spatiotemporal tactile si
This work presents a novel variable stiffness structure consisting of a fiber-constrained dielectric elastomer tube filled with insulating oil. The tensile stiffness of the structure can be adjusted by voltages and its initial value can be customized according to the initial pre-stretch of the mater…
This work presents a novel variable stiffness structure consisting of a fiber-constrained dielectric elastomer tube filled with insulating oil. The tensile stiffness of the structure can be adjusted by voltages and its initial value can be customized according to the initial pre-stretch of the mater
We address the problem of fingertip design by leveraging on the fact that most grasp contacts share a few classes of local geometries. In order to maximize the contact areas for achieving more robust grasps, we first define the concept of Contact Primitive, which represents a set of contacts of simi
Soft-bodied robots, due to their intrinsic compliance, have shown great potential for operating within unstructured environment and interacting with unknown objects. This letter deals with automatic design and fabrication of soft robots. From a structure point of view, we synthesize a soft cable-dri
Soft robots are inherently compliant and adaptive, therefore they are promising candidates for interacting with humans. However robotic manipulators utilizing soft actuators are often constrained by a series of actuator performance limitations. In this work we design a novel linear soft robotic actu…
Soft robots, primarily made out of intrinsically soft materials, have flourished greatly in the past decade due to their advantages such as flexibility and adaptability over rigid-bodied robots. A rich repertoire of soft robots designed from intuitive or biomimetic approaches have been developed to…
Soft actuators play an important role in producing motions in soft robots, and dielectric elastomers have shown great promise because of their considerable voltage-induced deformation. In particular, air-filled dielectric elastomer actuators have been well studied, where the air inside provides pres…
A dielectric elastomer is able to act as an actuator when subjected to an electric field. However, the allowable deformation of dielectric elastomer actuators is severely limited by the material failures. To extend the actuation range, this letter investigates a network of two interconnected dielect