70 accepted papers
Active scene reconstruction aims to autonomously recover the fine-grained appearance and structural details of a complex unknown scenes. Existing approaches based on 2D topological or voxel-based abstractions often scale poorly to large environments and rely heavily on handcrafted features and heuri…
Diffusion models are effective for waypoint prediction in visual navigation, but standard sampling and test time guidance can produce unsafe or inefficient trajectories when updates drift off the training manifold. We propose Fisher Preserving Guidance with Outer Product Span Projection, a training-…
Reinforcement learning (RL) has renewed interest in end-to-end robot navigation, yet dynamic, crowd-like scenes remain difficult due to partial observability and the fragility of online tracking. We introduce GPTS-Nav, a Graph-Privileged Teacher-Student RL framework for LiDAR-only navigation. A grap
Bipedal robots are inherently prone to falling due to their higher center of mass and narrower support polygon, making automatic fall recovery a long-standing challenge. Existing approaches often rely on posture-specific strategies or exhibit limited robustness and generalization, restricting their …
Accurate spatiotemporal calibration is crucial for effective data fusion in Visual-Inertial-LiDAR (VIL) systems. However, it remains challenging for setups involving omnidirectional cameras due to their ultra-wide Field-of-View (FoV) and severe distortion. Existing methods often rely on calibration
High-quality motion planning for mobile manipulators remains a challenging task due to the high dimensionality and complex constraints involved. While existing methods perform well in specific scenarios, their efficiency and the quality of the resulting paths and trajectories often degrade in dense
Visual navigation requires the robot to reach a specified goal such as an image, based on a sequence of first-person visual observations. While recent learning-based approaches have made significant progress, they often focus on improving policy heads or decision strategies while relying on simplist
Object navigation for mobile robots typically assumes that targets are visible and paths are unobstructed. However, real-world scenarios often involve occluded targets like objects hidden behind doors or inside containers. Such scenarios require interactive navigation and manipulation by mobile mani…
The biomimetic research of vertebrates is challenging in both mechanism design and control methods. Motivated by natural acrobatics exhibited by cats and humans, this paper presents a generic multi-joint continuous spinal system and a learning-based algorithm for agile and accurate control. The spin…
Achieving stable and natural locomotion in bipedal robots, comparable to that of humans and animals, remains a long-standing challenge in robotics. In this work, we propose a bio-inspired low-level control framework that streamlines the generation of naturalistic gait patterns while ensuring adaptab
This paper addresses the scarcity of affordable, fully-actuated five-fingered hands for dexterous teleoperation, which is crucial for collecting large-scale real-robot data within the "Learning from Demonstrations" paradigm. We introduce the prototype version of the RAPID Hand, the first low-cost, 2
Place recognition and relative localization are crucial for realizing the potential of collaboration in ground and aerial robot teams. Many existing works focus only on ground robots and are not well-suited for heterogeneous robot systems in large-scale environments. In this paper, we propose a nove
For quadrupeds, a flexible spine allows them to traverse space and make quick turns. From the perspective of mechanical design in quadruped robots, an active spine with 2 degrees of freedom (2-DOF) can achieve dynamic posture adjustment similar to biological organisms which allows for pitch and yaw
Autonomous exploration is essential for the effective deployment of quadrotors in various applications. However, existing approaches face significant challenges in large-scale environments, particularly in balancing global coverage efficiency and computational overhead. These limitations often resul
This study proposes GLO, a general LiDAR-only odometry method with high efficiency and low drift. First, we propose a map data structure using multilevel voxels to improve map update efficiency. Each voxel node actively maintains plane features, minimizing redundant fitting and enhancing matching ef
Aerial manipulation provides an effective alternative to human labor in high-risk outdoor situations. Complex and variable environments demand the system to respond quickly with minimal latency to external disturbances. To address this challenge, we propose a learning-based variable whole-body model
Autonomous exploration in large-scale, unknown environments remains a significant challenge in mobile robotics. In this paper, we propose a scalable exploration framework that integrates heterogeneous topological representations, lightweight global-local graph reasoning, and reinforcement learning.
Visual navigation, a fundamental challenge in mobile robotics, demands versatile policies to handle diverse environments. Classical methods leverage geometric solutions to minimize specific costs, offering adaptability to new scenarios but are prone to system errors due to their multi-modular design
Recent advancements in diffusion-based imitation learning, which shows impressive performance in modeling multimodal distributions and training stability, have led to substantial progress in various robot learning tasks. In visual navigation, previous diffusion-based policies typically generate acti…
This paper addresses the scarcity of low-cost but high-dexterity platforms for collecting real-world multi-fingered robot manipulation data towards generalist robot autonomy. To achieve it, we propose the RAPID Hand, a co-optimized hardware and software platform where the compact 20-DoF hand, robus…
Mobile manipulation is a crucial problem in various real-world applications. However, existing methods have demonstrated unsatisfactory training efficiency and sparse rewards, requiring complex coordination strategies between the mobile base and arm. In this paper, we propose RM-Planner, a planning
Continuum robots, inspired by biological structures such as spines and tails, have attracted significant attention due to their flexibility and ability to perform complex tasks in confined and dynamic environments. However, traditional flexible continuum robots often encounter challenges such as non
360-degree cameras streamline data collection for radiance field 3D reconstruction by capturing comprehensive scene data. However, traditional radiance field methods do not address the specific challenges inherent to 360-degree images. We present SC-OmniGS, a novel self-calibrating omnidirectional G…
Autonomous exploration in unknown environments is a crucial challenge for various applications of unmanned aerial vehicles (UAVs). However, in large-scale scenarios, existing methods suffer from inefficient environmental information acquisition, computationally expensive exploration planning, and in
Legged aerial-terrestrial robots have garnered significant research attention in recent years due to their enhanced environmental adaptability through combined aerial and terrestrial locomotion. However, existing passive spring-legged aerial robots exhibit limited motion versatility, demonstrating s
Differential-driven wheeled robots (DWR) represent the quintessential type of mobile robots and find extensive applications across the robotic field. Most high-performance control approaches for DWR explicitly utilize the linear and angular velocities of the trajectory as control references. However
Portable 360^\circ cameras are becoming a cheap and efficient tool to establish large visual databases. By capturing omnidirectional views of a scene these cameras could expedite building environment models that are essential for visual localization. However such an advantage is often overlooked due…
Quadrotors, while versatile, are vulnerable to unpredictable environmental disturbances, including turbulence, gusts, and ground effects, making precise path-following control a formidable challenge. This letter introduces an adaptive neural network-based predictive control framework tailored for qu
Federated Learning with Noisy Labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning by aggregating multiple client models trained with local noisy or clean samples. On the basis of a federated learning framework, recent advances primarily adopt label noise fil…
This paper studies the problem of estimating physical properties (system identification) through visual observations. To facilitate geometry-aware guidance in physical property estimation, we introduce a novel hybrid framework that leverages 3D Gaussian representation to not only capture explicit sh…
Object rearrangement, a fundamental challenge in robotics, demands versatile strategies to handle diverse objects, configurations, and functional needs. To achieve this, the AI robot needs to learn functional rearrangement priors to specify precise goals that meet the functional requirements. Previo
Goal-oriented grasping in dense clutter, a fundamental challenge in robotics, demands an adaptive policy to handle occluded target objects and diverse configurations. Previous methods typically learn policies based on partially observable segments of the occluded target to generate motions. However,…
The integration of neural rendering and the SLAM system recently showed promising results in joint localization and photorealistic view reconstruction. However existing methods fully relying on implicit representations are so resource-hungry that they cannot run on portable devices which deviates fr…
Real-time dense mapping with high-fidelity textures in large-scale environments is such a challenge in robots, digital twins, and AR/VR applications. Neural Radiance Field (NeRF) has demonstrated remarkable capabilities in capturing intricate details and saving memory space, which provides significa
Quadrotors may crash and cause severe accidents under instantaneously drastic disturbances. To mitigate the effect of such disturbances, these critical issues should be considered: efficient disturbance observation and compensation, full attitude controllability, and instant output power generation…
Aerial manipulation has received increasing research interest with wide applications of drones. To perform specific tasks, robotic arms with various mechanical structures will be mounted on the drone. It results in sudden disturbances to the aerial manipulator when switching the robotic arm or inter…
This paper tackles the challenge of autonomous target search using unmanned aerial vehicles (UAVs) in complex unknown environments. To fill the gap in systematic approaches for this task, we introduce Star-Searcher, an aerial system featuring specialized sensor suites, mapping, and planning modules
Autonomous exploration plays a crucial role in robotics applications like rescue and scene reconstruction. This work addresses the challenges of autonomous exploration in intricate unknown environments by presenting a novel UAV autonomous exploration method based on a new concept of the view-region.…
This paper addresses the scarcity of large-scale datasets for accurate object-in-hand pose estimation, which is crucial for robotic in-hand manipulation within the "Perception-Planning-Control" paradigm. Specifically, we introduce VinT-6D, the first extensive multi-modal dataset integrating vision,…
Paths generated by A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> and other graph-search-based planners are widely used in the robotic field. Due to the restricted node-expansion directions, the resulting paths are usually not the shortest. B
As an active spine introduces more degree of freedoms (DOFs) as well as time-varying inertia, locomotion control of spined quadruped robots is challenging. Direct optimization on the full dynamics model causes prohibitive calculation time and is difficult to apply to embedded platforms. Model predic…
Autonomous exploration in large-scale and complex environments is a challenging task. As the size of the environment increases, the significant overhead of exploration algorithms could overwhelm the computational capability of mobile platforms, prohibiting timely response to environmental changes. M
Compared with using only a single type of robot, the use of drones and ground vehicles to jointly explore unknown areas can bring efficiency improvements. However, due to the difficulty of ground-aerial loop detection and especially the lack of ground-air datasets in large outdoor scenes, there is n
Constructing a high-quality dense map in real-time is essential for robotics, AR/VR, and digital twins applications. As Neural Radiance Field (NeRF) greatly improves the mapping performance, in this letter, we propose a NeRF-based mapping method that enables higher-quality reconstruction and real-ti
In this letter, we propose a heterogeneous deep metric learning pipeline for ground and aerial point cloud-based place recognition in large-scale environments. The pipeline extracts local features from ground and aerial raw point clouds by a sparse convolution module. The local features are processe
In this letter, we study the problem of using mobile robots as relay nodes to keep moving clients and a fixed base station connected. Some applications, such as exploration and rescue, require this problem to be solved. However, existing methods are computationally time-consuming and insufficient to
Practical global path planning is critical for commercializing cleaning robots working in semi-structured environments. In the literature, global path planning methods for free space usually focus on path length and neglect the traffic rule constraints of the environments, which leads to high-freque…
We present VG-Swarm, a practical and effective method for aerial robots dynamic encirclement, which consists of a vision-based gene regulatory network (V-GRN) and a visual perception module. For each flying robot deployed with the proposed method, the relative spatial positions of the surrounding ro
The problem of multi-robot navigation of connectivity maintenance is challenging in multi-robot applications. This work investigates how to navigate a multi-robot team in unknown environments while maintaining connectivity. We propose a reinforcement learning (RL) approach to develop a decentralized…
We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source pretrained model is adapted to the target domain without access to the source data. Existing techniques mainly leverage self-supervised pseudo-labeling to achieve class-wise global align…
Learning-based methods are powerful in handling complex scenarios. However, it is still challenging to use learning-based methods under uncertain environments while stability, safety, and real-time performance of the system are desired to guarantee. In this letter, we propose a learning-based tracki
Grasping a wide range of novel objects in densely cluttered scenes is difficult due to irregular shapes of objects and the uncertainty in sensing. In this paper, a novel vacuum cup grasping method, based on uncertainty modeling of perception data and grasp geometric heuristics, is proposed to grasp…
In this paper, we propose a novel grasp pipeline based on contact point detection on the truncated signed distance function (TSDF) volume to achieve closed-loop 7-degree-of-freedom (7-DoF) grasping on cluttered environments. The key aspects of our method are that 1) the proposed pipeline exploits th…
Control of an aerial manipulator is challenging due to the decentralized dynamics of the aerial vehicle and the robotic arm. It is generally complex to adjust the controller of the aerial manipulator when replacing a different robotic arm. This paper presents a flexible control scheme for a quadroto…
A heterogeneous multi-robot system consisting of Unmanned Ground Vehicles (UGVs) and Unmanned Aerial Vehicles (UAVs) have advantages over a single-robot system in efficiency and flexibility, enabling them to perform a larger range of tasks. To allow heterogeneous platforms to work together in GPS-de
Accurate path following is challenging for autonomous robots operating in uncertain environments. Adaptive and predictive control strategies are crucial for a nonlinear robotic system to achieve high-performance path following control. In this letter, we propose a novel learning-based predictive con
Instance grasping is a challenging robotic grasping task when a robot aims to grasp a specified target object in cluttered scenes. In this paper, we propose a novel end-to-end instance grasping method using only monocular workspace and query images, where the workspace image includes several objects…
What is a proper representation for objects in manipulation? What would human try to perceive when manipulating a new object in a new environment? In fact, instead of focusing on the texture and illumination, human can infer the "affordance" [36] of the objects from vision. Here "affordance" describ…
This paper considers the multi-robot navigation problem where the geometric center of a multi-robot team aims to efficiently reach the waypoint without collisions in unknown complex environments while maintaining connectivity during the navigation. A novel Deep Reinforcement Learning (DRL)-based app
This letter addresses the full coverage problem of unknown convex and concave two-dimensional (2-D) areas by multiple robots with limited visibility sensing and communication range. The areas are initially unknown to the multiple robots, and the number of robots is not predefined. In order to accomp
Data-driven approach for grasping shows significant advance recently. But these approaches usually require much training data. To increase the efficiency of grasping data collection, this paper presents a novel grasp training system including the whole pipeline from data collection to model inferenc…
Currently, most deep learning based disparity estimation methods have the problem of over-smoothing at boundaries, which is unfavorable for some applications such as point cloud segmentation, mapping, etc. To address this problem, we first analyze the potential causes and observe that the estimated…
Accurate object 6D pose estimation is a core task for robot bin-picking applications, especially when objects are randomly stacked with heavy occlusion. To address this problem, this paper proposes a simple but novel Point-wise Pose Regression Network (PPR-Net). For each point in the point cloud, th…
For obstacles moving with high speeds, existing motion planning methods can rarely guarantee collision avoidance. This paper proposes a viable two-period velocity obstacle algorithm where one period predicts potential collisions within a limited time horizon, and the second period foresees collision…
Dense mapping is always the desire of simultaneous localization and mapping (SLAM), especially for the applications that require fast and dense scene information. Visual-inertial odometry (VIO) is a light-weight and effective solution to fast self-localization. However, VIO-based SLAM systems have d…
A cognitive robot usually needs to perform multiple tasks in practice and needs to locate the desired area for each task. Since deep learning has achieved substantial progress in image recognition, to solve this area detection problem, it is straightforward to label a functional area (affordance) im…
Random sample consensus (RANSAC) is a popular paradigm for parameter estimation with outlier detection, which plays an essential role in 3D robot vision, especially for LiDAR odometry. The success of RANSAC strongly depends on the probability of selecting a subset of pure inliers, which sets barrier…
In this paper, an autonomous vision-based tracking system is presented to track a maneuvering target for a rotorcraft unmanned aerial vehicle (UAV) with an onboard gimbal camera. To handle target occlusions or loss for real-time tracking, a robust and computationally efficient visual tracking scheme…
This paper presents a decentralized strategy for collision-free navigation of multiple agents. This strategy combines the Optimal Reciprocal Collision Avoidance (ORCA) algorithm and Model Predictive Control (MPC). Concretely, each agent applies the decentralized ORCA algorithm to compute the collisi…
3D human articulated pose recovery from monocular image sequences is very challenging due to the diverse appearances, viewpoints, occlusions, and also the human 3D pose is inherently ambiguous from the monocular imagery. It is thus critical to exploit rich spatial and temporal long-range dependencie…