84 accepted papers
Endoscopic submucosal dissection (ESD) is an effective technique to resect early cancers in the gastrointestinal (GI) tract. Bimanual telerobotic manipulation is an approach to performing ESD intuitively and efficiently, which requires two robotic instruments with flexibility, stiffness, dexterity a…
Stereo depth estimation has drawn widespread attention from the robotics and vision community due to its broad applications such as 3D reconstruction. Recently, stereo matching foundation models have made significant progress by being trained on the large-scale datasets containing natural images. Ho…
Quadruped robots have received increasing attention in recent years. Most existing trajectory planning algorithms for quadruped robots focus on how to avoid obstacles and achieve shortest trajectory or time, which is similar to the planning algorithms for mobile robots. These algorithms could not ta
Quadruped robots have received increasing attention in recent years. Most existing trajectory planning algorithms for quadruped robots focus on how to avoid obstacles and achieve shortest trajectory or time, which is similar to the planning algorithms for mobile robots. These algorithms could not ta…
In computer-assisted orthopedic surgery (CAOS), accurately registering sparse and partial intraoperative point sets with a complete preoperative model remains highly challenging due to limited overlap, extreme sparsity, and point localisation noise. In this paper, we propose a novel end-to-end compl…
3D point cloud registration is an essential problem in computer vision, robotics, surgical navigation and augmented reality. Accurate registration of partially overlapped intraoperative point clouds (e.g., femoral reconstruction) remains critical yet challenging in orthopedic navigation due to incom
Long-horizon composite task planning for multi-robot systems in cross-regional complex scenarios faces dual challenges: spatial-semantic comprehension of natural language described tasks and collaborative optimization of subtask al-location. To address these challenges, this paper proposes a progres
Human-robot interaction (HRI) encompasses a wide range of collaborative tasks, with handover being one of the most fundamental. As robots become more integrated into human environments, the potential for service robots to assist in handing objects to humans is increasingly promising. In robot-to-hum
Wheeled robots are highly efficient in human living environments. However, conventional wheeled designs, limited by degrees of freedom, struggle to meet varying footprint needs and achieve omnidirectional mobility. This paper proposes a novel robot drive model inspired by human movements, termed as
In computer-assisted orthopedic surgery (CAOS), accurate point set registration is essential for enhancing surgical accuracy. However, the sparse and low-overlap nature of intraoperative point sets presents significant challenges for reliable registration. To deal with these challenges, we propose a
In this paper, we present a novel curve-to-surface registration method, termed Bi-directional Hybrid Mixture Model Registration based on Dual-constrained Tangent and Normal Vectors (BiHMM-DTN), where two different tangent vectors at the intraoperative point are simultaneously used with the normal ve
Robust and accurate 2D/3D registration, which aligns preoperative models with intraoperative images of the same anatomy, is crucial for successful interventional navigation. To mitigate the challenge of a limited field of view in single-image intraoperative scenarios, multi-view 2D/3D registration i
In this paper, we propose a novel unsupervised intraoperative liver deformation correction method, called Learning Coherent point drift Network (LCNet), for image-guided liver surgery (IGLS). We first estimate the correspondences between the preoperative and intraoperative point sets in the optimal
Learning-based methods have improved locomotion skills of quadruped robots through deep reinforcement learning. However, the sim-to-real gap and low sample efficiency still limit the skill transfer. To address this issue, we propose an efficient model-based learning framework that combines a world m…
In this paper, we introduce Bidirectional Non-Overlapping Filtering Network (Bi-NOFNet), which registers the partial intraoperative point set with full preoperative point set for computer-assisted interventions (CAI). Our contributions are three-folds. First, Bi-NOFNet adopts customised feature extr…
In this paper, we introduce a novel normal-assisted learning-based rigid registration approach, i.e., Deep Bi-directional Hybrid Mixture Registration (DeepBHMR). Our approach utilises helpful normal vectors explicitly in both correspondence and transformation stages and formulates the optimization o
Precise and smooth control of an electromagnetic actuated micropositioner is a challenging task due to the system's nonlinear electromagnetic dynamics, as well as its unique structure and application scenarios. The main challenges include high-order nonlinearity, modeling uncertainties, input buffet
Continuum robots, characterized by their high flexibility and infinite degrees of freedom (DoFs), have gained prominence in applications such as minimally invasive surgery and hazardous environment exploration. However, the intrinsic complexity of continuum robots requires a significant amount of ti…
In this paper, we present a simultaneous exploration and object search framework for the application of autonomous trolley collection. For environment representation, a task-oriented environment partitioning algorithm is presented to extract diverse information for each sub-task. First, LiDAR data i…
In this paper, we introduce a novel overlap-based bidirectional point set registration approach, i.e., Overlap-guided Bidirectional Hybrid Mixture Registration (OBHMR), which incorporates geometric information (i.e., normal vectors) in both the correspondence and transformation stages and formulates…
Sampling-based kinodynamic motion planners (SKMPs) are powerful in finding collision-free trajectories for high-dimensional systems under differential constraints. Time-informed set (TIS) can provide the heuristic search domain to accelerate their convergence to the time-optimal solution. However, e
In recent years, great progress has been made in forecasting human motion in crowded scenes. However, current methods are far from practical applications due to the unbearable high computation costs, especially for encoding scene context. In addition, neglecting the partially detected trajectories m
Autonomous driving holds promise for increased safety, optimized traffic management, and a new level of convenience in transportation. While model-based reinforcement learning approaches such as MuZero enables long-term planning, the exponentially increase of the number of search nodes as the tree g
Learning-based grip force measurement methods in robot-assisted minimally invasive surgery (RAMIS) outperforms the traditional model-based methods and avoids the application issues of sensor-based approaches. However, few studies have investigated the problem of grip force measurement in mass-produc
In medical robotics and image-guided surgery (IGS), registration is needed in order to align together the coordinate frames of robots, medical imaging modalities, surgical tools, and patients. Existing registration algorithms often assume one point set to be a noise-free model while the other to con…
Planning a collision-free path efficiently among obstacles is crucial in robotics. Conventional one-shot unidirectional path planning algorithms work well in the static environment, but cannot respond to the environment changes timely in the dynamic environment. To tackle this issue and improve the…
Cooperative object transportation using multiple robots has been intensively studied in the control and robotics literature, but most approaches are either only applicable to omnidirectional robots or lack a complete navigation and decision-making framework that operates in real time. This paper pre…
Mobile robots have become more and more popular in large-scale and crowded environments, such as airports, shopping malls, etc. However, due to sparse landmarks and crowd noise, localization in this environment is a great challenge. Furthermore, it is unreliable for the robot to navigate safely in c
Optical Coherence Tomography (OCT) is a non-invasive imaging technique that is instrumental in retinal disease diagnosis and treatment. Segmentation of retinal layers in OCT is an essential step, but remains challenging for common pixel-wise segmentation methods usually fail to obtain the correct la…
This letter introduces an innovative multi-scale visual servoing framework for optical microscopy, engineered to automatically reposition the microscope for high-magnification target view across multiple magnifications, thereby facilitating repetitive and accurate histologic biopsies. The framework
This paper presents a novel augmented reality (AR)-assisted orthopedic surgical robotic system based on Head-Mounted Display (HMD) devices. The proposed system can overlay the preoperative plans over the patient's anatomy and provide useful guidance for surgeons during interventions, with integrated…
Surgical robots with variable stiffness can provide more stable configurations during minimally-invasive surgery.This letter presents a new design of modular lockable mechanism which can be used to change the stiffness of tendon-driven surgical robots. Locking and unlocking are simply actuated by pu
In this work, we propose A2DIO, a novel hybrid neural network model with a set of carefully designed attention mechanisms for pose invariant inertial odometry. The key idea is to extract both local and global features from the window of IMU measurements for velocity prediction. A2DIO leverages the c…
In recent years, simultaneous magnetic actuation and localization (SMAL) for active wireless capsule endoscopy (WCE) has been intensively studied to improve the efficiency and accuracy of gastrointestinal (GI) examination using WCE. In this letter, we draw inspiration from the conventional colonosco
Recently Koopman operator has become a promising data-driven tool to facilitate real-time control for unknown nonlinear systems. It maps nonlinear systems into equivalent linear systems in embedding space, ready for real-time linear control methods. However, designing an appropriate Koopman embeddin
This paper introduces a novel localization approach for active capsule endoscopy that, for the first time, combines external magnetic field sensing and internal inertial sensing to realize 6-DOF pose estimation of a magnetic capsule robot. It utilizes an inertial measurement unit embedded in the cap
This paper presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets(PSs) are aligned and that the…
Being able to explore unknown environments is a requirement for fully autonomous robots. Many learning-based methods have been proposed to learn an exploration strategy. In the frontier-based exploration, learning algorithms tend to learn the optimal or near-optimal frontier to explore. Most of thes
This paper proposes a new network for reconstructing multi-object point cloud. Different from previous networks which reconstruct multi-object point cloud as a whole, our network iteratively reconstructs each individual object point cloud from a frame of multi-object point cloud. To achieve this goa…
Optical Coherence Tomography (OCT) is a rapidly growing and promising imaging technique, enabling non-invasive high-resolution visualization of biological tissues. Segmentation of tissue structures from OCT scans is essen-tial for disease diagnosis but remains challenging for the blurry boundaries a…
Online state-time trajectory planning in highly dynamic environments remains an unsolved problem due to the curse of dimensionality of the state-time space. Existing state-time planners are typically implemented based on randomized sampling approaches or path searching on discrete graphs. The smooth…
Recently reinforcement learning (RL) has emerged as a promising approach for quadrupedal locomotion, which can save the manual effort in conventional approaches such as designing skill-specific controllers. However, due to the complex nonlinear dynamics in quadrupedal robots and reward sparsity, it
Autonomous mobile manipulation robots that can collect trolleys are widely used to liberate human resources and fight epidemics. Most prior robotic trolley collection solutions only detect trolleys with 2D poses or are merely based on spe-cific marks and lack the formal design of planning algorithms…
Most existing bipedal robots can only move with either their wheels or feet. Even if some of them are capable of transforming between these two motions, they need to change their configuration dramatically. In order to truly combine the advantages of wheeled and footed robots, in this letter, an <it
Magnetic soft robots have attracted widespread attention for their untethered, remotely operated, and compliant deformation characteristics. Earlier work has demonstrated magnetic origami robots' diverse locomotion capabilities. This letter will focus on the motion generation and open-loop control o
Human activities are hugely restricted by COVID-19, recently. Robots that can conduct inter-floor navigation attract much public attention since they can substitute human workers to conduct the service work. However, current robots either depend on human assistance or elevator retrofitting, and full…
Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US…
Accurate position feedback in a wide range is critical for medical microrobotics and robot-assisted examinations, such as colonoscopy, bronchoscopy and capsule endoscopy examination. Among the many modalities of positioning feedback, magnetic tracking is a preferable method due to the unique advanta…
Robot path planning is difficult to solve due to the contradiction between the optimality of results and the complexity of algorithms, even in 2D environments. To find an optimal path, the algorithm needs to search all the state space, which costs many computation resources. To address this issue, w…
This letter introduces a novel rigid point set registration (PSR) approach that accurately aligns the pre-operative space and the intra-operative space together in the scenario of computer-assisted orthopedic surgery (CAOS). Motivated by considering anisotropic positional localization noise and util
Point set registration (PSR) is an essential problem in communities of computer vision, medical robotics and biomedical engineering. This paper is motivated by considering the anisotropic characteristics of the error values in estimating both the positional and orientational vectors from the PSs to…
Grasp pose detection generates the position and orientation of the robot end-effector to grasp objects from the RGB or RGB-D image. In this paper, we propose a novel grasp pose detection network that generates 3-DOF grasp poses using the RGB image. The network follows the anchor-based object detecti…
Natural orifice transluminal endoscopic surgery (NOTES) has received significant attentions due to its minimal incision trauma compared with traditional multi-port robot assisted surgery. Continuum robot can be used in NOTES due to its high flexibility which can adapt to circuitous paths. However, t
Magnetically-actuated robotic catheter (MARC) has shown great potential in minimally-invasive surgery because they can be steered remotely and wirelessly. However, when driven by external permanent magnets (EPM), the MARC is subject to nonlinearly-coupled gravitational, magnetic, and elastic forces,
This paper presents a modular and reconfigurable mini-robotic system with 5 degrees of freedom (DoFs) towards minimally invasive surgery (MIS). The mini-robotic system consists of two modules, a 2-DoFs rotational end-effector, and a 3-DoFs positioning platform. The 2-DoFs rotational end-effector is…
As a common cause of anemia and gastrointestinal bleeding, angiodysplasia (AD) diagnosis in wireless capsule endoscopy (WCE) images is important in clinical. Current manual review requires undivided concentration of the gastroenterologists, which is laborious and time-consuming. The development of c…
Interactions with either environments or expert policies during training are needed for most of the current imitation learning (IL) algorithms. For IL problems with no interactions, a typical approach is Behavior Cloning (BC). However, BC-like methods tend to be affected by distribution shift. To mi…
Active wireless capsule endoscopy (WCE) under magnetic actuation is a promising technology to reduce the inspection time and relieve the burden of physicians. In this paper, we propose a reciprocally rotating magnetic actuation method for trajectory following of a capsule and develop its dynamic mod…
Brain needle intervention is a specific diagnosis and therapy procedure in brain disorders, such as brain tumors and Parkinson’s disease. Preoperative needle path planning is a vital step to guarantee the patient’s safety and reduce lesions. For positioning accuracy in the CT/MRI environment, we hav…
Point set registration (PSR) is an essential problem in surgical navigation and image-guided surgery (IGS). It can help align the pre-operative volumetric images with the intra-operative surgical space. The performances of PSR are susceptible to noise and outliers, which are the cases in real-world…
This paper presents a search-based partial motion planner for generating feasible trajectories of car-like robots in highly dynamic environments. The planner searches for smooth, safe, and near-time-optimal trajectories by exploring a state graph built on motion primitives. To enable fast online pla…
Wireless capsule endoscopy (WCE) is a novel imaging tool that allows the noninvasive visualization of the entire gastrointestinal (GI) tract without causing discomfort to the patients. Although convolutional neural networks (CNNs) have obtained promising performance for the automatic lesion recognit…
Learning-based grasp pose detection algorithms have boosted the performance of robot grasping, but they usually need manually fine-tuning steps to find the balance between detection accuracy and efficient. In this paper, we discard these intermediate procedures, like sampling grasps and generating g…
As one of the most promising areas, mobile robots draw much attention these years. Current work in this field is often evaluated in a few manually designed scenarios, due to the lack of a common experimental platform. Meanwhile, with the recent development of deep learning techniques, some researche…
Wireless Capsule Endoscopy (WCE) has the advantage of reducing the invasiveness and pain of gastrointestinal examinations. In this work, we propose a system aimed at autonomously accelerating and locating the WCE inside the intestine for clinical applications. A rotating magnet controlled by a robot…
In this paper, we propose a novel path planning algorithm for the nonholonomic multiple mobile robot system with applications to a robotic autonomous luggage trolley collection system at airports. We consider this path planning algorithm as a Multiple Traveling Salesman Problem (MTSP). Our path plan…
Inertial Measurement Unit (IMU) has long been a dream for stable and reliable motion estimation, especially in indoor environments where GPS strength limits. In this paper, we propose a novel method for position and orientation estimation of a moving object only from a sequence of IMU signals collec…
In this paper, we propose a novel algorithm that estimates the pose of the robot end effector using depth vision. The input to our system is the segmented robot hand point cloud from a depth sensor. Then a neural network takes a point cloud as input and outputs the position and orientation of the ro…
This paper presents a robust and accurate approach for the rigid registration of pre-operative and intraoperative point sets in image-guided surgery (IGS). Three challenges are identified in the pre-to-intraoperative registration: the intra-operative 3D data (usually forms a 3D curve in space) (1) i…
Modern agricultural robots like drones have been studied in automatic yield estimation in recent years. Fruit counting is a fundamental task in the automatic yield estimation, on which significant progress has been achieved by detection-based methods and segmentation-regression-based methods. Howeve…
This letter presents a novel and integrated framework for Next-Best-View (NBV) selection toward autonomous robotic exploration in indoor environments. A topological map, named semantic road map (SRM), is proposed to represent the explored environment during the exploration. The basic concept of the
Point set registration (PSR) is an important problem in computer vision, robotics and biomedical engineering communities. Usually, only positional information at each point is adopted in a registration. In this paper, the orientational vector (or normal vector) associated with each point is also uti…
Non-rigid point set registration (PSR) is an outstanding and fundamental problem in fields of robotics, computer vision, medical image analysis and imageguided surgery. The aim of a non-rigid registration problem is to align together two point sets that have been deformed. We have derived and presen…
Autonomous elevator operation is considered an intelligent solution in handling the inter-floor navigation problem of service robots. As one of the most fundamental steps, elevator button recognition starts to receive more and more attention. However, due to the challenging image conditions and seve…
Exploration region selection is an essential decision making process in autonomous robot exploration task. While a majority of greedy methods are proposed to deal with this problem, few efforts are made to investigate the importance of predicting long-term planning. In this paper, we present an algo…
In this paper, we study the problem of autonomous exploration in unknown indoor environments using mobile robot. We use mutual information (MI) to evaluate the information the robot would get at a certain location. In order to get the most informative sensing location, we first propose a sampling me…
This letter describes a pipeline for autonomous object search using a mobile robot. The robot is required to efficiently find an object in an unknown environment. In this letter, we formulate the object-search problem as a Partially Observable Markov Decision Process (POMDP). The semantic informatio
The-state-of-the-art of fruit detection with Faster R-CNN shows lack of detection advantage on small fruits. One of reasons is only single level features is used for localization of proposal candidates. In this paper, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CN…
Alignment of two point clouds is an essential problem in medical robotics and computer-assisted surgery. In this paper, we first formally formulate the generalized point cloud registration problem in a probabilistic manner. Specifically, not only positional but also the orientational information are…
Robots are increasingly operating in indoor environments designed for and shared with people. However, robots working safely and autonomously in uneven and unstructured environments still face great challenges. Many modern indoor environments are designed with wheelchair accessibility in mind. This…
This paper introduces a generic framework for field surveillance using consumer rotorcrafts and ground vehicles. Building such an autonomous system comes with two key challenges in persistent perception and obstacle avoidance. We begin with explaining two core algorithms to solve the challenges: an…
Flexible manipulator enables curvilinear accessibility through small incisions or natural orifices for minimally invasive surgery and diagnosis, which makes it a good choice for minimally invasive surgery. In order to control the robot precisely and safely, the real-time position and shape informati…
Target registration error (TRE) is widely adopted to evaluate the accuracy of a paired-point rigid registration (PPRR). However, TRE is defined in such a way that target localization error (TLE) is not considered. In this paper, we first propose a new type of error that is referred to as total targe…
Wireless Capsule Endoscopy (WCE) allows physicians to examine the entire digestive system without any surgical operation. Although it provides a noninvasive imaging approach to access the gastrointestinal (GI) tract, the biggest drawback of this technology is the large numbers of images need to be d…