20 accepted papers
Reinforcement learning (RL) is crucial for aligning Vision-Language Models (VLMs), but its practical application is hampered by significant system-level bottlenecks. The typical RL pipeline, encompassing data loading, inference-based rollouts, and model updates, suffers from severe inefficiencies wh…
Accurate segmentation of surgical instrument tip is an important task for enabling downstream applications in robotic surgery, such as surgical skill assessment, tool-tissue interaction and deformation modeling, as well as surgical autonomy. However, this task is very challenging due to the small si…
Three-dimensional reconstruction of dynamic objects is important for robotic applications, for example, the robotic recognition and manipulation. In this paper, we present a novel 3D surface reconstruction method for moving objects. The proposed method combines the spatial-multiplexing and time-mult
Conventional robot hand-eye calibration methods are impractical for localizing robotic instruments in minimally-invasive surgeries under intra-corporeal workspace after preoperative set-up. In this letter, we present a new approach to autonomously calibrate a robotic instrument relative to a monocul
Three-dimensional vision plays an important role in robotics. In this paper, we present a 3D surface reconstruction scheme based on combination of stereo matching and pattern projection. A two-step matching scheme is proposed to establish reliable correspondence between stereo images with high compu
Inherent compliance plays an enabling role in soft robots, which rely on it to mechanically conform to the environment. However, it also limits the payload of the robots. Various variable stiffness approaches have been adopted to limit compliance and provide structural stability, but most of them ca…
In robotic-assisted laparoscopic surgery, the first assistant (FA) stands at the bedside assisting the intervention, while the surgeon sits at the console teleoperating the robot. Tasks for the FA include navigating new instruments into the surgeon's field-of-view and passing in or retracting materi…
We present an autonomous manipulation approach for tissues with anisotropic deformation behavior using a continuum manipulator. The key feature of our vision-based study is an online learning and estimation method, which makes its implementation independent of any prior knowledge about the deformati
A major issue for needle insertion into soft tissue during suturing is the induced tissue deformation that hinders the minimization of tip-target positioning error. In this letter, we present a new robot control framework to solve target deviation by integrating active deformation control. We charac
In this paper, we propose a novel 3D laparoscopic imaging system based on stereo-photogrammetry which is assisted by projecting patterns on the tissue surface. The proposed laparoscopic imaging system has three optic channels, two of which are responsible for stereo vision feedback and the other one…
We present a generic data-driven method to address the problem of manipulating a three-dimensional (3-D) compliant object (CO) with heterogeneous physical properties in the presence of unknown disturbances. In this study, we do not assume a prior knowledge about the deformation behavior of the CO an
Emerging applications of robotic manipulation of deformable objects have opened up new challenges in robot control. While several control techniques have been developed to manipulate deformable objects, the performance of existing methods is commonly limited by two issues: 1) implicit assumption tha…
This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by th
In this letter, we propose a new method to fully control complete 4-image-DoF manipulation of laparoscopic instruments [with remote center of motion (RCM) mechanism] based on the geometric features of a designed marker in a 2-D image. Our marker encodes the configuration of the instruments by comput
In this paper, we address the problem of the image-based 3D pose computation of a semi-circle suturing needle using monocular image feedback for laparoscopy. We propose a constrained two-degree-of-freedom (2-DOF) geometry-based modelling method to parametrise the needle's 6-DOF pose, including depth…
In this paper, we address the 3D position control of RCM-constrained instruments with monocular cameras. To compute the instrument's position from a single 2D image, we develop an innovative gradient descent algorithm which rotates and translates a line segment (over the plane spanned by the imaged
In this paper, we present a new robotic uterine positioner for total laparoscopic hysterectomy. The robot is designed to actively position the patient's uterus during surgery, a lengthy and tedious task that is traditionally performed by a human assistant. Safety is simply the most important concern…
In this paper, we address the positioning problem of remote centre of motion (RCM) mechanisms with uncalibrated image feedback from a monocular camera. Nowadays, RCM mechanisms are widely used in minimally invasive robotic surgery due to their ability to distally rotate a tool around a fixed entry p…
In this paper, we propose a novel design of compliant safe joint, which has flexibility when the work load exceeds a predefined threshold. The compliance is generated by a spring. We design a special transmission mechanism to convert axial motion into circumferential motion such that the linear comp…
This paper presents the development of an endoscope manipulator with passive and active structures for functional endoscopic sinus surgery (FESS). The 5-DoF passive structure has three translations and two rotations (T3R2) that allows the surgeon to manually place the endoscope near to the entry poi…