67 accepted papers
Endoscopic submucosal dissection (ESD) is an effective technique to resect early cancers in the gastrointestinal (GI) tract. Bimanual telerobotic manipulation is an approach to performing ESD intuitively and efficiently, which requires two robotic instruments with flexibility, stiffness, dexterity a…
Accurate point tracking in surgical environments remains challenging due to complex visual conditions, including smoke occlusion, specular reflections, and tissue deformation. While existing surgical tracking datasets provide coordinate information, they lack the semantic context necessary to unders
Accurate depth estimation plays a critical role in the navigation of endoscopic surgical robots, forming the foundation for 3D reconstruction and safe instrument guidance. Fine-tuning pretrained models heavily relies on endoscopic surgical datasets with precise depth annotations. While existing self…
Language-guided grasping has emerged as a promising paradigm for enabling robots to identify and manipulate target objects through natural language instructions, yet it remains highly challenging in cluttered or occluded scenes. Existing methods often rely on multi-stage pipelines that separate obje…
Long-form clinical videos are central to visual evidence-based decision-making, with growing importance for applications such as surgical robotics and related settings. However, current multimodal large language models typically process videos with passive sampling or weakly grounded inspection, whi…
In-vivo environments, magnetically actuated soft robots offer advantages such as wireless operation and precise control, showing promising potential for painless detection and therapeutic procedures. We developed a trileg magnetically driven soft robot (TMR) whose multi-legged design enables more fl…
This work presents a generalizable framework to transfer relative depth to metric depth. Current monocular depth estimation methods are mainly divided into metric depth estimation (MMDE) and relative depth estimation (MRDE). MMDEs estimate depth in metric scale but are often limited to a specific do
Enabling intuitive, language-driven interaction with surgical scenes is a critical step toward intelligent operating rooms and autonomous surgical robotic assistance. However, the task of referring segmentation, localizing surgical instruments based on natural language descriptions, remains underexp
In situ tissue biopsy with an endoluminal catheter is an efficient approach for disease diagnosis, featuring low invasiveness and few complications. However, the endoluminal catheter struggles to adjust the biopsy direction by distal endoscope bending or proximal twisting for tissue sampling within
Simultaneous Localization and Mapping (SLAM) is essential for precise surgical interventions and robotic tasks in minimally invasive procedures. While recent advancements in 3D Gaussian Splatting (3DGS) have improved SLAM with high-quality novel view synthesis and fast rendering, these systems strug
Conventional in-ear electrodes face challenges such as inconsistent skin contact, motion artifacts, and discomfort, limiting their reliability in dynamic conditions. To overcome these limitations, this study presents a body-temperature responsive in-ear balloon actuator (BBA) based on liquid-to-vapo
Vision-Language-Action (VLA) models have emerged as a prominent research area, showcasing significant potential across a variety of applications. However, their performance in endoscopy robotics, particularly endoscopy capsule robots that perform actions within the digestive system, remains unexplor
Robot-assisted Endoscopic Submucosal Dissection (ESD) improves the surgical procedure by providing a more comprehensive view through advanced robotic instruments and bimanual operation, thereby enhancing dissection efficiency and accuracy. Accurate prediction of dissection trajectories is crucial fo
In endoscopic procedures, autonomous tracking of abnormal regions and following of circumferential cutting markers can significantly reduce the cognitive burden on endoscopists. However, conventional model-based pipelines are fragile—each component (e.g., detection, motion planning) requires manual…
Magnetic soft continuum robots are capable of bending with remote control in confined space environments, and they have been applied in various bioengineering contexts. As one type of ferromagnetic soft continuums, the Magnetically Induced Metamorphic Materials (MIMMs)-based continuum (MC) exhibits
Abstract— Robotic needle positioning tasks in neurosurgery often face challenges due to insufficient perception of planar guidance images during surgery. In this work, we propose an Augmented Reality (AR) interface to help perform the robotic needle positioning tasks by learning from demonstration (
This paper introduces a tangent line decomposition (TLD) algorithm that efficiently finds collision-free paths close to optimal in both 2D and 3D environments. Compared with the existing visibility line-based algorithms, the proposed algorithm innovatively proposed the concept of tangent line decomp
Deformable continuum robots (DCRs) present unique planning challenges due to nonlinear deformation mechanics and partial state observability, violating the Markov assumptions of conventional reinforcement learning (RL) methods. While Jacobian-Based approaches offer theoretical foundations for rigid
Nasotracheal intubation (NTI) is critical for establishing artificial airways in clinical anesthesia and critical care. Current manual methods face significant challenges, including cross-infection, especially during respiratory infection care, and insufficient control of endoluminal contact forces,
Open tracheostomy (OT) is considered the traditional way and golden standard for treating airway obstruction patients. However, OT has many unavoidable drawbacks, including strict performing scenarios, significant scarring, and the risk of surgeon infection. Percutaneous dilation tracheostomy (PDT)
Surgical phase recognition is critical for assisting surgeons in understanding surgical videos. Existing studies focused more on online surgical phase recognition, by leveraging preceding frames to predict the current frame. Despite great progress, they formulated the task as a series of frame-wise
In endoluminal surgeries, inserting a flexible endo-scope is one of the fundamental procedures. During this process, vision remains the primary feedback, while the perception of tactile magnitude and location is insufficient. This limitation can hinder the clinician's efficiency when navigating the
Soft robots driven by the external magnetic field can be widely applied in medical areas to realize noninvasive diagnosis and treatment. These robots gain advantages in flexibility, untethered remote control, and safe human interaction. Thus, they can achieve functions such as endo-imaging, drug del
Surgical instrument segmentation is crucial in surgical scene understanding, thereby facilitating surgical safety. Existing algorithms directly detected all instruments of predefined categories in the input image, lacking the capability to segment specific instruments according to the surgeon’s inte
Capsule endoscopes, predominantly serving diagnostic functions, provide lucid internal imagery but are devoid of surgical or therapeutic capabilities. Consequently, despite lesion detection, physicians frequently resort to traditional endoscopic or open surgical procedures for treatment, resulting i…
Needle interventions are crucial in neurosurgery, requiring high precision and stability. This paper presents a 5-DoF head-mounted hydraulic needle robot designed for accurate and targeted needle insertion and neuroimaging in the deep brain. The robot is compact and lightweight by utilizing a hydrau…
In the study of minimally invasive surgical robots, a mini parallel continuum robot has shown motion advantage after passing through a long and winding working channel. However, due to the interaction force between the elastic wires of the parallel robots during motion generation processes, the cons…
Helicobacter pylori, a pervasive bacterial infection associated with gastrointestinal disorders such as gastritis, peptic ulcer disease, and gastric cancer, impacts approximately 50% of the global population. The efficacy of standard clinical eradication therapies is diminishing due to the rise of a…
In the realm of automated robotic surgery and computer-assisted interventions, understanding robotic surgical activities stands paramount. Existing algorithms dedicated to surgical activity recognition predominantly cater to pre-defined closed-set paradigms, ignoring the challenges of real-world ope…
Sampling-based kinodynamic motion planners (SKMPs) are powerful in finding collision-free trajectories for high-dimensional systems under differential constraints. Time-informed set (TIS) can provide the heuristic search domain to accelerate their convergence to the time-optimal solution. However, e
Miniature rotation actuators have been extensively developed and utilized in optical coherence tomography (OCT) endoscopy, enabling distortion-free OCT imaging in complex and tortuous environments. However, the use of electrical-driven rotation actuators raises safety concerns. Although magnetic-dri…
Despite their impressive performance in various surgical scene understanding tasks, deep learning-based methods are frequently hindered from deploying to real-world surgical applications for various causes. Particularly, data collection, annotation, and domain shift in-between sites and patients are…
Despite the availability of computer-aided simulators and recorded videos of surgical procedures, junior residents still heavily rely on experts to answer their queries. However, expert surgeons are often overloaded with clinical and academic workloads and limit their time in answering. For this pur…
Endotrachealintubation is a common medical practice for patients requiring upper airway management. An important parameter while performing intubation is the depth of insertion of the tube. However, most current solutions rely on vision-based feedback for successfully performing remote intubation. W
Robot-assisted gastrointestinal endoscopic surgery (GES) as a kind of natural orifice transluminal endoscopic surgery (NOTES) is the next-generation minimally invasive surgery (MIS). Besides, rendering certain autonomy to a Gas-trointestinal Endoscopic Surgical Robot (GESR) is promising but highly c…
Global and local relational reasoning enable scene understanding models to perform human-like scene analysis and understanding. Scene understanding enables better semantic segmentation and object-to-object interaction detection. In the medical domain, a robust surgical scene understanding model allo
Magnetically-actuated flexible endoscopes (MAFE) have been well used in minimally-invasive surgery because they can be steered by a magnetic field thus more flexible than traditional endoscopes. Model-free and uncalibrated visual-feedback control makes it possible to manipulate MAFE with a magnetic…
Several approaches have been introduced to understand surgical scenes through downstream tasks like captioning and surgical scene graph generation. However, most of them heavily rely on an independent object detector and region-based feature extractor. Encompassing computationally expensive detectio
Surgical interaction recognition (SIR) plays a crucial role in navigation decision support for minimally invasive surgery (MIS) or robot-assisted MIS. Currently, the research in SIR is at a coarse-grained level and barely considers the surgical interaction dependencies unrelated to endoscopic images
Magnetic soft robots have attracted widespread attention for their untethered, remotely operated, and compliant deformation characteristics. Earlier work has demonstrated magnetic origami robots' diverse locomotion capabilities. This letter will focus on the motion generation and open-loop control o
This letter presents a miniature manipulator with variable stiffness towards minimally invasive transluminal endoscopic surgery, such as the endoscopic submucosal dissection (ESD). The manipulator comprises hollow modules with holes in the sidewall, compact with a 4 mm diameter, and dexterous with s
The feasibility of using chip-based RFID designs as wireless sensor tags open a wide range of application possibilities in the field of robotics. However, multi-step lithography manufacturing and/or MEMS techniques are often required for the industrial-grade fabrication of such sensors. In this pape…
Generating a surgical report in robot-assisted surgery, in the form of natural language expression of surgical scene understanding, can play a significant role in document entry tasks, surgical training, and post-operative analysis. Despite the state-of-the-art accuracy of the deep learning algorith…
This paper presents a modular and reconfigurable mini-robotic system with 5 degrees of freedom (DoFs) towards minimally invasive surgery (MIS). The mini-robotic system consists of two modules, a 2-DoFs rotational end-effector, and a 3-DoFs positioning platform. The 2-DoFs rotational end-effector is…
Multiphysics simulation of magnetically actuated origami robots promises a range of applications such as synthetic data generation, design parameter optimization, predicting the robot's performance, and implementing various control algorithms, but it has rarely been explored. This letter presents a
Origami, the art of folding paper, can impart useful design inspirations to the creation of mechanical structures and mechanisms. Bistability is a useful property for origami designs, which can help compartmentalize different actuations and stiffness tuning regimes. Given the benefits of bistability…
Brain needle intervention is a specific diagnosis and therapy procedure in brain disorders, such as brain tumors and Parkinson’s disease. Preoperative needle path planning is a vital step to guarantee the patient’s safety and reduce lesions. For positioning accuracy in the CT/MRI environment, we hav…
Surgical scene understanding and multi-tasking learning are crucial for image-guided robotic surgery. Training a real-time robotic system for the detection and segmentation of high-resolution images provides a challenging problem with the limited computational resource. The perception drawn can be a…
Robot-assisted surgery is an emerging technology that has undergone rapid growth with the development of robotics and imaging systems. Innovations in vision, haptics, and accurate movements of robot arms have enabled surgeons to perform precise minimally invasive surgeries. Real-time semantic segmen
This letter presents a robotic hand with modular antagonistic fingers and adhesive soft modulations for general purposes with grasping and manipulation capabilities. The anthropomorphic design includes the hand-like configuration, modular antagonistic fingers, soft modulations, compliant joints, and
Robots, either made from hard or soft materials, are being used increasingly for unknown object manipulation tasks in unstructured environments. To hold an object firmly and do the required task, the orientation of the object with respect to the robotic hand gripper is one of the necessary key infor…
Soft-bodied robots, due to their intrinsic compliance, have shown great potential for operating within unstructured environment and interacting with unknown objects. This letter deals with automatic design and fabrication of soft robots. From a structure point of view, we synthesize a soft cable-dri
The recent developments of soft robots have inspired a variety of applications that require compliance for safety and softness for biomimetics. With such prevalence and advantages of soft robots, researchers are diverting their interests toward applications like micromanipulation of biological tissu
Target registration error (TRE) is widely adopted to evaluate the accuracy of a paired-point rigid registration (PPRR). However, TRE is defined in such a way that target localization error (TLE) is not considered. In this paper, we first propose a new type of error that is referred to as total targe…
Flexible robotic manipulators have been widely used in minimally invasive surgery (MIS) and many other applications requiring closer inspection and operation. Although a variety of manipulators enabled by different mechanism have been developed, few of them can preserve softness, thinness, and decen
Surgical robotic systems with miniaturized flexible Tendon-driven Serpentine Manipulators (TSM) have enjoyed increasing popularities among surgeons and researchers for their advantages of working in constrained and torturous human lumen such as oral cavity and upper GI tract. However, they suffer fr
Minimally invasive surgery-based nasopharyngeal cancer treatment is promising, but currently, it is not a common treatment choice because of the absence of suitable tools. In this letter, a multi-channel concentric tube robot is proposed for the treatment of nasopharyngeal cancer based on natural or
Multiple-segment flexible and soft robotic arms composed by ionic polymer--metal composite (IPMC) flexible actuators exhibit compliance but suffer from the difficulty of path planning due to their redundant degrees of freedom, although they are promising in complex tasks such as crossing body caviti
Comprised of multiple telescoptic precurved tubes that can independently rotate and translate, concentric tube robots (CTRs) are favorable in minimally invasive surgeries thanks to their small size and considerable dexterity along with curvilinear accessibility. However, there is a lack of investiga…
In this paper, a novel constrained tendon-driven serpentine manipulator (CTSM) suited for minimally invasive surgery is presented. It comprises of a flexible backbone, a set of controlling tendons and a constraint. In the CTSM not only the curvature of the bending section can be controlled but also…
Central catadioptric omnidirectional images exhibit serious nonlinear distortions due to quadratic mirrors involved. Conventional visual features developed based on the perspective model are hard to achieve a satisfactory performance when directly applied to the distorted omnidirectional image. This…
Flexible serpentine manipulators are widely used in surgical robots as it can be operated inside the patient's body cavity by backbone bending. However, during the bending the manipulator sweeps over a region, where sensitive organs may locate. This raises the safety concern. In this paper, a motion…
Continuum tubular robots, which are constructed by telescoping pre-curved elastic tubes, are capable of balancing the force application and steerability during minimally invasive surgeries. These devices are able to reach the desired surgical sites in body cavities without colliding with critical bl…
Mandibular mobility plays a significant role in human daily lives, enabling food intake, respiration, speaking, and other oral activities. However, as for the patients suffering from mandibular mobility disorders, their mandible functions are deteriorated, which severely affects their quality of lif…