65 accepted papers
Continuum robots exhibit exceptional flexibility and multi-degree-of-freedom maneuverability, offering significant advantages for navigating confined luminal spaces. However, rapid simulation of their contact-influenced behavior remains challenging. This paper presents RLFEM, an innovative finite el
Autonomous ultrasound (US) probe initial localization is challenging when external cameras are obstructed or provide only partial patient views, as this limits the utility of anatomical priors. To overcome this, we introduce a novel robotic US probe localization framework featuring two key innovatio
Vision-Language Models (VLMs), such as CLIP, have demonstrated remarkable zero-shot capabilities in various computer vision tasks. However, their application to medical imaging remains challenging due to the high variability and complexity of medical data. Specifically, medical images often exhibit
Continual post-training (CPT) is a popular and effective technique for adapting foundation models like multimodal large language models to ever-evolving downstream tasks. While existing research primarily focus on methods like data replay, model expansion, or parameter regularization, the fundamenta…
Accurate three-dimensional (3D) shape sensing is vital for continuum robots in minimally invasive surgery. Conventional optical fiber methods depend on multi-fiber or multicore configurations, increasing integration complexity and associated costs. Single-fiber approaches support miniaturization, bu
Haptic feedback enhances user interaction with systems by adding the sense of touch, thereby improving immersion and realism in applications like virtual reality (VR), augmented reality (AR), video games, education, and robotic surgery. To address the challenges in mechanically actuated haptic feedb
Continuum robots offer high flexibility and multiple degrees of freedom, making them ideal for navigating narrow lumens. However, accurately modeling their behavior under large deformations and frequent environmental contacts remains challenging. Current methods for solving the deformation of these
Simultaneous Localization and Mapping (SLAM) is essential for precise surgical interventions and robotic tasks in minimally invasive procedures. While recent advancements in 3D Gaussian Splatting (3DGS) have improved SLAM with high-quality novel view synthesis and fast rendering, these systems strug
Test-time adaptation with pre-trained vision-language models, such as CLIP, aims to adapt the model to new, potentially out-of-distribution test data. Existing methods calculate the similarity between visual embedding and learnable class embeddings, which are initialized by text embeddings, for zer…
License plate (LP) recognition is crucial in intelligent traffic management systems. However, factors such as long distances and poor camera quality often lead to severe degradation of captured LP images, posing challenges to accurate recognition. The design of License Plate Image Restoration (LPIR)…
Motion planning for Continuum Surgical Robots (CSRs) faces significant challenges during minimally invasive surgery (MIS) when operating within highly constrained anatomical workspaces. Although traditional approaches aim to prevent tissue damage by finding collision-free paths, robot-tissue interac
Surgical phase recognition is critical for assisting surgeons in understanding surgical videos. Existing studies focused more on online surgical phase recognition, by leveraging preceding frames to predict the current frame. Despite great progress, they formulated the task as a series of frame-wise
Poisoning attacks compromise the training phase of federated learning (FL) such that the learned global model misclassifies attacker-chosen inputs called target inputs. Existing defenses mainly focus on protecting the training phase of FL such that the learnt global model is poison free. However, t…
In the era of Large Language Models (LLMs), embodied artificial intelligence presents transformative opportunities for robotic manipulation tasks. Ultrasound imaging, a widely used and cost-effective medical diagnostic procedure, faces. challenges due to the global shortage of professional sonograph
Vehicle license plate recognition is a crucial task in intelligent traffic management systems. However, the challenge of achieving accurate recognition persists due to motion blur from fast-moving vehicles. Despite the widespread use of image synthesis approaches in existing deblurring and recogniti…
Surgical instrument segmentation is crucial in surgical scene understanding, thereby facilitating surgical safety. Existing algorithms directly detected all instruments of predefined categories in the input image, lacking the capability to segment specific instruments according to the surgeon’s inte
The increasing realism of synthetic speech, driven by advancements in text-to-speech models, raises ethical concerns regarding impersonation and disinformation. Audio watermarking offers a promising solution via embedding human-imperceptible watermarks into AI-generated audios. However, the robustne…
Bronchoscopy plays a significant role in the early diagnosis and treatment of lung diseases. This process demands physicians to maneuver the flexible endoscope for reaching distal lesions, particularly requiring substantial expertise when examining the airways of the upper lung lobe. With the develo…
The discovery of new catalysts is essential for the design of new and more efficient chemical processes in order to transition to a sustainable future. We introduce an AI-guided computational screening framework unifying linguistic reasoning with quantum-chemistry based feedback from 3D atomistic re…
Transorbital Neuro Surgery (TNS) offers a novel treatment towards the lesion inside skull pursuing minimal invasiveness. Most conventional TNS tools are rigid and straight, limiting the dexterity and accessibility in passing a small port. Bendable and steerable surgical tools provides an alternative…
Real-time 6 DOF localization of bronchoscopes is crucial for enhancing intervention quality. However, current vision-based technologies struggle to balance between generalization to unseen data and computational speed. In this study, we propose a Depth-based Dual-Loop framework for real-time Visuall
Precisely determining the contact force during safe interaction in Minimally Invasive Robotic Surgery (MIRS) is still an open research challenge. Inspired by post-operative qualitative analysis from surgical videos, the use of cross-modality data driven deep neural network models has been one of the
Contrastive learning (CL) pre-trains general-purpose encoders using an unlabeled pre-training dataset which consists of images or image-text pairs. CL is vulnerable to data poisoning based backdoor attacks (DPBAs) in which an attacker injects poisoned inputs into the pre-training dataset so the enco…
Severe chronic pulmonary endarterectomy needs a dissector to delicately remove proliferative intima located in the depth of the pulmonary artery. This work proposed a novel endoscopic robotized steerable dissector for this surgery, enabling easier access to curved deep artery branches. The handheld…
Magnetically-actuated continuum robots (MCRs) have the potential to be miniaturized to submillimeter sizes. However, their limited deformation modes hinder their ability to navigate through narrow and tortuous lumens. In this letter, we introduce a novel 2-degrees of freedom (DoF) MCR with diverse d
Traditional rigid endoscopes have challenges in flexibly treating tumors located deep in the brain, and low operability and fixed viewing angles limit its development. This study introduces a novel dual-segment flexible robotic endoscope MicroNeuro, designed to perform biopsies with dexterous surgic…
Patient modelling has a wide range of applications in medicine and healthcare, such as clinical teaching, surgery navigation and automatic robotized scanning. While patients are typically covered or occluded in medical scenes, directly regressing human meshes from single RGB images is challenging. T
Robot-assisted minimally invasive surgery (MIS) using flexible endoscopy has emerged as a groundbreaking technology for improving traditional surgical approaches. However, a major challenge in advancing this technology is the lack of shape sensing, which leads to inaccurate navigation and control of
Cooperative perception (CP) is essential to enhance the efficiency and safety of future transportation systems, requiring extensive data sharing among vehicles on the road, which raises significant privacy concerns. Federated learning offers a promising solution by enabling data privacy-preserving c
Force feedback is of importance in Minimally Invasive Surgery (MIS) as it reduces surgical risks and enhances surgical safety. However, equipping force sensing to the tip of surgical instruments presents challenges due to their diminutive dimensions and often curved shapes. To address this issue, a
This work presents a novel shape evaluation and optimization approach for shape sensing, specifically targeting the constrained, irregular, and intricate spatial shapes of flexible bronchoscopes (FB) in human bronchial tree. The proposed evaluation criteria and optimization methods combine clinical…
Visual hallucination (VH) means that a multi-modal LLM (MLLM) imagines incorrect details about an image in visual question answering. Existing studies find VH instances only in existing image datasets, which results in biased understanding of MLLMs’ performance under VH due to limited diversity of s…
Intraoperative contact sensing has the potential to reduce the risk of surgical errors and enhance manipulation capabilities for medical robots, particularly in contact force control. Current intrinsic force sensing (IFS) methods are limited in application to medical instruments with arbitrary shape
Current imaging of the artery relies primarily on computed tomography angiography (CTA), which requires contrast injections and exposure to radiation. In this paper, we present a method for fully autonomous artery 3D image acquisition using a linear ultrasound (US) probe and a 6 DoFs robot arm with…
Point cloud classification is an essential component in many security-critical applications such as autonomous driving and augmented reality. However, point cloud classifiers are vulnerable to adversarially perturbed point clouds. Existing certified defenses against adversarial point clouds suffer f…
Soft robots that grow through eversion/apical extension can effectively navigate fragile environments such as ducts and vessels inside the human body. This paper presents the physics-based model of a miniature steerable eversion growing robot. We demonstrate the robot's growing, steering, stiffening
In this letter, a new 3-D electromagnetic position sensing method is proposed for localization of continuum medical robots. An electromagnet and magnetic sensors are placed outside the human body while only a piece of passive mu-metal with high magnetic permeability is attached to the robot moving i
Top-$k$ predictions are used in many real-world applications such as machine learning as a service, recommender systems, and web searches. $\ell_0$-norm adversarial perturbation characterizes an attack that arbitrarily modifies some features of an input such that a classifier makes an incorrect pred…
To realize dexterous robotic manipulation and enhance the human-machine interaction, nowadays increasing efforts have been made towards multi-dimensional force sensing. However, there are still bottlenecks in integrating these sensors into robots because of the limitation on conformability to comple
"Semi-supervised learning (SSL) leverages both labeled and unlabeled data to train machine learning (ML) models. State-of-the-art SSL methods can achieve comparable performance to supervised learning by leveraging much fewer labeled data. However, most existing works focus on improving the performan…
Optical colonoscopy is the gold standard for colorectal cancer screening, however even under optimal conditions only 81% of the internal tissue is inspected, in part causing up to 22% of early adenomas to be missed. Blindspots commonly occur at acute bends where the camera’s view is blocked. 3D reco…
Differentially private machine learning trains models while protecting privacy of the sensitive training data. The key to obtain differentially private models is to introduce noise/randomness to the training process. In particular, existing differentially private machine learning methods add noise t…
3D point cloud classification has many safety-critical applications such as autonomous driving and robotic grasping. However, several studies showed that it is vulnerable to adversarial attacks. In particular, an attacker can make a classifier predict an incorrect label for a 3D point cloud via care…
Emulating a human-like somatosensory system in instruments such as robotic hands and surgical grippers has the potential to revolutionize these domains. Using a combination of different sensing modalities is problematic due to the limited space and incompatibility of these sensing principles. Theref
Currently, optical tactile sensors propose solutions to measure contact forces at the tip of flexible medical instruments. However, the sensing capability of normal pressures applied to the surface along the tool body is still an open challenge. To deal with this challenge, this letter proposes a se
Endoscopic surgery is an increasingly popular alternative to laparoscopic techniques for many conditions, as the operation site can be reached without skin wounds. In many tasks, sufficient force generation is desired. As endoscopes must be highly flexible and slim, however, the force generation and…
Catheter-based intervention plays an important role in minimally invasive surgery. For the closed-loop control of catheter robot through contacts, the loss of contact sensing along the entire catheter might result in task failure. To deal with this problem, we propose a decoupled motion control stra
Intrinsic Tactile Sensing (ITS) is a well-established technique, relying on force/torque and geometric surface description to find contact centroids. The method works well for rigid surfaces. However, finding a solution for deformable surfaces is an open issue. This work presents two solutions to ex…
Enhancing haptic sensing allows more efficient control of interaction with unstructured and changing environments. Meanwhile, rich haptic information obtained during the interaction is useful for recognizing the environment. The haptic information contains distinctive physical features like friction
This letter introduces the design and evaluation of a novel worm-inspired, multisegment robotic endoscope with multiple degrees of freedom segments. The novelty of this design is that the robot is able to drive forwards and backwards, anchor itself, steer while inside a tubular structure and control
Medical ultrasound imaging robotics systems often comprise of complex control architectures and hardware integration to enable safe human robot interaction. In this paper, we investigate the applicability of a soft robotic end-effector for ultrasound intervention. A novel, parallel design is derived…
Knowledge of externally applied forces is crucial for compliant robotic manipulators in minimally-invasive and endoluminal robotic surgery for both patient safety and controllability of the device. We developed a novel continuum manipulator which comprises hydraulic actuation. In this work we invest…
This paper presents a methodology for the development of a multi-axis force/torque sensor based on optoelectronic technology. The advantages of using this sensing principle are the low manufacturing costs, the simple fabrication, and the immunity to electrical noise. The force/ torque sensor makes u…
This letter presents a strategy to represent data from a tactile array sensor and match it to an object's geometric features. Using that representation, a method is presented to localise a grasped object within a robot hand. The method consists of computing the covariance matrix in the tactile senso
Tactile data and kinesthetic cues are two important sensing sources in robot object recognition and are complementary to each other. In this paper, we propose a novel algorithm named Iterative Closest Labeled Point (iCLAP) to recognize objects using both tactile and kinesthetic information. The iCLA…
Contact force play is a significant role in success of the cardiac ablation. However, it is still challenging to estimate contact force when a catheter is under large bending and multiple contacts. This paper develops a new multi-section static model of the tendon-driven catheters for both real-time…
In this paper, a potential-field-based real-time path planning algorithm for a multi-segment continuum manipulator is proposed. This planner is employed to enable a continuum-style manipulator to move autonomously in dynamic environments in real-time. The classic potential field method is modified t
In this paper, we present a novel pose estimation and obstacle avoidance approach for tendon-driven multi-segment continuum manipulators moving in dynamic environments. A novel multi-stage implementation of an Extended Kalman Filter is used to estimate the pose of every point along the manipulator's…
This work investigates the applicability of stiffness-based modelling in soft robotics manipulation. The methodology is introduced and applied to model a soft robotics manipulator as single 3d Timoshenko beam element. The model is then utilized to solve the forward kinematics problem for the manipul…
This paper proposes a novel Cosserat rod model based method to estimate contact forces based on the shape analysis of the catheter. We simplify the original Cosserat rod model to achieve real-time estimation of the force magnitude and directions applied to the catheter tip. The simplified model cont…
The absence of touch of sense is a widely known drawback of robotic minimally invasive surgery (MIS). This paper proposes a design of optic soft tactile arrays which is promising to be adapted for MIS. The proposed design consists of multiple soft material channels. Each channel is designed using th…
This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondences, e.g., prominent features, between visual and tactile object identification. To apply it in robotics, we propose to l…