44 accepted papers
Stereo disparity estimation is crucial for obtaining depth information in robot-assisted minimally invasive surgery (RAMIS). While current deep learning methods have made significant advancements, challenges remain in achieving an optimal balance between accuracy, robustness, and inference speed. To…
Image-guided surgery demands adaptive, real-time decision support, yet static AI models struggle with structured task planning and providing interactive guidance. Large language models (LLMs)-powered agents offer a promising solution by enabling dynamic task planning and predictive decision support.
Minimally invasive surgery (MIS) offers significant benefits, such as reduced recovery time and minimised patient trauma, but poses challenges in visibility and access, making accurate 3D reconstruction a significant tool in surgical planning and navigation. This work introduces a robotic arm platfo
Automated detection of surgical errors can improve robotic-assisted surgery. Despite promising progress, existing methods still face challenges in capturing rich temporal context to establish long-term dependencies while maintaining computational efficiency. In this letter, we propose a novel hierar
Accurate tracking of tissues and instruments in videos is crucial for Robotic-Assisted Minimally Invasive Surgery (RAMIS), as it enables the robot to comprehend the surgical scene with precise locations and interactions of tissues and tools. Traditional keypoint-based sparse tracking is limited by f
Tumors affecting abdominal organs rank among the deadliest malignancies. In this context, Magnetic Resonance Imaging (MRI) serves as an effective diagnostic tool with a strong potential to support image-guided minimally invasive interventions for treating these tumours, offering an ionizing-radiatio…
Soft robotics, an emerging domain in modern robotics, introduces innovative possibilities alongside challenges in controllability, particularly with multi-degree inflatable actuators. We present a novel manipulation method using underactuated soft fingers that addresses these challenges by harnessin…
In this paper, we introduce a method to tackle the problem of robot local path planning among pushable objects –an open problem in robotics. In particular, we simultaneously train multiple agents in a physics-based simulation environment, utilizing an Advantage Actor-Critic algorithm coupled with a…
Screening of the lower gastrointestinal (GI) tract is of paramount importance for the early detection of precancerous lesions in the intestine, with an impact on reducing the high death rate of patients affected by cancer worldwide. Colonoscopy, i.e. standard procedure for screening the colon, is ef…
Despite advancements in robotic systems and surgical data science, ensuring safe execution in robot-assisted minimally invasive surgery (RMIS) remains challenging. Current methods for surgical error detection typically involve two parts: identifying gestures and then detecting errors within each ges
Most state-of-the-art methods for action segmentation are based on single input modalities or naive fusion of multiple data sources. However, effective fusion of complementary information can potentially strengthen segmentation models and make them more robust to sensor noise and more accurate with…
Colonoscopy is considered the golden standard for cancer screening of the lower gastrointestinal (GI) tract. Nonetheless, conventional colonoscopy can cause discomfort to patients due to the forces occurring between colonoscopes and the walls of the colon. Robotic solutions have been proposed to red
This letter proposes a novel video-based, contrastive regression architecture, Contra-Sformer, for automated surgical skill assessment in robot-assisted surgery. The proposed framework is structured to capture the differences in the surgical performance, between a test video and a reference video wh
We introduce a novel approach for learning a complex multi-stage needle pick-and-place manipulation task for surgical applications using Reinforcement Learning without expert demonstrations or explicit curriculum. The proposed method is based on a recursive decomposition of the original task into a
Concentric Tube Robots (CTRs) are promising for minimally invasive interventions due to their miniature diameter, high dexterity, and compliance with soft tissue. CTRs comprise individual pre-curved tubes usually composed of NiTi and are arranged concentrically. As each tube is relatively rotated an
In ophthalmic surgery, particularly in procedures involving the posterior segment, clinicians face significant challenges in maintaining precise control of hand-held instruments without damaging the fundus tissue. Typical targets of this type of surgery are the internal limiting membrane (ILM) and t
Learning a disentangled, interpretable, and structured latent representation in 3D generative models of faces and bodies is still an open problem. The problem is particularly acute when control over identity features is required. In this paper, we propose an intuitive yet effective self-supervised a…
In retinal surgery clinicians access the internal volume of the eyeball through small scale trocar ports, typically 0.65 mm in diameter, to treat vitreoretinal disorders like idiopathic epiretinal membrane and age-related macular holes. The treatment of these conditions involves the removal of thin
The optoelectronic microrobot is an advanced light-controlled micromanipulation technology which has particular promise for collecting and transporting sensitive microscopic objects such as biological cells. However, wider application of the technology is currently limited by a reliance on manual co
Determining the externally-induced motion of a soft robot in minimally-invasive procedures is highly challenging and commonly demands specific tools and dedicated sensors. Intrinsic force sensing paired with a model describing the robot's compliance offers an alternative pathway which relies heavily
Minimally invasive surgery requires real-time tool tracking to guide the surgeon where depth perception and visual occlusion present navigational challenges. Although vision-based and external sensor-based tracking methods exist, fibre-optic sensing can overcome their limitations as they can be inte…
While mobile navigation has been focused on obstacle avoidance, Navigation Among Movable Obstacles (NAMO) via interaction with the environment, is a problem that is still open and challenging. This paper, presents a novel system integration to handle NAMO using visual feedback. In order to explore t…
Intratympanic steroid injections are commonly employed in treating ear diseases, such as sudden sensorineural hearing loss or Meniere's disease through drug delivery via the middle ear. Whilst being an effective treatment, the procedure has to be performed by a trained surgeon to avoid delicate regi
Optoelectronic tweezer-driven microrobots (OETdMs) are a versatile micromanipulation technology based on the application of light induced dielectrophoresis to move small dielectric structures (microrobots) across a photoconductive substrate. The microrobots in turn can be used to exert forces on sec…
Concentric Tube Robots (CTRs), a type of continuum robot, are a collection of concentric, pre-curved tubes composed of super elastic nickel titanium alloy. CTRs can bend and twist from the interactions between neighboring tubes causing the kinematics and therefore control of the end-effector to be v…
Fetoscopic laser ablation surgery could be guided using a high-quality panorama of the operating site, representing a map of the placental vasculature. This can be achieved during the initial inspection phase of the procedure using image mosaicking techniques. Due to the lack of camera calibration i
Hand-eye calibration is a method to determine the transformation linking between the robot and camera coordinate systems. Conventional calibration algorithms use a calibration grid to determine camera poses, corresponding to the robot poses, both of which are used in the main calibration procedure.
The semantic segmentation of robotic surgery video and the delineation of robotic instruments are important for enabling automation. Despite major recent progresses, the majority of the latest deep learning models for instrument detection and segmentation rely on large datasets with ground truth lab
Surgical gesture recognition is important for surgical data science and computer-aided intervention. Even with robotic kinematic information, automatically segmenting surgical steps presents numerous challenges because surgical demonstrations are characterized by high variability in style, duration…
To what extent are two images picturing the same 3D surfaces? Even when this is a known scene, the answer typically requires an expensive search across scale space, with matching and geometric verification of large sets of local features. This expense is further multiplied when a query image is eval…
Surgical-tool joint detection from laparoscopic images is an important but challenging task in computer-assisted minimally invasive surgery. Illumination levels, variations in background and the different number of tools in the field of view, all pose difficulties to algorithm and model training. Ye
In the eye-in-hand robot configuration, hand-eye calibration plays a vital role in completing the link between the robot and camera coordinate systems. Calibration algorithms are mature and provide accurate transformation estimations for an effective camera-robot link but rely on a sufficiently wide
Robotic-assisted partial nephrectomy is a surgical operation in which part of a kidney is removed typically because of the presence of a mass. Pre-operative and intraoperative imaging techniques are used to identify and outline the target mass, thus the margins of the resection area on the kidney su
In robotic surgery the motion of instruments and the laparoscopic camera is constrained by their insertion ports, i. e. a remote centre of motion (RCM). We propose a Simultaneous Localisation and Mapping (SLAM) approach that estimates laparoscopic camera motion under RCM constraints. To achieve this…
Fetoscopy is a technically challenging surgery, due to the dynamic environment and low diameter endoscopes often resulting in a limited field of view. In this paper, we report on the design and operation of a robotic multimodal endoscope with optical ultrasound and white light stereo camera. The man…
Kinematic trajectories recorded from surgical robots contain information about surgical gestures and potentially encode cues about surgeon's skill levels. Automatic segmentation of these trajectories into meaningful action units could help to develop new metrics for surgical skill assessment as well…
Hand-eye calibration is a classic problem in robotics that aims to find the transformation between two rigidly attached reference frames, usually a camera and a robot end-effector or a motion tracker. Most hand-eye calibration techniques require two data streams, one containing the eye (camera) moti
This letter proposes an algorithm to estimate the relative pose between two image view-points assuming that a camera is moving under a remote center of motion constraint. This is useful in minimally invasive robotic surgery, where the motion of a laparoscopic camera is constrained by the keyhole ins
Twin-twin transfusion syndrome requires interventional treatment using a fetoscopically introduced laser to sever the shared blood supply between the fetuses. This is a delicate procedure relying on small instrumentation with limited articulation to guide the laser tip and a narrow field of view to
Surgical interventions are increasingly executed minimal invasively. Surgeons insert instruments through tiny incisions in the body and pivot slender instruments to treat organs or tissue below the surface. While a blessing for patients, surgeons need to pay extra attention to overcome the fulcrum e…
Recovering 3D scene geometry from underwater images involves the Refractive Structure-from-Motion (RSfM) problem, where the image distortions caused by light refraction at the interface between different propagation media invalidates the single view point assumption. Direct use of the pinhole camera…
Real-time tool segmentation from endoscopic videos is an essential part of many computer-assisted robotic surgical systems and of critical importance in robotic surgical data science. We propose two novel deep learning architectures for automatic segmentation of non-rigid surgical instruments. Both…
In a robot mounted camera arrangement, hand-eye calibration estimates the rigid relationship between the robot and camera coordinate frames. Most hand-eye calibration techniques use a calibration object to estimate the relative transformation of the camera in several views of the calibration object…
In endovascular surgery and cardiology, robotic catheters are emerging as a promising technology for enhanced catheter manipulation and navigation while reducing radiation exposure. For robotic catheter systems especially with tendon actuation, a key challenge is the localisation of the catheter sha