42 accepted papers
In robotic-assisted minimally invasive surgery, an assistant surgeon stands at the bedside to insert and manipulate instruments while the primary surgeon operates the robot. Augmented reality (AR) head-mounted displays (HMDs) may improve the assistant's spatial awareness, but require tracking of sur…
Robotic surgery has revolutionized minimally invasive procedures by offering enhanced precision, dexterity, and patient outcomes. However, the training and operational paradigms in robotic surgery have not evolved in parallel. Current apprenticeship models fall short in this domain, as robotic surge…
Most existing robotic surgery systems adopt a human-in-the-loop paradigm, often with the surgeon directly teleoperating the robotic system. Adding intelligence to these robots would enable higher-level control, such as supervised autonomy or even full autonomy. However, artificial intelligence (AI) …
The use of robots in high-risk and extreme environments is crucial for tasks that are dangerous or inaccessible to humans and require high precision. Particularly in scenarios where the cost of failure is high, remote human teleoperation can be the preferred method of robot control due to the adapta
Accurate and efficient surgical robotic tool pose estimation is of fundamental significance to downstream applications such as augmented reality (AR) in surgical training and learning-based autonomous manipulation. While significant advancements have been made in pose estimation for humans and anima
Data scarcity has long been an issue in the robot learning community. Particularly, in safety-critical domains like surgical applications, obtaining high-quality data can be especially difficult. It poses challenges to researchers seeking to exploit recent advancements in reinforcement learning and
Skull base surgery is a demanding field in which surgeons operate in and around the skull while avoiding critical anatomical structures including nerves and vasculature. While image-guided surgical navigation is the prevailing standard, limitation still exists requiring personalized planning and rec…
Automation in surgical robotics has the potential to improve patient safety and surgical efficiency, but it is difficult to achieve due to the need for robust perception algorithms. In particular, 6D pose estimation of surgical instruments is critical to enable the automatic execution of surgical ma…
The da Vinci surgical robot introduced remote control of instruments, providing surgeons with increased dexterity and precision. A major drawback, however, is the loss of sense of touch due to a lack of kinesthetic coupling between the surgical field and the surgeon. This paper presents a framework
Despite advancements in robotic-assisted surgery, automating complex tasks like suturing remains challenging due to the need for adaptability and precision. Learning-based approaches, particularly reinforcement learning (RL) and imitation learning (IL), require realistic simulation environments for…
Teleoperation of robots in space is challenging due to high latency and limited workspace visibility. Previously, the Interactive Planning and Supervised Execution (IPSE) and Augmented Virtuality systems were developed to reduce failure risk. These tools were visualized on a 3D da Vinci surgical con…
Camera navigation in minimally invasive surgery changed significantly since the introduction of robotic assistance. Robotic surgeons are subjected to a cognitive workload increase due to the asynchronous control over tools and camera, which also leads to interruptions in the workflow. Camera motion
The introduction of image-guided surgical navigation (IGSN) has greatly benefited technically demanding surgical procedures by providing real-time support and guidance to the surgeon during surgery. To develop effective IGSN, a careful selection of the surgical information and the medium to present…
Studies of the human brain during natural activities, such as locomotion, would benefit from the ability to image deep brain structures during these activities. While Positron Emission Tomography (PET) can image these structures, the bulk and weight of current scanners are not compatible with the de…
Telesurgery has a clear potential for providing high-quality surgery to medically underserved areas like rural areas, battlefields, and spacecraft; nevertheless, effective methods to overcome unreliable communication systems are still lacking. Furthermore, it is not well understood how users react a…
Automation has the potential to improve the standard of care but is difficult to realize due to perceptual challenges, especially in soft-tissue surgery. Machine learning can provide solutions, but typically requires large amounts of training data, which is time-consuming to collect. Even with share
Traditional teleoperation (leader/follower) systems primarily focus on one operator controlling one remote robot, but as robots become ubiquitous, there is an increasing need for multiple operators, including autonomous agents, to collaboratively control multiple robots. However, existing teleoperat
Learning high-performance deep neural networks for dynamic modeling of high Degree-Of-Freedom (DOF) robots remains challenging due to the sampling complexity. Typical unknown system disturbance caused by unmodeled dynamics (such as internal compliance, cables) further exacerbates the problem. In thi
Teleoperation platforms often require the user to be situated at a fixed location to both visualize and control the movement of the robot and thus do not provide the operator with much mobility. One example is in existing robotic surgery solutions that require the surgeons to be away from the patien…
The da Vinci Research Kit (dVRK) was introduced in 2012 to provide an affordable, open-source platform for research in robotic minimally-invasive surgery. It provides access to all levels of control but, until now, has relied on an analog controller for the motor current, which cannot easily be cust…
A flexible endoscope introduces more dexterity to the image capturing in endoscopic surgery. However, manual control or automatic control based on instrument tracking does not handle the misorientation between the endoscopic video and the surgeon. We propose an automatic flexible endoscope control m…
Ground-based teleoperation of robot manipulators for on-orbit servicing of spacecraft represents an example of high-payoff, high-risk operations that are challenging to perform due to high latency communications, with telemetry time delays of several seconds. In these scenarios, confidence of operat…
Most current surgical robotic systems lack the ability to sense tool/tissue interaction forces, which motivates research in methods to estimate these forces from other available measurements, primarily joint torques. These methods require the internal joint torques, due to the robot inverse dynamics…
Robot-Assisted systems for Minimally Invasive Surgery enhance the surgeon capability, however, direct control over both the surgical tools and the endoscope results in an increased workload that leads to longer operation times. This work investigates the introduction of SCAN (System for Camera Auton…
We propose a system for visually monitoring and servoing the cutting of a multi-layer insulation (MLI) blanket that covers the envelope of satellites and spacecraft. The main contributions of this paper are: 1) to propose a model for relating visual features describing the engagement depth of the bl…
External disturbance forces caused by nonlinear springy electrical cables in the master tool manipulator (MTM) of the da Vinci Research Kit (dVRK) limits the usage of the existing gravity compensation methods. Significant motion drifts at the MTM tip are often observed when the MTM is located far fr
In adult laparoscopy, robot-aided surgery is a reality in thousands of operating rooms worldwide, owing to the increased dexterity provided by the robotic tools. Many robots and robot control techniques have been developed to aid in more challenging scenarios, such as pediatric surgery and microsurg…
In robotic-assisted laparoscopic surgery, the first assistant (FA) stands at the bedside assisting the intervention, while the surgeon sits at the console teleoperating the robot. Tasks for the FA include navigating new instruments into the surgeon's field-of-view and passing in or retracting materi…
On-orbit servicing of satellites is complicated by the fact that almost all existing satellites were not designed to be serviced. This creates a number of challenges, one of which is to cut and partially remove the protective thermal blanketing that encases a satellite prior to performing the servic…
Robot control algorithms often rely on measurements of robot joint velocities, which can be estimated by measuring the time between encoder edges. When encoder edges occur infrequently, such as at low velocities and/or with low resolution encoders, this measurement delay may affect the stability of…
Teleoperation in extreme environments can be hindered by limitations in telemetry and in operator perception of the remote environment. Often, the primary mode of perception is visual feedback from remote cameras, which do not always provide suitable views and are subject to telemetry delays. To add
This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by th
Ground-based teleoperation of robots in space is subject to time delays of several seconds or more. This leads to the use of model-based approaches, where the operator interacts with a model (simulation) of the remote environment and the remote robot attempts to reproduce the results of that interac…
In this letter, we propose a new method to fully control complete 4-image-DoF manipulation of laparoscopic instruments [with remote center of motion (RCM) mechanism] based on the geometric features of a designed marker in a 2-D image. Our marker encodes the configuration of the instruments by comput
One of the risks of the endonasal approach to skull base surgery is inadvertent damage to one of the two carotid arteries that are located behind the bone being drilled. Photoacoustic imaging, which combines a pulsed laser with an ultrasound receiver probe, has been shown to be able to image blood v…
Medical and surgical robot systems are examples of safety-critical systems due to the potential hazards that can lead to severe injury or the loss of human life. Furthermore, robot safety is becoming increasingly important in other domains as robots begin to share their workspace with humans. At the
Model-based teleoperation is suitable for systems with large communication delay because the operator interacts with a model of the task while the remote robot uses sensor-based control to replicate that interaction on the physical system. When the model geometry is accurately known, it is only nece
Suturing is a challenging and highly dexterous task in minimally invasive surgery, even with the assistance of robotic surgical systems. In this work, we propose a simple yet versatile impedance virtual fixture framework, which can be applied on the master manipulator in a tele-operated robotic surg…
The ability to refuel satellites on-orbit using teleoperated robots could extend the life of satellite missions, potentially saving time and resources for both industry and space agencies. One critical step in the refueling process is to gain access to the satellite's fueling port by cutting through…
For satellite servicing, it is necessary to remove a patch of multi-layer insulation (MLI) that covers the access panel. We consider the case where this patch is secured by tape and desire to use ground-based teleoperation to carefully cut the tape on three sides of the patch. Communication delays o…
We are investigating the use of photoacoustic (PA) imaging to detect critical structures, such as the carotid artery, that may be located behind the bone being drilled during robot-assisted endonasal transsphenoidal surgery. In this system, the laser is mounted on the drill (via an optical fiber) an…
Direct teleoperation with multisecond time-delayed telemetry between master and slave is challenging for humans to perform. When controlling a holonomic robot with many degrees of freedom, operators may incidentally provide commands in an intended direction without realizing their mistake until rece…