41 accepted papers
Multi-robot systems provide the parallelism and redundancy necessary for long-horizon tasks, while Large Language Models (LLMs) offer the reasoning capabilities to decompose these objectives into actionable plans. However, effectively grounding this high-level reasoning in physical multi-agent execu…
Multi-agent robotic manipulation remains challenging due to the combined demands of coordination, grasp stability, and collision avoidance in shared workspaces. To address these challenges, we propose the Adaptive Dynamic Modality Diffusion Policy (ADM-DP), a framework that integrates vision, tactil…
Modular small-scale robots offer the potential for on-demand assembly and disassembly, enabling task-specific adaptation in dynamic and constrained environments. However, existing modular magnetic platforms often depend on workspace collisions for reconfiguration, employ bulky three-dimensional elec…
Acquiring aligned visuo-tactile datasets is slow and costly, requiring specialised hardware and large-scale data collection. Synthetic generation is promising, but prior methods are typically single-modality, limiting cross-modal learning. We present MultiDiffSense, a unified diffusion model that sy…
Precise pose estimation of optical microrobots is essential for enabling high-precision object tracking and autonomous biological studies. However, current methods rely heavily on large, high-quality microscope image datasets, which are difficult and costly to acquire due to the complexity of micror…
Contact-rich robotic manipulation requires representations that encode local geometry. Vision provides global context but lacks direct measurements of properties such as texture and hardness, whereas touch supplies these cues. Modern visuo-tactile sensors capture both modalities in a single fused im…
Parallel diffusion decoding can accelerate diffusion language model inference by unmasking multiple tokens per step, but aggressive parallelism often harms quality. Revocable decoding mitigates this by rechecking earlier tokens, yet we observe that existing verification schemes frequently trigger fl…
Accurate perception of object hardness is essential for safe and dexterous contact-rich robotic manipulation. Here, we present TactEx, an explainable multimodal robotic interaction framework that unifies vision, touch, and language for human-like hardness estimation and interactive guidance. While t…
In vivo image-guided multi-pipette patch-clamp is essential for studying cellular interactions and network dynamics in neuroscience. However, current procedures mainly rely on manual expertise, which limits accessibility and scalability. Robotic automation presents a promising solution, but achievin
Magnetic micro-robots have demonstrated immense potential in biomedical applications, such as in vivo drug delivery, non-invasive diagnostics, and cell-based therapies, owing to their precise maneuverability and small size. However, current micromanipulation techniques often rely solely on a two-dim
Recently, Transformer-based methods have gained popularity in skeleton-based action recognition due to their advantages in modeling long-range dependencies. However, Transformer lacks the inductive biases towards skeletal topology and tends to capture salient features, potentially overlooking subtle…
Optical tweezers (OT) offer unparalleled capabilities for micromanipulation with submicron precision in biomedical applications. However, controlling conventional multi-trap OT to achieve cooperative manipulation of multiple complexshaped microrobots in dynamic environments poses a significant chall
Spatial reasoning is a core component of human cognition, enabling individuals to perceive, comprehend, and interact with the physical world. It relies on a nuanced understanding of spatial structures and inter-object relationships, serving as the foundation for complex reasoning and decision-making
Bimanual robotic manipulation is an emerging and critical topic in the robotics community. Previous works primarily rely on integrated control models that take the perceptions and states of both arms as inputs to directly predict their actions. However, we think bimanual manipulation involves not on…
Dexterous teleoperation plays a crucial role in robotic manipulation for real-world data collection and remote robot control. Previous dexterous teleoperation mostly relies on hand retargeting to closely mimic human hand postures. However, these approaches may fail to fully leverage the inherent dex…
This paper presents a novel digital twin (DT)-driven framework for immersive teleoperation in the domain of robot-assisted microsurgery (RAMS). The proposed method leverages the power of DT with mixed reality (MR) technology to create an interactive, immersive teleoperation environment for surgeons
Motion scaling is essential to empower users to conduct precise manipulation during teleoperation for robot-assisted microsurgery (RAMS). A constant, small motion scaling ratio can enhance the precision of teleoperation but hinder the operator from quickly reaching distant targets. The concept of se…
Accurate robotic control over interactions with the environment is fundamentally grounded in understanding tactile contacts. In this paper, we introduce MagicTac, a novel high-resolution grid-based tactile sensor. This sensor employs a 3D multi-layer grid-based design, inspired by the Magic Cube str…
The burgeoning demand for home-care services, driven by a rapidly aging global population, necessitates innovative solutions to alleviate the burden on caregivers and enhance care quality. This paper introduces the development of an Inter-net of Human and Robotic Things (IoHRT) framework, which syne…
Tactile sensing is significant for robotics since it can obtain physical contact information during manipulation. To capture multimodal contact information within a compact framework, we designed a novel sensor called ViTacTip, which seamlessly integrates both tactile and visual perception capabilit…
High-resolution tactile sensing can provide accurate information about local contact in contact-rich robotic tasks. However, the deployment of such tasks in unstructured environments remains under-investigated. To improve the robustness of tactile robot control in unstructured environments, we propo…
Bimanual manipulation with tactile feedback will be key to human-level robot dexterity. However, this topic is less explored than single-arm settings, partly due to the availability of suitable hardware along with the complexity of designing effective controllers for tasks with relatively large stat
Teleoperation has widely contributed to many applications. Consequently, the design of intuitive and ergonomic control interfaces for teleoperation has become crucial. The rapid advancement of Mixed Reality (MR) has yielded tangible benefits in human-robot interaction. MR provides an immersive envir
Segmenting and recognizing surgical operation trajectories into distinct, meaningful gestures is a critical preliminary step in surgical workflow analysis for robot-assisted surgery. This step is necessary for facilitating learning from demonstrations for autonomous robotic surgery, evaluating surgi…
Object pushing presents a key non-prehensile manipulation problem that is illustrative of more complex robotic manipulation tasks. While deep reinforcement learning (RL) methods have demonstrated impressive learning capabilities using visual input, a lack of tactile sensing limits their capability f
Microsurgery involves the dexterous manipulation of delicate tissue or fragile structures, such as small blood vessels and nerves, under a microscope. To address the limitations of imprecise manipulation of human hands, robotic systems have been developed to assist surgeons in performing complex mic…
Tactile pose estimation and tactile servoing are fundamental capabilities of robot touch. Reliable and precise pose estimation can be provided by applying deep learning models to high-resolution optical tactile sensors. Given the recent successes of Graph Neural Network (GNN) and the effectiveness o…
Multi-robot platforms are playing an increasingly important role in warehouse automation for efficient goods transport. This paper proposes a novel customization of a multi-robot system, called Tactile Mobile Manipulators (TacMMs). Each TacMM integrates a soft optical tactile sensor and a mobile rob
In endoluminal surgery, the miniature instruments shall be of high accuracy and flexibility for minimal invasive diagnosis and surgical intervention. To this end, continuum robots with flexible joints have been proposed as the mechanism of endoscopic instruments. The compliance and deformability of…
Human-robot shared control, which integrates the advantages of both humans and robots, is an effective approach to facilitate efficient surgical operation. Learning from demonstration (LfD) techniques can be used to automate some of the surgical sub tasks for the construction of the shared control m…
In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant is normally required to control the position and the zooming ratio of the laparoscope, following the surgeon’s instructions. However, moving the laparoscope frequently may lead to unstable and suboptimal views, while the adjust…
Minimally invasive surgery mainly consists of a series of sub-tasks, which can be decomposed into basic gestures or contexts. As a prerequisite of autonomic operation, surgical gesture recognition can assist motion planning and decision-making, and build up context-aware knowledge to improve the sur…
Surgical knot tying is one of the most fundamental and important procedures in surgery, and a high-quality knot can significantly benefit the postoperative recovery of the patient. However, a longtime operation may easily cause fatigue to surgeons, especially during the tedious wound closure task. I…
Master-Slave control is commonly used for RobotAssisted Minimally Invasive Surgery (RAMIS). The configuration, as well as the placement of the master manipulators, can influence the remote control performance. An ergonomic shared workspace analysis framework is proposed in this letter. Combined with
The assessment of microsurgical skills for Robot-Assisted Microsurgery (RAMS) still relies primarily on subjective observations and expert opinions. A general and automated evaluation method is desirable. Deep neural networks can be used for skill assessment through raw kinematic data, which has the
The recent development of Robot-Assisted Minimally Invasive Surgery (RAMIS) has brought much benefit to ease the performance of complex Minimally Invasive Surgery (MIS) tasks and lead to more clinical outcomes. Compared to direct master-slave manipulation, semi-autonomous control for the surgical ro…
Accurate master-slave control is important for Robot-Assisted Microsurgery (RAMS). This paper presents a handheld master controller for the operation and training of RAMS. A 9-axis Inertial Measure Unit (IMU) and a micro camera are utilized to form the sensing system for the handheld controller. A n…
Master-slave control is a common form of human-robot interaction for robotic surgery. To ensure seamless and intuitive control, a mechanism of self-adaptive motion scaling during teleoperaton is proposed in this letter. The operator can retain precise control when conducting delicate or complex mani
Master manipulators represent a key component of Robot-Assisted Minimally Invasive Surgery (RAMIS). In this paper, an Analytic Hierarchy Process (AHP) method is used to construct an effective workspace analysis framework, which can assist the configuration selection and design evaluation of a portab…
Workspace analysis is essential for robotic manipulators, which helps researchers study, evaluate, and optimize their designs based on specific criteria with due consideration of ergonomics and usability. Although workspace analysis is a common research topic, current solutions provide design-specif
Based on the integrated design concept of using the robot's mechanical arm for both locomotion and manipulation, we proposed a compact and transformable robot, i.e. the WALS-Robot, which utilizes combination and switch among four components: wheels, arms, legs and suckers. The WALS-Robot could trans…