58 accepted papers
Operating drones in urban environments often means they need to land on rooftops, which can have different geometries and surface irregularities. Accurately detecting roof inclination using conventional sensing methods, such as vision based or acoustic techniques, can be unreliable, as measurement q…
We present a tactile-sensorized Fin-Ray finger that enables simultaneous detection of contact location and indentation depth through an indirect sensing approach. A hinge mechanism is integrated between the soft Fin-Ray structure and a rigid sensing module, allowing deformation and translation infor
Robotic pushing can be a complicated process that is indicative of the techniques needed for general object manipulation. Here we propose a novel coarse-to-fine approach that combines visual localization with a pushing strategy using tactile and proprioceptive feedback. In the coarse control stage,
Humans can easily manipulate soft, deformable objects, relying on the intrinsic deformability of their fingerpads and their capabilities to infer item compliance. Transferring these skills into robotic systems is still an open challenging task. Recently, the introduction of soft biomimetic tactile s
This paper introduces an anthropomorphic robot hand built entirely using LEGO MINDSTORMS: the Educational SoftHand-A, a tendon-driven, highly-underactuated robot hand based on the Pisa/IIT SoftHand and related hands. To be suitable for an educational context, the design is constrained to use only st
We present SimShear: a sim-to-real pipeline for tactile control that allows use of shear information without explicitly modeling shear dynamics in simulation. Shear, which arises from lateral movements across contact surfaces, are critical for tasks involving dynamic object interactions but are chal…
Large language models (LLMs) are beginning to automate reward design for dexterous manipulation. However, no prior work has considered tactile sensing, which is known to be critical for human-like dexterity. We present Text2Touch, bringing LLM-crafted rewards to the challenging task of multi-axis in…
Robotic pushing is a fundamental manipulation task that requires tactile feedback to capture subtle contact forces and dynamics between the end-effector and the object. However, real tactile sensors often face hardware limitations such as high costs and fragility, and deployment challenges involving
Human hands are capable of in-hand manipulation in the presence of different hand motions. For a robot hand, harnessing rich tactile information to achieve this level of dexterity still remains a significant challenge. In this paper, we present AnyRotate, a system for gravity-invariant multi-axis in…
In this study, we introduce a new soft biomimetic optical tactile sensor based on mimicking the interlocking structure of the epidermal-dermal boundary: the BioTacTip. The primary sensing unit comprises a sharp white tip surrounded by four black cover tips that when subjected to an external force em
Imitation learning for robot dexterous manipulation, especially with a real robot setup, typically requires a large number of demonstrations. In this paper, we present a data-efficient learning from demonstration framework which exploits the use of rich tactile sensing data and achieves fine bimanua…
Humans rely on their visual and tactile senses to develop a comprehensive 3D understanding of their physical environment. Recently, there has been a growing interest in exploring and manipulating objects using data-driven approaches that utilise high-resolution vision-based tactile sensors. However,
High-resolution tactile sensing can provide accurate information about local contact in contact-rich robotic tasks. However, the deployment of such tasks in unstructured environments remains under-investigated. To improve the robustness of tactile robot control in unstructured environments, we propo…
Bimanual manipulation with tactile feedback will be key to human-level robot dexterity. However, this topic is less explored than single-arm settings, partly due to the availability of suitable hardware along with the complexity of designing effective controllers for tasks with relatively large stat
Teleoperation has widely contributed to many applications. Consequently, the design of intuitive and ergonomic control interfaces for teleoperation has become crucial. The rapid advancement of Mixed Reality (MR) has yielded tangible benefits in human-robot interaction. MR provides an immersive envir
Incipient slip is defined as the slippage of part, but not all, of the contact surface between a sensor and an object. Reliably detecting incipient slip in artificial tactile sensors would benefit autonomous robot handling capabilities by helping prevent object slippage during manipulation. Here, we…
Soft tactile optical sensors have opened up new possibilities for endowing artificial robotic hands with advanced touch-related properties; however, their use for compliance discrimination has been poorly investigated and mainly relies on data-driven methods. Discrimination of object compliance is c
Object pushing presents a key non-prehensile manipulation problem that is illustrative of more complex robotic manipulation tasks. While deep reinforcement learning (RL) methods have demonstrated impressive learning capabilities using visual input, a lack of tactile sensing limits their capability f
Tactile pose estimation and tactile servoing are fundamental capabilities of robot touch. Reliable and precise pose estimation can be provided by applying deep learning models to high-resolution optical tactile sensors. Given the recent successes of Graph Neural Network (GNN) and the effectiveness o…
Multi-robot platforms are playing an increasingly important role in warehouse automation for efficient goods transport. This paper proposes a novel customization of a multi-robot system, called Tactile Mobile Manipulators (TacMMs). Each TacMM integrates a soft optical tactile sensor and a mobile rob
This paper presents a control scheme for force sensitive, gentle grasping with a Pisa/IIT anthropomorphic SoftHand equipped with a miniaturised version of the TacTip optical tactile sensor on all five fingertips. The tactile sensors provide high-resolution information about a grasp and how the finge…
The soft fingers and strategic grasping skills enable the human hands to grasp objects in a stable manner. This letter is to model human grasping skills and transfer the learned skills to robots to improve grasping quality and success rate. First, we designed a wearable tool-like parallel hand exosk
This letter introduces the BRL/Pisa/IIT (BPI) SoftHand: a single actuator-driven, low-cost, 3D-printed, tendon-driven, underactuated robot hand that can be used to perform a range of grasping tasks. Based on the adaptive synergies of the Pisa/IIT SoftHand, we design a new joint system and tendon rou
Deep learning combined with high-resolution tactile sensing could lead to highly capable dexterous robots. However, progress is slow because of the specialist equipment and expertise. The DIGIT tactile sensor offers low-cost entry to high-resolution touch using GelSight-type sensors. Here we customi
Mid-air haptics create a new mode of feedback to allow people to feel tactile sensations in the air. Ultrasonic arrays focus acoustic radiation pressure in space, to induce tactile sensation from the resulting skin deflection. In this work, we present a low-cost tactile robot to test mid-air haptics
The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. This complicates tactile tasks like shape reconstruction and exploration that require information about…
High-resolution optical tactile sensors are increasingly used in robotic learning environments due to their ability to capture large amounts of data directly relating to agent-environment interaction. However, there is a high barrier of entry to research in this area due to the high cost of tactile
This letter proposes a controller for stable grasping of unknown-shaped objects by two robotic fingers with tactile fingertips. The grasp is stabilised by rolling the fingertips on the contact surface and applying a desired grasping force to reach an equilibrium state. The validation is both in simu
Robotic touch, particularly when using soft optical tactile sensors, suffers from distortion caused by motion-dependent shear. The manner in which the sensor contacts a stimulus is entangled with the tactile information about the stimulus geometry. In this work, we propose a supervised convolutional…
Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs. Tactile information is not usually considered despite its direct relation to environment interaction. In this work, we pres…
In this work, we report on how a sense of touch can be used to control an underactuated anthropomorphic robot hand, based on an integration that respects the hand’s mechanical functionality. Our focus is on integrating the sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot…
This work investigates uncertainty-aware deep learning (DL) in tactile robotics based on a general framework introduced recently for robot vision. For a test scenario, we consider optical tactile sensing in combination with DL to estimate the edge pose as a feedback signal to servo around various 2D…
We present a modified TacTip biomimetic optical tactile sensor design which demonstrates the ability to induce and detect incipient slip, as confirmed by recording the movement of markers on the sensor's external surface. Incipient slip is defined as slippage of part, but not all, of the contact sur…
Restoring tactile sensation is essential to enable in-hand manipulation and the smooth, natural control of upper-limb prosthetic devices. Here we present a platform to contribute to that long-term vision, combining an anthropomorphic robot hand (QB SoftHand) with a neuromorphic optical tactile senso…
Artificial touch would seem well-suited for Reinforcement Learning (RL), since both paradigms rely on interaction with an environment. Here we propose a new environment and set of tasks to encourage development of tactile reinforcement learning: learning to type on a braille keyboard. Four tasks are
Tactile sensing has been used for a variety of robotic exploration and manipulation tasks but a common constraint is a requirement for a large amount of training data. This paper addresses the issue of data-efficiency by proposing a novel method for online learning based on a Gaussian Process Latent…
Developing artificial tactile sensing capabilities that rival human touch is a long-term goal in robotics and prosthetics. Gradually more elaborate biomimetic tactile sensors are being developed and applied to grasping and manipulation tasks to help achieve this goal. Here we present the neuroTac, a…
Deep learning and reinforcement learning methods have been shown to enable learning of flexible and complex robot controllers. However, the reliance on large amounts of training data often requires data collection to be carried out in simulation, with a number of sim-to-real transfer methods being d…
Little research into tactile feet has been done for walking robots despite the benefits such feedback could give when walking on uneven terrain. This paper describes the development of a simple, robust and inexpensive tactile foot for legged robots based on a high-resolution biomimetic TacTip tactil…
In this letter, we have designed and built a set of tactile fingertips for integration with a commercial three-fingered robot hand, the Shadow Modular Grasper. The fingertips are an evolution of an established optical biomimetic tactile sensor, the TacTip. In developing the tactile fingertips, we ha
A common approach in the field of tactile robotics is the development of a new perception algorithm for each new application of existing hardware solutions. In this letter, we present a method of dimensionality reduction of an optical-based tactile sensor image output using a convolutional neural ne
Deep learning has the potential to have same the impact on robot touch as it has had on robot vision. Optical tactile sensors act as a bridge between the subjects by allowing techniques from vision to be applied to touch. In this letter, we apply deep learning to an optical biomimetic tactile sensor
Manipulation tasks often require robots to be continuously in contact with an object. Therefore tactile perception systems need to handle continuous contact data. Shear deformation causes the tactile sensor to output path-dependent readings in contrast to discrete contact readings. As such, in some…
Tactile sensing is required for human-like control with robotic manipulators. Multimodality is an essential component for these tactile sensors, for robots to achieve both the perceptual accuracy required for precise control, as well as the robustness to maintain a stable grasp without causing damag
Our human sense of touch enables us to manipulate our surroundings; therefore, complex robotic manipulation will require artificial tactile sensing. Typically tactile sensor arrays are used in robotics, implying that a straightforward way of interpreting multidimensional data is required. In this pa…
Slip detection helps to prevent robotic hands from dropping grasped objects and would thus enable complex object manipulation. Here we present a method of detecting slip with a biomimetic optical tactile sensor-the TacTip-that operates by measuring the positions of internal pins embedded in its comp
Here we propose and investigate a novel vibrissal tactile sensor - the Tac Whisker array - based on modifying a 3D-printed optical cutaneous (fingertip) tactile sensor - the TacTip. Two versions are considered: a static Tac Whisker array analogous to immotile tactile vibrissae (e.g. rodent microvib-…
There are a wide range of features that tactile contact provides, each with different aspects of information that can be used for object grasping, manipulation, and perception. In this paper inference of some key tactile features, tip displacement, contact location, shear direction and magnitude, is…
The fingerprint is a morphological aspect of the human fingertip that has interesting implications for our sense of touch. Previous studies focused on how the fingerprint affects the perception of stimuli that excite high temporal frequencies, such as for texture perception. These studies also only
Tactile perception methods rely on creating mappings from tactile data to percepts. In many approaches to artificial tactile perception, this involves extensive sampling of the object during the training phase. We introduce here a method to instead generalize tactile features across different orient
A key unsolved problem in tactile robotics is how to combine tactile perception and control to interact robustly and intelligently with the surroundings. Here, we focus on a prototypical task of tactile exploration over surface features such as edges or ridges, which is a principal exploratory proce
The use of tactile feedback for precision manipulation in robotics still lags far behind human capabilities. This study has two principal aims: 1) to demonstrate in-hand reorientation of grasped objects through active tactile manipulation; and 2) to present the development of a novel TacTip sensor a
Achieving object exploration with passive vision and active touch has been under investigation for thirty years. We build upon recent progress in biomimetic active touch that combines perception via Bayesian evidence accumulation with controlling the tactile sensor using perceived stimulus location.…
Tactile manipulation will be essential for automating industrial and service tasks currently done by humans. However, the application of tactile feedback to dexterous manipulation remains a challenging unsolved problem, with robot capabilities lagging far behind those of humans. Here, we present the
Fully autonomous factories of the future will need automated quality control processes to monitor products during manufacture. Here, we demonstrate that an artificial tactile system offers a solution to autonomous quality inspection, using a biomimetic tactile fingertip mounted as end-effector on an
Tactile manipulation is the ability to control objects in real-time using the sense of touch. Here we examine tactile manipulation from the perspective of active touch with a biomimetic tactile sensor, which combines tactile perception with control of sensor location. Experiments are performed with…
Decision making under uncertainty is commonly modelled as a process of competitive stochastic evidence accumulation to threshold (the drift-diffusion model). However, it is unknown how animals learn these decision thresholds. We examine threshold learning by constructing a reward function that avera…
Although superresolution has been studied to huge impact in visual imaging, it is relatively unexplored in tactile robotics. Here we demonstrate a novel optical sensor design (the TacTip) capable of achieving 40-fold localization superresolution to 0.1mm accuracy compared with a 4mm resolution betwe…