39 accepted papers
Locating and identifying objects in vision-denied environments is a critical challenge for intelligent robot systems. To address the limitation of vision, we present a tactile-only method for object search and recognition using custom tactile skin sensors on robot hands. The method involves searchin…
The human somatosensory system integrates multimodal sensory feedback, including tactile, proprioceptive, and thermal signals, to enable comprehensive perception and effective interaction with the environment. Inspired by the biological mechanism, we present a sensorized soft anthropomorphic hand eq
Accurate temporal segmentation of human actions is critical for intelligent robots in collaborative settings, where a precise understanding of sub-activity labels and their temporal structure is essential. However, the inherent noise in both human pose estimation and object detection often leads to
Current methods for human-object interaction segmentation excel in closed-world settings but struggle to generalize to open-world scenarios where novel actions emerge. Since collecting exhaustive training data for all possible dynamic human activities is impractical, a model capable of detecting and
Driven by the need for rapid and reliable heavy payload transport in logistics and manufacturing, researchers are increasingly exploring early applications of humanoid robotics in these domains. Although bipedal locomotion excels on challenging terrain, wheeled modes of transportation remain signifi
This work presents a novel method to keep stable contact and balance while stepping over partial footholds for biped humanoid robots with flat feet. We exploit plantar tactile feedback to detect the geometry of the terrain and reconstruct online the new supporting polygon after landing every step. P
In this work, we present a novel bilateral teleoperation architecture with variable impedance control for orientational contact tasks. We exploit Unit Quaternions and tools from Lie Theory to model and manipulate robot orientations, as well as Learning-from-Demonstration to learn a stiffness adaptat
Intuitive teleoperation enables operators to embody remote robots, providing the sensation that the robot is part of their own body during control. The sense of agency (SoA), i.e., the feeling of controlling the robot, contributes to enhanced motivation and embodiment during teleoperation. However,
Robots that autonomously operate in human living environments require the ability to adapt to unpredictable changes and flexibly handle a variety of tasks. Particularly, coordinated bimanual motions are essential for enabling tasks that are difficult with just one hand, such as grasping bulky object…
In this letter, we present a method for realizing the heat-evoked nociceptive withdrawal reflex (NWR) in the upper limb of a humanoid robot so that it can avoid the potential damage caused by noxious heat. We use a spiking neural network whose structure, encoding scheme, and form of information tran
This paper presents an optimization of an admittance control model for gait balance assistance offered by a walker-type assistive robot. We previously introduced the notion of quasi-passive physical Human-Robot Interaction (pHRI) where a non-wearable assistive device adaptively achieves supportabili…
An intelligent assistive robotic walker can be a tremendous help to patients with mobility disorders, e.g., hemiplegia patients. However, at the same time, a full-size assistive walker could be difficult to steer due to the reduced balancing ability of the patient. An intuitive interface can provide
Learning object affordances enables robots to plan and perform purposeful actions. However, a fundamental challenge for the utilization of affordance knowledge lies in its generalization to unknown objects and environments. In this letter we present a new method for learning causal relationships bet
In this work, we present a novel method for ankle/foot compliance for biped humanoid robots walking over uneven terrain. Based on distributed plantar proximity sensing, we developed the Preemptive Foot Compliance (PFC) control that generates a Preemptive Ground Reaction Wrench that modifies the foot
In this letter, we present a fast method for autonomously planing manipulation tasks for mobile manipulators. The planner defines an optimal order to perform pick-and-place operations for taking objects from a cluttered scene to specific deposit areas considering both, manipulator and mobile base mo
Multiple approaches to the estimation of high-order motion derivatives for innovative control applications now rely on the data collected by redundant arrays of inertial sensors mounted on robots, with promising results. However, most of these works suffer scalability issues induced by the considera
In this paper, we propose a real-time human-robot interface (HRI) system, where Electrooculography (EOG) and Electromyography (EMG) signals were decoded to perform reach-to-grasp movements. For that, five different eye movements (up, down, left, right and rest) were classified in real-time and trans…
In this work, we propose a new estimation method for second-order kinematics for floating-base robots, based on highly redundant distributed inertial feedback. The linear acceleration of each robot link is measured at multiple points using a multimodal, self-configuring and self-calibrating artifici…
In a physical Human-Robot Interaction for industrial scenarios is paramount to guarantee the safety of the user while keeping the robot's performance. Hierarchical task approaches are not sufficient since they tend to sacrifice the low priority tasks in order to guarantee the consistency of the main…
To act effectively in its environment, a cognitive robot needs to understand the causal dependencies of all intermediate actions leading up to its goal. For example, the system has to infer that it is instrumental to open a cupboard door before trying to grasp an object inside the cupboard. In this…
This work presents a new contact-based 3D path planning approach for manipulators using robot skin. We make use of the Stochastic Functional Gradient Path Planner, extending it to the 3D case, and assess its usefulness in combination with multi-modal robot skin. Our proposed algorithm is verified on…
This letter evaluates and describes the large-scale integration of our multi-modal event-driven robot skin system on our humanoid robot H1 (REEM-C, PAL robotics). The robot skin is powered by the robot and all processing of tactile perception and control are executed onboard the robot. The robot ski
In this letter, we introduce our approach to walking assistance for elderly adults through predictive optimization of gait assistive force. We focus on providing supportive interaction force to the user during walking with a robotic assistive device with an admittance controlled mobile base. Appropr
Skin can provide rich multi-modal contact information about the interaction forces of a robot with its environment. With this new way of sensing, a new generation of compliant controllers can be developed to enable different kinds of interactions. In this letter, a pressure-driven compliance control
Humanoid robots are expected to interact in human environments, where physical interactions are unavoidable. Therefore, whole-body control methods that include multi-contact interactions are required. The new emerging technologies in touch sensing are fundamental to acquire online and rich informati…
Artificial robotic skins are continuously in contact with their environment, and therefore highly rely on proper connections in their skin cells' network. With a static network protocol approach, the affected skin area is unusable after a connection failure. Therefore, we developed a dynamic network…
The predictive functions that permit humans to infer their body state by sensorimotor integration are critical to perform safe interaction in complex environments. These functions are adaptive and robust to non-linear actuators and noisy sensory information. This paper introduces a computational per…
The realization of a kinesthetic robot behavior using robot skin requires a reactive skin torque controller, which fuses skin information and robot information to an appropriate skin joint torque in real-time. This fusion of information in real-time is challenging when deploying large scale skin. In…
This paper presents novel event-driven forward kinematics algorithms for open kinematic chains with O(log n) complexity. This event-driven algorithm can efficiently update forward kinematics only when new sensory data comes. This will also contribute to localization of computational resources at sen…
In this letter, we propose a complete probabilistic tactile-based framework to enable robots to autonomously explore unknown workspaces and recognize objects based on their physical properties. Our framework consists of three components: 1) an active pretouch strategy to efficiently explore unknown
In this work we present a novel efficient event-driven reactive skin controller for large scale robot skin. The novel event-driven controller derives from a standard Jacobian torque controller and fully takes advantage of our multi-modal event-driven robot skin. Event-driven systems only sample, tra…
This paper presents a new algorithm for forward kinematics, called Asynchronous Forward Kinematics (AFK). The algorithm has the complexity of O(log n) for updating one joint angle, and O(logn) for obtaining a homogeneous transformation matrix between links. AFK enables computation for efficient forw…
Capturing realistic human behaviors is essential to learn human models that can later be transferred to robots. Recent improvements in virtual reality (VR) head-mounted displays provide a viable way to collect natural examples of human behavior without the difficulties often associated with capturin…
The concept of hybrid zero dynamics is a promising approach for designing exponentially stabilizing controllers for dynamic walking with some degrees of underactuation. By this approach a feedback controller is designed such that a stable periodic orbit, within an invariant submanifold for the hybri…
In this paper, we present the mechatronic design of our Tactile Omnidirectional Robot Manipulator (TOMM), which is a dual arm wheeled humanoid robot with 6DoF on each arm, 4 omnidirectional wheels and 2 switchable end-effectors (1 DoF grippers and 12 DoF Hands). The main feature of TOMM is its arms…
Skin technology enabled a powerful way to sense the environment in robotic systems. It allows simplifying the formulation of safety tasks such as collision avoidance between the robot, the environment and surrounding objects. In this paper, a hierarchy policy based on tactile feedback is proposed to…
In this paper we describe how we realized event-based signaling for large scale artificial robotic skin. We developed a new algorithm for the event generation on multi-modal skin cells. The skin cells have two modes, the conventional data sampling mode and the event mode. A comprehensive performance…
This paper proposes an online tactile transfer learning strategy for discriminating objects through the surface texture properties via a robotic hand and an artificial robotic skin. The proposed method has the ability to autonomously select and exploit the previously learned multiple texture models
In this paper we propose event-based signaling for large-scale artificial robotic skin to reduce bandwidth requirements on data transmission and processing power. We use the send-on-delta principle to trigger the event generation only when tactile sensors are stimulated and transduce novel informati…