38 accepted papers
Real-world contact-rich manipulation demands robots to perceive temporal tactile feedback, capture subtle surface deformations, and reason about object properties and force dynamics. Although optical tactile sensors are uniquely capable of providing such rich information, existing tactile datasets a…
This paper explores the challenges of integrating tactile sensing into intelligent systems for multimodal reasoning, particularly in enabling commonsense reasoning about the open-ended physical world. We identify two key challenges: modality discrepancy, where existing touch-language models often tr
Vision-based tactile sensors (VBTS) recover high-resolution contact geometry but typically rely on opaque elastomer layers that prevent visual transparency, while RGB-D cameras provide global depth perception yet degrade significantly at close range. To address this limitation, we present TransTac, …
This paper presents a novel multi-mode bionic robotic hand. Its bionic finger (BIF) ingeniously combines a magnetic-silica-gel skin with a rigid skeletal framework and integrates a vacuum suction cup at the fingertip. This design enables the bionic manipulator to execute multiple grasping modes, nam
Visuo-tactile sensors aim to emulate human tactile perception, enabling robots to precisely understand and manipulate objects. Over time, numerous meticulously designed visuo-tactile sensors have been integrated into robotic systems, aiding in completing various tasks. However, the distinct data cha…
Mobile manipulation typically entails the base for mobility, the arm for accurate manipulation, and the camera for perception. The principle of Distant Mobility, Close Grasping(DMCG) is essential for holistic control. We propose Embod-ied Holistic Control for Mobile Manipulation(EHC-MM) with the emb
Force estimation is the core indicator for evaluating the performance of tactile sensors, and it is also the key technical path to achieving precise force feedback mechanisms. This study proposes a design method for a visual tactile sensor (VBTS) that integrates a magnetic perception mechanism, and
This paper presents a novel Bionic Soft Finger (BSF) that aims to overcome the limitations of conventional rigid manipulators in terms of adaptability and safety, as well as the challenges faced by soft hands regarding carrying capacity and stability. The BSF design uses a hybrid variable stiffness…
In this paper, we propose an end-to-end no-reference image quality assessment (NR-IQA) method that perceives multilayer representations from low-level to high-level stages. First, multi-layer representations (MR) of the distorted images are extracted from different layers of multiple feature extract…
The unauthorized use of personal data for commercial purposes and the covert acquisition of private data for training machine learning models continue to raise concerns. To address these issues researchers have proposed availability attacks that aim to render data unexploitable. However many availab…
We present RobotGPT, an innovative decision framework for robotic manipulation that prioritizes stability and safety. The execution code generated by ChatGPT cannot guarantee the stability and safety of the system. ChatGPT may provide different answers for the same task, leading to unpredictability.
Drawing inspiration from human fine tactile and proprioceptive kinaesthetic sensing pathways, we propose a soft magnetic skin (m-skin) with multimodal sensing functions integrated into the anthropomorphic robotic finger. This paper mainly explores the magnetic tactile sensor's structural design, per
This letter introduces a switching control method for a soft rehabilitation glove using pneumatic bellows actuators. The design and manufacture are introduced for the soft rehabilitation glove. A dynamic nonlinear system is established for deflection angles of finger joints when using the soft rehab
3D human reconstruction from RGB images achieves decent results in good weather conditions but degrades dramatically in rough weather. Complementary, mmWave radars have been employed to reconstruct 3D human joints and meshes in rough weather. However, combining RGB and mmWave signals for robust all-…
Robust and efficient grasping of different objects is still an open problem due to the difficulty of integrating multidisciplinary knowledge such as gripper ontology design, perception, control, and learning. In recent years, learning-based methods have achieved excellent results in grasping various…
Global co-occurrence information is the primary source of structural information on multilingual corpora, and we find that analogical/parallel compound words across languages have similar co-occurrence counts/frequencies (normalized) giving weak but stable self-supervision for cross-lingual transfer…
In multilingual pre-training with the objective of MLM (masked language modeling) on multiple monolingual corpora, multilingual models only learn cross-linguality implicitly from isomorphic spaces formed by overlapping different language spaces due to the lack of explicit cross-lingual forward pass.…
Current state-of-the-art semantic segmentation methods usually require high computational resources for accurate segmentation. Knowledge distillation has been one promising way to achieve a good trade-off between accuracy and efficiency. However, current distillation methods focus on transferring th…
This letter proposes a vision-based tactile sensor named TIRgel, leveraging visual integration to simplify the sensing system. First, the sensor achieves conversion of visual and tactile modality via focus adjustment. Under far-focus imaging, the camera can observe external environments; Under near-
Simulation is widely applied in robotics research to save time and resources. There have been several works to simulate optical tactile sensors that leverage either a smoothing method or Finite Element Method (FEM). However, elastomer deformation physics is not considered in the former method, where
In this letter, we propose a C-shaped bidirectional stiffness joint for an anthropomorphic hand. The spring steel piece is used to connect the knuckles with inner concave C-shape as a rotational joint. The finger is bent by the tendon driven with low stiffness and reset by its own elasticity. The re
Skill generalization in force fields is quite challenging and has not been fully investigated yet in the domain of robot learning. In this letter, we present a novel adaptation strategy that allows a robot to generalize the learned skill to deal with new task conditions with different force fields.
In sequence modeling, certain tokens are usually less ambiguous than others, and representations of these tokens require fewer refinements for disambiguation. However, given the nature of attention-based models like Transformer and UT (universal transformer), all tokens are equally processed towards…
During postharvest storage, fruit firmness usually decreases due to respiration and bruise, the former of which indicates the fruit ripeness while the latter negatively influence consumers' taste preference. This paper presents a portable and low-cost device using vision-based tactile information to…
A Multilingual model relies on language encodings to identify input languages because the multilingual model has to distinguish between the input and output languages or among all the languages for cross-lingual tasks. Furthermore, we find that language encodings potentially refine multiple morpholo…
Translation quality can be improved by global information from the required target sentence because the decoder can understand both past and future information. However, the model needs additional cost to produce and consider such global information. In this work, to inject global information but al…
UNMT tackles translation on monolingual corpora in two required languages. Since there is no explicitly cross-lingual signal, pre-training and synthetic sentence pairs are significant to the success of UNMT. In this work, we empirically study the core training procedure of UNMT to analyze the synthe…
This paper proposes a method of fabric structure defect detection based on tactile information. Different from traditional visual-based detection methods, the proposed method uses a tactile sensor to attain the information of fabric. The advantage of using tactile information is to avoid different i…
In this paper, we proposed a novel hybrid active contour driven by double-weighted signed pressure force method for image segmentation. First, the Legendre polynomials and global information are integrated into the signed pressure force (SPF) function and a coefficient is applied to weight the effec…
Unsupervised image-to-image translation is a central task in computer vision. Current translation frameworks will abandon the discriminator once the training process is completed. This paper contends a novel role of the discriminator by reusing it for encoding the images of the target domain. The pr…
Different functional areas of the human brain play different roles in brain activity, which has not been paid sufficient research attention in the brain-computer interface (BCI) field. This paper presents a new approach for electroencephalography (EEG) classification that applies attention-based tra…
In this paper, a novel robotic grasping system is established to automatically pick up objects in cluttered scenes. A composite robotic hand composed of a suction cup and a gripper is designed for grasping the object stably. The suction cup is used for lifting the object from the clutter first and t…
In this work, we investigate the lifelong learning problem from the viewpoint of heterogeneous multi-modal fusion. The main challenges come from the fact that the common representation between heterogeneous modalities should be persistently learned and the learned classifier for each multi-modal tas…
In this paper, we propose an end-to-end grasp evaluation model to address the challenging problem of localizing robot grasp configurations directly from the point cloud. Compared to recent grasp evaluation metrics that are based on handcrafted depth features and a convolutional neural network (CNN),…
In this paper, we present TeachNet, a novel neural network architecture for intuitive and markerless vision-based teleoperation of dexterous robotic hands. Robot joint angles are directly generated from depth images of the human hand that produce visually similar robot hand poses in an end-to-end fa…
Humans' fingertips can perceive not only the magnitude and the direction of force but also the texture of object. When we grasp an object, the surface texture sensing of the fingertip helps us recognize the object and the force feeling that is parallel to the skin helps us grasp stably. Focusing on…
The robotic grasp detection is a great challenge in the area of robotics. Previous work mainly employs the visual approaches to solve this problem. In this paper, a hybrid deep architecture combining the visual and tactile sensing for robotic grasp detection is proposed. We have demonstrated that th…
In this paper, we exploit the intrinsic relation between different adjective labels and develop a novel multilabel dictionary learning and sparse coding method which is improved by introducing the structured output association information. Such a method makes use of the label correlation information…