20 accepted papers
Reinforcement Learning (RL) in partially observable environments poses significant challenges due to the complexity of learning under uncertainty. While additional information, such as that available in simulations, can enhance training, effectively leveraging it remains an open problem. To address…
Due to practical constraints such as partial observability and limited communication, Centralized Training with Decentralized Execution (CTDE) has become the dominant paradigm in cooperative Multi-Agent Reinforcement Learning (MARL). However, existing CTDE methods often underutilize centralized tra…
Multi-step motion prediction for continuum robots is difficult, especially under actuation distribution shift, where error accumulation can distort the predicted steady response and destabilize rollouts. This paper introduces a hybrid equilibrium-anchored plus residual-learning framework for a tendo…
Cooperative Multi-Agent Reinforcement Learning (MARL) has become a critical tool for addressing complex real-world problems. However, off-policy MARL methods, which rely on joint Q-functions, face significant scalability challenges due to the exponentially growing joint action space. In this work,…
An increasing number of underwater robots inspired by Carangidae are developed, which is characterized by high efficiency and flexibility. However, estimating the swimming direction of these robotic fish is challenging due to the constant swinging of the head during movement, which complicates preci…
Spherical robots have many desirable traits when designing mass efficient systems interacted with unstructured terrain. In this letter, we propose a hybrid bionic spherical robot based on the morphological properties of sea urchins and the movement characteristics of tumbleweeds. This robot enables
Multi-agent policy gradient methods in centralized training with decentralized execution recently witnessed many progresses. During centralized training, multi-agent credit assignment is crucial, which can substantially promote learning performance. However, explicit multi-agent credit assignment in…
Accurate pose estimation and localization technology is always a challenge for small underwater robots, since the underwater lighting conditions could limit the use of cameras while the cramped environments restrict the use of sonars. In nature, some fishes perceive other creatures by sensing the we
In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the kn…
It has long been proposed that animals flying in the air and swimming in the water could extract energy from neighbour-induced flows. A large number of mechanisms have been proposed to explain whether, and if so how, animals can save energy by moving in two-dimensional (2D) formations-individuals sw
Value decomposition recently injects vigorous vitality into multi-agent actor-critic methods. However, existing decomposed actor-critic methods cannot guarantee the convergence of global optimum. In this paper, we present a novel multi-agent actor-critic method, FOP, which can factorize the optimal…
Value function decomposition (VFD) methods under the popular paradigm of centralized training and decentralized execution (CTDE) have promoted multi-agent reinforcement learning progress. However, existing VFD methods proceed from a group's value function decomposition to only solve cooperative task…
Novel robotic grippers have captured increasing interests recently because of their abilities to adapt to varieties of circumstances and their powerful functionalities. Differing from traditional gripper with mechanical components-made fingers, novel robotic grippers are typically made of novel stru…
Underwater communication is extremely challenging for small underwater robots which typically have stringent power and size constraints. In our previous work, we developed an artificial electrocommunication system which could be an alternative for the communication of small underwater robots. This p…
For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs’ vast view scope. However, its limitations on flight altitude prevent the U…
Lateral line system (LLS) is a sensory organ system which serves functions in varieties of flow-relative fish behaviors. Inspired by excellent performances of LLS in fish behaviors, multiple artificial lateral line systems (ALLSs) have been designed and applied to promote underwater robot technology…
Underwater communication is particularly challenging for small submarine robots that have limited power and size constraints. Inspired by weakly electric fish, a novel electric current communication (termed electrocommunication) system has been developed for small underwater robots in our previous s…
Artificial lateral line has been drawing an increasing attention recently for its potential applications in robotics. Experiments are usually conducted with a bioinspired robot in a controlled environment, where the sensing platform is held stationary or slowly driven with a simple linear motion. In
Fish possesses a unique sensory organ called the lateral line. The lateral line provides fish with flow-related information. It is accepted that fish can use the lateral line to sense states of its neighbours in schooling behaviors. In this study, we investigate how a focal robotic fish senses the s…
Communication is challenging for underwater robots. This paper presents the first research into developing an underwater electric current communication system and integrating the system into a small robotic fish. It is notable for its potential for a group of underwater robots communicating within a…