98 accepted papers
This letter presents a distributed trajectory planning method for multi-agent aerial tracking. The proposed method uses a Dynamic Buffered Voronoi Cell (DBVC) and a Dynamic Inter-Visibility Cell (DIVC) to formulate the distributed trajectory generation. Specifically, the DBVC and the DIVC are time-v…
We present EigenSafe, an operator-theoretic framework for safety assessment of learning-enabled stochastic systems. In many robotic applications, the dynamics are inherently stochastic due to factors such as sensing noise and environmental disturbances, and it is challenging for conventional methods…
This work presents a geometric backstepping controller for a variable-tilt omnidirectional multirotor that explicitly accounts for both servo and rotor dynamics. Considering actuator dynamics is essential for more effective and reliable operation, particularly during aggressive flight maneuvers or r…
Driving without considering the preferred separation distance from surrounding vehicles may cause discomfort for users. To address this limitation, we propose a planning framework that explicitly incorporates user preferences regarding the desired level of safe clearance from surrounding vehicles. W…
Maintaining the visibility of the target is one of the major objectives of aerial tracking missions. This paper proposes a target-visible trajectory planning pipeline using quadratic programming (QP). Our approach can handle various tracking settings, including 1) single- and dual-target following a…
Reinforcement learning (RL) has enabled robots to develop complex skills, but its success in image-based tasks often depends on effective representation learning. Prior works have primarily focused on 2D representations, often overlooking the inherent 3D geometric structure of the world, or have att…
Autonomous robotic systems should reason about resource control and its impact on subsequent maneuvers, especially when operating with limited energy budgets or restricted sensing. Learning-based control is effective in handling complex dynamics and represents the problem as a hybrid action space un…
This letter presents a distributed trajectory planning method for multi-agent aerial tracking. The proposed method uses a Dynamic Buffered Voronoi Cell (DBVC) and a Dynamic Inter-Visibility Cell (DIVC) to formulate the distributed trajectory generation. Specifically, the DBVC and the DIVC are time-v
Vision sensors are extensively used for localizing a robot's pose, particularly in environments where global localization tools such as GPS or motion capture systems are unavailable. In many visual navigation systems, localization is achieved by detecting and tracking visual features or landmarks, w
Multi-task reinforcement learning (MTRL) offers a promising approach to improve sample efficiency and generalization by training agents across multiple tasks, enabling knowledge sharing between them. However, applying MTRL to robotics remains challenging due to the high cost of collecting diverse ta
Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependency between states and skills or maximizing…
Aerial manipulation for safe physical interaction with their environments is gaining significant momentum in robotics research. In this paper, we present a disturbance-observer-based safety-critical control for a fully actuated aerial manipulator interacting with both static and dynamic structures.
Reinforcement learning (RL) has enabled robots to develop complex skills, but its success in image-based tasks often depends on effective representation learning. Prior works have primarily focused on 2D representations, often overlooking the inherent 3D geometric structure of the world, or have att
Aerial unperching of multirotors has received little attention as opposed to perching that has been investigated to elongate operation time. This study presents a new aerial robot capable of both perching and unperching autonomously on/from a ferromagnetic surface during flight, and a switching cont…
This letter presents a versatile trajectory planning pipeline for aerial tracking. The proposed tracker is capable of handling various chasing settings such as complex unstructured environments, crowded dynamic obstacles and multiple-target following. Among the entire pipeline, we focus on developin
Generative modeling of complex behaviors from labeled datasets has been a longstanding problem in decision-making. Unlike language or image generation, decision-making requires modeling actions – continuous-valued vectors that are multimodal in their distribution, potentially drawn from uncurated so…
Neural implicit representation has attracted attention in 3D reconstruction through various success cases. For further applications such as scene understanding or editing, several works have shown progress towards object-compositional reconstruction. Despite their superior performance in observed re
Learning-based depth estimation requires a large amount of real-world training data, which can be both expensive and time-consuming to acquire. As a result, utilizing fully-annotated synthetic data from virtual environments has emerged as a promising alternative. However, networks trained with synth
The use of aerial manipulators for safe and efficient physical interaction with their surrounding environments has been gaining attention within the aerial robotics research community. In this letter, we present an image-based time-varying force tracking controller for an aerial manipulator conducti
Chasing a dynamic target in a dense environment is one of the challenging applications of autonomous drones. The task requires multi-modal data, such as RGB and depth, to accomplish safe and robust maneuver. However, using different types of modalities can be difficult due to the limited capacity of
Control barrier functions (CBFs) are powerful tools for ensuring safety in controlled systems, commonly employed through the construction of a safety filter using quadratic programming (QP), known as CBF-QP. However, synthesizing a CBF specifically for the navigation tasks of mobile robots, where sa
Safety verification in motion planning is known to be computationally burdensome, despite its importance in robotics. In this paper, we investigate the behavior of safe receding horizon motion planners when the update interval becomes infinitesimal. By requiring the trajectory parameters to evolve c…
Recent curriculum Reinforcement Learning (RL) has shown notable progress in solving complex tasks by proposing sequences of surrogate tasks. However, the previous approaches often face challenges when they generate curriculum goals in a high-dimensional space. Thus, they usually rely on manually spe…
This paper presents a decentralized multi-agent trajectory planning (MATP) algorithm that guarantees to generate a safe, deadlock-free trajectory in an obstacle-rich environment under a limited communication range. The proposed algorithm utilizes a grid-based multi-agent path planning (MAPP) algorit…
While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to…
Thanks to the development of 2D keypoint detectors, monocular 3D human pose estimation (HPE) via 2D-to-3D uplifting approaches have achieved remarkable improvements. Still, monocular 3D HPE is a challenging problem due to the inherent depth ambiguities and occlusions. To handle this problem, many pr…
Reinforcement learning (RL) often faces the challenges of uninformed search problems where the agent should explore without access to the domain knowledge such as characteristics of the environment or external rewards. To tackle these challenges, this work proposes a new approach for curriculum RL c…
This study presents a globally defined dynamics for a conventional multirotor equipped with a single n\mathbf{-DOF}n\mathbf{-DOF} manipulator using modified Lagrangian dynamics. This enables the reformulation of entire dynamics directly on \text{SO}(3)\text{SO}(3) without exploiting any local coordi…
This study presents a new hardware design and control of a minimally actuated 5 control degrees of freedom (CDoF) quadrotor-based tiltrotor. The proposed tiltrotor possesses several characteristics distinct from those found in existing works, including: 1) minimal number of actuators for 5 CDoF, 2)…
Existence of symmetric objects, whose observation at different viewpoints can be identical, can deteriorate the performance of simultaneous localization and mapping (SLAM). This work proposes a system for robustly optimizing the pose of cameras and objects even in the presence of symmetric objects.…
Current reinforcement learning (RL) often suffers when solving a challenging exploration problem where the desired outcomes or high rewards are rarely observed. Even though curriculum RL, a framework that solves complex tasks by proposing a sequence of surrogate tasks, shows reasonable results, most…
Event cameras can show better performance than frame cameras in challenging scenarios, such as fast-moving environments or high-dynamic-range scenes. However, it is still difficult for event cameras to replace frame cameras in non-challenging normal scenarios. In order to leverage the advantages of
As previous representations for reinforcement learning cannot effectively incorporate a human-intuitive understanding of the 3D environment, they usually suffer from sub-optimal performances. In this paper, we present Semantic-aware Neural Radiance Fields for Reinforcement Learning (SNeRL), which jo…
The motion planning problem for multiple unstop-pable agents is of interest in many robotics applications, for example, autonomous traffic management for multiple fixed-wing aircraft. Unfortunately, many of the existing algorithms cannot provide safety for such agents, because they require the agent…
This study aims to design a motion/force controller for an aerial manipulator which guarantees the tracking of time-varying motion/force trajectories as well as the stability during the transition between free and contact motions. To this end, we model the force exerted on the end-effector as the Ke…
Monocular depth estimation plays a critical role in various computer vision and robotics applications such as localization, mapping, and 3D object detection. Recently, learning-based algorithms achieve huge success in depth estimation by training models with a large amount of data in a supervised ma…
Deep reinforcement learning has enabled robots to learn motor skills from environmental interactions with minimal to no prior knowledge. However, existing reinforcement learning algorithms assume an episodic setting, in which the agent resets to a fixed initial state distribution at the end of each
Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, $i.e.$, goal space and length of episodes,…
This letter presents a new online multi-agent trajectory planning algorithm that guarantees to generate safe, dynamically feasible trajectories in a cluttered environment. The proposed algorithm utilizes a linear safe corridor (LSC) to formulate the distributed trajectory optimization problem with o
Offline reinforcement learning (Offline RL) suffers from the innate distributional shift as it cannot interact with the physical environment during training. To alleviate such limitation, state-based offline RL leverages a learned dynamics model from the logged experience and augments the predicted…
Current reinforcement learning (RL) in robotics often experiences difficulty in generalizing to new downstream tasks due to the innate task-specific training paradigm. To alleviate it, unsupervised RL, a framework that pre-trains the agent in a task-agnostic manner without access to the task-specifi
This letter deals with the problem of an aerial manipulator pushing a movable structure. Contrary to physical interaction with a static structure, suitable consideration of the interacting force during the motion of the structure is required to stably perform this movable structure interaction. To a
This paper proposes an automatic and accuracy- enhanced extrinsic calibration method for 3D LiDARs with a range offset correction, which needs only an arbitrarily-shaped single planar board. One of the most exhaustive parts of existing LiDAR calibration procedures is to manually find target objects…
Guaranteeing safety of robotic systems is essential for a reliable operation. A funnel, the set of states that can be reached from the initial set of states in the presence of disturbances, can be used as a means of safety verification. This letter presents a tractable algorithm for computing the fu
Estimating a depth map from a single RGB image has been investigated widely for localization, mapping, and 3- dimensional object detection. Recent studies on a single-view depth estimation are mostly based on deep Convolutional neural Networks (ConvNets) which require a large amount of training data…
Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity…
This letter presents a new distributed multi-agent trajectory planning algorithm that generates safe, dynamically feasible trajectories considering the uncertainty of obstacles in dynamic environments. We extend the relative safe flight corridor (RSFC) presented in previous work to replace time-vari
Automation of excavation tasks requires real-time trajectory planning satisfying various constraints. To guarantee both constraint feasibility and real-time trajectory re-plannability, we present an integrated framework for real-time optimization-based trajectory planning of a hydraulic excavator. T…
Contrast maximization is an event camera application that can estimate angular velocity, depth, and optical-flow using a subset of events observed in a temporal window. In the estimation of rotational motion, we can compute the angular position by integrating the angular velocity. However, the accum
Motion planners for mobile robots in unknown environments face the challenge of simultaneously maintaining both robustness against unmodeled uncertainties and persistent feasibility of the trajectory-finding problem. That is, while dealing with uncertainties, a motion planner must update its traject…
In this paper, an autonomous aerial manipulation task of pulling a plug out of an electric socket is conducted, where maintaining the stability and robustness is challenging due to sudden disappearance of a large interaction force. The abrupt change in the dynamical model before and after the separa…
Autonomous aerial videography is a challenging task, which involves collision avoidance against obstacles and visibility guaranteed target tracking in unstructured environments. In this paper, we organize a two micro aerial vehicle (MAV) team, which consists of a target agent responsible for a speci…
Visual navigation has been widely used for state estimation of micro aerial vehicles (MAVs). For stable visual navigation, MAVs should generate perception-aware paths which guarantee enough visible landmarks. Many previous works on perception-aware path planning focused on sampling-based planners. H…
Existing studies for environment interaction with an aerial robot have been focused on interaction with static surroundings. However, to fully explore the concept of an aerial manipulation, interaction with moving structures should also be considered. In this paper, a multirotor-based aerial manipul…
This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background i…
We propose an efficient edge-based stereo visual odometry (VO) using multiple quadtrees created according to image gradient orientations. To characterize edges, we classify them into eight orientation groups according to their image gradient directions. Using the edge groups, we construct eight quad…
This paper presents a new efficient algorithm which guarantees a solution for a class of multi-agent trajectory planning problems in obstacle-dense environments. Our algorithm combines the advantages of both grid-based and optimization-based approaches, and generates safe, dynamically feasible traje…
In this letter, we introduce a new quadrotor fail-safe flight solution that can perform the same four controllable degrees-of-freedom flight as a standard multirotor even when a single thruster fails. The new solution employs a novel multirotor platform known as the T <sup xmlns:mml="http://www.w3.o
This work suggests an integrated approach for a drone (or multirotor) to perform an autonomous videography task in a 3-D obstacle environment by following a moving object. The proposed system includes 1) a target motion prediction module which can be applied to dense environments and 2) a hierarchic…
Deep neural networks facilitate visuosensory inputs for robotic systems. However, the features encoded in a network without specific constraints have little physical meaning. In this research, we add constraints on the network so that the trained features are forced to represent the actual twist coo
Detection of moving objects is an essential capability in dealing with dynamic environments. Most moving object detection algorithms have been designed for color images without depth. For robotic navigation where real-time RGBD data is often readily available, utilization of the depth information wo…
This letter deals with the problem of multiple robots working together to explore and gather at the global maximum of the unknown field. Given noisy sensor measurements obtained at the location of robots with no prior knowledge about the environmental map, Gaussian process regression can be an effic
In micro aerial vehicle (MAV) operations, the success of a mission is highly dependent on navigation performance, which has raised recent interests on navigation-aware path planning. One of the challenges lies in that optimal motions for successful navigation and the designated mission are often dif…
This letter presents real-time optimization algorithms on trajectory generation and control for a multi-rotor with a suspended load. Since the load is suspended through a cable without any actuator, movement of the load must be controlled via maneuvers of the multi-rotor, which brings about difficul
Flapping-wing micro air vehicles (FWMAVs) inspired by the nature are interesting flight platforms due to their efficiency, concealment and agility. However, most studies have been conducted in indoor environments where external disturbance is excluded because these FWMAVs are susceptible to disturba
Planning a trajectory with guaranteed safety is a core part for a risk-free flight of a multirotor. If a trajectory planner only aims to ensure safety, it may generate trajectories which overly bypass risky regions and prevent the system from achieving specific missions. This work presents a robust…
In this paper, we introduce a cargo transportation method with a new type of multi-rotor UAV platform known as T3-multirotor, to achieve stable and constant flight performance regardless of the type of cargo attached to the fuselage. The T3-multirotor, which consists of the `Thrust Generating Part'…
This paper presents a new trajectory planning method for multiple quadrotors in obstacle-dense environments. We suggest a relative safe flight corridor (RSFC) to model safe region between a pair of agents, and it is used to generate linear constraints for inter-collision avoidance by utilizing the c…
Applying deep reinforcement learning to physical systems, as opposed to learning in simulation, presents additional challenges in terms of sample efficiency and safety. Collecting large amounts of hardware demonstration data is time-consuming and the exploratory behavior of reinforcement learning al…
Parametric dynamic movement primitives (PDMPs) are powerful motion representation algorithms, which encode multiple demonstrations and generalize them. As an online trajectory from PDMPs emulates the provided demonstrations, managing the safety guarantee of the demonstrations for a given scenario is
This study addresses an operation of unmanned aerial vehicles (UAVs) in a network environment where there is time-varying network delay. The network delay entails undesirable effects on the stability of the UAV control system due to delayed state feedback and outdated control input. Although several…
This work deals with a moving target chasing mission of an aerial vehicle equipped with a vision sensor in a cluttered environment. In contrast to obstacle-free or sparse environments, the chaser should be able to handle collision and occlusion together with flight efficiency. In order to tackle the…
Visual servoing, i.e. control with visual information, is a valuable capability in many robotic applications. In particular, position based visual servoing (PBVS) estimates position information from the observed image to generate visual servo control. However, the estimation of the position of an un…
This paper presents planning and control algorithms for a multi-rotor with a suspended load. The suspended load cannot be controlled easily by the multi-rotor due to severe dynamic coupling between them. Difficulties are exacerbated by under-actuated, highly nonlinear nature of multi-rotor dynamics.…
In the paper, we propose a robust real-time visual odometry in dynamic environments via rigid-motion model updated by scene flow. The proposed algorithm consists of spatial motion segmentation and temporal motion tracking. The spatial segmentation first generates several motion hypotheses by using a…
Ensuring safety in trajectory planning of multirotor systems is an essential element for risk-free operation. Even if the generated trajectory is known to be safe in the planning phase, unknown disturbance during an actual operation can lead to a dangerous situation. This paper proposes safety-guara…
This paper presents a motion planning approach for an aerial pick-and-place task where an aerial manipulator is supposed to pick up or place an object at locations specified as way-points. In particular, we focus on situations where such way-point constraints are imposed on certain partial state var…
This letter presents a viable framework that integrates control, estimation of unknown payload, safety management, and obstacle avoidance for cooperative transportation in unknown environments using multiple aerial manipulators. Unlike existing approaches for cooperative manipulation based on force
This paper presents a new design of multirotor, named as `Tilting Thruster Type' (T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> )-multirotor. The new platform is equipped with mechanically separated thrusters, which can take any fuselage pos
This paper presents a new design of multirotor, named as `Tilting Thruster Type' (T3)-multirotor. The new platform is equipped with mechanically separated thrusters, which can take any fuselage posture within a specified range regardless of any direction of translational acceleration. A specially de…
This paper proposes an edge-based robust RGB-D visual odometry (VO) using 2-D edge divergence minimization. Our approach focuses on enabling the VO to operate in more general environments subject to low texture and changing brightness, by employing image edge regions and their image gradient vectors…
We propose a novel approach to estimate the three degrees of freedom (DoF) drift-free rotational motion of an RGB-D camera from only a single line and plane in the Manhattan world (MW). Previous approaches exploit the surface normal vectors and vanishing points to achieve accurate 3-DoF rotation est…
This paper presents an approach to learn and generalize robotic skills from a demonstration using deep reinforcement learning (deep RL). Dynamic Movement Primitives (DMPs) formulate a nonlinear differential equation and produce the observed movement from a demonstration. However, it is hard to gener…
Flapping-wing micro air vehicles (FWMAVs) become promising research platforms due to their advantages such as various maneuverability, and concealment. However, unsteady flow at low Reynolds number around the wings makes their dynamics time-varying and highly non-linear. It makes autonomous flight o…
We propose a new formulation for including orthogonal planar features as a global model into a linear SLAM approach based on sequential Bayesian filtering. Previous planar SLAM algorithms estimate the camera poses and multiple landmark planes in a pose graph optimization. However, since it is formul…
We present a low-drift visual odometry algorithm that separately estimates rotational and translational motion from lines, planes, and points found in RGB-D images. Previous methods estimate drift-free rotational motion from structural regularities to reduce drift in the rotation estimate, which is…
This paper investigates a multi-rotor with a suspended load in perspectives of 1) real-time path planning, 2) obstacle avoidance, and 3) transportation of a suspended object. A suspended load cannot be controlled with conventional controllers designed for nominal multi-rotors due to the dynamic coup…
This paper describes a vision-based target tracking method for a skid-steer vehicle. With the development of deep reinforcement learning, many researchers have tried to generate an end-to-end policy to control the mobile robot from a raw pixel image data. However, the action in most research only co…
This paper concentrates on design of a vision-based guidance command for aerial manipulation of a cylindrical object, using a stochastic model predictive approach. We first develop an image-based cylinder detection algorithm that utilizes a geometric characteristic of perspectively projected circles…
This paper presents a practical swing-angle estimation method for slung load operations of the multi-rotor unmanned aerial vehicle (UAV), which is essential to maintain the safety during the operation. In order not to rely on extra sensors for monitoring the swing angle, the proposed method in this…
Hovering flight is one of the valuable capabilities for unmanned aerial vehicles (UAVs) to perform various missions. It is easily attainable by rotary-wing aircrafts such as a helicopter and multirotor. This flight maneuver, however, remains a significant challenge to a flapping wing micro air vehic…
Aerial manipulation tasks necessitate a reliable trajectory planning algorithms to perform complicated tasks. This paper provides a method of developing the locally optimal trajectory for aerial manipulation in constrained environments. We first show differential flatness of the aerial manipulation…
This paper presents a motion planning approach for cooperative transportation using aerial robots. We describe a framework based on Parametric Dynamic Movement Primitives (PDMPs) for coordinating multiple aerial robots and their manipulators quickly in an environment cluttered with obstacles. In ord…
We present an illumination-robust visual localization algorithm for Astrobee, a free-flying robot designed to autonomously navigate on the International Space Station (ISS). Astrobee localizes with a monocular camera and a pre-built sparse map composed of natural visual features. Astrobee must perfo…
In this paper, we design an autonomous flight controller for height regulation and bang-bang controller for directional control of a light-weight flapping-wing micro air vehicle (FWMAV) with limited payload. We also present autonomous vision-based target tracking for a FWMAV equipped with a low-cost…
This paper is about opening and closing an unknown drawer using an aerial manipulator. To accommodate practical applications, it is assumed that the direction of motion and mechanical properties of the drawer are not given beforehand. A multirotor combined with a robotic arm is used for the manipula…
This paper presents planning and control of multiple aerial manipulators for cooperative transportation. Individual aerial manipulators which consist of a hexacopter and 2-DOF robotic arm are controlled by an augmented adaptive sliding mode controller based on a closed-chain robot dynamics. The desi…
Sensitivity to illumination conditions poses a challenge when utilizing visual odometry (VO) in various applications. To make VO robust with respect to illumination conditions, they need to be considered explicitly. In this paper, we propose a direct visual odometry method which can handle illuminat…