AIConfPaper
← Search

Michael Neunert

23 accepted papers

2024

Mastering Stacking of Diverse Shapes with Large-Scale Iterative Reinforcement Learning on Real Robots

Thomas Lampe, Abbas Abdolmaleki, Sarah Bechtle, Sandy H. Huang, Jost Tobias Springenberg, Michael Bloesch +9
ICRA 2024poster

Reinforcement learning solely from an agent’s self-generated data is often believed to be infeasible for learning on real robots, due to the amount of data needed. However, if done right, agents learning from real data can be surprisingly efficient through re-using previously collected sub-optimal d…

Cited by 6Source ↗Scholar ↗
2024

The Design of the Barkour Benchmark for Robot Agility

Wenhao Yu, Ken Caluwaerts, Atil Iscen, J. Chase Kew, Tingnan Zhang, Daniel Freeman +33
IROS 2024poster

In this paper, we describe the design of the Barkour benchmark for measuring robot agility in navigating complex environments. Despite the growing interest in developing agile robot locomotion skills, the field lacks systematic benchmarks to measure the performance of robotic control systems and har…

Cited by 1Source ↗Scholar ↗
2021

A Constrained Multi-Objective Reinforcement Learning Framework

Sandy Huang, Abbas Abdolmaleki, Giulia Vezzani, Philemon Brakel, Daniel J Mankowitz, Michael Neunert +5
CoRL 2021poster

Many real-world problems, especially in robotics, require that reinforcement learning (RL) agents learn policies that not only maximize an environment reward, but also satisfy constraints. We propose a high-level framework for solving such problems, that treats the environment reward and costs as se…

Cited by 34Source ↗Scholar ↗
2021

Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes

Alex X. Lee, Coline Manon Devin, Yuxiang Zhou, Thomas Lampe, Konstantinos Bousmalis, Jost Tobias Springenberg +15
CoRL 2021poster

We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategies beyond a simple “pick-and-place” solution. Our method is a reinforcement learning (RL) approach combined with vision-b…

Cited by 118Source ↗code ↗Scholar ↗
2021

Data-efficient Hindsight Off-policy Option Learning

Markus Wulfmeier, Dushyant Rao, Roland Hafner, Thomas Lampe, Abbas Abdolmaleki, Tim Hertweck +5
ICML 2021spotlight

We introduce Hindsight Off-policy Options (HO2), a data-efficient option learning algorithm. Given any trajectory, HO2 infers likely option choices and backpropagates through the dynamic programming inference procedure to robustly train all policy components off-policy and end-to-end. The approach o…

Cited by 52Source ↗PDF ↗Scholar ↗
2020

A distributional view on multi-objective policy optimization

Abbas Abdolmaleki, Sandy Huang, Leonard Hasenclever, Michael Neunert, Francis Song, Martina Zambelli +4
ICML 2020poster

Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for practitioners to express numerical preferences over objectives in their native units. In this paper we propose a novel a…

Cited by 97Source ↗PDF ↗code ↗Scholar ↗
2020

Compositional Transfer in Hierarchical Reinforcement Learning

Markus Wulfmeier, Abbas Abdolmaleki, Roland Hafner, Jost Tobias Springenberg, Michael Neunert, Noah Siegel +4
RSS 2020poster

The successful application of general reinforcement learning algorithms to real-world robotics applications is often limited by their high data requirements. We introduce Regularized Hierarchical Policy Optimization (RHPO) to improve data-efficiency for domains with multiple dominant tasks and ultim…

Cited by 41Source ↗PDF ↗Scholar ↗
2020

Keep Doing What Worked: Behavior Modelling Priors for Offline Reinforcement Learning

Noah Siegel, Jost Tobias Springenberg, Felix Berkenkamp, Abbas Abdolmaleki, Michael Neunert, Thomas Lampe +3
ICLR 2020poster

Off-policy reinforcement learning algorithms promise to be applicable in settings where only a fixed data-set (batch) of environment interactions is available and no new experience can be acquired. This property makes these algorithms appealing for real world problems such as robot control. In pract…

Cited by 330Source ↗Scholar ↗
2020

Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion

Roland Hafner, Tim Hertweck, Philipp Kloeppner, Michael Bloesch, Michael Neunert, Markus Wulfmeier +3
CoRL 2020

Modern Reinforcement Learning (RL) algorithms promise to solve difficult motor control problems directly from raw sensory inputs. Their attraction is due in part to the fact that they can represent a general class of methods that allow to learn a solution with a reasonably set reward and minimal pri

Cited by 0Source ↗PDF ↗Scholar ↗
2019

Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics

Michael Neunert, Abbas Abdolmaleki, Markus Wulfmeier, Thomas Lampe, Tobias Springenberg, Roland Hafner +4
CoRL 2019

Many real-world control problems involve both discrete decision variables – such as the choice of control modes, gear switching or digital outputs – as well as continuous decision variables – such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding opti

Cited by 0Source ↗PDF ↗Scholar ↗
2019

Imagined Value Gradients: Model-Based Policy Optimization with Tranferable Latent Dynamics Models

Arunkumar Byravan, Jost Tobias Springenberg, Abbas Abdolmaleki, Roland Hafner, Michael Neunert, Thomas Lampe +3
CoRL 2019

Humans are masters at quickly learning many complex tasks, relying on an approximate understanding of the dynamics of their environments. In much the same way, we would like our learning agents to quickly adapt to new tasks. In this paper, we explore how model-based Reinforcement Learning (RL) can f

Cited by 0Source ↗PDF ↗Scholar ↗
2019

Simultaneously Learning Vision and Feature-Based Control Policies for Real-World Ball-In-A-Cup

Devin Schwab, Jost Tobias Springenberg, Murilo Fernandes Martins, Michael Neunert, Thomas Lampe, Abbas Abdolmaleki +4
RSS 2019poster

We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particul…

Cited by 31Source ↗PDF ↗Scholar ↗
2018

A Family of Iterative Gauss-Newton Shooting Methods for Nonlinear Optimal Control

Markus Giftthaler, Michael Neunert, Markus Stäuble, Jonas Buchli, Moritz Diehl
IROS 2018poster

This paper introduces a family of iterative algorithms for unconstrained nonlinear optimal control. We generalize the well-known iLQR algorithm to different multiple shooting variants, combining advantages like straightforward initialization and a closed-loop forward integration. All algorithms have…

Cited by 117Source ↗Scholar ↗
2018

Learning by Playing Solving Sparse Reward Tasks from Scratch

Martin Riedmiller, Roland Hafner, Thomas Lampe, Michael Neunert, Jonas Degrave, Tom Wiele +3
ICML 2018oral

We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary ta…

Cited by 549Source ↗PDF ↗code ↗Scholar ↗
2018

Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds

Michael Neunert, Markus Stäuble, Markus Giftthaler, Carmine Dario Bellicoso, Jan Carius, Christian Gehring +2
RA-L 2018

In this letter, we present a whole-body nonlinear model predictive control approach for rigid body systems subject to contacts. We use a full-dynamic system model which also includes explicit contact dynamics. Therefore, contact locations, sequences, and timings are not prespecified but optimized by

Cited by 292Source ↗Scholar ↗
2017

An efficient optimal planning and control framework for quadrupedal locomotion

Farbod Farshidian, Michael Neunert, Alexander W. Winkler, Gonzalo Rey, Jonas Buchli
ICRA 2017poster

In this paper, we present an efficient Dynamic Programing framework for optimal planning and control of legged robots. First we formulate this problem as an optimal control problem for switched systems. Then we propose a multi-level optimization approach to find the optimal switching times and the o…

Cited by 195Source ↗Scholar ↗
2017

Fast Trajectory Optimization for Legged Robots Using Vertex-Based ZMP Constraints

Alexander W. Winkler, Farbod Farshidian, Diego Pardo, Michael Neunert, Jonas Buchli
RA-L 2017

This letter combines the fast zero-moment-point approaches that work well in practice with the broader range of capabilities of a trajectory optimization formulation, by optimizing over body motion, footholds, and center of pressure simultaneously. We introduce a vertex-based representation of the s

Cited by 83Source ↗Scholar ↗
2017

Hybrid direct collocation and control in the constraint-consistent subspace for dynamic legged robot locomotion

Diego Pardo, Michael Neunert, Alexander Winkler, Ruben Grandia, Jonas Buchli
RSS 2017poster

In this paper, we present an algorithm for planning and control of legged robot locomotion. Given the desired contact sequence, this method generates gaits and dynamic motions for legged robots without resorting to simplified stability criteria. The method uses direct collocation for searching for s…

Cited by 47Source ↗PDF ↗Scholar ↗
2017

Online walking motion and foothold optimization for quadruped locomotion

Alexander W. Winkler, Farbod Farshidian, Michael Neunert, Diego Pardo, Jonas Buchli
ICRA 2017poster

We present an algorithm that generates walking motions for quadruped robots without the use of an explicit footstep planner by simultaneously optimizing over both the Center of Mass (CoM) trajectory and the footholds. Feasibility is achieved by imposing stability constraints on the CoM related to th…

Cited by 66Source ↗Scholar ↗
2017

Trajectory Optimization Through Contacts and Automatic Gait Discovery for Quadrupeds

Michael Neunert, Farbod Farshidian, Alexander W. Winkler, Jonas Buchli
RA-L 2017

In this letter, we present a trajectory optimization framework for whole-body motion planning through contacts. We demonstrate how the proposed approach can be applied to automatically discover different gaits and dynamic motions on a quadruped robot. In contrast to most previous methods, we do not

Cited by 138Source ↗Scholar ↗
2016

Evaluating Direct Transcription and Nonlinear Optimization Methods for Robot Motion Planning

Diego Pardo, Lukas Möller, Michael Neunert, Alexander W. Winkler, Jonas Buchli
RA-L 2016

This letter studies existing direct transcription methods for trajectory optimization applied to robot motion planning. There are diverse alternatives for the implementation of direct transcription. In this study, we analyze the effects of such alternatives when solving a robotics problem. Different

Cited by 78Source ↗Scholar ↗
2016

Fast nonlinear Model Predictive Control for unified trajectory optimization and tracking

Michael Neunert, Cedric de Crousaz, Fadri Furrer, Mina Kamel, Farbod Farshidian, Roland Siegwart +1
ICRA 2016

This paper presents a framework for real-time, full-state feedback, unconstrained, nonlinear model predictive control that combines trajectory optimization and tracking control in a single, unified approach. The proposed method uses an iterative optimal control algorithm, namely Sequential Linear Qu

Cited by 219Source ↗Scholar ↗
2015

Unified motion control for dynamic quadrotor maneuvers demonstrated on slung load and rotor failure tasks

Cédric de Crousaz, Farbod Farshidian, Michael Neunert, Jonas Buchli
ICRA 2015poster

In recent years impressive results have been presented illustrating the potential of quadrotors to solve challenging tasks. Generally, the derivation of the controllers involve complex analytical manipulation of the dynamics and are very specific to the task at hand. In addition, most approaches con…

Cited by 63Source ↗Scholar ↗