← Search

Ingmar Posner

67 accepted papers

2026

Grasp-MPC: Closed-Loop Visual Grasping Via Value-Guided Model Predictive Control

ICRA 2026poster

Grasping of diverse objects in unstructured environments remains a significant challenge. Open-loop grasping methods, effective in controlled settings, struggle in cluttered environments. Grasp prediction errors and object pose changes during grasping are the main causes of failure. In contrast, clo…

2026

Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spaces

ICML 2026poster

We analyze the Bayesian regret of the Gaussian process posterior sampling reinforcement learning (GP-PSRL) algorithm. Posterior sampling is an effective heuristic for decision-making under uncertainty that has been used to develop successful algorithms for a variety of continuous control problems. H…

Cited by 0SourceScholar
2026

TactGen: Tactile Sensory Data Generation via Zero-Shot Sim-to-Real Transfer (Abstract Reprint)

AAAI 2026technical

Recent advances in machine learning have driven a step-change in robot perception with modalities such as vision, where large amounts of training data are readily available or cheap to collect. However, in tactile perception, the relatively high cost of data collection still largely impedes the adop

Cited by 0SourcePDFScholar
2026

XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning

ICLR 2026poster

Sample efficiency is a central property of effective deep reinforcement learning algorithms. Recent work has improved this through added complexity, such as larger models, exotic network architectures, and more complex algorithms, which are typically motivated purely by empirical performance. We tak…

Cited by 0SourcecodeScholar
2025

An End-to-End Framework for Modeling Pneumatic Soft Robots Based on Differentiable Finite Element Methods

RA-L 2025

Soft robots present significant modelling challenges due to their non-linearity, complex dynamics and potentially intricate geometries. These difficulties in accurate system identification and dynamics modelling limit their applications in precise robotics tasks. Prior modelling approaches typically

Cited by 0SourceScholar
2025

COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping

CoRL 2025poster

This paper addresses the challenge of occluded robot grasping, i.e. grasping in situations where the desired grasp poses are kinematically infeasible due to environmental constraints such as surface collisions. Existing RL methods struggle with task complexity, and collecting expert demonstrations i…

Cited by 0SourceScholar
2025

D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation

CoRL 2025poster

Mastering deformable object manipulation often necessitates the use of anthropomorphic, high-degree-of-freedom robot hands capable of precise, contact-rich control. However, current trajectory optimisation methods often struggle in these settings due to the large search space and the sparse task inf…

Cited by 0SourceScholar
2025

Kaputt: A Large-Scale Dataset for Visual Defect Detection

ICCV 2025poster

We present a novel large-scale dataset for defect detection in a logistics setting. Recent work on industrial anomaly detection has primarily focused on manufacturing scenarios with highly controlled poses and a limited number of object categories. Existing benchmarks like MVTec-AD (Bergmann et al.,…

Cited by 0SourcePDFScholar
2025

LUMOS: Language-Conditioned Imitation Learning with World Models

ICRA 2025

We introduce LUMOS, a language-conditioned multi-task imitation learning framework for robotics. LUMOS learns skills by practicing them over many long-horizon rollouts in the latent space of a learned world model and transfers these skills zero-shot to a real robot. By learning on-policy in the late

Cited by 13SourceScholar
2025

Offline Adaptation of Quadruped Locomotion Using Diffusion Models

ICRA 2025

We present a diffusion-based approach to quadrupedal locomotion that simultaneously addresses the limitations of learning and interpolating between multiple skills (modes) and of offline adapting to new locomotion behaviours after training. This is the first framework to apply classifier-free guided

Cited by 5SourcecodeScholar
2024

DreamUp3D: Object-Centric Generative Models for Single-View 3D Scene Understanding and Real-to-Sim Transfer

RA-L 2024

3D scene understanding for robotic applications exhibits a unique set of requirements including real-time inference, object-centric latent representation learning, accurate 6D pose estimation and 3D reconstruction of objects. Current methods for scene understanding typically rely on a combination of

Cited by 2SourceScholar
2024

Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion

CoRL 2024poster

The current state-of-the-art in quadruped locomotion is able to produce a variety of complex motions. These methods either rely on switching between a discrete set of skills or learn a distribution across gaits using complex black-box models. Alternatively, we present Gaitor, which learns a disentan…

Cited by 3SourceScholar
2024

RAMP: A Benchmark for Evaluating Robotic Assembly Manipulation and Planning

RA-L 2024

We introduce RAMP, an open-source robotics benchmark inspired by real-world industrial assembly tasks. RAMP consists of beams that a robot must assemble into specified goal configurations using pegs as fasteners. As such, it assesses planning and execution capabilities, and poses challenges in perce

Cited by 24SourceScholar
2024

TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer

ICRA 2024poster

Model-based RL is a promising approach for real-world robotics due to its improved sample efficiency and generalization capabilities compared to model-free RL. However, effective model-based RL solutions for vision-based real-world applications require bridging the sim-to-real gap for any world mode…

Cited by 6SourceScholar
2023

Leveraging Scene Embeddings for Gradient-Based Motion Planning in Latent Space

ICRA 2023poster

Motion planning framed as optimisation in structured latent spaces has recently emerged as competitive with traditional methods in terms of planning success while significantly outperforming them in terms of computational speed. However, the real-world applicability of recent work in this domain rem…

Cited by 10SourceScholar
2023

Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization

NeurIPS 2023poster

Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curv…

Cited by 10SourcePDFScholar
2023

Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learning

ICLR 2023poster

The ability to discover behaviours from past experience and transfer them to new tasks is a hallmark of intelligent agents acting sample-efficiently in the real world. Equipping embodied reinforcement learners with the same ability may be crucial for their successful deployment in robotics. While hi…

Cited by 4SourcePDFScholar
2022

Fast-MbyM: Leveraging Translational Invariance of the Fourier Transform for Efficient and Accurate Radar Odometry

ICRA 2022poster

Masking by Moving (MByM), provides robust and accurate radar odometry measurements through an exhaustive correlative search across discretised pose candidates. However, this dense search creates a significant computational bottleneck which hinders real-time performance when high-end GPUs are not ava…

Cited by 31SourceScholar
2022

Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion

ICRA 2022poster

Quadruped locomotion is rapidly maturing to a degree where robots now routinely traverse a variety of unstructured terrains. However, while gaits can be varied typically by selecting from a range of pre-computed styles, current planners are unable to vary key gait parameters continuously while the r…

Cited by 6SourceScholar
2022

Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation

RA-L 2022

We present a novelapproach to path planning for robotic manipulators, in which paths are produced via iterative optimisation in the latent space of a generative model of robot poses. Constraints are incorporated through the use of constraint satisfaction classifiers operating on the same space. Opti

Cited by 17SourceScholar
2022

Semantically Grounded Object Matching for Robust Robotic Scene Rearrangement

ICRA 2022poster

Object rearrangement has recently emerged as a key competency in robot manipulation, with practical solutions generally involving object detection, recognition, grasping and high-level planning. Goal-images describing a desired scene configuration are a promising and increasingly used mode of instru…

Cited by 44SourcecodeScholar
2022

Touching a NeRF: Leveraging Neural Radiance Fields for Tactile Sensory Data Generation

CoRL 2022poster

Tactile perception is key for robotics applications such as manipulation. However, tactile data collection is time-consuming, especially when compared to vision. This limits the use of the tactile modality in machine learning solutions in robotics. In this paper, we propose a generative model to sim…

Cited by 36SourceScholar
2022

You Only Look at One: Category-Level Object Representations for Pose Estimation From a Single Example

CoRL 2022poster

In order to meaningfully interact with the world, robot manipulators must be able to interpret objects they encounter. A critical aspect of this interpretation is pose estimation: inferring quantities that describe the position and orientation of an object in 3D space. Most existing approaches to po…

Cited by 10SourceScholar
2022

Zero-Shot Category-Level Object Pose Estimation

ECCV 2022poster

"Object pose estimation is an important component of most vision pipelines for embodied agents, as well as in 3D vision more generally. In this paper we tackle the problem of estimating the pose of novel object categories in a zero-shot manner. This extends much of the existing literature by removin…

2021

APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation

IROS 2021poster

Recent advances in unsupervised learning for object detection, segmentation, and tracking hold significant promise for applications in robotics. A common approach is to frame these tasks as inference in probabilistic latent-variable models. In this paper, however, we show that the current state-of-t…

Cited by 23SourceScholar
2021

E(n) Equivariant Normalizing Flows

NeurIPS 2021oral

This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs). To construct E-NFs, we take the discriminative E(n) graph neural networks and integrate them as a differential equation to obtain an invertible equivariant function: a continuous…

Cited by 231SourcePDFScholar
2021

GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement

NeurIPS 2021poster

Advances in unsupervised learning of object-representations have culminated in the development of a broad range of methods for unsupervised object segmentation and interpretable object-centric scene generation. These methods, however, are limited to simulated and real-world datasets with limited vis…

2021

Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives

ICRA 2021poster

Visuomotor control (VMC) is an effective means of achieving basic manipulation tasks such as pushing or pick- and-place from raw images. Conditioning VMC on desired goal states is a promising way of achieving versatile skill primitives. However, common conditioning schemes either rely on task-specif…

Cited by 18SourcecodeScholar
2021

Introspective Visuomotor Control: Exploiting Uncertainty in Deep Visuomotor Control for Failure Recovery

ICRA 2021poster

End-to-end visuomotor control is emerging as a compelling solution for robot manipulation tasks. However, imitation learning-based visuomotor control approaches tend to suffer from a common limitation, lacking the ability to recover from an out-of-distribution state caused by compounding errors. In…

Cited by 4SourceScholar
2021

There and Back Again: Learning to Simulate Radar Data for Real-World Applications

ICRA 2021poster

Simulating realistic radar data has the potential to significantly accelerate the development of data-driven approaches to radar processing. However, it is fraught with difficulty due to the notoriously complex image formation process. Here we propose to learn a radar sensor model capable of synthes…

Cited by 25SourceScholar
2020

First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion

IROS 2020poster

Traditional approaches to quadruped control frequently employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In addition, kinodynamic constraints are often non-differentiable and difficult to implement in an o…

Cited by 12SourceScholar
2020

GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations

ICLR 2020poster

Generative latent-variable models are emerging as promising tools in robotics and reinforcement learning. Yet, even though tasks in these domains typically involve distinct objects, most state-of-the-art generative models do not explicitly capture the compositional nature of visual scenes. Two recen…

Cited by 316SourcecodeScholar
2020

Localising Faster: Efficient and precise lidar-based robot localisation in large-scale environments

ICRA 2020poster

This paper proposes a novel approach for global localisation of mobile robots in large-scale environments. Our method leverages learning-based localisation and filtering-based localisation, to localise the robot efficiently and precisely through seeding Monte Carlo Localisation (MCL) with a deeplear…

Cited by 51SourceScholar
2020

RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces

NeurIPS 2020poster

We present RELATE, a model that learns to generate physically plausible scenes and videos of multiple interacting objects. Similar to other generative approaches, RELATE is trained end-to-end on raw, unlabeled data. RELATE combines an object-centric GAN formulation with a model that explicitly accou…

2020

The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset

ICRA 2020poster

In this paper we present The Oxford Radar RobotCar Dataset, a new dataset for researching scene understanding using Millimetre-Wave FMCW scanning radar data. The target application is autonomous vehicles where this modality is robust to environmental conditions such as fog, rain, snow, or lens flare…

Cited by 494SourceScholar
2020

Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar

ICRA 2020poster

This paper presents a self-supervised framework for learning to detect robust keypoints for odometry estimation and metric localisation in radar. By embedding a differentiable point-based motion estimator inside our architecture, we learn keypoint locations, scores and descriptors from localisation…

Cited by 137SourceScholar
2019

On the Limitations of Representing Functions on Sets

ICML 2019oral

Recent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectured that the dimension of this latent space may remain fixed as the cardinality of the sets under consideration increases…

Cited by 229SourcePDFScholar
2018

Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments

ICRA 2018poster

We present a self-supervised approach to ignoring “distractors” in camera images for the purposes of robustly estimating vehicle motion in cluttered urban environments. We leverage offline multi-session mapping approaches to automatically generate a per-pixel ephemerality mask and depth map for each…

Cited by 85SourceScholar
2018

Incremental Adversarial Domain Adaptation for Continually Changing Environments

ICRA 2018poster

Continuous appearance shifts such as changes in weather and lighting conditions can impact the performance of deployed machine learning models. While unsupervised domain adaptation aims to address this challenge, current approaches do not utilise the continuity of the occurring shifts. In particular…

Cited by 161SourceScholar
2018

Resource-Performance Tradeoff Analysis for Mobile Robots

RA-L 2018

The design of mobile autonomous robots is challenging due to the limited on-board resources such as processing power and energy. A promising approach is to generate intelligent schedules that reduce the resource consumption while maintaining best performance, or more interestingly, to tradeoff reduc

Cited by 33SourceScholar
2018

Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects

NeurIPS 2018spotlight

We present Sequential Attend, Infer, Repeat (SQAIR), an interpretable deep generative model for image sequences. It can reliably discover and track objects through the sequence; it can also conditionally generate future frames, thereby simulating expected motion of objects. This is achieved by expl…

2018

ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking

ECCV 2018poster

Physical intuition is pivotal for intelligent agents to perform complex tasks. In this paper we investigate the passive acquisition of an intuitive understanding of physical principles as well as the active utilisation of this intuition in the context of generalised object stacking. To this end, we…

2018

TACO: Learning Task Decomposition via Temporal Alignment for Control

ICML 2018oral

Many advanced Learning from Demonstration (LfD) methods consider the decomposition of complex, real-world tasks into simpler sub-tasks. By reusing the corresponding sub-policies within and between tasks, we can provide training data for each policy from different high-level tasks and compose them to…

Cited by 117SourcePDFScholar
2017

Addressing appearance change in outdoor robotics with adversarial domain adaptation

IROS 2017poster

Appearance changes due to weather and seasonal conditions represent a strong impediment to the robust implementation of machine learning systems in outdoor robotics. While supervised learning optimises a model for the training domain, it will deliver degraded performance in application domains that…

Cited by 92SourceScholar
2017

Find your own way: Weakly-supervised segmentation of path proposals for urban autonomy

ICRA 2017poster

We present a weakly-supervised approach to segmenting proposed drivable paths in images with the goal of autonomous driving in complex urban environments. Using recorded routes from a data collection vehicle, our proposed method generates vast quantities of labelled images containing proposed paths…

Cited by 158SourceScholar
2017

Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

ICRA 2017poster

This paper proposes a computationally efficient approach to detecting objects natively in 3D point clouds using convolutional neural networks (CNNs). In particular, this is achieved by leveraging a feature-centric voting scheme to implement novel convolutional layers which explicitly exploit the spa…

Cited by 719SourceScholar
2017

What makes a place? Building bespoke place dependent object detectors for robotics

IROS 2017poster

This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the places through which a robot moves. We leverage the concept of `experiences' in visual perception for robotics, accounting fo…

Cited by 2SourceScholar
2016

Choosing a time and place for calibration of lidar-camera systems

ICRA 2016

We propose a calibration method that automatically estimates the extrinsic calibration between a sensor pose-graph from natural scenes. The sensor pose-graph represents a system of sensors comprising of lidars and cameras, without sensor co-visibility constraints. The method addresses the fact that

Cited by 32SourceScholar
2016

Enabling intelligent energy management for robots using publicly available maps

IROS 2016poster

Energy consumption represents one of the most basic constraints for mobile robot autonomy. We propose a new framework to predict energy consumption using information extracted from publicly available maps. This method avoids having to model internal robot configurations, which are often unavailable,…

Cited by 11SourceScholar
2016

Off the beaten track: Predicting localisation performance in visual teach and repeat

ICRA 2016

This paper proposes an appearance-based approach to estimating localisation performance in the context of visual teach and repeat. Specifically, it aims to estimate the likely corridor around a taught trajectory within which a vision-based localisation system is still able to localise itself. In con

Cited by 25SourceScholar
2016

Watch this: Scalable cost-function learning for path planning in urban environments

IROS 2016poster

In this work, we present an approach to learn cost maps for driving in complex urban environments from a large number of demonstrations of human driving behaviour. The learned cost maps are constructed directly from raw sensor measurements, bypassing the effort of manually designing cost maps as wel…

Cited by 153SourceScholar
2015

Exploiting known unknowns: Scene induced cross-calibration of lidar-stereo systems

IROS 2015poster

We propose an automatic, targetless, data-driven, extrinsic calibration method to calibrate push-broom 2D lidars with a multi-camera system. The calibration problem is decoupled into alternating optimisers over two hierarchical levels, where both levels are linked with a penalty term. The lower-leve…

Cited by 25SourceScholar
2015

From dusk till dawn: Localisation at night using artificial light sources

ICRA 2015poster

This paper is about localising at night in urban environments using vision. Despite it being dark exactly half of the time, surprisingly little attention has been given to this problem. A defining aspect of night-time urban scenes is the presence and effect of artificial lighting - be that in the fo…

Cited by 42SourceScholar
2015

Integrating metric and semantic maps for vision-only automated parking

ICRA 2015poster

We present a framework for integrating two layers of map which are often required for fully automated operation: metric and semantic. Metric maps are likely to improve with subsequent visitations to the same place, while semantic maps can comprise both permanent and fluctuating features of the envir…

Cited by 34SourceScholar
2015

Know your limits: Embedding localiser performance models in teach and repeat maps

ICRA 2015poster

This paper is about building maps which not only contain the traditional information useful for localising — such as point features — but also embeds a spatial model of expected localiser performance. This often overlooked second-order information provides vital context when it comes to map use and…

Cited by 24SourceScholar
2015

Learning to assess terrain from human demonstration using an introspective Gaussian-process classifier

ICRA 2015poster

This paper presents an approach to learning robot terrain assessment from human demonstration. An operator drives a robot for a short period of time, supervising the gathering of traversable and untraversable terrain data. After this initial training period, the robot can then predict the traversabi…

Cited by 41SourceScholar
2015

Scheduled perception for energy-efficient path following

ICRA 2015poster

This paper explores the idea of reducing a robot's energy consumption while following a trajectory by turning off the main localisation subsystem and switching to a lower-powered, less accurate odometry source at appropriate times. This applies to scenarios where the robot is permitted to deviate fr…

Cited by 40SourceScholar