172 accepted papers
Visuomotor policies trained on human expert demonstrations have recently shown strong performance across a wide range of robotic manipulation tasks. However, these policies remain highly sensitive to domain shifts stemming from background or robot embodiment changes, which limits their generalizatio
Visuomotor policies trained on human expert demonstrations have recently shown strong performance across a wide range of robotic manipulation tasks. However, these policies remain highly sensitive to domain shifts stemming from background or robot embodiment changes, which limits their generalizatio…
Generative navigation policies have made rapid progress in improving end-to-end learned navigation. Despite their promising results, this paradigm has two structural problems. First, the sampled trajectories exist in an abstract, unscaled space without metric grounding. Second, the control strategy …
Vision-Language-Action models (VLAs) mark a major shift in robot learning. They replace specialized architectures and task-tailored components of expert policies with large-scale data collection and setup-specific fine-tuning. In this machine learning-focused workflow that is centered around models …
A key challenge for the widespread application of learning-based models for robotic perception is to significantly reduce the required amount of annotated training data while achieving accurate predictions. This is essential not only to decrease operating costs but also to speed up deployment time.
Understanding the 3D motion of articulated objects is essential in robotic scene understanding, mobile manipulation, and motion planning. Prior methods for articulation estimation have primarily focused on controlled settings, assuming either fixed camera viewpoints or direct observations of various…
Dynamic scene understanding remains a persistent challenge in robotic applications. Early dynamic mapping methods focused on mitigating the negative influence of short-term dynamic objects on camera motion estimation by masking or tracking specific categories, which often fall short in adapting to l
Nowadays, unmanned aerial vehicles (UAVs) are commonly used in search and rescue scenarios to gather information in the search area. The automatic identification of the person searched for in aerial footage could increase the autonomy of such systems, reduce the search time, and thus increase the mi
Fine-tuning diffusion policies with reinforcement learning (RL) presents significant challenges. The long denoising sequence for each action prediction impedes effective reward propagation. Additionally, standard RL methods require millions of physical interaction steps, making fine-tuning even more…
Effective robot navigation in unseen environments is a challenging task that requires precise control actions at high frequencies. Recent advances have framed it as an image-goal-conditioned control problem, where the robot generates navigation actions using frontal RGB images. Current state-of-the-
We introduce LUMOS, a language-conditioned multi-task imitation learning framework for robotics. LUMOS learns skills by practicing them over many long-horizon rollouts in the latent space of a learned world model and transfers these skills zero-shot to a real robot. By learning on-policy in the late
A main bottleneck of learning-based robotic scene understanding methods is the heavy reliance on extensive annotated training data, which often limits their generalization ability. In LiDAR panoptic segmentation, this challenge becomes even more pronounced due to the need to simultaneously address b
LiDAR registration is a fundamental task in robotic mapping and localization. A critical component of aligning two point clouds is identifying robust point correspondences using point descriptors, which becomes particularly challenging in scenarios involving domain shifts, seasonal changes, and vari…
Loop closures are essential for correcting odometry drift and creating consistent maps, especially in the context of large-scale navigation. Current methods using dense point clouds for accurate place recognition do not scale well due to computationally expensive scan-to-scan comparisons. Alternativ
Vision-Language-Action Models (VLAs) have demonstrated remarkable generalization capabilities in real-world experiments. However, their success rates are often not on par with expert policies, and they require fine-tuning when the setup changes. In this work, we introduce Refined Policy Distillation
In this paper, we propose VLM-Vac, a novel framework designed to enhance the autonomy of smart robot vacuum cleaners. Our approach integrates the zero-shot object detection capabilities of a Vision-Language Model (VLM) with a Knowledge Distillation (KD) strategy. By leveraging the VLM, the robot can
Active traffic management with autonomous vehicles offers the potential for reduced congestion and improved traffic flow. However, developing effective algorithms for real-world scenarios requires overcoming challenges related to infinite-horizon traffic flow and partial observability. To address th…
Lane graph estimation is a long-standing problem in the context of autonomous driving. Previous works aimed at solving this problem by relying on large-scale, hand-annotated lane graphs, introducing a data bottleneck for training models to solve this task. To overcome this limitation, we propose to
Sensor setups of robotic platforms commonly include both camera and LiDAR as they provide complementary information. However, fusing these two modalities typically requires a highly accurate calibration between them. In this letter, we propose MDPCalib which is a novel method for camera-LiDAR calibr
Semantic scene segmentation from a bird’s-eye-view (BEV) perspective plays a crucial role in facilitating planning and decision-making for mobile robots. Although recent vision-only methods have demonstrated notable advancements in performance, they often struggle under adverse illumination conditio…
Sample efficient learning of manipulation skills poses a major challenge in robotics. While recent approaches demonstrate impressive advances in the type of task that can be addressed and the sensing modalities that can be incorporated, they still require large amounts of training data. Especially w…
Reliable object grasping is a crucial capability for autonomous robots. However, many existing grasping approaches focus on general clutter removal without explicitly modeling objects and thus only relying on the visible local geometry. We introduce CenterGrasp, a novel framework that combines objec
Maps have played an indispensable role in enabling safe and automated driving. Although there have been many advances on different fronts ranging from SLAM to semantics, building an actionable hierarchical semantic representation of urban dynamic scenes and processing information from multiple agent…
Current state-of-the-art methods for panoptic segmentation require an immense amount of annotated training data that is both arduous and expensive to obtain posing a significant challenge for their widespread adoption. Concurrently, recent breakthroughs in visual representation learning have sparked…
Recent open-vocabulary robot mapping methods enrich dense geometric maps with pre-trained visual-language features. While these maps allow for the prediction of point-wise saliency maps when queried for a certain language concept, large-scale environments and abstract queries beyond the object level…
Robustness against Out-of-Distribution (OoD) samples is a key performance indicator of a trajectory prediction model. However, the development and ranking of state-of-the-art (SotA) models are driven by their In-Distribution (ID) performance on individual competition datasets. We present an OoD test…
Geometric regularity, which leverages data symmetry, has been successfully incorporated into deep learning architectures such as CNNs, RNNs, GNNs, and Transformers. While this concept has been widely applied in robotics to address the curse of dimensionality when learning from high-dimensional data,…
"Semantic Bird’s Eye View (BEV) maps offer a rich representation with strong occlusion reasoning for various decision making tasks in autonomous driving. However, most BEV mapping approaches employ a fully supervised learning paradigm that relies on large amounts of human-annotated BEV ground truth…
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for man
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for man…
The availability of a robust map-based localization system is essential for the operation of many autonomously navigating vehicles. Since uncertainty is an inevitable part of perception, it is beneficial for the robustness of the robot to consider it in typical downstream tasks of navigation stacks.
Accurate value estimates are important for off-policy reinforcement learning. Algorithms based on temporal difference learning typically are prone to an over- or underestimation bias building up over time. In this letter, we propose a general method called Adaptively Calibrated Critics (ACC) that us
Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments. Optimally, the robot is able to adapt by itself to new conditions without human supervision, e.g., automatically adjusting its perception system to changing lighting conditions. In…
Early stopping based on the validation set performance is a popular approach to find the right balance between under- and overfitting in the context of supervised learning. However, in reinforcement learning, even for supervised sub-problems such as world model learning, early stopping is not applic…
Uncertainty estimation is crucial in safety-critical settings such as automated driving as it provides valuable information for several downstream tasks including high-level decision making and path planning. In this work, we propose EvCenterNet, a novel uncertainty-aware 2D object detection framewo…
Visual place recognition is essential for vision-based robot localization and SLAM. Despite the tremendous progress made in recent years, place recognition in changing environments remains challenging. A promising approach to cope with appearance variations is to leverage high-level semantic feature…
Recent works have shown that Large Language Models (LLMs) can be applied to ground natural language to a wide variety of robot skills. However, in practice, learning multi-task, language-conditioned robotic skills typically requires large-scale data collection and frequent human intervention to rese…
The safe deployment of autonomous vehicles relies on their ability to effectively react to environmental changes. This can require maneuvering on varying surfaces which is still a difficult problem, especially for slippery terrains. To address this issue we propose a new approach that learns a surfa…
Lane graph estimation is an essential and highly challenging task in automated driving and HD map learning. Existing methods using either onboard or aerial imagery struggle with complex lane topologies, out-of-distribution scenarios, or significant occlusions in the image space. Moreover, merging ov…
A key component of graph-based SLAM systems is the ability to detect loop closures in a trajectory to reduce the drift accumulated over time from the odometry. Most LiDAR-based methods achieve this goal by using only the geometric information, disregarding the semantics of the scene. In this work, w
Simultaneous localization and mapping (SLAM) in slowly varying scenes is important for long-term robot task completion in GPS-denied environments. Failing to detect scene changes may lead to inaccurate maps and, ultimately, lost robots. Classical SLAM algorithms assume static scenes, and recent work…
Bird's-Eye-View (BEV) semantic maps have become an essential component of automated driving pipelines due to the rich representation they provide for decision-making tasks. However, existing approaches for generating these maps still follow a fully supervised training paradigm and hence rely on larg…
In policy learning for robotic manipulation, sample efficiency is of paramount importance. Thus, learning and extracting more compact representations from camera observations is a promising avenue. However, current methods often assume full observability of the scene and struggle with scale invarian
Reliable scene understanding is indispensable for modern autonomous systems. Current learning-based methods typically try to maximize their performance based on segmentation metrics that only consider the quality of the segmentation. However, for the safe operation of a system in the real world it i
Modern autonomous systems often rely on LiDAR scanners, in particular for autonomous driving scenarios. In this context, reliable scene understanding is indispensable. Conventional learning-based methods generally try to achieve maximum performance for this task, while neglecting a proper estimation…
Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visual-language models pretrained on Internet-scale data (e.g., image captions). While this is useful for matching images to natural language descriptions of object goals, it remains disjoint fro…
Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visu…
General-purpose robots coexisting with humans in their environment must learn to relate human language to their perceptions and actions to be useful in a range of daily tasks. Moreover, they need to acquire a diverse repertoire of general-purpose skills that allow composing long-horizon tasks by fol
Learning to solve complex manipulation tasks from visual observations is a dominant challenge for real-world robot learning. Although deep reinforcement learning algorithms have recently demonstrated impressive results in this context, they still require an impractical amount of time-consuming trial
The transition from today's mostly human-driven traffic to a purely automated one will be a gradual evolution, with the effect that we will likely experience mixed traffic in the near future. Connected and automated vehicles can benefit human-driven ones and the whole traffic system in different way
Service robots in the future need to execute abstract instructions such as “fetch the milk from the fridge”. To translate such instructions into actionable plans, robots require in-depth background knowledge. With regards to interactions with doors and drawers, robots require articulation models tha
Everyday tasks of long-horizon and comprising a sequence of multiple implicit subtasks still impose a major challenge in offline robot control. While a number of prior methods aimed to address this setting with variants of imitation and offline reinforcement learning, the learned behavior is typical…
Simulation remains one of the key methods for testing and validation of robotic perception systems and it also becomes increasingly important for training visuomotor policies for autonomous driving or manipulation. Further, as perception pipelines tend to leverage increasing amounts of modalities, i…
AA core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner through traje…
Robust localization in dense urban scenarios using a low-cost sensor setup and sparse HD maps is highly relevant for the current advances in autonomous driving, but remains a challenging topic in research. We present a novel monocular localization approach based on a sliding-window pose graph that l…
Robust detection of moving vehicles is a critical task for any autonomously operating outdoor robot or self-driving vehicle. Most modern approaches for solving this task rely on training image-based detectors using large-scale vehicle detection datasets such as nuScenes or the Waymo Open Dataset. Pr
For autonomous skill acquisition, robots have to learn about the physical rules governing the 3D world dynamics from their own past experience to predict and reason about plausible future outcomes. To this end, we propose a transformation-based 3D video prediction (T3VIP) approach that explicitly mo…
For autonomous skill acquisition, robots have to learn about the physical rules governing the 3D world dynamics from their own past experience to predict and reason about plausible future outcomes. To this end, we propose a transformation-based 3D video prediction (T3VIP) approach that explicitly mo
Robustly classifying ground infrastructure such as roads and street crossings is an essential task for mobile robots operating alongside pedestrians. While many semantic segmentation datasets are available for autonomous vehicles, models trained on such datasets exhibit a large domain gap when deplo…
A long-standing goal in robotics is to build robots that can perform a wide range of daily tasks from perceptions obtained with their onboard sensors and specified only via natural language. While recently substantial advances have been achieved in language-driven robotics by leveraging end-to-end l
Lane-level scene annotations provide invaluable data in autonomous vehicles for trajectory planning in complex environments such as urban areas and cities. However, obtaining such data is time-consuming and expensive since lane annotations have to be annotated manually by humans and are as such hard
Tracking by detection, the dominant approach for online multi-object tracking, alternates between localization and association steps. As a result, it strongly depends on the quality of instantaneous observations, often failing when objects are not fully visible. In contrast, tracking in humans is un…
Dynamic illumination is a challenging problem for visual robot localization and tracking. In indoor environments, the main source of light during the day is outdoor illumination. We propose a method that estimates the appearance of an indoor scene in real-time based on a reflectance map and the curr
Estimating scene geometry from cost-effective sensors is key for robots. In this paper, we study the problem of predicting dense depth from a single RGB image (monodepth) with optional sparse measurements from low-cost active depth sensors. We introduce Sparse Auxiliary Networks (SAN), a new module…
We propose Adaptive Curriculum Generation from Demonstrations (ACGD) for reinforcement learning in the presence of sparse rewards. Rather than designing shaped reward functions, ACGD adaptively sets the appropriate task difficulty for the learner by controlling where to sample from the demonstration…
Key challenges for the deployment of reinforcement learning (RL) agents in the real world are the discovery, representation and reuse of skills in the absence of a reward function. To this end, we propose a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view video…
Tracking the pose of a camera is at the core of visual localization methods used in many applications. As the observations of a camera are inherently affected by lighting, it has always been a challenge for these methods to cope with varying lighting conditions. Thus far, this issue has mainly been…
In this paper, we present an integrated, model-based system for state estimation and control in dynamic manipulation tasks with partial observability. We track a belief over the system state using a particle filter from which we extract a Gaussian Mixture Model (GMM). This compressed representation…
Understanding the semantic characteristics of the environment is a key enabler for autonomous robot operation. In this paper, we propose a deep convolutional neural network (DCNN) for semantic segmentation of a LiDAR scan into the classes car, pedestrian and bicyclist. This architecture is based on…
Safe autonomous driving requires robust detection of other traffic participants. However, robust does not mean perfect, and safe systems typically minimize missed detections at the expense of a higher false positive rate. This results in conservative and yet potentially dangerous behavior such as av…
Traffic signal controllers play an essential role in today's traffic system. However, the majority of them currently is not sufficiently flexible or adaptive to generate optimal traffic schedules. In this paper we present an approach to learn policies for signal controllers using deep reinforcement…
The majority of learning-based semantic segmentation methods are optimized for daytime scenarios and favorable lighting conditions. Real-world driving scenarios, however, entail adverse environmental conditions such as nighttime illumination or glare which remain a challenge for existing approaches.…
A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces. In order to scale learning through interaction to many objects and scenes, robots should be able to improve their own p…
The current state of the art in automotive high definition digital (HD) map generation based on dedicated mapping vehicles cannot reliably keep these maps up to date because of the low traversal frequencies. Anonymized data collected from the fleet of vehicles that is already on the road provides a…
Robots perceive their environment using various sensor modalities, e.g., vision, depth, sound or touch. Each modality provides complementary information for perception. However, while it can be assumed that all modalities are available for training, when deploying the robot in real-world scenarios t…
Autonomous systems, such as delivery robots, are increasingly employed in indoor spaces to carry out activities alongside humans. This development poses the question of how robots can carry out their tasks while, at the same time, behaving in a socially compliant manner. Further, humans need to be a…
Robots coexisting with humans in their environment and performing services for them need the ability to interact with them. One particular requirement for such robots is that they are able to understand spatial relations and can place objects in accordance with the spatial relations expressed by the…
Real-time object detection in videos using lightweight hardware is a crucial component of many robotic tasks. Detectors using different modalities and with varying computational complexities offer different trade-offs. One option is to have a very lightweight model that can predict from all modaliti…
In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning. However, this perception task is difficult, particularly for generic obstacles/objects, due to appearance and occlusion changes. To tackle this problem, we propose an end-to…
Horizontally scanning 2D laser rangefinders are a popular approach for indoor robot localization because of the high accuracy of the sensors and the compactness of the required 2D maps. As the scanners in this configuration only provide information about one slice of the environment, the measurement…
Whether it is object detection, model reconstruction, laser odometry, or point cloud registration: Plane extraction is a vital component of many robotic systems. In this paper, we propose a strictly probabilistic method to detect finite planes in organized 3-D laser range scans. An agglomerative hie…
Learning from demonstration is a powerful tool for teaching manipulation actions to a robot. It is, however, an unsolved problem how to consider knowledge about the world and action-induced reactions such as forces imposed onto the gripper or measured liquid levels during pouring without explicit an…
Learning from demonstrations is a promising paradigm for transferring knowledge to robots. However, learning mobile manipulation tasks directly from a human teacher is a complex problem as it requires learning models of both the overall task goal and of the underlying actions. Additionally, learning…
In this paper, we present a change detection algorithm that can run in real time as part of a backend-based stream processing pipeline. It can process the floating car data collected by series-production vehicles to detect changes in an automotive high definition digital (HD) map used for automated…
In this paper, we propose a new algorithm that derives lane marking maps from crowdsourced data. We process the data in four steps: (i) We make use of a point landmark map and, if available, an existent lane marking map for trajectory optimization and alignment, (ii) use a custom DBSCAN variant to c…
In this paper we propose a new model for sequential manipulation tasks that also considers robot dynamics and time-variant environments. From this model we automatically derive constraint-based controllers and use them as steering functions in a kinodynamic manipulation planner. The resulting plan i…
The Bayes filter is the basis for many state-of-the-art robot localization algorithms. In the literature, its derivation typically requires the robot controls to be chosen independently of all other variables. However, this assumption is not valid for a robotic system that is to act purposefully. Th
When robots perform manipulation tasks, they need to determine their own movement, as well as how to make and break contact with objects in their environment. Reasoning about the motions of robots and objects simultaneously leads to a constrained planning problem in a high-dimensional state-space. A…
Indoor localization is one of the crucial enablers for deployment of service robots. Although several successful techniques for indoor localization have been proposed, the majority of them relies on maps generated from data gathered with the same sensor modality used for localization. Typically, ted…
In this paper, we present an approach to generate robot motions for robust parts assembly. The computation of motions for parts assembly usually requires an exact model of all relevant objects. Generating detailed object models, including friction and dynamics, is often complex and time-consuming, e…
We present a convolutional neural network for joint 3D shape prediction and viewpoint estimation from a single input image. During training, our network gets the learning signal from a silhouette of an object in the input image-a form of self-supervision. It does not require ground truth data for 3D…
Instance segmentation of unknown objects from images is regarded as relevant for several robot skills including grasping, tracking and object sorting. Recent results from computer vision have shown that large hand-labeled datasets enable high segmentation performance. To overcome the time-consuming…
This paper introduces a Bayesian state estimator for contact-rich manipulation tasks with application in non-prehensile manipulation, industrial assembly or in-hand localization. The core idea of our approach is to explicitly model both the contact dynamics and a torque-based robot controller as par…
In this letter, we deal with the reality gap from a novel perspective, targeting transferring deep reinforcement learning (DRL) policies learned in simulated environments to the real-world domain for visual control tasks. Instead of adopting the common solutions to the problem by increasing the visu
We propose an approach to estimate 3D human pose in real world units from a single RGBD image and show that it exceeds performance of monocular 3D pose estimation approaches from color as well as pose estimation exclusively from depth. Our approach builds on robust human keypoint detectors for color…
Man-made environments such as households, offices, or factory floors are typically composed of linear structures. Accordingly, polylines are a natural way to accurately represent their geometry. In this paper, we propose a novel probabilistic method to extract polylines from raw 2-D laser range scan…
The ability to build models of the environment is an essential prerequisite for many robotic applications. In recent years, mapping of dense surface geometry using RGB-D cameras has seen extensive progress. Many approaches build colored models, typically directly using the intensity values provided…
Dynamic systems are a practical alternative to motion planning in executing robot actions. They are of particular interest in Learning from Demonstration, as here we aim to carry out actions in a certain fashion, without a model or in-depth knowledge about the world, which might be difficult to achi…
In sustainable farming, robotic solutions are in rising demand. Specifically robots for precision agriculture open up possibilities for new applications. Such applications typically require a high accuracy of the underlying navigation system. A cornerstone for reliable navigation is the robust detec
Most robot mapping techniques for lidar sensors tessellate the environment into pixels or voxels and assume uniformity of the environment within them. Although intuitive, this representation entails disadvantages: The resulting grid maps exhibit aliasing effects and are not differentiable. In this l
Densely connected networks for classification enable feature exploration and result in state-of-the-art performance on multiple classification tasks. The alternative to dense networks is the residual network which enables feature re-usage. In this work, we combine these orthogonal concepts for encod…
Localization is an indispensable component of a robot's autonomy stack that enables it to determine where it is in the environment, essentially making it a precursor for any action execution or planning. Although convolutional neural networks have shown promising results for visual localization, the…
To detect changes in an environment, one has to decide whether a set of recent observations is incompatible with a set of previous observations. For binary, lidar-based grid maps, this is essentially the case when the laser beam traverses a voxel that has been observed as occupied, or when the beam
Interpreting the brain activity to identify user goals or to ground a robot's hypotheses about them is a promising direction for non-intrusive and intuitive communication. Such a capability can be of particular relevance in the context of human-robot cooperation scenarios. This paper proposes a nove…
Robust data association is necessary for virtually every SLAM system and finding corresponding points is typically a preprocessing step for scan alignment algorithms. Traditionally, handcrafted feature descriptors were used for these problems but recently learned descriptors have been shown to perfo…
Pouring is a fundamental skill for robots in both domestic and industrial environments. Ideally, a robot should be able to pour with high accuracy to specific, pre-defined heights and without spilling. However, due to the complex dynamics of liquids, it is difficult to learn how to pour to achieve t…
Various autonomous robotic systems require maps for robust and safe navigation. Particularly when robots are employed in dynamic environments, accurate knowledge about which components of the robot perceptions belong to dynamic and static aspects in the environment can greatly improve navigation fun…
To operate intelligently in domestic environments, robots require the ability to understand arbitrary spatial relations between objects and to generalize them to objects of varying sizes and shapes. In this work, we present a novel end-to-end approach to generalize spatial relations based on distanc…
As robots are increasingly entering human environments, many subtleties of socially compliant navigation are still unsolved. To behave in a socially compliant way, robots need to have an understanding of the natural motion paths of humans in the shared environment. Humans intuitively follow social n
In automated manufacturing, robots must reliably assemble parts of various geometries and low tolerances. Ideally, they plan the required motions autonomously. This poses a substantial challenge due to high-dimensional state spaces and non-linear contact-dynamics. Furthermore, object poses and model…
We present an approach for mobile robots to learn to navigate in dynamic environments with pedestrians via raw depth inputs, in a socially compliant manner. To achieve this, we adopt a generative adversarial imitation learning (GAIL) strategy, which improves upon a pre-trained behavior cloning polic…
Semantic understanding and localization are fundamental enablers of robot autonomy that have been tackled as disjoint problems for the most part. While deep learning has enabled recent breakthroughs across a wide spectrum of scene understanding tasks, its applicability to state estimation tasks has
Most of the conventional approaches to mobile robot navigation avoid any kind of contact with the environment or with humans. As nowadays distance sensors typically have a limited - and often only two-dimensional - field of view, collisions with the environment or contacts with humans cannot be full…
Robust scene understanding of outdoor environments using passive optical sensors is a onerous and essential task for autonomous navigation. The problem is heavily characterized by changing environmental conditions throughout the day and across seasons. Robots should be equipped with models that are…
Two core competencies of a mobile robot are to build a map of the environment and to estimate its own pose on the basis of this map and incoming sensor readings. To account for the uncertainties in this process, one typically employs probabilistic state estimation approaches combined with a model of
An objective performance measure for movement tasks is widely regarded as having utmost relevance for the therapy of movement disorders. Existing systems typically rely on human experts, which is known to produce substantial inter- and intra-rater variability. Present solutions are either based on s…
A popular class of lidar-based grid mapping algorithms computes for each map cell the probability that it reflects an incident laser beam. These algorithms typically determine the map as the set of reflection probabilities that maximizes the likelihood of the underlying laser data and do not compute…
Precise localization of robots is imperative for their safe and autonomous navigation in both indoor and outdoor environments. In outdoor scenarios, the environment typically undergoes significant perceptual changes and requires robust methods for accurate localization. Monocular camera-based approa…
In this paper we consider the problem of robot navigation in simple maze-like environments where the robot has to rely on its onboard sensors to perform the navigation task. In particular, we are interested in solutions to this problem that do not require localization, mapping or planning. Additiona…
Robots are expected to operate autonomously in dynamic environments. Understanding the underlying dynamic characteristics of objects is a key enabler for achieving this goal. In this paper, we propose a method for pointwise semantic classification of 3D LiDAR data into three classes: non-movable, mo…
Efficient navigation planning for mobile robots in complex environments is a challenging problem. In this paper we consider the path planning problem for mobile robots with adjustable relative wheel positions, which further increase the navigation capabilities. In particular we account for changes o…
Publicly available map services are widely used by humans for navigation and nowadays provide almost complete road network data. When utilizing such maps for autonomous navigation with mobile robots one is faced with the problem of inaccuracies of the map and the uncertainty about the position of th…
Over the past years learning from demonstration has become a popular method to intuitively teach new skills to service robots without explicit programming. However, most teaching approaches in literature use kinesthetic training and do not include mobile platforms. Here, we present a novel approach…
Human-centered environments are rich with a wide variety of spatial relations between everyday objects. For autonomous robots to operate effectively in such environments, they should be able to reason about these relations and generalize them to objects with different shapes and sizes. For example,…
When robots perform manipulation tasks, they need to determine their own movement, as well as how to grasp and release an object. Reasoning about the motion of the robot and the object simultaneously leads to a multi-modal planning problem in a high-dimensional configuration space. In this paper we…
Occupancy grid maps are a popular method for representing the environment in the context of robot navigation tasks. However, occupancy grid maps can have a high memory demand that grows quadratically with the range of the sensor. In this paper, we introduce a memory-efficient map representation that…
Modern automation demands mobile robots to be robustly localized in complex scenarios. Current localization systems typically use maps that require to be built and interpreted by experienced operators, increasing deployment costs as well as reducing the adaptability of robots to rearrangements in th…
Interpreting the semantics and motion of objects are prerequisites for autonomous robots that enable them to reason and operate in dynamic real-world environments. Existing approaches that tackle the problem of semantic motion segmentation consist of long multistage pipelines and typically require s…
Visual place recognition under difficult perceptual conditions remains a challenging problem due to changing weather conditions, illumination and seasons. Long-term visual navigation approaches for robot localization should be robust to these dynamics of the environment. Existing methods typically l…
Shakey the robot was one of the first autonomous robots that showed impressive capabilities of navigation and mobile manipulation. Since then, robotics research has made great progress, showing more and more capable robotic systems for a large variety of application domains and tasks. In this letter
We consider the problem of developing robots that navigate like pedestrians on sidewalks through city centers for performing various tasks including delivery and surveillance. One particular challenge for such robots is crossing streets without pedestrian traffic lights. To solve this task the robot…
The objective characterization of human motion is required in a variety of fields including competitive sports, rehabilitation and the detection of motor deficits. Nowadays, typically human experts evaluate the motor behavior. These evaluations are based on their individual experience which leads to…
Robotic assistants have the potential to greatly improve our quality of life by supporting us in our daily activities. A service robot acting autonomously in an indoor environment is faced with very complex tasks. Consider the problem of pouring a liquid into a cup, the robot should first determine…
Motion analysis is important in a broad range of contexts, including animation, bio-mechanics, robotics and experiments investigating animal behavior. For applications, in which tracking accuracy is one of the main requirements, passive optical motion capture systems are widely used. Many skeleton t
State-of-the-art neural microprobes contain hundreds of electrodes within a single shaft. Due to hardware and wiring restrictions, it is usually only possible to measure a small subset of the available electrodes simultaneously. The selection of the best channels is typically performed offline eithe…
Hand-Drawn sketches are natural means by which abstract descriptions of environments can be provided. They represent weak prior information about the scene, thereby enabling a robot to perform autonomous navigation and exploration when a full metrical description of the environment is not available
Mobile manipulators installed in warehouses and factories for conveying goods between working stations need to meet the requirements of time-critical workflows. Moreover, the systems are expected to deal with changing tasks, cluttered environments and constraints imposed by the goods to be delivered…
Object detection is an essential task for autonomous robots operating in dynamic and changing environments. A robot should be able to detect objects in the presence of sensor noise that can be induced by changing lighting conditions for cameras and false depth readings for range sensors, especially…
This paper addresses the problem of human body part segmentation in conventional RGB images, which has several applications in robotics, such as learning from demonstration and human-robot handovers. The proposed solution is based on Convolutional Neural Networks (CNNs). We present a network archite
Text is one of the richest sources of information in an urban environment. Although textual information is heavily relied on by humans for a majority of the daily tasks, its usage has not been completely exploited in the field of robotics. In this work, we propose a localization approach utilizing t
This paper addresses the problem of road scene segmentation in conventional RGB images by exploiting recent advances in semantic segmentation via convolutional neural networks (CNNs). Segmentation networks are very large and do not currently run at interactive frame rates. To make this technique app…
Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent’s behavior originates in its policy and MDP policies depend on both the stochastic system dynamics as well as the r…
Learning from demonstration is a popular approach for teaching robots as it allows service robots to acquire new skills without explicit programming. However, for manipulation actions mostly kinesthetic teaching is used as these actions require precise knowledge about the interactions between the ro…
Localizing a camera in a given map is essential for vision-based navigation. In contrast to common methods for visual localization that use maps acquired with cameras, we propose a novel approach, which tracks the pose of monocular camera with respect to a given 3D LiDAR map. We employ a visual odom…
Robots are expected to operate autonomously in increasingly complex scenarios such as crowded streets or heavy traffic situations. Perceiving the dynamics of moving objects in the environment is crucial for safe and smart navigation and therefore a key enabler for autonomous driving. In this paper w
This paper provides a new fully-decentralized al- gorithm for Collaborative Localization based on the extended Kalman filter. The major challenge in decentralized collaborative localization is to track inter-robot dependencies â which is particularly difficult in situations where sustained synchro…
The perception of the dynamic aspects of the environment is a highly relevant precondition for the realization of autonomous robot system acting in the real world. In this paper, we propose a novel method for estimating dense rigid scene flow in 3D LiDAR scans. We formulate the problem as an energy…
The ability to autonomously learn a model of an environment is an important capability of a mobile robot. In this paper, we investigate the problem of exploring a scene given background information in form of a topo-metric graph of the environment. Our method is relevant for several real-world appli
Reliable detection and avoidance of obstacles is a crucial prerequisite for autonomously navigating robots as both guarantee safety and mobility. To ensure safe mobility, the obstacle detection needs to run online, thereby taking limited resources of autonomous systems into account. At the same time…
Observing human activities can reveal a lot about the structure of the environment, the objects contained therein and also their functionality. This knowledge, in turn, can be useful for robots interacting with humans or for robots performing mobile manipulation tasks. In this paper, we present an a…
We study the contact-aware state estimation (CASE) problem, i.e., the problem of estimating the state of an object while it is being actively manipulated by a robot. Several researchers have developed particle filters for this problem. They estimate the state (pose and velocity) of manipulated objec…
Accurate localization in indoor environments is widely regarded as a key opener for various location-based services. Despite tremendous advancements in the development of innovative sensor concepts, the most effective and accurate solutions to this problem make use of a map computed from sensory dat…
Many mobile manipulation tasks require the robot to be accurately localized with respect to the object where the manipulation has to be executed. These tasks include autonomous docking and positioning as well as pick and place or logistics tasks. State-of-the-art approaches to the problem commonly a…
Stroke and neurodegenerative diseases, among a range of other neurologic disorders, can cause chronic paralysis. Patients suffering from paralysis may remain unable to achieve even basic everyday tasks such as liquid intake. Currently, there is a great interest in developing robotic assistants contr…
Networks of laser range finders are a popular tool for monitoring large cluttered areas and to track people. Whenever multiple scanners are used for this purpose, one major problem is how to determine the relative positions of all the scanners. In this paper, we present a novel approach to calibrate…
The ability to track skeletal movements is important in a variety of applications including animation, biological studies and animal experiments. To detect even small movements, such a method should provide highly accurate estimates. Besides that it should not impede the mammal in its motion. This m…
The ability to localize a robot is an important capability and matching of observations under substantial changes is a prerequisite for robust long-term operation. This paper investigates the problem of efficiently coping with seasonal changes in image data. We present an extension of a recent appro…
To increase the acceptance of autonomous systems in populated environments, it is indispensable to teach them social behavior. We would expect a social robot, which plans its motions among humans, to consider both the social acceptability of its behavior as well as task constraints, such as time lim…
It is expected that autonomous vehicles capable of driving without human supervision will be released to market within the next decade. For user acceptance, such vehicles should not only be safe and reliable, they should also provide a comfortable user experience. However, individual perception of c…
Although the neurological impairments of Parkinson's disease (PD) patients are well known to go along with motor control deficits, e.g., tremor, rigidity, and reduced movement, not much is known about the motor control parameters affected by the disease. In this paper, we therefore present a novel a…
In the last years, many researchers started to consider teach-and-repeat approaches for reliable autonomous navigation. The paradigm, in all its proposed forms, is deeply rooted in the idea that the robot should autonomously follow a route that has been demonstrated by a human during a teach phase.…
To determine the pose of a vehicle is a fundamental problem in mobile robotics. Most approaches relate the current sensor observations to a map generated with previously acquired data of the same system or by another system with a similar sensor setup. Unfortunately, previously acquired data is not…
Laser range scanners are commonly used in mobile robotics to enable a robot to sense the spatial configuration of its environment. In addition to the range measurements, most scanners provide remission values, representing the intensity of the returned light pulse. These values add a visual componen…
Accurate metrical localization is one of the central challenges in mobile robotics. Many existing methods aim at localizing after building a map with the robot. In this paper, we present a novel approach that instead uses geo-tagged panoramas from the Google Street View as a source of global positio…
Robot localization is one of the most important problems in robotics. Most of the existing approaches assume that the map of the environment is available beforehand and focus on accurate metrical localization. In this paper, we address the localization problem when the map of the environment is not…
Robust object recognition is a crucial ingredient of many, if not all, real-world robotics applications. This paper leverages recent progress on Convolutional Neural Networks (CNNs) and proposes a novel RGB-D architecture for object recognition. Our architecture is composed of two separate CNN proce…
Navigation in complex and unknown environments is a major challenge for blind people. The most popular, conventional navigation aids such as white canes and guide dogs, however, provide limited assistance in such settings as they are constrained to interpret the local environment only. At the same t…
As service robots become more and more capable of performing useful tasks for us, there is a growing need to teach robots how we expect them to carry out these tasks. However, learning our preferences is a nontrivial problem, as many of them stem from a variety of factors including personal taste, c…
In this paper, we present an appearance-based visual SLAM approach that focuses on detecting loop closures across seasons. Given two image sequences, our method first extracts one descriptor per image for both sequences using a deep convolutional neural network. Then, we compute a similarity matrix…
The travel range of a mobile robot is directly linked to its energy consumption. Apart from the actuators, the computing resources are the main energy consumers. Thus, robots that can navigate with minimal computational resources would be able to travel longer distances and, hence, would be valuable…
The ability to safely navigate is a crucial prerequisite for truly autonomous systems. A robot has to distinguish obstacles from traversable ground. Failing on this task can cause great damage or restrict the robots movement unnecessarily. Due to the security relevance of this problem, great effort…
Quickly finding a free parking spot that is close to a desired target location can be a difficult task. This holds for human drivers and autonomous cars alike. In this paper, we investigate the problem of predicting the occupancy of parking spaces and exploiting this information during route plannin…