AIConfPaper
← Search

Steven L Waslander

40 accepted papers

2026

A Photorealistic Dataset and Vision-Based Algorithm for Anomaly Detection During Proximity Operations in Lunar Orbit

Selina Leveugle, Chang Won Lee, Svetlana Stolpner, Chris Langley, Paul Grouchy, Steven L. Waslander +1
RA-L 2026

NASA's forthcoming Lunar Gateway space station, which will be uncrewed most of the time, will need to operate with an unprecedented level of autonomy. One key challenge is enabling the Canadarm3, the Gateway's external robotic system, to detect hazards in its environment using its onboard inspection

Cited by 0Source ↗code ↗Scholar ↗
2026

CLIP Is Shortsighted: Paying Attention Beyond the First Sentence

Marc-Antoine Lavoie, Anas Mahmoud, Aldo Zaimi, Arsene Fansi Tchango, Steven L. Waslander
CVPR 2026

CLIP models learn transferable multi-modal features via image-text contrastive learning on internet-scale data. They are widely used in zero-shot classification, multi-modal retrieval, text-to-image diffusion, and as image encoders in large vision-language models. However, CLIP's pretraining is domi

Cited by 0Source ↗code ↗Scholar ↗
2026

DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving

Yang Zhou, Hao Shao, Letian Wang, Zhuofan Zong, Hongsheng Li, Steven L. Waslander
ICLR 2026poster

Video generation models, as one form of world models, has emerged as one of the most exciting frontiers in AI, promising agents the ability to imagine the future by modeling the temporal evolution of complex scenes. In autonomous driving, this vision gives rise to driving world models—generative si…

Cited by 0Source ↗Scholar ↗
2025

ForeSight: Multi-View Streaming Joint Object Detection and Trajectory Forecasting

Sandro Papais, Letian Wang, Brian Cheong, Steven L. Waslander
ICCV 2025poster

We introduce ForeSight, a novel joint detection and forecasting framework for vision-based 3D perception in autonomous vehicles. Traditional approaches treat detection and forecasting as separate sequential tasks, limiting their ability to leverage temporal cues. ForeSight addresses this limitation…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Large Self-Supervised Models Bridge the Gap in Domain Adaptive Object Detection

Marc-Antoine Lavoie, Anas Mahmoud, Steven L. Waslander
CVPR 2025poster

The current state-of-the-art methods in domain adaptive object detection (DAOD) use Mean Teacher self-labelling, where a teacher model, directly derived as an exponential moving average of the student model, is used to generate labels on the target domain which are then used to improve both models i…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

PSA-SSL: Pose and Size-aware Self-Supervised Learning on LiDAR Point Clouds

Barza Nisar, Steven L. Waslander
CVPR 2025poster

Self-supervised learning (SSL) on 3D point clouds has the potential to learn feature representations that can transfer to diverse sensors and multiple downstream perception tasks. However, recent SSL approaches fail to define pretext tasks that retain geometric information such as object pose and sc…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Prediction

Yang Zhou, Hao Shao, Letian Wang, Steven L. Waslander, Hongsheng Li, Yu Liu
ICLR 2025poster

Predicting the future motion of surrounding agents is essential for autonomous vehicles (AVs) to operate safely in dynamic, human-robot-mixed environments. However, the scarcity of large-scale driving datasets has hindered the development of robust and generalizable motion prediction models, limitin…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2024

Active Pose Refinement for Textureless Shiny Objects using the Structured Light Camera

Jun Yang, Jian Yao, Steven L. Waslander
IROS 2024poster

6D pose estimation of textureless shiny objects has become an essential problem in many robotic applications. Many pose estimators require high-quality depth data, often measured by structured light cameras. However, when objects have shiny surfaces (e.g., metal parts), these cameras fail to sense c…

Cited by 3Source ↗Scholar ↗
2024

DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features

Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic, Steven L. Waslander +4
NeurIPS 2024poster

We propose DistillNeRF, a self-supervised learning framework addressing the challenge of understanding 3D environments from limited 2D observations in outdoor autonomous driving scenes. Our method is a generalizable feedforward model that predicts a rich neural scene representation from sparse, sing…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2024

LMDrive: Closed-Loop End-to-End Driving with Large Language Models

Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L. Waslander, Yu Liu +1
CVPR 2024poster

Despite significant recent progress in the field of autonomous driving modern methods still struggle and can incur serious accidents when encountering long-tail unforeseen events and challenging urban scenarios. On the one hand large language models (LLM) have shown impressive reasoning capabilities…

Cited by 148Source ↗PDF ↗code ↗Scholar ↗
2024

Multiple View Geometry Transformers for 3D Human Pose Estimation

Ziwei Liao, Jialiang Zhu, Chunyu Wang, Han Hu, Steven L. Waslander
CVPR 2024poster

In this work we aim to improve the 3D reasoning ability of Transformers in multi-view 3D human pose estimation. Recent works have focused on end-to-end learning-based transformer designs which struggle to resolve geometric information accurately particularly during occlusion. Instead we propose a no…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2024

SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Prediction

Yang Zhou, Hao Shao, Letian Wang, Steven L. Waslander, Hongsheng Li, Yu Liu
CVPR 2024poster

Predicting the future motion of surrounding agents is essential for autonomous vehicles (AVs) to operate safely in dynamic human-robot-mixed environments. Context information such as road maps and surrounding agents' states provides crucial geometric and semantic information for motion behavior pred…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2024

Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors

Ziwei Liao, Binbin Xu, Steven L. Waslander
CoRL 2024poster

Object-level mapping builds a 3D map of objects in a scene with detailed shapes and poses from multi-view sensor observations. Conventional methods struggle to build complete shapes and estimate accurate poses due to partial occlusions and sensor noise. They require dense observations to cover all…

Cited by 3Source ↗code ↗Scholar ↗
2024

Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priors

Ziwei Liao, Jun Yang, Jingxing Qian, Angela P. Schoellig, Steven L. Waslander
ICRA 2024poster

3D object-level mapping is a fundamental problem in robotics, which is especially challenging when object CAD models are unavailable during inference. We propose a framework that can reconstruct high-quality object-level maps for unknown objects. Our approach takes multiple RGB-D images as input and…

Cited by 8Source ↗code ↗Scholar ↗
2024

UncertaintyTrack: Exploiting Detection and Localization Uncertainty in Multi-Object Tracking

Chang Won Lee, Steven L. Waslander
ICRA 2024poster

Multi-object tracking (MOT) methods have seen a significant boost in performance recently, due to strong interest from the research community and steadily improving object detection methods. The majority of tracking methods, which follow the tracking-by-detection (TBD) paradigm, blindly trust the in…

Cited by 2Source ↗code ↗Scholar ↗
2023

6D Pose Estimation for Textureless Objects on RGB Frames using Multi-View Optimization

Jun Yang, Wenjie Xue, Sahar Ghavidel, Steven L. Waslander
ICRA 2023poster

6D pose estimation of textureless objects is a valuable but challenging task for many robotic applications. In this work, we propose a framework to address this challenge using only RGB images acquired from multiple viewpoints. The core idea of our approach is to decouple 6D pose estimation into a s…

Cited by 16Source ↗Scholar ↗
2023

Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors

Letian Wang, Jie Liu, Hao Shao, Wenshuo Wang, Ruobing Chen, Yu Liu +1
RSS 2023poster

When autonomous vehicles are deployed on public roads, they will encounter countless and diverse driving situations. Many manually designed driving policies are difficult to scale to the real world. Fortunately, reinforcement learning has shown great success in many tasks by automatic trial and erro…

Cited by 42Source ↗PDF ↗code ↗Scholar ↗
2023

POV-SLAM: Probabilistic Object-Aware Variational SLAM in Semi-Static Environments

Jingxing Qian, Veronica Chatrath, James Servos, Aaron Mavrinac, Wolfram Burgard, Steven L Waslander +1
RSS 2023poster

Simultaneous localization and mapping (SLAM) in slowly varying scenes is important for long-term robot task completion in GPS-denied environments. Failing to detect scene changes may lead to inaccurate maps and, ultimately, lost robots. Classical SLAM algorithms assume static scenes, and recent work…

Cited by 12Source ↗PDF ↗code ↗Scholar ↗
2023

ReasonNet: End-to-End Driving With Temporal and Global Reasoning

Hao Shao, Letian Wang, Ruobing Chen, Steven L. Waslander, Hongsheng Li, Yu Liu
CVPR 2023poster

The large-scale deployment of autonomous vehicles is yet to come, and one of the major remaining challenges lies in urban dense traffic scenarios. In such cases, it remains challenging to predict the future evolution of the scene and future behaviors of objects, and to deal with rare adverse events…

Cited by 94Source ↗PDF ↗Scholar ↗
2023

Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss

Anas Mahmoud, Jordan S. K. Hu, Tianshu Kuai, Ali Harakeh, Liam Paull, Steven L. Waslander
CVPR 2023poster

An effective framework for learning 3D representations for perception tasks is distilling rich self-supervised image features via contrastive learning. However, image-to-point representation learning for autonomous driving datasets faces two main challenges: 1) the abundance of self-similarity, whic…

Cited by 35Source ↗PDF ↗code ↗Scholar ↗
2021

Categorical Depth Distribution Network for Monocular 3D Object Detection

Cody Reading, Ali Harakeh, Julia Chae, Steven L. Waslander
CVPR 2021poster

Monocular 3D object detection is a key problem for autonomous vehicles, as it provides a solution with simple configuration compared to typical multi-sensor systems. The main challenge in monocular 3D detection lies in accurately predicting object depth, which must be inferred from object and scene…

Cited by 621PDF ↗code ↗Scholar ↗
2021

Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors

Ali Harakeh, Steven L. Waslander
ICLR 2021poster

Predictive uncertainty estimation is an essential next step for the reliable deployment of deep object detectors in safety-critical tasks. In this work, we focus on estimating predictive distributions for bounding box regression output with variance networks. We show that in the context of object de…

Cited by 56Source ↗PDF ↗code ↗Scholar ↗
2021

Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching

Justin Tomasi, Brandon Wagstaff, Steven L. Waslander, Jonathan Kelly
RA-L 2021

Successful visual navigation depends upon capturing images that contain sufficient useful information. In this letter, we explore a data-driven approach to account for environmental lighting changes, improving the quality of images for use in visual odometry (VO) or visual simultaneous localization

Cited by 38Source ↗Scholar ↗
2021

ROBI: A Multi-View Dataset for Reflective Objects in Robotic Bin-Picking

Jun Yang, Yizhou Gao, Dong Li, Steven L. Waslander
IROS 2021poster

In robotic bin-picking applications, the perception of texture-less, highly reflective parts is a valuable but challenging task. The high glossiness can introduce fake edges in RGB images and inaccurate depth measurements, especially in heavily cluttered bin scenarios. In this paper, we present the…

Cited by 55Source ↗Scholar ↗
2021

Uncertainty-Constrained Differential Dynamic Programming in Belief Space for Vision Based Robots

Shatil Rahman, Steven L. Waslander
RA-L 2021

Most mobile robots follow a modular sense-plan-act system architecture that can lead to poor performance or even catastrophic failure for visual inertial navigation systems due to trajectories devoid of feature matches. Planning in belief space provides a unified approach to tightly couple the perce

Cited by 13Source ↗Scholar ↗
2020

AC/DCC : Accurate Calibration of Dynamic Camera Clusters for Visual SLAM

Jason Rebello, Angus Fung, Steven L. Waslander
ICRA 2020poster

In order to relate information across cameras in a Dynamic Camera Cluster (DCC), an accurate time-varying set of extrinsic calibration transformations need to be determined. Previous calibration approaches rely solely on collecting measurements from a known fiducial target which limits calibration a…

Cited by 5Source ↗code ↗Scholar ↗
2020

BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors

Ali Harakeh, Michael Smart, Steven L. Waslander
ICRA 2020poster

When incorporating deep neural networks into robotic systems, a major challenge is the lack of uncertainty measures associated with their output predictions. Methods for uncertainty estimation in the output of deep object detectors (DNNs) have been proposed in recent works, but have had limited succ…

Cited by 159Source ↗code ↗Scholar ↗
2019

Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation

Jason Ku, Alex D. Pon, Sean Walsh, Steven L. Waslander
IROS 2019poster

Accurately estimating the orientation of pedestrians is an important and challenging task for autonomous driving because this information is essential for tracking and predicting pedestrian behavior. This paper presents a flexible Virtual Multi-View Synthesis module that can be adopted into 3D objec…

Cited by 29Source ↗Scholar ↗
2019

Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction

Jason Ku, Alex D. Pon, Steven L. Waslander
CVPR 2019poster

We present MonoPSR, a monocular 3D object detection method that leverages proposals and shape reconstruction. First, using the fundamental relations of a pinhole camera model, detections from a mature 2D object detector are used to generate a 3D proposal per object in a scene. The 3D location of the…

Cited by 328PDF ↗Scholar ↗
2018

Encoderless Gimbal Calibration of Dynamic Multi-Camera Clusters

Christopher L. Choi, Jason Rebello, Leonid Koppel, Pranav Ganti, Arun Das, Steven L. Waslander
ICRA 2018poster

Dynamic Camera Clusters (DCCs) are multi-camera systems where one or more cameras are mounted on actuated mechanisms such as a gimbal. Existing methods for DCC calibration rely on joint angle measurements to resolve the time-varying transformation between the dynamic and static camera. This informat…

Cited by 11Source ↗Scholar ↗
2018

Joint 3D Proposal Generation and Object Detection from View Aggregation

Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, Steven L. Waslander
IROS 2018poster

We present AVOD, an Aggregate View Object Detection network for autonomous driving scenarios. The proposed neural network architecture uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network.…

Cited by 1858Source ↗code ↗Scholar ↗