AIConfPaper
← Search

Wentao Yuan

16 accepted papers

2026

GraspGen: A Diffusion-Based Framework for 6-DOF Grasping with On-Generator Training

Adithyavairavan Murali, Balakumar Sundaralingam, Yu-Wei Chao, Wentao Yuan, Jun Yamada, Mark Carlson +4
ICRA 2026poster

Grasping is a fundamental robot skill, yet despite significant research advancements, learning-based 6-DOF grasping approaches are still not turnkey and struggle to generalize across different embodiments and in-the-wild settings. We build upon the recent success on modeling the object-centric grasp…

Cited by 0Source ↗PDF ↗Scholar ↗
2026

RoboVista: Evaluating Vision Language Models for Diverse Robot Applications

Shuangyu Xie, Kaiyuan Chen, Ziyang Chen, Simeon Adebola, Yixuan Huang, Zehan Ma +5
RSS 2026poster

Diverse applications for robotics, such as industry and agriculture, require robots to operate across various embodiments, changing visual conditions, and complex planning. Vision–Language Models (VLMs) offer a promising foundation for general-purpose and interpretable robotic reasoning. Aligning VL…

Cited by 0Source ↗Scholar ↗
2025

AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation

Jiafei Duan, Wilbert Pumacay, Nishanth Kumar, Yi Ru Wang, Shulin Tian, Wentao Yuan +4
ICLR 2025poster

Robotic manipulation in open-world settings requires not only task execution but also the ability to detect and learn from failures. While recent advances in vision-language models (VLMs) and large language models (LLMs) have improved robots' spatial reasoning and problem-solving abilities, they sti…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2024

Avoid Everything: Model-Free Collision Avoidance with Expert-Guided Fine-Tuning

Adam Fishman, Aaron Walsman, Mohak Bhardwaj, Wentao Yuan, Balakumar Sundaralingam, Byron Boots +1
CoRL 2024poster

The world is full of clutter. In order to operate effectively in uncontrolled, real world spaces, robots must navigate safely by executing tasks around obstacles while in proximity to hazards. Creating safe movement for robotic manipulators remains a long-standing challenge in robotics, particularly…

Cited by 3Source ↗Scholar ↗
2024

Evaluating Robustness of Visual Representations for Object Assembly Task Requiring Spatio-Geometrical Reasoning

Chahyon Ku, Carl Winge, Ryan Diaz, Wentao Yuan, Karthik Desingh
ICRA 2024poster

This paper primarily focuses on evaluating and benchmarking the robustness of visual representations in the context of object assembly tasks. Specifically, it investigates the alignment and insertion of objects with geometrical extrusions, commonly referred to as a peg-in-hole task. The accuracy req…

Cited by 2Source ↗Scholar ↗
2024

Manipulate-Anything: Automating Real-World Robots using Vision-Language Models

Jiafei Duan, Wentao Yuan, Wilbert Pumacay, Yi Ru Wang, Kiana Ehsani, Dieter Fox +1
CoRL 2024poster

Large-scale endeavors like RT-1 and widespread community efforts such as Open-X-Embodiment have contributed to growing the scale of robot demonstration data. However, there is still an opportunity to improve the quality, quantity, and diversity of robot demonstration data. Although vision-language m…

Cited by 39Source ↗code ↗Scholar ↗
2024

RoboPoint: A Vision-Language Model for Spatial Affordance Prediction in Robotics

Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali +2
CoRL 2024poster

From rearranging objects on a table to putting groceries into shelves, robots must plan precise action points to perform tasks accurately and reliably. In spite of the recent adoption of vision language models (VLMs) to control robot behavior, VLMs struggle to precisely articulate robot actions usin…

Cited by 53Source ↗code ↗Scholar ↗
2023

M2T2: Multi-Task Masked Transformer for Object-centric Pick and Place

Wentao Yuan, Adithyavairavan Murali, Arsalan Mousavian, Dieter Fox
CoRL 2023poster

With the advent of large language models and large-scale robotic datasets, there has been tremendous progress in high-level decision-making for object manipulation. These generic models are able to interpret complex tasks using language commands, but they often have difficulties generalizing to out-…

Cited by 23Source ↗code ↗Scholar ↗
2023

TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

Xiangyun Meng, Nathan Hatch, Alexander Lambert, Anqi Li, Nolan Wagener, Matthew Schmittle +8
RSS 2023poster

Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene prediction have yet to be successfully adapted for complex…

Cited by 62Source ↗PDF ↗Scholar ↗
2022

KD-MVS: Knowledge Distillation Based Self-Supervised Learning for Multi-View Stereo

Yikang Ding, Qingtian Zhu, Xiangyue Liu, Wentao Yuan, Haotian Zhang, Chi Zhang
ECCV 2022poster

"Supervised multi-view stereo (MVS) methods have achieved remarkable progress in terms of reconstruction quality, but suffer from the challenge of collecting large-scale ground-truth depth. In this paper, we propose a novel self-supervised training pipeline for MVS based on knowledge distillation, t…

Cited by 36Source ↗PDF ↗code ↗Scholar ↗
2022

Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives

Wentao Yuan, Qingtian Zhu, Xiangyue Liu, Yikang Ding, Haotian Zhang, Chi Zhang
ECCV 2022poster

"Recently, Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful and promising tool to represent different kinds of signals due to its continuous, differentiable properties, showing superiorities to classical discretized representations. However, the trai…

Cited by 6Source ↗PDF ↗code ↗Scholar ↗
2021

SORNet: Spatial Object-Centric Representations for Sequential Manipulation

Wentao Yuan, Chris Paxton, Karthik Desingh, Dieter Fox
CoRL 2021oral

Sequential manipulation tasks require a robot to perceive the state of an environment and plan a sequence of actions leading to a desired goal state, where the ability to reason about spatial relationships among object entities from raw sensor inputs is crucial. Prior works relying on explicit state…

Cited by 86Source ↗code ↗Scholar ↗
2021

STaR: Self-Supervised Tracking and Reconstruction of Rigid Objects in Motion With Neural Rendering

Wentao Yuan, Zhaoyang Lv, Tanner Schmidt, Steven Lovegrove
CVPR 2021poster

We present STaR, a novel method that performs Self-supervised Tracking and Reconstruction of dynamic scenes with rigid motion from multi-view RGB videos without any manual annotation. Recent work has shown that neural networks are surprisingly effective at the task of compressing many views of a sce…

Cited by 172PDF ↗Scholar ↗
2021

Self-Supervised Learning on 3D Point Clouds by Learning Discrete Generative Models

Benjamin Eckart, Wentao Yuan, Chao Liu, Jan Kautz
CVPR 2021poster

While recent pre-training tasks on 2D images have proven very successful for transfer learning, pre-training for 3D data remains challenging. In this work, we introduce a general method for 3D self-supervised representation learning that 1) remains agnostic to the underlying neural network architect…

Cited by 74PDF ↗Scholar ↗
2020

DeepGMR: Learning Latent Gaussian Mixture Models for Registration

Wentao Yuan, Benjamin Eckart, Kihwan Kim, Varun Jampani, Dieter Fox, Jan Kautz
ECCV 2020poster

Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics. For the last few decades, existing registration algorithms have struggled in situations with large transformations, noise, and time constraints. In this paper, we introduce Deep Gaussian Mixture Registrat…

Cited by 298Source ↗PDF ↗code ↗Scholar ↗
2018

Intelligent Shipwreck Search Using Autonomous Underwater Vehicles

Jeffrey Rutledge, Wentao Yuan, Jane Wu, Sam Freed, Amy Lewis, Zoë Wood +2
ICRA 2018poster

This paper presents an autonomous robot system that is designed to autonomously search for and geo-localize potential underwater archaeological sites. The system, based on Autonomous Underwater Vehicles, invokes a multi-step pipeline. First, the AUV constructs a high altitude scan over a large area…

Cited by 30Source ↗Scholar ↗