AIConfPaper
← Search

Sai Vemprala

17 accepted papers

2026

DreamControl: Human-Inspired Whole-Body Humanoid Control for Scene Interaction Via Guided Diffusion

Dvij Kalaria, Sudarshan S Harithas, Pushkal Katara, Sangkyung Kwak, Sarthak Bhagat, Shankar Sastry +4
ICRA 2026poster

We introduce DreamControl, a novel methodology for learning autonomous whole-body humanoid skills. DreamControl leverages the strengths of diffusion models and Reinforcement Learning (RL): our core innovation is the use of a diffusion prior trained on human motion data, which subsequently guides an …

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2023

Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training

Yao Wei, Yanchao Sun, Ruijie Zheng, Sai Vemprala, Rogerio Bonatti, Shuhang Chen +4
ICCV 2023poster

We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning. DualMind uses a novel "Dual-phase" training strategy that emulates how humans lear…

Cited by 17PDF ↗code ↗Scholar ↗
2023

LATTE: LAnguage Trajectory TransformEr

Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Sai Vemprala +1
ICRA 2023poster

Natural language is one of the most intuitive ways to express human intent. However, translating instructions and commands towards robotic motion generation and deployment in the real world is far from being an easy task. The challenge of combining a robot's inherent low-level geometric and kinodyna…

Cited by 79Source ↗code ↗Scholar ↗
2023

PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training

Rogerio Bonatti, Sai Vemprala, Shuang Ma, Felipe Frujeri, Shuhang Chen, Ashish Kapoor
IROS 2023poster

Robotics has long been a field riddled with complex systems architectures whose modules and connections, whether traditional or learning-based, require significant human expertise and prior knowledge. Inspired by large pre-trained language models, this work introduces a paradigm for pretraining a ge…

Cited by 20Source ↗Scholar ↗
2022

3DB: A Framework for Debugging Computer Vision Models

Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet +6
NeurIPS 2022accept

We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation. We demonstrate, through a wide range of use cases, that 3DB allows users to discover vulnerabilities in computer vision systems and gain insights into how models make decision…

Cited by 49Source ↗PDF ↗code ↗Scholar ↗
2022

COMPASS: Contrastive Multimodal Pretraining for Autonomous Systems

Shuang Ma, Sai Vemprala, Wenshan Wang, Jayesh K. Gupta, Yale Song, Daniel McDufft +1
IROS 2022poster

Learning representations that generalize across tasks and domains is challenging yet necessary for autonomous systems. Although task-driven approaches are appealing, de-signing models specific to each application can be difficult in the face of limited data, especially when dealing with highly varia…

Cited by 10Source ↗code ↗Scholar ↗
2022

Learning to Simulate Realistic LiDARs

Benoît Guillard, Sai Vemprala, Jayesh K. Gupta, Ondrej Miksik, Vibhav Vineet, Pascal Fua +1
IROS 2022poster

Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling. To alleviate this, we introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We propose a m…

Cited by 19Source ↗Scholar ↗
2022

Missingness Bias in Model Debugging

Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala +1
ICLR 2022poster

Missingness, or the absence of features from an input, is a concept fundamental to many model debugging tools. However, in computer vision, pixels cannot simply be removed from an image. One thus tends to resort to heuristics such as blacking out pixels, which may in turn introduce bias into the deb…

Cited by 33Source ↗PDF ↗code ↗Scholar ↗
2021

Unadversarial Examples: Designing Objects for Robust Vision

Hadi Salman, Andrew Ilyas, Logan Engstrom, Sai Vemprala, Aleksander Madry, Ashish Kapoor
NeurIPS 2021poster

We study a class of computer vision settings wherein one can modify the design of the objects being recognized. We develop a framework that leverages this capability---and deep networks' unusual sensitivity to input perturbations---to design ``robust objects,'' i.e., objects that are explicitly opti…

Cited by 58Source ↗PDF ↗Scholar ↗
2020

Safety Considerations in Deep Control Policies with Safety Barrier Certificates Under Uncertainty

Tom Hirshberg, Sai Vemprala, Ashish Kapoor
IROS 2020poster

Recent advances in Deep Machine Learning have shown promise in solving complex perception and control loops via methods such as reinforcement and imitation learning. However, guaranteeing safety for such learned deep policies has been a challenge due to issues such as partial observability and diffi…

Cited by 5Source ↗Scholar ↗
2018

Real-Time Tumor Tracking for Pencil Beam Scanning Proton Therapy

Sai Vemprala, Srikanth Saripalli, Carlos Vargas, Martin Bues, Yanle Hu, Jiajian Shen
IROS 2018poster

In this paper, we describe the method and implementation of a real-time tumor tracking system for a pencil beam scanning (PBS) proton therapy system. PBS is an advanced cancer treatment system that can benefit from precise localization of the tumors through motion. We utilize techniques such as cros…

Cited by 6Source ↗Scholar ↗