AIConfPaper
← Search

Balakumar Sundaralingam

29 accepted papers

2026

Grasp-MPC: Closed-Loop Visual Grasping Via Value-Guided Model Predictive Control

Jun Yamada, Adithyavairavan Murali, Ajay Uday Mandlekar, Clemens Eppner, Ingmar Posner, Balakumar Sundaralingam
ICRA 2026poster

Grasping of diverse objects in unstructured environments remains a significant challenge. Open-loop grasping methods, effective in controlled settings, struggle in cluttered environments. Grasp prediction errors and object pose changes during grasping are the main causes of failure. In contrast, clo…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2026

GraspGen: A Diffusion-Based Framework for 6-DOF Grasping with On-Generator Training

Adithyavairavan Murali, Balakumar Sundaralingam, Yu-Wei Chao, Wentao Yuan, Jun Yamada, Mark Carlson +4
ICRA 2026poster

Grasping is a fundamental robot skill, yet despite significant research advancements, learning-based 6-DOF grasping approaches are still not turnkey and struggle to generalize across different embodiments and in-the-wild settings. We build upon the recent success on modeling the object-centric grasp…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning

Neel Jawale, Byron Boots, Balakumar Sundaralingam, Mohak Bhardwaj
ICRA 2025

We investigate the problem of teaching a robot manipulator to perform dynamic non-prehensile object transport, also known as the ‘robot waiter’ task, from a limited set of real-world demonstrations. We propose an approach that combines batch reinforcement learning (RL) with modelpredictive control (

Cited by 4Source ↗Scholar ↗
2025

Inference-Time Policy Steering Through Human Interactions

Yanwei Wang, Lirui Wang, Yilun Du, Balakumar Sundaralingam, Xuning Yang, Yu-Wei Chao +3
ICRA 2025

Generative policies trained with human demonstrations can autonomously accomplish multimodal, longhorizon tasks. However, during inference, humans are often removed from the policy execution loop, limiting the ability to guide a pre-trained policy towards a specific sub-goal or trajectory shape amon

Cited by 37Source ↗code ↗Scholar ↗
2025

VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning

Binghao Huang, Jie Xu, Iretiayo Akinola, Wei Yang, Balakumar Sundaralingam, Rowland O'Flaherty +5
CoRL 2025poster

Humans excel at bimanual assembly tasks by adapting to rich tactile feedback—a capability that remains difficult to replicate in robots through behavioral cloning alone, due to the suboptimality and limited diversity of human demonstrations. In this work, we present VT-Refine, a visuo-tactile policy…

Cited by 0Source ↗code ↗Scholar ↗
2024

Avoid Everything: Model-Free Collision Avoidance with Expert-Guided Fine-Tuning

Adam Fishman, Aaron Walsman, Mohak Bhardwaj, Wentao Yuan, Balakumar Sundaralingam, Byron Boots +1
CoRL 2024poster

The world is full of clutter. In order to operate effectively in uncontrolled, real world spaces, robots must navigate safely by executing tasks around obstacles while in proximity to hazards. Creating safe movement for robotic manipulators remains a long-standing challenge in robotics, particularly…

Cited by 3Source ↗Scholar ↗
2024

DiffusionSeeder: Seeding Motion Optimization with Diffusion for Rapid Motion Planning

Huang Huang, Balakumar Sundaralingam, Arsalan Mousavian, Adithyavairavan Murali, Ken Goldberg, Dieter Fox
CoRL 2024poster

Running optimization across many parallel seeds leveraging GPU compute [2] have relaxed the need for a good initialization, but this can fail if the problem is highly non-convex as all seeds could get stuck in local minima. One such setting is collision-free motion optimization for robot manipulatio…

Cited by 33Source ↗Scholar ↗
2023

CuRobo: Parallelized Collision-Free Robot Motion Generation

Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis +6
ICRA 2023poster

This paper explores the problem of collision-free motion generation for manipulators by formulating it as a global motion optimization problem. We develop a parallel optimization technique to solve this problem and demonstrate its effectiveness on massively parallel GPUs. We show that combining simp…

Cited by 74Source ↗Scholar ↗
2023

DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality

Ankur Handa, Arthur Allshire, Viktor Makoviychuk, Aleksei Petrenko, Ritvik Singh, Jingzhou Liu +5
ICRA 2023poster

Recent work has demonstrated the ability of deep reinforcement learning (RL) algorithms to learn complex robotic behaviours in simulation, including in the domain of multi-fingered manipulation. However, such models can be challenging to transfer to the real world due to the gap between simulation a…

Cited by 146Source ↗Scholar ↗
2023

RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control

Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay, Bowen Wen, Ye Yuan, Stephen Tyree +3
ICRA 2023poster

We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera (without depth) that is either handheld or mounted on the robot end effector. A NeRF-like process is used…

Cited by 14Source ↗code ↗Scholar ↗
2023

Ready, Set, Plan! Planning to Goal Sets Using Generalized Bayesian Inference

Jana Pavlasek, Stanley Robert Lewis, Balakumar Sundaralingam, Fabio Ramos, Tucker Hermans
CoRL 2023poster

Many robotic tasks can have multiple and diverse solutions and, as such, are naturally expressed as goal sets. Examples include navigating to a room, finding a feasible placement location for an object, or opening a drawer enough to reach inside. Using a goal set as a planning objective requires tha…

Cited by 4Source ↗Scholar ↗
2023

VaPr: Variable-Precision Tensors to Accelerate Robot Motion Planning

Yu-Shun Hsiao, Siva Kumar Sastry Hari, Balakumar Sundaralingam, Jason Yik, Thierry Tambe, Charbel Sakr +2
IROS 2023poster

High-dimensional motion generation requires nu-merical precision for smooth, collision-free solutions. Typically, double-precision or single-precision floating-point (FP) formats are utilized. Using these for big tensors imposes a strain on the memory bandwidth provided by the devices and alters the…

Cited by 5Source ↗Scholar ↗
2022

Correcting Robot Plans with Natural Language Feedback

Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Torralba +2
RSS 2022poster

When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, under-specified, or beyond planners’ ability to solve. In these cases, corrections provide a valuable tool for human-in-the-loop robot control. Corrections might take the form of new goal…

Cited by 110Source ↗PDF ↗Scholar ↗
2022

DefGraspSim: Physics-Based Simulation of Grasp Outcomes for 3D Deformable Objects

Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Ruzena Bajcsy +2
RA-L 2022

Robotic grasping of 3D deformable objects (e.g., fruits/vegetables, internal organs, bottles/boxes) is critical for real-world applications such as food processing, robotic surgery, and household automation. However, developing grasp strategies for such objects is uniquely challenging. Unlike rigid

Cited by 37Source ↗Scholar ↗
2022

Geometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior

Karl Van Wyk, Mandy Xie, Anqi Li, Muhammad Asif Rana, Buck Babich, Bryan Peele +6
RA-L 2022

Classical mechanical systems are central to controller design in energy shaping methods of geometric control. However, their expressivity is limited by position-only metrics and the intimate link between metric and geometry. Recent work on Riemannian Motion Policies (RMPs) has shown that shedding th

Cited by 49Source ↗Scholar ↗
2022

HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers

Yu-Wei Chao, Chris Paxton, Yu Xiang, Wei Yang, Balakumar Sundaralingam, Tao Chen +3
ICRA 2022poster

We introduce a new simulation benchmark “Han-doverSim” for human-to-robot object handovers. To simulate the giver's motion, we leverage a recent motion capture dataset of hand grasping of objects. We create training and evaluation environments for the receiver with standardized protocols and metrics…

Cited by 29Source ↗code ↗Scholar ↗
2022

Learning Perceptual Concepts by Bootstrapping From Human Queries

Andreea Bobu, Chris Paxton, Wei Yang, Balakumar Sundaralingam, Yu-Wei Chao, Maya Cakmak +1
RA-L 2022

When robots operate in human environments, it's critical that humans can quickly teach them new <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">concepts:</i> object-centric properties of the environment that they care about (e.g., objects <italic xml

Cited by 17Source ↗Scholar ↗
2022

Model Predictive Control for Fluid Human-to-Robot Handovers

Wei Yang, Balakumar Sundaralingam, Chris Paxton, Iretiayo Akinola, Yu-Wei Chao, Maya Cakmak +1
ICRA 2022poster

Human-robot handover is a fundamental yet challenging task in human-robot interaction and collaboration. Recently, remarkable progressions have been made in human-to-robot handovers of unknown objects by using learning-based grasp generators. However, how to responsively generate smooth motions to t…

Cited by 31Source ↗Scholar ↗
2021

Joint Space Control via Deep Reinforcement Learning

Visak Kumar, David Hoeller, Balakumar Sundaralingam, Jonathan Tremblay, Stan Birchfield
IROS 2021poster

The dominant way to control a robot manipulator uses hand-crafted differential equations leveraging some form of inverse kinematics / dynamics. We propose a simple, versatile joint-level controller that dispenses with differential equations entirely. A deep neural network, trained via model-free rei…

Cited by 25Source ↗Scholar ↗
2021

STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation

Mohak Bhardwaj, Balakumar Sundaralingam, Arsalan Mousavian, Nathan D. Ratliff, Dieter Fox, Fabio Ramos +1
CoRL 2021oral

Sampling-based model-predictive control (MPC) is a promising tool for feedback control of robots with complex, non-smooth dynamics, and cost functions. However, the computationally demanding nature of sampling-based MPC algorithms has been a key bottleneck in their application to high-dimensional ro…

Cited by 152Source ↗code ↗Scholar ↗
2021

Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections

Yashraj Narang, Balakumar Sundaralingam, Miles Macklin, Arsalan Mousavian, Dieter Fox
ICRA 2021poster

Tactile sensing is critical for robotic grasping and manipulation of objects under visual occlusion. However, in contrast to simulations of robot arms and cameras, current simulations of tactile sensors have limited accuracy, speed, and utility. In this work, we develop an efficient 3D finite elemen…

Cited by 69Source ↗Scholar ↗
2020

Benchmarking In-Hand Manipulation

Silvia Cruciani, Balakumar Sundaralingam, Kaiyu Hang, Vikash Kumar, Tucker Hermans, Danica Kragic
RA-L 2020

The purpose of this benchmark is to evaluate the planning and control aspects of robotic in-hand manipulation systems. The goal is to assess the system's ability to change the pose of a hand-held object by either using the fingers, environment or a combination of both. Given an object surface mesh f

Cited by 45Source ↗Scholar ↗
2020

Learning Continuous 3D Reconstructions for Geometrically Aware Grasping

Mark Van der Merwe, Qingkai Lu, Balakumar Sundaralingam, Martin Matak, Tucker Hermans
ICRA 2020poster

Deep learning has enabled remarkable improvements in grasp synthesis for previously unseen objects from partial object views. However, existing approaches lack the ability to explicitly reason about the full 3D geometry of the object when selecting a grasp, relying on indirect geometric reasoning de…

Cited by 108Source ↗Scholar ↗
2019

Joint Inference of Kinematic and Force Trajectories with Visuo-Tactile Sensing

Alexander Sasha Lambert, Mustafa Mukadam, Balakumar Sundaralingam, Nathan Ratliff, Byron Boots, Dieter Fox
ICRA 2019poster

To perform complex tasks, robots must be able to interact with and manipulate their surroundings. One of the key challenges in accomplishing this is robust state estimation during physical interactions, where the state involves not only the robot and the object being manipulated, but also the state…

Cited by 38Source ↗Scholar ↗
2019

Learning Latent Space Dynamics for Tactile Servoing

Giovanni Sutanto, Nathan Ratliff, Balakumar Sundaralingam, Yevgen Chebotar, Zhe Su, Ankur Handa +1
ICRA 2019poster

To achieve a dexterous robotic manipulation, we need to endow our robot with tactile feedback capability, i.e. the ability to drive action based on tactile sensing. In this paper, we specifically address the challenge of tactile servoing, i.e. given the current tactile sensing and a target/goal tact…

Cited by 39Source ↗Scholar ↗
2019

Robust Learning of Tactile Force Estimation through Robot Interaction

Balakumar Sundaralingam, Alexander Sasha Lambert, Ankur Handa, Byron Boots, Tucker Hermans, Stan Birchfield +2
ICRA 2019poster

Current methods for estimating force from tactile sensor signals are either inaccurate analytic models or task-specific learned models. In this paper, we explore learning a robust model that maps tactile sensor signals to force. We specifically explore learning a mapping for the SynTouch BioTac sens…

Cited by 71Source ↗Scholar ↗
2018

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Yu Xiang, Dieter Fox, Stan Birchfield
CoRL 2018

Using synthetic data for training deep neural networks for robotic manipulation holds the promise of an almost unlimited amount of pre-labeled training data, generated safely out of harm’s way. One of the key challenges of synthetic data, to date, has been to bridge the so-called reality gap, so tha

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2018

Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation

Balakumar Sundaralingam, Tucker Hermans
ICRA 2018poster

This paper explores the problem of autonomous, in-hand regrasping-the problem of moving from an initial grasp on an object to a desired grasp using the dexterity of a robot's fingers. We propose a planner for this problem which alternates between finger gaiting, and in-grasp manipulation. Finger gai…

Cited by 55Source ↗Scholar ↗
2017

Relaxed-Rigidity Constraints: In-Grasp Manipulation using Purely Kinematic Trajectory Optimization

Balakumar Sundaralingam, Tucker Hermans
RSS 2017poster

This paper proposes a novel approach to performing in-grasp manipulation planning: the problem of moving an object with reference to the palm from an initial pose to a goal pose without breaking or making contacts. Our method to perform in-grasp manipulation uses kinematic trajectory optimization wh…

Cited by 47Source ↗PDF ↗Scholar ↗