12 accepted papers
We introduce SeaSplat, a method to enable real-time rendering of underwater scenes leveraging recent advances in 3D radiance fields. Underwater scenes are challenging visual environments, as rendering through a medium such as water introduces both range and color dependent effects on image capture.
Efficient and accurate motion prediction is crucial for ensuring safety and informed decision-making in autonomous driving, particularly under dynamic real-world conditions that necessitate multi-modal forecasts. We introduce TrajFlow, a novel flow matching-based motion prediction framework that add
Video summarization remains a huge challenge in computer vision due to the size of the input videos to be summarized. We propose an efficient, language-only video summarizer that achieves competitive accuracy with high data efficiency. Using only textual captions obtained via a zero-shot approach, w…
In this work, we propose a novel framework for image segmentation guided by visual prompting which leverages the power of vision foundation models. Inspired by recent advancements in computer vision, our approach integrates multiple large-scale pretrained models to address the challenges of segment…
Teaching robots novel skills with demonstrations via human-in-the-loop data collection techniques like kinesthetic teaching or teleoperation puts a heavy burden on human supervisors. In contrast to this paradigm, it is often significantly easier to provide raw, action-free visual data of tasks being…
This paper studies instrument classification of solo sheet music. Whereas previous work has focused on instrument recognition in audio data, we instead approach the instrument classification problem using raw sheet music images. Our approach first converts the sheet music image into a sequence of mu…
We present a novel approach to robotic grasp planning using both a learned grasp proposal network and a learned 3D shape reconstruction network. Our system generates 6-DOF grasps from a single RGB-D image of the target object, which is provided as input to both networks. By using the geometric recon…
A major challenge in reinforcement learning is exploration, when local dithering methods such as epsilon-greedy sampling are insufficient to solve a given task. Many recent methods have proposed to intrinsically motivate an agent to seek novel states, driving the agent to discover improved reward. H…
The state estimation and control of a ball-balancing robot under high yaw rate is a challenging problem due to its highly nonlinear 3D dynamic. The small size and low-cost components in our Micro Ball-Balancing Robot makes the system inherently very noisy which further increases the complexity of th…
This paper presents a (patent-pending) small, quasi-static, minimal-complexity Stair Climbing Robot (SCR). The vehicle design is given simply by adding a third motor to a (Segway-like) Mobile Inverted Pendulum (MIP), enabling it to maneuver up stairs, leveraging feedback control, by planting it's “f…
Watching expert demonstrations is an important way for humans and robots to reason about affordances of unseen objects. In this paper, we consider the problem of reasoning object affordances through the feature embedding of demonstration videos. We design the Demo2Vec model which learns to extract e…
Ball-balancing robots (BBRs) are endowed with rich dynamics. When properly designed and stabilized via feedback to eliminate jitter, and intuitively coordinated with a well-designed smartphone interface, BBRs exhibit a uniquely fluid and organic motion. Unlike mobile inverted pendulums (MIPs, akin t…