17 accepted papers
A key challenge in manipulation is learning a policy that can robustly generalize to diverse visual environments. A promising mechanism for learning robust policies is to leverage video generative models, which are pretrained on large-scale datasets of internet videos. In this paper, we propose a vi…
Scene-aware Complementary Item Retrieval (CIR) is a challenging task which requires to generate a set of compatible items across domains. Due to the subjectivity, it is difficult to set up a rigorous standard for both data collection and learning objectives. To address this challenging task, we prop…
"We propose an efficient Video-Language Alignment (ViLA) network. Our ViLA model addresses both efficient frame sampling and effective cross-modal alignment in a unified way. In our ViLA network, we design a new learnable text-guided Frame-Prompter together with a cross-modal distillation (QFormer-D…
Over the last decade, a plethora of soft robotic devices have been proposed for the execution of complex grasping and dexterous manipulation tasks. Tasks requiring such increased dexterity are typically executed using fully-actuated, rigid end-effectors equipped with sophisticated sensing and contro…
Unmanned aerial vehicles (UAVs) have revolutionised various industries, such as agriculture, remote sensing, and infrastructure inspection. To explore new designs and improve UAV flight performance, roboticists are seeking inspiration from nature. In this paper, we present a bioinspired tailsitter U…
Soft robotic devices have been popular in handling intricate grasping and dexterous manipulation tasks, serving as an alternative to conventional, rigid end-effectors. These devices are relatively simple, lightweight, and cost-effective. Recently, kirigami based structures have been used to create l…
"Fabric materials are central to recreating realistic appearance of avatars in a virtual world and many VR applications, ranging from virtual try-on, teleconferencing, to character animation. We propose an end-to-end network model that uses video input to estimate the fabric materials of the garment…
Soft robots are capable of effortlessly adapting to their environment using elastic materials that impart structural compliance into their designs, allowing them to execute complex tasks with minimal sensing and control. However, soft robots cannot exert high forces and can only handle low deformati…
Grasping and manipulation are complex and demanding tasks, especially when executed in dynamic and unstructured environments. Typically, such tasks are executed by rigid articulated end-effectors, with a plethora of actuators that need sophisticated sensing and complex control laws to execute them e…
We introduce an efficient differentiable fluid simulator that can be integrated with deep neural networks as a part of layers for learning dynamics and solving control problems. It offers the capability to handle one-way coupling of fluids with rigid objects using a variational principle that natura…
We present a method for differentiable simulation of soft articulated bodies. Our work enables the integration of differentiable physical dynamics into gradient-based pipelines. We develop a top-down matrix assembly algorithm within Projective Dynamics and derive a generalized dry friction model for…
We present a method for efficient differentiable simulation of articulated bodies. This enables integration of articulated body dynamics into deep learning frameworks, and gradient-based optimization of neural networks that operate on articulated bodies. We derive the gradients of the contact solver…
The generation of realistic apparel model has become increasingly popular as a result of the rapid pace of change in fashion trends and the growing need for garment models in various applications such as virtual try-on. For such application requirements, it is important to have a general cloth model…
Differentiable physics is a powerful approach to learning and control problems that involve physical objects and environments. While notable progress has been made, the capabilities of differentiable physics solvers remain limited. We develop a scalable framework for differentiable physics that can…
We propose a differentiable cloth simulator that can be embedded as a layer in deep neural networks. This approach provides an effective, robust framework for modeling cloth dynamics, self-collisions, and contacts. Due to the high dimensionality of the dynamical system in modeling cloth, traditiona…
We propose a scalable neural network framework to reconstruct the 3D mesh of a human body from multi-view images, in the subspace of the SMPL model. Use of multi-view images can significantly reduce the projection ambiguity of the problem, increasing the reconstruction accuracy of the 3D human body…
Image understanding enables better reconstruction of the physical world from images and videos. Existing methods focus largely on geometry and visual appearance of the reconstructed scene. In this paper, we extend the frontier in image understanding and present a new technique to recover the materia…