AIConfPaper
← Search

Amit Raj

16 accepted papers

2025

Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generation

Agneet Chatterjee, Rahim Entezari, Maksym Zhuravinskyi, Maksim Lapin, Reshinth Adithyan, Amit Raj +3
NeurIPS 2025poster

Recent advances in video generation have enabled high-fidelity video synthesis from user provided prompts. However, existing models and benchmarks fail to capture the complexity and requirements of professional video generation. Towards that goal, we introduce Stable Cinemetrics, a structured evalua…

Cited by 0Source ↗Scholar ↗
2024

3D Congealing: 3D-Aware Image Alignment in the Wild

Yunzhi Zhang*, Zizhang Li, Amit Raj, Andreas Engelhardt, Yuanzhen Li, Tingbo Hou +2
ECCV 2024poster

"We propose , a novel problem of 3D-aware alignment for 2D images capturing semantically similar objects. Given a collection of unlabeled Internet images, our goal is to associate the shared semantic parts from the inputs and aggregate the knowledge from 2D images to a shared 3D canonical space. We…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

Diffusion Sampling with Momentum for Mitigating Divergence Artifacts

Suttisak Wizadwongsa, Worameth Chinchuthakun, Pramook Khungurn, Amit Raj, Supasorn Suwajanakorn
ICLR 2024poster

Despite the remarkable success of diffusion models in image generation, slow sampling remains a persistent issue. To accelerate the sampling process, prior studies have reformulated diffusion sampling as an ODE/SDE and introduced higher-order numerical methods. However, these methods often produce d…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2024

DiffusionLight: Light Probes for Free by Painting a Chrome Ball

Pakkapon Phongthawee, Worameth Chinchuthakun, Nontaphat Sinsunthithet, Varun Jampani, Amit Raj, Pramook Khungurn +1
CVPR 2024poster

We present a simple yet effective technique to estimate lighting in a single input image. Current techniques rely heavily on HDR panorama datasets to train neural networks to regress an input with limited field-of-view to a full environment map. However these approaches often struggle with real-worl…

Cited by 24Source ↗PDF ↗code ↗Scholar ↗
2024

MagicMirror: Fast and High-Quality Avatar Generation with Constrained Search Space

Armand Comas, Di Qiu*, Menglei Chai, Marcel C. Bühler, Amit Raj, Ruiqi Gao +5
ECCV 2024poster

"We introduce a novel framework for 3D human avatar generation and personalization, leveraging text prompts to enhance user engagement and customization. Central to our approach are key innovations aimed at overcoming the challenges in photo-realistic avatar synthesis. Firstly, we utilize a conditio…

Cited by 3Source ↗PDF ↗code ↗Scholar ↗
2024

Probing the 3D Awareness of Visual Foundation Models

Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Abhishek Kar, Yuanzhen Li, Michael Rubinstein +4
CVPR 2024poster

Recent advances in large-scale pretraining have yielded visual foundation models with strong capabilities. Not only can recent models generalize to arbitrary images for their training task their intermediate representations are useful for other visual tasks such as detection and segmentation. Given…

Cited by 83Source ↗PDF ↗code ↗Scholar ↗
2024

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild

Andreas Engelhardt, Amit Raj, Mark Boss, Yunzhi Zhang, Abhishek Kar, Yuanzhen Li +5
CVPR 2024poster

We present SHINOBI an end-to-end framework for the reconstruction of shape material and illumination from object images captured with varying lighting pose and background. Inverse rendering of an object based on unconstrained image collections is a long-standing challenge in computer vision and grap…

Cited by 6Source ↗PDF ↗Scholar ↗
2024

WordRobe: Text-Guided Generation of Textured 3D Garments

Astitva Srivastava*, Pranav Manu, Amit Raj, Varun Jampani, Avinash Sharma
ECCV 2024poster

"In this paper, we tackle a new and challenging problem of text-driven generation of 3D garments with high-quality textures. We propose, WordRobe, a novel framework for the generation of unposed & textured 3D garment meshes from user-friendly text prompts. We achieve this by first learning a latent…

Cited by 10Source ↗PDF ↗Scholar ↗
2023

ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections

Chun-Han Yao, Amit Raj, Wei-Chih Hung, Michael Rubinstein, Yuanzhen Li, Ming-Hsuan Yang +1
NeurIPS 2023poster

Estimating 3D articulated shapes like animal bodies from monocular images is inherently challenging due to the ambiguities of camera viewpoint, pose, texture, lighting, etc. We propose ARTIC3D, a self-supervised framework to reconstruct per-instance 3D shapes from a sparse image collection in-the-wi…

Cited by 23Source ↗PDF ↗Scholar ↗
2023

DreamBooth3D: Subject-Driven Text-to-3D Generation

Amit Raj, Srinivas Kaza, Ben Poole, Michael Niemeyer, Nataniel Ruiz, Ben Mildenhall +6
ICCV 2023poster

We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject. Our approach combines recent advances in personalizing text-to-image models (DreamBooth) with text-to-3D generation (DreamFusion). We find that naively combining…

Cited by 211PDF ↗Scholar ↗
2021

ANR: Articulated Neural Rendering for Virtual Avatars

Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, Christoph Lassner
CVPR 2021poster

Deferred Neural Rendering (DNR) uses a three-step pipeline to translate a mesh representation into an RGB image. The combination of a traditional rendering stack with neural networks hits a sweet spot in terms of computational complexity and realism of the resulting images. Using skinned meshes for…

Cited by 70PDF ↗Scholar ↗
2019

Kernel Mean Matching for Content Addressability of GANs

Wittawat Jitkrittum, Patsorn Sangkloy, Muhammad Waleed Gondal, Amit Raj, James Hays, Bernhard Schölkopf
ICML 2019oral

We propose a novel procedure which adds "content-addressability" to any given unconditional implicit model e.g., a generative adversarial network (GAN). The procedure allows users to control the generative process by specifying a set (arbitrary size) of desired examples based on which similar sample…

Cited by 12Source ↗PDF ↗code ↗Scholar ↗
2018

Deep Forward and Inverse Perceptual Models for Tracking and Prediction

Alexander Lambert, Amirreza Shaban, Amit Raj, Zhen Liu, Byron Boots
ICRA 2018poster

We consider the problems of learning forward models that map state to high-dimensional images and inverse models that map high-dimensional images to state in robotics. Specifically, we present a perceptual model for generating video frames from state with deep networks, and provide a framework for i…

Cited by 25Source ↗Scholar ↗
2018

SwapNet: Garment Transfer in Single View Images

Amit Raj, Patsorn Sangkloy, Huiwen Chang, Jingwan Lu, Duygu Ceylan, James Hays
ECCV 2018poster

We present SwapNet, a framework to transfer garments across images of people with arbitrary body pose, shape, and clothing. Garment transfer is a challenging task that requires (i) disentangling the features of the clothing from the body pose and shape and (ii) realistic synthesis of the garment tex…

Cited by 61Source ↗PDF ↗Scholar ↗
2018

TextureGAN: Controlling Deep Image Synthesis With Texture Patches

Wenqi Xian, Patsorn Sangkloy, Varun Agrawal, Amit Raj, Jingwan Lu, Chen Fang +2
CVPR 2018poster

In this paper, we investigate deep image synthesis guided by sketch, color, and texture. Previous image synthesis methods can be controlled by sketch and color strokes but we are the first to examine texture control. We allow a user to place a texture patch on a sketch at arbitrary locations and sca…

Cited by 353Source ↗PDF ↗Scholar ↗