Qiyuan Hu — accepted AI-conference papers

2023

Make-A-Video: Text-to-Video Generation without Text-Video Data

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang +7

ICLR 2023poster

We propose Make-A-Video -- an approach for directly translating the tremendous recent progress in Text-to-Image (T2I) generation to Text-to-Video (T2V). Our intuition is simple: learn what the world looks like and how it is described from paired text-image data, and learn how the world moves from un…

Cited by 1412 Source ↗PDF ↗Scholar ↗

2022

MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENeration

Thomas Hayes, Songyang Zhang, Xi Yin, Guan Pang, Sasha Sheng, Harry Yang +3

ECCV 2022poster

"Multimodal video-audio-text understanding and generation can benefit from datasets that are narrow but rich. The narrowness allows bite-sized challenges that the research community can make progress on. The richness ensures we are making progress along the core challenges. To this end, we present a…

Cited by 31 Source ↗PDF ↗code ↗Scholar ↗