Hsiao-Yu Fish Tung — accepted AI-conference papers

2020

Embodied Language Grounding With 3D Visual Feature Representations

Mihir Prabhudesai, Hsiao-Yu Fish Tung, Syed Ashar Javed, Maximilian Sieb, Adam W. Harley, Katerina Fragkiadaki

CVPR 2020poster

We propose associating language utterances to 3D visual abstractions of the scene they describe. The 3D visual abstractions are encoded as 3-dimensional visual feature maps. We infer these 3D visual scene feature maps from RGB images of the scene via view prediction: when the generated 3D scene feat…

Cited by 23 PDF ↗Scholar ↗

2020

Learning from Unlabelled Videos Using Contrastive Predictive Neural 3D Mapping

Adam W. Harley, Shrinidhi K. Lakshmikanth, Fangyu Li, Xian Zhou, Hsiao-Yu Fish Tung, Katerina Fragkiadaki

ICLR 2020poster

Predictive coding theories suggest that the brain learns by predicting observations at various levels of abstraction. One of the most basic prediction tasks is view prediction: how would a given scene look from an alternative viewpoint? Humans excel at this task. Our ability to imagine and fill in m…

Cited by 30 Source ↗code ↗Scholar ↗

2019

Learning Spatial Common Sense With Geometry-Aware Recurrent Networks

Hsiao-Yu Fish Tung, Ricson Cheng, Katerina Fragkiadaki

CVPR 2019oral

We integrate two powerful ideas, geometry and deep visual representation learning, into recurrent network architectures for mobile visual scene understanding. The proposed networks learn to "lift" 2D visual features and integrate them over time into latent 3D feature maps of the scene. They are equi…

Cited by 83 PDF ↗Scholar ↗

2017

Adversarial Inverse Graphics Networks: Learning 2D-To-3D Lifting and Image-To-Image Translation From Unpaired Supervision

Hsiao-Yu Fish Tung, Adam W. Harley, William Seto, Katerina Fragkiadaki

ICCV 2017poster

Researchers have developed excellent feed-forward models that learn to map images to desired outputs, such as to the images' latent factors, or to other images, using supervised learning. Learning such mappings from unlabelled data, or improving upon supervised models by exploiting unlabelled data,…

Cited by 170 PDF ↗Scholar ↗