Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization
Miika Aittala, Prafull Sharma, Lukas Murmann, Adam Yedidia, Gregory Wornell, Bill Freeman, Fredo Durand
Abstract
We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by factoring the observed video into a matrix product between the unknown hidden scene video and an unknown light transport matrix. This task is extremely ill-posed, as any non-negative factorization will satisfy the data. Inspired by recent work on the Deep Image Prior, we parameterize the factor matrices using randomly initialized convolutional neural networks trained in a one-off manner, and show that this results in decompositions that reflect the true motion in the hidden scene.
BibTeX
@inproceedings{NEURIPS2019_5a2afca6,
author = {Aittala, Miika and Sharma, Prafull and Murmann, Lukas and Yedidia, Adam and Wornell, Gregory and Freeman, Bill and Durand, Fredo},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/5a2afca61e35f45a7dd44ca46e0225f4-Paper.pdf},
volume = {32},
year = {2019}
}