A Connected Auto-Encoders Based Approach for Image Separation with Side Information: With Applications to Art Investigation
Wei Pu, Barak Sober, Nathan Daly, Catherine Higgitt, Ingrid Daubechies, Miguel R. D. Rodrigues
Abstract
X-radiography is a widely used imaging technique in art investigation, whether to investigate the condition of a painting or provide insights into artists' techniques and working methods. In this paper, we propose a new architecture based on the use of `connected' auto-encoders in order to separate mixed X-ray images acquired from double-sided paintings, where in addition to the mixed X-ray image one can also exploit the two RGB images associated with the front and back of the painting. This proposed architecture uses convolutional auto-encoders that extract features from the RGB images that can be employed to (1) reproduce both of the original RGB images, (2) reconstruct the associated separated X-ray images, and (3) regenerate the mixed X-ray image. It operates in a totally self-supervised fashion without the need for examples containing both the mixed X-ray images and the separated ones. Based on images from the double-sided wing panels from the famous Ghent Altarpiece, painted in 1432 by the brothers Hubert and Jan Van Eyck, the proposed algorithm has been experimentally verified to outperform state-of-the-art X-ray separation methods in art investigation applications.
BibTeX
@inproceedings{icassp2020_aconnectedautoen,
title = {A Connected Auto-Encoders Based Approach for Image Separation with Side Information: With Applications to Art Investigation},
author = {Wei Pu and Barak Sober and Nathan Daly and Catherine Higgitt and Ingrid Daubechies and Miguel R. D. Rodrigues},
booktitle = {ICASSP 2020},
year = {2020}
}