Beyond Reassembly: Fractured Object Recovery with Missing Parts

Abstract

We propose a novel learning-based task named fractured object recovery. Unlike the previous fractured object reassembly task that only aligns existing parts with overlaps, our task aims to recover the complete shape by not only reassembling irrelevant parts but also predicting missing parts. Our task coincides with practical experiences, where the prior knowledge of similar shapes can be leveraged such that even non-overlapping parts can be reasoned into adequate locations. We also present the first learning model for the proposed task by correlating features of both existing and missing parts using a transformer, where the latter is naturally represented as missing tokens. To facilitate the task, we introduce a new dataset based on the existing fractured object benchmark by imposing different configurations of missing parts. We perform extensive evaluations to demonstrate the performance of the proposed model over baselines. The results show that joint part reassembly and prediction can be made possible and also have mutual benefits, which we believe can inspire future research and favor real applications.