Animation Anycolor: Enhancing Line Drawing Colorization with Keypoint Matching

Abstract

Colorization is a crucial but labor-intensive and time-consuming process of animation production. The automation of animation line-drawing colorization has become a prominent research topic. Recently, methods based on pre-trained text-to-image models have been explored for the task of line-drawing colorization. However, these approaches may result in colorization errors when dealing with complex situations such as positional changes and extensive motion commonly encountered in animated scenes. These issues are primarily attributed to the inadequate semantic correspondence between the reference images and line-drawings. To tackle this problem, we introduce the Animation Anycolor framework. This approach leverages spatial attention mechanisms to integrate the appearance features of reference images, ensuring consistent feature transmission. Furthermore, we present a novel technique that employs keypoint matching to explicitly direct the network to recognize the feature correspondence areas between reference and target images, thus effectively mitigating color confusion. Our method preserves the accuracy and naturalness of color results in scenes characterized by positional shifts and character movement. Comparative evaluations indicate that our method outperforms the baseline by an average of 11.3% on the FID metric. Notably, this method improves the efficiency of line-drawing colorization and reduces production costs. It also introduces new insights by combining in-context correspondences with knowledge from the pre-trained model. This approach has broad application prospects in the animation industry.