AnimateSketches: Animate Sketches with Instance-Aware Mask

Abstract

Sketch animation, the transformation of static sketches into dynamic experiences, is an essential tool for visual expression. Current methods rely on global-level optimization and neglect instance-level priors, resulting in optimization mismatch and overfitting across multiple sketches. To solve this problem, information about the attention map, representing different instances is needed to guide the optimization. In this work, we propose AnimateSketches, a novel optimization-based framework that focuses on animating complex vectorized sketches. We introduce prompt-guided instance-aware mask generation (PGIM), which leverages attention maps from a pretrained diffusion model to guide the optimization of individual sketches. In addition, we use mask-based score distillation sampling (MSDS) to maintain the integrity of untargeted sketches. Quantitative and qualitative evaluations show the superiority of our approach over baseline methods in terms of visual quality and instance-level prompt-guided correspondence.