AS-Net: Adaptive Style-aware Network for Handwritten Text Generation

Abstract

Handwritten text generation (HTG) is a challenging task due to the vast diversity of handwriting styles. In addition, even the same writer can show subtle differences when writing the same character, further aggravating the difficulties of HTG. In this paper, we propose a novel Adaptive Style-aware Network (AS-Net) to address the challenging HTG task. Specifically, we propose a Style-induced Context-aware Generator (SCG) to capture handwriting styles and generate new images. The SCG learns a lightweight neural network to generate a style token (vector) for each image. Then, the context tokens with text information are element-wise multiplied by the style tokens to construct mixed tokens. Finally, we employ multiple attention mechanisms to capture dependencies among style features and entanglement with style and text features. Extensive experiments on two widely used benchmark datasets demonstrate that our AS-Net is an effective HTG model and outperforms state-of-the-art methods markedly. Our code is publicly available at: https://github.com/Fyzjym/AS-Net.