Disentangled Representation Learning for Chinese Handwriting Recognition

Abstract

Deep learning-based sequence modeling methods have improved the performance in Chinese handwriting recognition tasks. However, the implicit representations learned in current deep neural network models usually lack explainability and generalization ability for practical handwriting samples with diverse styles. This paper proposes a disentangled representation learning method for Chinese handwriting recognition by using a latent Gaussian process, which models the features in latent space as Gaussian distributions conditioned on the content and style of handwriting samples. A deep learning model with one encoder and multiple decoders is designed. The style embeddings in a style decoder are used to convert the hidden representations output by the encoder into style-agnostic representations for content decoding. Objective functions to decouple content and style representations are introduced in the training stage. Experiments are conducted on the public CASIA-HWDB Chinese handwriting dataset and a self-built historical Chinese local genealogies dataset, which demonstrate that the proposed method achieves high accuracy in recognizing handwritten Chinese text images with diverse styles.