Improving Irregular Text Recognition with Adaptive Feature Compression

Abstract

Scene text recognition models typically do not handle text irregularities well, especially for connectionist temporal classification (CTC)-based ones. In CTC models, visual features must be compressed into a one-dimensional sequence to fit the CTC decoding. Current solutions adopt simple average pooling or feature shrinking for this compression, which is a bottleneck restricting their recognition capabilities. To tackle this, we introduce an adaptive feature compression block to compress the features adaptively. It leverages the attention mechanism to selectively preserve features related to text foreground and discard those belonging to text background. As a result, the compression adaptively retains features the mostly important to recognition, and CTC models could better deal with text irregularities when equipped with this block. Correspondingly, we design a novel text recognition model termed AFCTR by appending this adaptive feature compression block to existing CTC models. Experimental results on typical English benchmarks show that AFCTR outperforms existing popular models in terms of accuracy under multiple evaluation protocols. Moreover, AFCTR also preserves the efficiency advantage of CTC models.