Long-tailed Oracle Character Recognition Based on Convolutional Neural Networks and Vision Transformers

Abstract

Oracle bone inscriptions, which are among the oldest known hieroglyphics in China, encompass rich historical and cultural information. However, the automatic recognition of oracle characters faces substantial challenges due to issues with data quality and long-tail distribution. This study introduces a hybrid model based on a convolutional neural network (CNN) and a visual transformer (ViT), augmented with a multi-expert learning strategy, to enhance the recognition performance of long-tail oracle bone characters. Initially, the CNN extracts local features, while the ViT captures global dependencies, thereby improving the model’s feature representation capability. Subsequently, through the multi-expert learning mechanism, the prediction results of multiple models are aggregated, effectively mitigating the effects of data imbalance and alleviating the challenges associated with the long-tail distribution of the dataset. Experimental results demonstrate that the proposed model achieves new state-of-the-art performance on two largescale Oracle datasets (Oracle-AYNU and OBC306) in terms of Top-1 accuracy, Top-5 accuracy, F1-Score, and average accuracy, with respective values of 87.97% (93.61%), 96.58% (98.84%), 86.83% (93.49%), and 82.81% (85.23%).