Part in Part Embedding Network for Zero-Shot Learning

Abstract

Zero-shot learning (ZSL) seeks to utilize semantic information from seen classes encountered during training to effectively recognize unseen classes during testing. When dealing with fine-grained images, capturing local features heavily influences the accuracy of semantic descriptions. Meanwhile, local features are represented at various scales across different layers of a neural network, making it hard to capture their local details fully. To address these challenges, we propose a novel part in part embedding network, termed PPEN. Specifically, PPEN consists of two key modules: the cross-layer aggregation (CLA) module and the part in part attention (PIPA) module. The CLA module is designed to fuse and preserve features from multiple layers of the network, thereby maintaining the richness of information across different scales. Further, the PIPA module focuses on identifying local features that are most pertinent to the class semantic vectors, enhancing the alignment between visual features and semantic descriptions. We evaluate our approach on three ZSL benchmarks, i.e., CUB, SUN, and AWA2, and demonstrate the superiority and competitiveness of our proposed approach. Code is available at https://github.com/zhou834177226/PPENet.