A Novel Transformer-Based Pipeline for Lung Cytopathological Whole Slide Image Classification

Abstract

We propose a novel three-stage Transformer-based methodology for entire cytopathological whole slide image (WSI) classification. The key idea is to leverage Transformer to extract the fine-grained lesion-level features and then progressively aggregate them into intermediate-grained patch-level features and coarse-grained WSI-level features for classification. Specifically, we first extract multi-scale lesion features from each patch image via Transformer-based lesion detection, and then adaptively aggregate the extracted lesion features into the corresponding patch feature with an MLP-Mixer. Finally, we select the most representative patch features and feed them into the Vision Transformer (ViT) for the final WSI-level classification. We collect a dataset consisting of 961 lung cytopathological WSIs of pleural effusions cytology specimens and conduct extensive experiments on it. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art (SOTA) methods for cytopathological WSI classification.