Multi-level Feature Adaptation and Embeddings Alignment for Zero-shot Anomaly Detection

Abstract

Anomaly Detection (AD) is an important and challenging task in computer vision. Recently, Contrastive Language-Image Pre-training (CLIP) has shown impressive generalization capability in zero-shot AD tasks. However, the inherent global semantic consistency of CLIP fails to accurately identify subtle anomalies, and the lack of fine-grained alignment leads to inaccurate defect localization. In this study, we propose a hierarchical semantic feature adaptation framework designed to better leverage intermediate layer information from the visual encoder, enabling the effective capture of subtle and diverse anomaly-aware semantic features, thus improving CLIP’s adaptation to AD task. We further introduce an alignment and mask refinement co-optimization decoder to locate key anomalous regions from coarse to fine, thereby alleviating the misalignment between visual and textual embeddings. The experimental results demonstrate the effectiveness of our method.