Domain-Aware Knowledge Debiasing for Generalizable Video Understanding in CLIP

Abstract

The pre-trained models contain multitudinous knowledge from huge amount of data. However, when applying these models to downstream tasks, they may mis-locate to wrong knowledge distribution due to a lack of domain or contextual knowledge. To address the distribution bias between the pre-trained model and the downstream domain, an innovative domain-aware knowledge de-biasing strategy, DKD, is introduced to improve the generalization performances on downstream tasks. Specifically, we use a CLIP-based video understanding framework to demonstrate the proposed approach, which dynamically adjusts the model’s representation space using the knowledge distribution of the target domain, effectively mitigating bias. Experimental results show that the method significantly improves model accuracy in action recognition tasks on standard datasets such as UCF101 and HMDB51, while also demonstrating superior generalization in cross-domain tasks. A comparison with state-of-the-art algorithms further validates the method’s remarkable advantages in the field of video understanding.