Meta-Conscious Driven Domain-Aware Federated Learning

Abstract

Cross-domain collaboration can drive comprehensive knowledge innovation and foster synergistic advancements. Federated learning (FL) enables such collaboration while ensuring data security. However, cross-domain FL often faces challenges due to knowledge interference between domains, which can result in the inclusion of irrelevant or incorrect information, ultimately affecting decision-making. Our findings suggest that extracting key information from raw data and compressing it into a highly compact set of synthetic data can significantly enhance data privacy. Moreover, utilizing cross-domain data to train a teacher model or incorporating multiple teachers from different domains proves advantageous for knowledge distillation by effectively integrating cross-domain information. In response to these challenges, we propose a meta-conscious driven domain-aware FL framework (FedMC). In particular, meta-teachers are employed to extract equivalent information across different domains, safeguarding data privacy through meta-data extraction. They also isolate and refine domain-specific knowledge through meta-representation extraction. This process provides both generalized and essential knowledge, aiding cross-domain meta-students in refining their knowledge. Experimental results demonstrate that the proposed method not only improves model performance on cross-domain tasks across multiple datasets but also offers substantial benefits in terms of privacy protection.