FedTG: Text-guided Federated Domain Generalization

Abstract

Federated Domain Generalization is aimed at learning a global model that generalizes well to new clients with domain shift in a privacy-preserving manner. Inspired by the powerful generation capability of Large language models (LLMs), we propose a novel method for this problem, named Text-guided Federated Domain Generalization (FedTG), which utilizes text generated by LLMs to guide the training of visual models. In the client, we align the image embeddings to the text embeddings, which provides strong generalization knowledge. In the server, we generate abundant textual descriptions about different classes, subsequently converting them into embeddings for fine-tuning the global classifier, which helps the global classifier not overfit to a particular domain. Moreover, we utilize the powerful image generation capabilities of LLMs to generate a cross-domain validation dataset to help select the most generalized model. We conduct experiments on several datasets, and the results prove that FedTG outperforms relevant baselines.