← Search

zekun shi

9 accepted papers

2025

An Empirical Study of Federated Prompt Learning for Vision Language Model

IJCAI 2025

The Vision Language Model (VLM) excels in aligning vision and language representations, and prompt learning has emerged as a key technique for adapting such models to downstream tasks. However, the application of prompt learning with VLM in federated learning (FL) scenarios remains underexplored. Th

2025

Learn from Downstream and Be Yourself in Multimodal Large Language Models Fine-Tuning

ICML 2025poster

Multimodal Large Language Model (MLLM) has demonstrated strong generalization capabilities across diverse distributions and tasks, largely due to extensive pre-training datasets. Fine-tuning MLLM has become a common practice to improve performance on specific downstream tasks. However, during fine-t…

Cited by 9SourcePDFScholar
2025

Pixel-wise Divide and Conquer for Federated Vessel Segmentation

IJCAI 2025

Accurate vessel segmentation is essential for diagnosing and managing vascular and ophthalmic diseases. Traditional learning-based vessel segmentation methods heavily rely on high-quality, pixel-level annotated datasets. However, segmentation performance suffers significantly when applied in federat

Cited by 0SourcePDFScholar
2024

Fisher Calibration for Backdoor-Robust Heterogeneous Federated Learning

ECCV 2024poster

"Federated learning presents massive potential for privacy-friendly vision task collaboration. However, the federated visual performance is deeply affected by backdoor attacks, where malicious clients optimize on triggered samples to mislead the global model into targeted mispredictions. Existing ba…

2024

Parameter Disparities Dissection for Backdoor Defense in Heterogeneous Federated Learning

NeurIPS 2024poster

Backdoor attacks pose a serious threat to federated systems, where malicious clients optimize on the triggered distribution to mislead the global model towards a predefined target. Existing backdoor defense methods typically require either homogeneous assumption, validation datasets, or client optim…

Cited by 3SourcePDFScholar
2024

Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning

ICML 2024poster

Federated learning presents massive potential for privacy-friendly collaboration. However, the performance of federated learning is deeply affected by byzantine attacks, where malicious clients deliberately upload crafted vicious updates. While various robust aggregations have been proposed to defen…

Cited by 5SourcePDFScholar
2024

Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators

NeurIPS 2024oral

Optimizing neural networks with loss that contain high-dimensional and high-order differential operators is expensive to evaluate with back-propagation due to $\mathcal{O}(d^{k})$ scaling of the derivative tensor size and the $\mathcal{O}(2^{k-1}L)$ scaling in the computation graph, where $d$ is t…

2023

Rethinking Federated Learning With Domain Shift: A Prototype View

CVPR 2023poster

Federated learning shows a bright promise as a privacy-preserving collaborative learning technique. However, prevalent solutions mainly focus on all private data sampled from the same domain. An important challenge is that when distributed data are derived from diverse domains. The private model pre…