Integer Is Enough: When Vertical Federated Learning Meets Rounding
Pengyu Qiu, Yuwen Pu, Yongchao Liu, Wenyan Liu, Yun Yue, Xiaowei Zhu, Lichun Li, Jinbao Li
Abstract
Vertical Federated Learning (VFL) is a solution increasingly used by companies with the same user group but differing features, enabling them to collaboratively train a machine learning model. VFL ensures that clients exchange intermediate results extracted by their local models, without sharing raw data. However, in practice, VFL encounters several challenges, such as computational and communication overhead, privacy leakage risk, and adversarial attack. Our study reveals that the usage of floating-point (FP) numbers is a common factor causing these issues, as they can be redundant and contain too much information. To address this, we propose a new architecture called rounding layer, which converts intermediate results to integers. Our theoretical analysis and empirical results demonstrate the benefits of the rounding layer in reducing computation and memory overhead, providing privacy protection, preserving model performance, and mitigating adversarial attacks. We hope this paper inspires further research into novel architectures to address practical issues in VFL.
BibTeX
@article{Qiu_Pu_Liu_Liu_Yue_Zhu_Li_Li_Ji_2024, title={Integer Is Enough: When Vertical Federated Learning Meets Rounding}, volume={38}, url={https://ojs.aaai.org/index.php/AAAI/article/view/29388}, DOI={10.1609/aaai.v38i13.29388}, abstractNote={Vertical Federated Learning (VFL) is a solution increasingly used by companies with the same user group but differing features, enabling them to collaboratively train a machine learning model. VFL ensures that clients exchange intermediate results extracted by their local models, without sharing raw data. However, in practice, VFL encounters several challenges, such as computational and communication overhead, privacy leakage risk, and adversarial attack. Our study reveals that the usage of floating-point (FP) numbers is a common factor causing these issues, as they can be redundant and contain too much information. To address this, we propose a new architecture called rounding layer, which converts intermediate results to integers. Our theoretical analysis and empirical results demonstrate the benefits of the rounding layer in reducing computation and memory overhead, providing privacy protection, preserving model performance, and mitigating adversarial attacks. We hope this paper inspires further research into novel architectures to address practical issues in VFL.}, number={13}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Qiu, Pengyu and Pu, Yuwen and Liu, Yongchao and Liu, Wenyan and Yue, Yun and Zhu, Xiaowei and Li, Lichun and Li, Jinbao and Ji, Shouling}, year={2024}, month={Mar.}, pages={14704-14712} }