An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference
Yu Lin, Qizhi Zhang, Quanwei Cai, Jue Hong, Wu Ye, Huiqi Liu, Bing Duan
Abstract
With the rapidly-growing deployment of large language model (LLM) inference services, privacy concerns have arisen regarding to the user input data. Recent studies are exploring transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service provides. However, in this paper we show that again, without a solid and deliberate security design and analysis, such embedded vector obfuscation failed to protect users’ privacy. We demonstrate the conclusion via conducting a novel inversion attack called Element-wise Differential Nearest Neighbor (EDNN) on the glide-reflection proposed in (CITATION), and the result showed that the original user input text can be 100% recovered from the obfuscated embedded vectors. We further analyze security requirements on embedding obfuscation and present several remedies to our proposed attack.
BibTeX
@inproceedings{lin-etal-2024-inversion,
title = "An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference",
author = "Lin, Yu and
Zhang, Qizhi and
Cai, Quanwei and
Hong, Jue and
Ye, Wu and
Liu, Huiqi and
Duan, Bing",
editor = "Al-Onaizan, Yaser and
Bansal, Mohit and
Chen, Yun-Nung",
booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2024",
address = "Miami, Florida, USA",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.emnlp-main.126/",
doi = "10.18653/v1/2024.emnlp-main.126",
pages = "2100--2104"
}