MAEM: A Multi-Aspect Extraction Model for Enhanced Embedding in RAG

Abstract

Retrieval Augmented Generation can effectively reduce hallucinations in LLMs during question-answering, with embedding models directly influencing its performance. While current embedding models improve encoding through large-scale training, they often overlook the potential or explicit multi-aspect information within the text. To address this, we propose the Multi-Aspect Extraction Model (MAEM), an eigen decomposition-based approach that extracts and integrates text aspects into a unified vector for enhanced retrieval, and introduce a regularization loss function to assist in training. We utilized LLMs to create the Policy-Corpus dataset and validated the model on both Policy-Corpus and FiQA. Incorporating MAEM and regularization into GTE<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">base</inf> improved NDCG@10 by 3.8 points on FiQA and 4.56 points on Policy-Corpus. Respectively, achieving results comparable to larger models using a smaller parameter model.