Hybrid Feature Fusion for Enhancing Medical Document Embedding

Abstract

Despite the strong capabilities of large language models in generative tasks, issues related to information unreliability and hallucinations pose significant challenges in high-precision fields, such as drug analysis and recommendations in the medical domain. In this work, we introduce the HFFN model, a retrieval-augmented framework designed for medical document retrieval tasks, which combines an embedding backbone with a Hybrid Feature Fusion module to enhance retrieval quality. HFFN improves model representation by learning to control the weight parameters of nonlinear features, thereby avoiding the instability associated with using the same activation function across different datasets. Experimental results demonstrate that, compared to a single-hidden-layer MLP, HFFN improves NDCG score across various baseline embedding models by 2.3%-15.1%.