← Search

Guangyuan Ma

7 accepted papers

2026

LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference

ICLR 2026poster

Large Language Models (LLMs)-based text retrieval retrieves documents relevant to search queries based on vector similarities. Documents are pre-encoded offline, while queries arrive in real-time, necessitating an efficient online query encoder. Although LLMs significantly enhance retrieval capabili…

Cited by 0SourcecodeScholar
2025

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts

NAACL 2025long

Large language models (LLM) have been attracting much attention from the community recently, due to their remarkable performance in all kinds of downstream tasks. According to the well-known scaling law, scaling up a dense LLM enhances its capabilities, but also significantly increases the computati…

2025

Task-level Distributionally Robust Optimization for Large Language Model-based Dense Retrieval

AAAI 2025technical

Large Language Model-based Dense Retrieval (LLM-DR) optimizes over numerous heterogeneous fine-tuning collections from different domains. However, the discussion about its training data distribution is still minimal. Previous studies rely on empirically assigned dataset choices or sampling ratios, w…

2024

Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems

NAACL 2024long

Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history. Most existing works primarily focus on post-training and fine-tuning tailored for cross-encoders. However, there are no post-training methods tailored for de…

2023

ConTextual Masked Auto-Encoder for Dense Passage Retrieval

AAAI 2023technical

Dense passage retrieval aims to retrieve the relevant passages of a query from a large corpus based on dense representations (i.e., vectors) of the query and the passages. Recent studies have explored improving pre-trained language models to boost dense retrieval performance. This paper proposes CoT…

2023

PUNR: Pre-training with User Behavior Modeling for News Recommendation

EMNLP 2023long findings

News recommendation aims to predict click behaviors based on user behaviors. How to effectively model the user representations is the key to recommending preferred news. Existing works are mostly focused on improvements in the supervised fine-tuning stage. However, there is still a lack of PLM-ba…

Cited by 0SourcecodeScholar
2023

Query-as-context Pre-training for Dense Passage Retrieval

EMNLP 2023long main

Recently, methods have been developed to improve the performance of dense passage retrieval by using context-supervised pre-training. These methods simply consider two passages from the same document to be relevant, without taking into account the potential negative impacts of weakly correlated pair…

Cited by 0SourcecodeScholar