Enhanced Data Synthesis for LLM through Reasoning Structures Generated by Hierarchical GFlowNet
Tianpeng Bu, Minying Zhang, Hongtao Duan, Shurui Li, Lulu Hu, Yu Li
Abstract
Large language models (LLMs) excel in problem-solving but require training data with diverse reasoning processes. Existing methods mainly optimize instruction-response pairs but lack a systematic design for the underlying reasoning structure. This paper proposes RSS: a Reasoning Structure driven data Synthesis method. We first proactively develop a hierarchical GFlowNet to construct reasoning structures efficiently through a coarse-to-fine directed acyclic graph (DAG) growth process. Then reasoning DAGs are leveraged to actively guide the instruction generation via an iterative suggester-editor workflow and enhance response quality using a structure-aware strategy. Experiments show that LLMs trained on our synthetic datasets achieve 48.50%, 84.00%, 79.90% for AlpacaEval2, GSM8K and HumanEval, outperforming existing data synthesis methods.
BibTeX
@inproceedings{bu-etal-2025-enhanced,
title = "Enhanced Data Synthesis for {LLM} through Reasoning Structures Generated by Hierarchical {GF}low{N}et",
author = "Bu, Tianpeng and
Zhang, Minying and
Duan, Hongtao and
Li, Shurui and
Hu, Lulu and
Li, Yu",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.821/",
doi = "10.18653/v1/2025.findings-acl.821",
pages = "15931--15958",
ISBN = "979-8-89176-256-5"
}