A Survey of Joint Online-Offline Fine-tuning for Large Language Models
Taihang Zhen, Guang Yang, Chenzhang Li, Nuo Yan, Shilong Zhou, Guangyu Liu, Xiaotong Tang, Jing Huo
Abstract
Post-training for Large Language Models (LLMs) can be mainly categorized into offline Supervised Fine-Tuning (SFT) for knowledge acquisition and online Reinforcement Fine-Tuning (RFT) for adaptive refinement. Current state-of-the-art approaches typically employ a sequential cold-start pipeline (SFT-Then-RFT). However, we argue that this disjoint transition imposes an “alignment tax", leading to catastrophic forgetting and reward hacking during the unregularized exploration phase. In this work, we advocate for Joint Online-Offline Fine-Tuning as a superior paradigm that breaks the convention of restricting offline data to SFT and online data to RFT. By integrating full offline response generation with online rollouts—particularly within the realm of Reinforcement Learning with Verifiable Rewards (RLVR)—this approach mitigates the limitations of isolated training phases. We provide the first comprehensive survey focusing specifically on the synchronization of data provenance. We introduce a novel taxonomy for related works, analyze their theoretical advantages in balancing stability with plasticity, and outline a roadmap for next-generation post-training frameworks.
BibTeX
@inproceedings{ijcai2026_asurveyofjointon,
title = {A Survey of Joint Online-Offline Fine-tuning for Large Language Models},
author = {Taihang Zhen and Guang Yang and Chenzhang Li and Nuo Yan and Shilong Zhou and Guangyu Liu and Xiaotong Tang and Jing Huo and Boyan Wang and Junlan Feng and Yuyao Zhang},
booktitle = {IJCAI 2026},
year = {2026}
}