Huimu Yu

CODEPMP: SCALABLE PREFERENCE MODEL PRETRAINING FOR LARGE LANGUAGE MODEL REASONING

Large language models (LLMs) have made significant progress in natural language understanding and generation, driven by scalable pretraining and advanced finetuning. However, enhancing reasoning abilities in LLMs, particularly via reinforcement learning from human feedback (RLHF), remains challengin…