← Search

Jiaxuan Peng

2 accepted papers

2025

Synergistic Tensor and Pipeline Parallelism

NeurIPS 2025poster

In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communicat…

Cited by 0SourcecodeScholar