Houming Wu

2 accepted papers

2026

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training

ICML 2026poster

Pipeline parallelism is essential for large-scale model training, but existing asynchronous approaches often degrade convergence due to parameter mismatch between forward and backward passes. We propose Asynchronous Multi-Directional Pipeline parallelism (AMDP) to mitigate this issue while sustainin…

Cited by 0 Source ↗Scholar ↗

2026

TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training

Houming Wu, Ling Chen

AAAI 2026technical

Training large language models (LLMs) is fundamentally constrained by limited device memory and costly inter-device communication. Although pipeline parallelism alleviates memory pressure by partitioning models across devices, it incurs activation communication overhead that scales linearly with seq

Cited by 0 Source ↗PDF ↗Scholar ↗