Paper: SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD
Listen to this article.
Audio is available for 30 days and will be removed automatically.
Problem
Training massive, trillion-parameter Mixture of Experts (MoE) language models like DeepSeek-V4 presents significant engineering challenges when using distributed training systems. The paper highlights issues including intense memory usage, communication bottlenecks, and inefficient processing during the post-training phase—specifically, Full Parameter Post-Training (CPT) and Supervised Fine Tuning (SFT). While most existing solutions rely on GPU clusters, this research explores an alternative approach leveraging Ascend Neural Processing Units (NPUs).




