摘要
大模型训练如火如荼,参数网络是提升模型训练效率的关键网络架构。本文主要研究参数网络轨道优化设计,通过对比多轨方案和单轨方案的特点、优劣以及各自适用的场景,给出参数网络选型建议,并探讨未来参数网络设计的优化研究方向。
The training of large models is in full swing,and the parameter network is a key network architecture for improving the eficiency of model training.This paper mainly studies the optimization design of the parameter network rail,compares the characteristics,advantages and disadvantages of multi-rail and single-rail schemes,as well as their respective applicable scenarios,provides suggestions for the selection of the parameter network,and explores the future research directions for the optimization design of the parameter network.
作者
田雯
吴启迪
王祎玮
魏逸哲
丁国强
TIAN Wen;WU Qi-di;WANG Yi-wei;WEI Yi-zhe;DING Guo-qiang(China Mobile Group Design Institute Co.,Ltd.,Beijing 100080,China)
出处
《电信工程技术与标准化》
2024年第S02期18-22,共5页
Telecom Engineering Technics and Standardization
