Inspur Information released the "Source 2.0-M32" open source model...
DIYuan | 2024-05-28 17:19
【数据猿导读】 Inspur Information released the "Source 2.0-M32" open source model
On May 28, Inspur Information released the "Source 2.0-M32" open source model. Based on the Source 2.0 series of large models, the model adopts the technology of "attention mechanic-based gated network" to build a hybrid expert model (MoE) containing 32 experts, and greatly improves the computing efficiency of the model. With a specially optimized model architecture, the M32 achieves the same level of performance as the 70-billion-parameter LLaMA3 with only 3.7 billion parameters activated, and consumes only 1/19 of the computing power of the LLaMA3.
来源：DIYuan
刷新相关文章
我要评论
不容错过的资讯
大家都在搜
