https://x7peeps.com/tags/%E7%A8%80%E7%96%8F%E6%BF%80%E6%B4%BB/index.htmlHugozh-CNWed, 15 Jul 2026 11:44:35 +0000https://x7peeps.com/AI/01-LLM%E5%8E%9F%E7%90%86%E4%B8%8E%E5%B7%A5%E7%A8%8B/Mixture-of-Experts%E6%9E%B6%E6%9E%84%E7%A8%80%E7%96%8F%E6%BF%80%E6%B4%BB%E5%A6%82%E4%BD%95%E9%87%8D%E5%A1%91%E5%A4%A7%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%95%88%E7%8E%87%E4%B8%8E%E8%A7%84%E6%A8%A1/index.htmlWed, 15 Jul 2026 11:44:35 +0000https://x7peeps.com/AI/01-LLM%E5%8E%9F%E7%90%86%E4%B8%8E%E5%B7%A5%E7%A8%8B/Mixture-of-Experts%E6%9E%B6%E6%9E%84%E7%A8%80%E7%96%8F%E6%BF%80%E6%B4%BB%E5%A6%82%E4%BD%95%E9%87%8D%E5%A1%91%E5%A4%A7%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%95%88%E7%8E%87%E4%B8%8E%E8%A7%84%E6%A8%A1/index.htmlMixture of Experts 架构：稀疏激活如何重塑大语言模型的效率与规模 大语言模型的 Scaling Law 曾长期遵循一个简单假设：更大的参数量意味着更强的能力，但代价是成比例增长的计算开销。GPT-4 据报训练成本超过 1 亿美元，Llama 2 耗费 330 万 A100 GPU 小时——当参数规模逼近万亿级别，Dense 模型的线性扩展路径正变得难以持续。