Kimi K3 Architecture Explained: How the World’s Largest Open Source AI Model Works | Kimi K3 架构解析:全球最大开源AI模型是如何运作的
Moonshot AI's Kimi K3 is the first open-weight model to cross 3 trillion parameters, activating just 104B per token. Three innovations power it: a latent-compressed mixture-of-experts, Kimi Delta Attention for 1M-token context, and attention residuals across 93 layers — together delivering 2.5x better scaling efficiency than K2, at frontier-competitive benchmark performance. Moonshot AI 推出的 Kimi K3,是首个突破3万亿参数的开放权重模型,每个token仅激活1040亿参数。其核心依托三大创新:潜空间压缩混合专家系统、支持百万token上下文的Kimi Delta Attention,以及贯穿93层网络的注意力残差机制——三者协同,使扩展效率较K2提升2.5倍,性能达到前沿水平。











