Moonshot AI's Kimi K3 Is Here: A 2.8 Trillion Parameter Open MoE Model That Pushes Long-Context AI Forward

Moonshot AI has introduced Kimi K3, an open Mixture-of-Experts model featuring 2.8 trillion parameters. The model supports a 1 million token context window and utilizes Kimi Delta Attention to improve training efficiency, inference speed, and long-context reasoning. Due to its open nature and high-level coding skills, Kimi K3 is positioned as a significant competitor to OpenAI. The engineering focus emphasizes solving practical bottlenecks rather than relying solely on scaling parameters to advance large language model performance.