Scaling Kimi K2.5: How agent swarms and token efficiency redefine AI

Decoding the architecture behind the new open AI.

The pursuit of democratizing artificial intelligence relies heavily on pushing the boundaries of open-source models beyond mere parameter scaling. By optimizing token efficiency, extending context lengths, and deploying highly orchestrated agent swarms, open architectures like Kimi K2.5 are rapidly closing the capabilities gap with proprietary systems. Ultimately, fusing modalities like vision and text from day one establishes a resilient intelligence capable of tackling unprecedentedly complex, real-world economic tasks.

Points clés

Zhilin Yang unveiled the architecture of Kimi K2.5 at the GTC 2026 Keynote, focusing on scaling token efficiency, context length, and agent swarms.
The team successfully scaled the Muon optimizer for a one-trillion parameter model by utilizing a novel QK clip technique to prevent training instability and massive max logit explosions.
Utilizing the Muon optimizer effectively doubles token efficiency, meaning 50 trillion high-quality tokens yield the mathematical equivalent of 100 trillion tokens.
To address long-context challenges, they introduced Kimi linear, mixing linear attention with full attention at a 1:3 ratio to effortlessly process sequences exceeding 1 million tokens.
The revolutionary agent swarms paradigm allows a main orchestrator to deploy up to 1,000 sub-agents in parallel to execute complex workloads in a fraction of the time.
Kimi K2.5 was rigorously pre-trained on over 15 trillion tokens on a powerful Nvidia H800 GPU cluster connected via NVLink, maintaining a perfectly stable training curve.
The model employs an “early fusion” approach by mixing text and vision tokens from day one, achieving state-of-the-art vision performance with zero vision SFT data while proving that vision training actively enhances textual reasoning.
A newly open-sourced architecture called Attention residue was introduced, adapting residual connections into the depth dimension to further improve token efficiency by an additional 24%.

À retenir

If you are a non-expert hoping to build the next groundbreaking AI model in your garage, my recommendation is simple: just casually pick up a few thousand Nvidia H800 GPUs, gather a modest 15 trillion high-quality tokens, and you will be perfectly fine. For the rest of us mere mortals, perhaps it is best to stop worrying about coding logic from scratch and simply let a swarm of 1,000 AI sub-agents do our jobs while we nod thoughtfully and pretend to understand what a Muon optimizer is. After all, if the AI is already teaching itself to see just by reading text, it probably will not need our help hitting the “compile” button for much longer.

Sources