KimiK3 MaxKimi K3 Specs & Architecture
The engineering behind the largest open-weight model yet — and why its architecture matters for real workloads.
Parameter scale
Kimi K3 uses a sparse Mixture-of-Experts (MoE) design (Stable LatentMoE) with 896 total experts, activating only 16 per token. Total parameters are reported at ~2.8 trillion. The selective activation is what makes serving a 2.8T model practical at scale.
Attention: KDA + AttnRes
Two innovations underpin the 1M-token context. Kimi Delta Attention (KDA) combines cheaper linear attention with periodic full-attention layers, cutting KV-cache memory at long context. Attention Residuals (AttnRes) let each layer selectively pull from earlier representations instead of carrying everything forward through a fixed residual path.
Multimodal & reasoning
K3 accepts text, image, and video input, with reasoning always on and a tunable reasoning_effort control (launched at max effort).
| Component | Detail |
|---|---|
| Experts | 896 total, 16 active per token |
| Attention | KDA + Attention Residuals |
| Context | 1,048,576 tokens |
| Input | Text, image, video |
| Quantization | MXFP4 weights / MXFP8 activations (from SFT) |
Next: see how K3 performs or what it costs.
Frequently Asked Questions
How many parameters does Kimi K3 have?
Moonshot reports roughly 2.8 trillion total parameters using a sparse Mixture-of-Experts design. The exact active-parameter count has not been disclosed.
What is Kimi Delta Attention?
KDA is Moonshot's attention mechanism that uses lower-cost linear attention for most layers with periodic full-attention layers, sharply reducing KV-cache memory at 1M-token context.
What is the Kimi K3 context window?
Kimi K3 supports a context window of up to 1,000,000 tokens.