← All posts

Explainer

Kimi K3 open weights: the self-hosting reality check

Kimi K3's weights are genuinely open, and almost nobody should self-host them. Moonshot pushed the checkpoint to Hugging Face on July 27, 2026 — 2.8 trillion total parameters, 104 billion activated per token, 1.56 TB on disk. vLLM's day-0 post puts the hardware floor at one 8x B300 node or a GB300 NVL72; Moonshot's production guidance is 64+ accelerators. Against a list API price of $3 per million input and $15 per million output, the economics only invert at enormous, steady volume. What the weights buy most teams is optionality: auditability, no vendor lock, a private deployment path if regulation demands one, and the right to modify — not a cheaper Tuesday.

What actually shipped

Get the release date right first, because the coverage does not. The Hugging Face repo's initial commit is timestamped 2026-07-27T13:31:26Z and the GitHub repo was created earlier the same day. The wave of press dated July 28 is pickup, not release.

Two numbers in circulation are simply wrong. Activated parameters are 104B, not ~50B — the vendor's own model card says so. And the native download is 1.56 TB, not 594 GB; the 594 GB figure is Unsloth's 1-bit GGUF quant, a community artifact, not the release.

Why 2.8T at 4 bits is still 1.56 TB

Naive arithmetic says 2.8T parameters at 4 bits is 1.4 TB. The extra 160 GB is explained by config.json: the quantization config is mxfp4-pack-quantized at num_bits: 4, group_size: 32, but its ignore list excludes self-attention, the shared experts, the dense MLP projections, lm_head, the vision tower and the multimodal projector. Only the MoE expert weights are MXFP4.

Moonshot also applied quantization-aware training from the SFT stage onward, with MXFP8 activations — the model was trained to be served at 4 bits. So for self-hosters, the usual compression dividend has already been spent, and squeezing further costs quality.

The hardware floor, from the people who wrote the serving stack

vLLM's day-0 post is blunt: at least one 8x B300 node or a GB300 NVL72; 16x B200 also supported; AMD needs 8x MI355X/MI350X. Docker is currently mandatory — no standalone pip install, because of pre-release FlashInfer dependencies. The recipes page pins a minimum vLLM version, the vllm/vllm-openai:kimi-k3 image (CUDA 13 only), a recent NVIDIA driver, and TP8/TEP16 profiles — read the exact versions off that page rather than off any blog post, this one included, because they moved inside the first week. The serve command vLLM published at launch:

vllm serve moonshotai/Kimi-K3 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --load-format fastsafetensors \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

Diff those flags against the recipes page before you paste them — parser and load-format options are the first things to churn on a day-0 model. Two caveats vLLM discloses itself: prefix caching is disabled by default for K3, so that flag is not decorative, and tool calling needs validating on your own traffic, because vLLM has seen K3 emit a tool-call format its own parser does not expect.

Production shapes run above that floor. Baseten describes each K3 instance as 8 GPUs on GB300 NVL72 (2 nodes x 4 GPUs), TP8 including cross-node over the NVLink fabric, nine replicas per rack; NVIDIA's Dynamo recipe lists 16x GB200 aggregated over MNNVL and a disaggregated 8x GB300 prefill + 16x GB300 decode topology. At 288 GB of HBM3e per GB300, an 8-GPU node is 2,304 GB — the headroom that ~1.4 TB of resident weights plus KV cache needs. Moonshot itself recommends 64+ accelerators.

The workstation path exists and it hurts

Unsloth ships GGUF quants and an llama.cpp fork. The sizes tell the story:

Even 2-bit lands past 700 GB — the quantization-aware training bill coming due. The build needs Unsloth's fork on a specific PR branch, not upstream llama.cpp, and Unsloth quotes roughly 20 tokens per second on B200s. At the other end, vLLM's own launch benchmark — vendor-reported, on hardware almost nobody reading this owns — put 111 tok/s/user at TP8 on a GB300 NVL72 at batch size 1, rising to about 331 with DSpark speculative decoding, roughly a 3x decode gain.

The release ships less than the production stack

That draft model illustrates what "open weights" does and does not mean. The draft is not in the release: num_nextn_predict_layers is 0 and there are no MTP or draft weights in the file list, even though the tech report describes fine-tuning the pre-trained MTP layer into an EAGLE-3-style draft. Want the ~3x decode speedup? Pull a third-party one — vLLM points at Inferact/Kimi-K3-DSpark, SGLang's cookbook at RadixArk/Kimi-K3-DSpark.

Also absent: training data, pre-training code, RL environments. The K3 repo contains exactly a LICENSE, a README, a 47-page tech report, a logo, and the weights and configs. Infrastructure was open-sourced separately under MIT — MoonEP for expert parallelism, FlashKDA for the Kimi Delta Attention kernels, plus minitriton and nano-kpu. AgentENV, which several write-ups attribute to the MoonshotAI org, actually lives at kvcache-ai/AgentENV.

The license, read properly

K2 used a modified-MIT label; K3 does not. It is a bespoke "Kimi K3 License" — if you are making a compliance decision, read the file, not the blog posts. Two conditions have teeth:

• MaaS revenue gate. If you or your affiliates operate a Model-as-a-Service business and aggregate revenue exceeds USD 20 million over any consecutive 12 months, you must sign a separate agreement with Moonshot before any commercial use. MaaS means giving a third party access to inference or fine-tuning where they control inputs, parameters or training data — which explicitly excludes end-user products embedding the capability in specific features.
• Attribution gate. If the software is used in a commercial product with more than 100 million monthly active users, or more than USD 20 million in monthly revenue, "Kimi K3" must be displayed prominently in that product's UI.

Section 4 exempts purely internal use that does not expose the software, its outputs or its capabilities to third parties, and use via Moonshot's official products or certified inference partners. For most engineering teams neither condition triggers. Licensing questions go to license@moonshot.ai.

Self-host, official API, or aggregator

Published cost comparisons for a 2.8T model do not reconcile, so be suspicious of any single break-even. One analysis puts 64x H200 between roughly $212k and $495k per month depending on provider — break-even from 17 to 118 billion tokens per month, explicitly assuming 100% utilization and excluding headcount, redundancy, storage and egress. Another puts 8x B200 near $32k/month, break-even around 2.1 billion output tokens. An order of magnitude apart. Treat both as bounds, not answers; the decision is usually made by constraint, not arithmetic:

• Official API ($3.00 in / $0.30 cached in / $15.00 out per 1M, flat to 1M context). Correct for essentially everyone. Rate limits ladder by cumulative recharge — Tier0 at $1 gives 1 concurrency and 3 RPM, Tier1 at $10 jumps to 50 concurrency and 200 RPM with unlimited daily tokens, up to Tier5 at $3,000 for 1,000 concurrency and 10,000 RPM.
• Aggregator when you want one bill across vendors or a fallback provider. Day-0 hosting spanned vLLM, Baseten, Modal, Fireworks, Nebius, Together, DigitalOcean, Cursor, Ollama Cloud and Dell Enterprise Hub; list price is broadly matched, with Together at $3.00/$0.30/$15.00 in FP4. Cents drift weekly, and do not assume a given aggregator passes through the $0.30 cached-input rate. Vercel's AI Gateway added a kimi-k3-fast variant at roughly 50% higher per-token with a zero-data-retention flag — provider-side only; Moonshot's own model list has just kimi-k3.
• Self-host when a regulator, an air gap, or a data-residency contract makes the API impossible, or when you need to modify the model. Within 72 hours the community shipped requantizations (RedHatAI FP8-BLOCK, vessl W4AFP8), REAP-pruned MLX builds, an abliterated variant, and a Kazakh continued-pretraining checkpoint. No API grants that.

On fine-tuning, temper expectations: providers advertise LoRA adapters on K3, but nobody has published full fine-tune hardware requirements, optimizer-state math, or a worked cost. Read "you can fine-tune it" as "you can LoRA it on a provider."

If you are just going to call the API

One gotcha dominates. K3 always reasons — thinking cannot be disabled — and it was trained in preserved-thinking-history mode, so multi-turn and tool-calling flows must pass the complete assistant message back into messages verbatim, including reasoning_content and tool_calls, not just content. Getting that wrong quietly degrades agent behavior over a long session. Note also that reasoning_effort is a top-level field taking "low" | "high" | "max" and defaults to "max", that Moonshot flags web_search as mid-update, and that kimi-k2.5 and moonshot-v1 sunset on August 31, 2026.

Setup and agent behavior are covered in installing Kimi Code CLI and Kimi K3 as a coding agent; for K3 against the frontier closed models see Fable 5 vs Kimi K3 and the wider Chinese coding models roundup. Run your own numbers with the agent cost calculator and the model pricing tracker.

Frequently asked questions

Where Backgrind fits

Backgrind is not a model and not an agent. It is a desktop overlay for macOS and Windows that PTY-wraps the CLI you already run — Claude Code, Cursor, Codex, OpenCode, or Kimi Code — and floats it above your other windows, including games running in borderless fullscreen (exclusive fullscreen is out of scope). It pings you when the agent needs a decision or finishes, so a long K3 run does not need watching.

The open-weights angle matters in one specific way: because the same checkpoint is served by many providers, switching endpoints is a config change in your CLI rather than a migration — and Backgrind never sees that layer. It renders and forwards whatever your CLI does, whether that CLI is pointed at Moonshot, an aggregator, or your own vLLM instance. In BYO-CLI mode your agent's content never touches our servers; Live mode (Plus/Pro) answers prompts from a phone or browser. The Fusion council runs your own CLIs on one hard question in parallel when you want a second opinion before committing.

Sources

License text in full: Kimi K3 License. Architecture, release contents and the 47-page technical report: MoonshotAI/Kimi-K3 and the model card, config.json and file listing at huggingface.co/moonshotai/Kimi-K3 (repo size and commit timestamps read from the Hugging Face API). Pricing, rate-limit tiers, model list, reasoning-history semantics and the CLI installer: platform.kimi.ai docs. Hardware floor, serve command, caveats and measured decode throughput: vLLM day-0 post and vLLM recipes; SGLang's generator at docs.sglang.io. Production topologies: Baseten and NVIDIA Dynamo recipes. GGUF quant sizes and build path: Unsloth and unsloth/Kimi-K3-GGUF. Provider pricing and variants: Together, Fireworks, Vercel AI Gateway. Self-hosting cost ranges (secondary, cost math only, stated assumptions apply): Digital Applied. Benchmark figures are deliberately omitted here — Moonshot's are vendor-run, several on its own Kimi Code harness, and the report itself concedes K3 still trails the leading closed models overall.