CPU-first infrastructure guide · 19 July 2026

Build a server
for Kimi K3.

A 2.8-trillion-parameter sparse model does not belong in a workstation. To make K3 local, think three terabytes of ECC memory, 24 populated DDR5 channels, enterprise NVMe, NUMA-aware execution—and GPUs if “interactive” matters.

Residency · Bandwidth · AMX · MoE offload · Power · Procurement
2.8Treported total parameters
16 / 896active / routed experts per token
≈1.49 TBideal 4.25-bit weight payload
July 27planned full-weight release
01 / The answer before the shopping list

Can a CPU server run K3?
Probably. Not yet proven.

On July 19, the API exists but the full weights do not. Moonshot says they are planned for July 27. Until the checkpoint, tensor shapes, license, reference kernels and quantized files are public, nobody can honestly give a validated K3 CPU BOM or measured tokens per second.

What we can do is size the machine from the disclosed architecture: 2.8 trillion total parameters, a 1-million-token context, native vision, 896 routed experts with 16 selected per token, Kimi Delta Attention, and quantization-aware MXFP4 weights with MXFP8 activations. Those numbers are enough to establish orders of magnitude.

A sparse MoE saves compute, not model residency. Every expert can be selected by a future token, so the server must keep all expert weights somewhere reachable. CPU DRAM is the only economically plausible single-node pool at roughly 1.5 TB. NVMe can store the checkpoint but cannot feed arbitrary experts on the token critical path without severe stalls.

Thus “CPU server” has two meanings. A pure CPU experiment keeps and computes everything in DRAM. A CPU-first heterogeneous server keeps most experts in DRAM while GPUs execute dense/KDA layers and selected hot experts. The second is far more likely to feel usable.

Procurement verdict: design the rack, power and memory topology now; do not place a non-returnable order specifically for K3 until the July 27 artifacts demonstrate a supported CPU or CPU+GPU path. K2.5 is evidence that KTransformers-style offload can work—not proof that K3 will work unchanged.
02 / Residency math

“Only 16 experts run”
does not mean only 16 experts load.

BF16 / FP165.6 TB

2 bytes × 2.8T parameters. Outside a practical single-node build.

FP8 / INT82.8 TB

1 byte per parameter before scales, metadata and runtime workspace.

Raw four-bit1.40 TB

Half a byte per parameter, before any block scales or non-4-bit tensors.

MXFP4 idealized1.49 TB

E2M1 values plus one 8-bit scale per 32 values: 4.25 effective bits.

2.8 × 10¹² parameters × 4.25 bits ÷ 8 = 1.4875 × 10¹² bytes ≈ 1.49 TB decimal ≈ 1.35 TiB
Minimum that loads

About 2 TB of installed RAM may hold a fully optimized 4-bit checkpoint plus runtime state, but it leaves limited safety margin for file cache, conversion, vision components, KDA states, global-attention KV cache, concurrent requests and allocator fragmentation.

Recommended capacity

3 TB of ECC DRAM is the sane target for a first CPU node. It accommodates a roughly 1.5–1.7 TB model image, duplicated or repacked buffers during startup, operating-system headroom and meaningful context/cache experiments.

Expert sparsity

The reported 16-of-896 routing means about 1.8% of routed experts execute for each token. It reduces matrix multiplication work. It does not allow the other experts to disappear, because routing decisions vary by token, layer, sequence and batch.

Unknown until release

The exact byte count depends on which tensors are MXFP4, scale granularity, padding, shared experts, embeddings, vision encoder, KDA/MLA parameters and checkpoint container overhead. Treat 1.49 TB as a floor calculation—not a download-size promise.

03 / Recommended single-node foundation

Buy memory bandwidth
before buying core count.

For CPU LLM inference, a core waiting on weights is not productive. The platform’s memory channels, DIMM population, NUMA behavior and vector/matrix instructions matter more than an impressive aggregate core number.

Processors2 × Intel Xeon 6 6900P-classPrefer a SKU with 12 memory channels per socket and AMX. The 6980P offers 128 cores, 500 W, 12 channels and up to 8800 MT/s MRDIMM support; a lower-core 6972P/6960P can be sensible if bandwidth, not cores, is limiting.
Memory24 × 128 GB ECC modules = 3 TBPopulate one DIMM per channel symmetrically across both sockets. Use server-vendor-qualified RDIMM/MRDIMM. Verify capacity and rated speed together—maximum capacity and maximum frequency are rarely simultaneous.
MotherboardDual-socket, 24+ DIMM slots, PCIe 5Require BMC/IPMI, bifurcation, large BAR, enough x16 slots for future accelerators, and a topology diagram showing which PCIe roots and NVMe bays attach to each NUMA node.
Storage4 × 3.84 TB enterprise NVMe + mirrored boot15.36 TB raw gives room for original weights, converted AMX/GGUF artifacts, temporary shards and rollback copies. Stripe scratch/model volumes for load and conversion speed; keep the OS on a separate mirrored pair.
NetworkingDual-port 100/200 GbE or InfiniBandOptional for a standalone node, essential if a second node or external GPU box is plausible. Put each port near the socket that owns its worker and avoid routing every expert transfer over the socket interconnect.
Chassis / PSU4U enterprise chassis; redundant 2–3 kW PSUA CPU-only dual-500 W node with 24 DIMMs, NVMe and fans can approach 1.5–2 kW at the wall. Select a GPU-ready chassis only if its airflow, slot spacing and power backplane match the intended passive accelerators.
SoftwareLinux, NUMA tooling, huge pages, pinned workersStart with the runtime Moonshot validates. KTransformers is the leading precedent because its K2/K2.5 paths place experts in CPU memory and use AMX INT4, but K3 support is not established as of publication.

Why Intel is the lead option

KTransformers already exposes AMXINT4 paths on Xeon for K2.5. Xeon 6900P supplies 12 channels per socket and high-bandwidth MRDIMMs. That is the shortest plausible path from K3’s low-bit experts to optimized CPU matrix kernels.

  • AMX tile acceleration for low-precision GEMM
  • 24 aggregate channels in a 2P node
  • Up to 8800 MT/s MRDIMM on specified SKUs
  • But: validate K3 kernels and packing first

AMD is a credible alternative

EPYC 9005 provides 12 DDR5 channels per socket, up to 128 PCIe 5 lanes per CPU, AVX-512 with a full 512-bit data path, and large DRAM/CXL capacity. It may win on platform economics or CPU throughput.

  • 24 aggregate DDR5 channels in 2P
  • Up to DDR5-6000/6400 depending configuration
  • Strong AVX-512 and vast PCIe connectivity
  • But: no Intel AMX; runtime kernel maturity decides
Do not leave channels empty to save money. Twelve 256 GB DIMMs may provide the same 3 TB capacity as twenty-four 128 GB DIMMs in a dual-socket system, but only half the channels would be populated. For a bandwidth-bound model, that can sacrifice the very resource the expensive CPUs were purchased to provide.
04 / Performance reality

Capacity lets K3 start.
Bandwidth decides whether it talks.

A rough decode roofline divides sustainable memory bandwidth by the low-bit bytes touched per generated token. It is an upper bound, not a prediction.

If the reported active path is roughly 50B parameters and weights average 4.25 bits, one token may stream about 26.6 GB of active weights before cache reuse and implementation details. A node sustaining 600 GB/s would then have a 22.6 token/s bandwidth ceiling. Real performance is lower after routing, NUMA traffic, non-quantized tensors, KDA/MLA, synchronization and imperfect kernels.

Batching changes the equation. When several tokens select the same expert, its weights can serve multiple matrix rows after one fetch. Throughput can rise while per-request latency worsens. Always specify concurrency when quoting tokens per second.

Decode roofline calculator

Ideal bandwidth ceiling22.6 tok/sbefore compute, NUMA, routing and runtime overhead

Pure CPU

All dense and expert work competes for DRAM and CPU execution. It is the cheapest way to prove residency and correctness, but likely single-user research territory rather than production serving.

CPU + GPU

GPU-resident KDA/MLA, dense layers and selected experts reduce latency; CPU DRAM supplies the vast cold expert pool. This matches the successful KTransformers pattern for earlier Kimi models.

All accelerator

Enough aggregate device memory and high-bandwidth expert parallelism delivers the intended throughput. Moonshot reportedly recommends supernodes with 64+ accelerators for optimal K3 serving—far beyond a personal server.

05 / Three realistic build tiers

Choose the outcome,
then choose the machine.

Tier A · residency lab

CPU-only proof

≈ $50K–$100K
  • 2 × 12-channel server CPUs
  • 3 TB ECC DDR5/MRDIMM
  • 8–16 TB enterprise NVMe
  • 100/200 GbE for later expansion
  • No promise of interactive speed
Goal: load, validate outputs, profile kernels, test short prompts and establish the real bottleneck.
Tier B · recommended direction

CPU-first hybrid

≈ $150K–$250K
  • The same 3 TB host foundation
  • 4–8 × 96 GB RTX PRO 6000 Server Edition
  • 384–768 GB aggregate GDDR7
  • Dense/KDA and hot experts on GPU
  • 200 GbE and GPU-ready 4U/5U chassis
Goal: interactive experimentation and modest concurrency—conditional on K3 runtime support and a workable PCIe/offload schedule.
Tier C · datacenter appliance

8 × H200 node

≈ $300K–$500K+
  • 1.128 TB aggregate HBM3e
  • 4.8 TB/s bandwidth per H200
  • NVLink within HGX
  • Host DRAM still holds overflow experts
  • Multi-node path for full residency/scale
Goal: serious serving. Still below Moonshot’s reported 64-accelerator optimal environment and not a “CPU server” in spirit.
Planning bands, not quotes: enterprise memory, accelerator, chassis, support and integration prices move sharply by region and vendor. A validated OEM configuration costs more than parts, but avoids discovering that 24 DIMMs down-clock, passive GPUs cannot cool, or the desired PCIe slots share lanes.
06 / The hot path through the machine

The server is a
data-movement pipeline.

01 · NVMeCold persistenceCheckpoint shards and converted formats live here. Load them at startup; do not page experts from SSD token by token.
02 · CPU DRAMCapacity tierAll or most of the 896 experts remain resident across two NUMA domains.
03 · RouterSelect 16Each MoE layer routes token representations to a sparse subset of experts.
04 · CPU / PCIeExecute or moveCPU AMX/AVX kernels run resident experts, or activations cross PCIe to/from GPU-owned layers and experts.
05 · GPU memoryLatency tierKDA/MLA, dense weights, vision and popular experts use high-bandwidth device memory when available.

Moving activations to an expert is usually cheaper than moving that expert’s weights. The scheduler should therefore keep experts stationary and dispatch token rows to the device or NUMA node that owns them.

With 896 choices, routing locality becomes a first-class systems problem. If a CPU worker frequently reads expert weights attached to the other socket, the socket interconnect becomes an accidental bottleneck. Shard experts by NUMA node, allocate memory locally, pin threads, and route token batches to ownership.

GPU offload has the same principle. A 4-bit expert that is hundreds of megabytes cannot be copied across PCIe for every selection. Keep a chosen expert set resident on GPUs, execute the rest on CPU, and move compact activations. Which experts deserve device residency depends on routing frequency, batch composition and latency objectives.

KDA reduces the growing cache burden in most token-mixing layers, but K3’s periodic global attention still retains token-indexed state. One-million-token capability is not the same as one-million-token practicality on the first local build. Begin at 8K–32K, measure memory per request, then increase deliberately.

07 / Power, cooling and physical reality

This is rack equipment,
not a quiet tower.

Electrical

Use 208/240 V power

A CPU-only build may draw 1.5–2 kW under sustained inference. Eight 400–600 W GPUs can push a full system into the 6–8 kW class. Size branch circuits, PDUs and redundant PSUs for continuous load with margin—not nameplate arithmetic alone.

Thermal

Plan heat rejection

Every kilowatt consumed becomes roughly one kilowatt of heat. Passive server GPUs require engineered front-to-back airflow. A garage or office needs ventilation or dedicated cooling; a datacenter needs the rack’s per-cabinet limit checked.

Acoustics

Expect 60–80 dBA behavior

High-static-pressure fans protecting 500 W CPUs and dense DIMM banks are loud. Remote the machine, use a proper rack, and never reduce fan policy until temperature telemetry proves safe.

Reliability

ECC and enterprise SSDs are justified

Multi-terabyte memory increases the surface area for faults. Use ECC, scrubbing, machine-check monitoring, redundant boot, power-loss-protected NVMe and checkpoint hashes. Consumer parts turn a research appliance into a recovery exercise.

Boot / conversion

Budget minutes, not seconds

Reading 1.5 TB at an ideal 10 GB/s still takes 150 seconds before parsing, repacking and allocation. Conversion may need another full model copy. Preserve the original shards and automate a reproducible build artifact.

CXL

Capacity rescue, not first choice

CXL Type-3 memory can expand capacity, but it usually offers a different latency/bandwidth tier than local DDR5. Use it for cold experts or overflow only after profiling; do not replace populated local channels with CXL capacity.

08 / The July 27 validation gate

Prove the software path
before buying the metal.

  1. Inspect the official checkpoint.Record actual shard size, tensor dtypes, expert layout, layer count, active parameters, vision weights, context configuration, license and checksums.
  2. Confirm a supported runtime.Look for Moonshot guidance and merged K3 support in KTransformers, SGLang, vLLM or another engine. “Architecture looks similar” is not a compatibility guarantee.
  3. Confirm CPU quantization semantics.MXFP4 is not automatically an AMXINT4 or GGUF kernel format. Verify a quality-preserving conversion, scale handling, packing layout and reference output test.
  4. Prototype on rented hardware.Use a multi-terabyte cloud/bare-metal node for one day. Measure peak RAM, load time, prefill and decode at batch 1 and realistic concurrency before committing capital.
  5. Run correctness comparisons.Compare logits or deterministic outputs against Moonshot’s reference/API over text, tool use, long context and vision. A model that “generates text” can still be numerically or templating-wrong.
  6. Profile the roofline.Measure local and remote NUMA bandwidth, expert GEMM throughput, routing skew, PCIe traffic, cache growth, CPU utilization and power. Identify the limiting tier with counters—not intuition.
  7. Freeze the BOM last.Choose CPU family, DIMM type and GPU count only after the kernels reveal whether the system is bandwidth-, compute-, capacity- or transfer-bound.
My practical recommendation: if the goal is learning, first rent and test. If the goal is ownership, build the 3 TB/24-channel dual-socket foundation with expansion-ready PCIe and power, then add GPUs only after K3’s runtime demonstrates exactly what belongs in device memory. If the goal is production SLA, use Moonshot’s API or a specialist inference provider until validated reference deployments emerge.
09 / Final proposed specification

The shortest honest BOM.

CPU-first K3 research node: dual 12-channel AMX-capable Xeons, 3 TB ECC memory populated across all 24 channels, 15 TB enterprise NVMe, dual 200 GbE, a 4U GPU-ready chassis, redundant 240 V power—and space for at least four 96 GB GPUs.

This configuration is intentionally larger than the idealized weight payload. The margin is what turns a one-off load into a usable laboratory: room for conversion, KV/recurrent state, longer prompts, the vision path, concurrent requests and debugging tools.

Pure CPU mode should be treated as a feasibility and systems-research target. It may generate correctly, but there is no public K3 CPU benchmark yet. The most credible path to interactive behavior is heterogeneous: CPU DRAM supplies economical residency while accelerators handle the dense, attention and latency-sensitive path.

And the most important component is not yet orderable: a validated K3 runtime. On a model this large, software determines which hardware is valuable. Wait for the weights, test the kernel path, then spend.

10 / Sources & evidence standard

Published facts, precedents
and explicit estimates.

K3 launch facts are current to 19 July 2026. The full weights were announced for 27 July and were not available when this was written. Memory figures labeled idealized are calculations; prices are broad planning bands, not vendor quotations; no K3 CPU throughput benchmark is claimed.

  1. 01Moonshot Kimi K3 launch brief↗
  2. 02K3 disclosures, weight-release date and serving guidance↗
  3. 03Official Kimi K2.5 deployment guidance↗
  4. 04KTransformers CPU/GPU K2.5 precedent↗
  5. 05Intel Xeon 6 P-core SKU and memory-channel summary↗
  6. 06Intel Xeon 6 platform and MRDIMM overview↗
  7. 07AMD EPYC 9005 architecture and memory capacity↗
  8. 08AMD EPYC 9005 channels, PCIe and AVX-512↗
  9. 09RTX PRO 6000 Server Edition capacity and power↗
  10. 10NVIDIA 8-GPU RTX PRO server reference↗
  11. 11NVIDIA H200 memory, bandwidth and interconnect↗
  12. 12Companion: KTransformers systems deep dive→
Evidence boundary: reported K3 architecture and release details come from Moonshot disclosures and launch reporting. The 3 TB recommendation, storage plan, roofline calculation, build tiers and procurement gate are engineering estimates derived from those disclosures and prior KTransformers deployments. They must be revalidated against the released checkpoint.