Yes — technically, Kimi Linear could become a CPU-only model. In some respects its architecture is genuinely attractive for CPU long-context inference. But open weights alone aren't enough. What's missing isn't theory or memory — it's optimized CPU implementations of KDA, MLA, and its MoE layers.
01 Why It Could Work
the parameter count is misleading in a good way
Kimi Linear's headline numbers look intimidating until you notice how the model actually uses them:
Because it's an MoE model, every token only executes a subset of the 48B weights — see the Kimi Linear model card.
Its arithmetic cost per token is therefore much closer to a small dense model — although every expert's weights still have to stay accessible somewhere, even the ones that aren't firing this step. That distinction — light compute, heavy storage — is the whole tension of running MoE models on CPU.
The weight-size math actually works out
| Weight format | Approx. raw size |
|---|---|
| BF16 | 96 GB |
| INT8 | 48 GB |
| INT4 | 24 GB |
| 3-bit | 18 GB |
Add metadata, embeddings, alignment, and runtime buffers, and a practical INT4 deployment lands around 28–35 GB — comfortably inside a workstation with 64–128 GB of RAM. Capacity, in other words, was never going to be the blocker.
02 Linear Attention Helps CPUs With Long Context
a fixed-size state beats a growing KV cache
For 75% of its attention layers, Kimi doesn't scan an ever-growing KV cache at all. It processes fixed-sized recurrent states instead:
Normal attention CPU work: scan historical KV for every generated token KDA CPU work: read/update fixed state matrices
At extremely long contexts this stops being a minor optimization and becomes the whole ballgame. A conventional CPU deployment gets progressively slower as context grows, because attention has to read more and more KV history for every new token. KDA's token-generation cost for those layers stays approximately constant regardless of how long the context gets.
The caveat: the remaining MLA layers still carry a context-growing KV cache, so total decoding cost isn't fully constant — just far flatter than a conventional architecture's.
03 The Real Obstacle: CPU Kernels
theoretical feasibility isn't a runtime
Here's where the optimism has to meet reality. The released fast KDA implementation is built entirely around GPU-oriented tooling:
- Triton kernels
- CUDA kernels
- TileLang kernels
Official usage and deployment paths emphasize Flash Linear Attention and vLLM acceleration. Kimi Linear does not currently appear among llama.cpp's supported model architectures.
A serious CPU runtime would need native implementations covering all of the following, not just one:
- KDA state read/update
- MLA attention
- MoE routing
- Token-to-expert permutation
- Quantized grouped matrix multiplication
- RMSNorm, RoPE, and activation fusions
A slow PyTorch reference implementation would technically run on a CPU. That is a categorically different claim from "usable" — the difference between a proof of concept and a deployable model.
04 MoE: Both the Advantage and the Difficulty
sparse compute, scattered memory
That routing is what lowers total computation — but the selected experts can change on every single token, which scatters memory access across a weight set that may span 24–100 GB depending on quantization. On a GPU, HBM's raw bandwidth absorbs that chaos. On a CPU, performance instead hinges on a long list of hardware specifics:
- DRAM channel count and bandwidth
- Whether expert weights are NUMA-local
- INT4 / INT8 vector kernels
- AVX-512, VNNI, or AMX support
- Batch size
- Expert-routing locality
- Whether hot experts stay cache-resident
The model may fit comfortably in RAM and still run slowly — inference at this scale is mostly about moving weights, not merely storing them.
05 KDA Is Genuinely Implementable on CPU
a regular access pattern, not a sparse lookup
Unlike the scattered MoE access pattern, KDA's core recurrent-state operation is dense and regular — it works with 128×128 matrices, which CPUs are well equipped to handle:
- AVX-512
- Intel AMX
- ARM SVE/SME
- oneDNN or specialized GEMM microkernels
- Fused state-read/update/output operations
A well-built implementation would process state tiles entirely in cache rather than round-tripping to DRAM repeatedly:
That avoids physically reading the state from DRAM three separate times — and this regular, predictable access pattern is far friendlier to a CPU than sparse, random KV retrieval ever is.
Where CPU-Only Would Actually Be Compelling
Makes sense for
- Private local deployment
- Offline batch processing
- Long-running agents
- High-throughput servers with large batches
- Machines with abundant, inexpensive RAM
- Latency-tolerant environments (a few tokens/sec is fine)
- Hybrid CPU/NPU systems
Poor fit for
- Very fast interactive generation
- Million-token prompt ingestion
- Large concurrent batches on ordinary desktop DDR5
- Workloads needing tens or hundreds of tok/s per user
A Particularly Interesting Implementation
tiering memory the way GPUs tier bandwidth
The most promising CPU-only architecture would likely tier data across memory the way a well-designed GPU pipeline tiers across bandwidth levels:
Notice that SSD offload for KV only makes sense for the remaining MLA layers and for persistent cold contexts — KDA's fixed-size state has no need for it.
So, yes: open weights could enable a practical CPU-only version of Kimi Linear, especially after INT4 quantization. The missing ingredient was never theoretical feasibility or RAM capacity — it's a llama.cpp- or oneDNN-class CPU runtime with native quantized MoE, MLA, and fused KDA kernels. Until that runtime exists, "runs on CPU" and "usefully runs on CPU" remain two very different claims.