llama-modchip Runs 284B DeepSeek on 64GB RAM Without GPU

An Apache 2.0 open-source project called llama-modchip, released by GitHub user synchopate, claims to run DeepSeek V4-Flash's 284.33 billion parameters entirely on a 64GB CPU-only machine with no GPU hardware. The project trended to 417 points on Hacker News as developers verified throughput figures of 44.4 tokens per second prefill and 9.1 to 14.7 tok/s decode at 256K context on a 16-core Zen 5 CPU.

Officially released on November 9, 2026, the repository reports peak resident memory of 57.61 to 58.9 GiB with zero swap usage, down from a 76.87 GiB baseline. The compression achievement places frontier mixture-of-experts inference within reach of consumer-grade hardware for the first time.

Fractal-64 Compression Architecture

The core innovation is a custom weight format the project calls Fractal-64, which packs DeepSeek V4-Flash's weights at 1.71 bits per weight using block-64 BF16-scale tensors. Routed experts drop as low as IQ1_S at 1.5625 bpw, cutting the model's footprint from 82.5GB to 60.73GB. The README reports a lossless configuration with 1.00000 cosine similarity and 0.0000 KL-divergence against the full-precision reference, which the author labels a bit-exact match on logits.

The quantization scheme targets mixture-of-experts models specifically, exploiting sparse activation patterns where only 13.17 billion of DeepSeek's 284.33 billion parameters are active per forward pass. By aggressively compressing inactive experts and preserving precision in the routing layer and active shards, llama-modchip balances memory reduction with output fidelity.

CPU Kernel Optimizations

The throughput claims rely on what the README describes as a four-level register tiling pattern using ZMM AVX-512BW registers, L1D (32 KB), L2 (1 MB), and L3 (64 MB) caches. The project implements register-stationary MQA8 FlashAttention and credits these kernel patches with a 77 percent prefill uplift and 53 percent decode uplift versus the unmodified baseline.

A speculative-decoding scheme borrowed from the Stockfish chess engine, called Stockfish-PVS v2, uses principal-variation search and a 1.64 MB Zobrist hash table to predict likely token sequences. On the 16-core Zen 5 test system, this drafter pushes decode throughput from 9.1 tok/s to 14.7 tok/s at 256K context. The repo also lists verified runs on Apple M4 Max and Pro with Metal and NEON acceleration, AWS Graviton3 Neoverse-V1 cores, and a Threadripper PRO 7985WX that achieves 111.62 tok/s on the smaller Nemotron-3.5-30B-A3B model across eight CCDs.

Cross-Platform Performance

Beyond x86-64, the project reports ARM64 and Apple Silicon compatibility. On AWS Graviton3, the same DeepSeek V4-Flash configuration runs with the documented memory footprint and prefill speeds within 10 percent of the Zen 5 baseline. The M4 Max and Pro implementations use Metal for matrix operations and NEON for quantized arithmetic, delivering comparable decode throughput on unified memory architectures.

The Threadripper PRO 7985WX run demonstrates scaling behavior: the smaller Nemotron model saturates eight CCDs at 111.62 tok/s, suggesting that memory bandwidth rather than compute becomes the bottleneck at frontier model sizes. Every figure above is the author's; no independent third party has reproduced the benchmarks at publication time.

Availability and Licensing

The llama-modchip repository is available under the Apache 2.0 license on GitHub, with build instructions for Linux, macOS, and Windows. The README includes step-by-step quantization commands for converting Hugging Face checkpoints to the Fractal-64 format and launching inference with the Stockfish-PVS drafter enabled. Dependencies include a C++20 compiler, AVX-512 support, and 64GB system RAM.

The project arrives one day after a separate report on DeepSeek's inference harness crossing 1.54 million daily users, indicating sustained interest in efficient deployment of the V4-Flash model family. No commercial support or hosted API is announced; the project targets developers who build local inference infrastructure.

What This Means

llama-modchip demonstrates that frontier mixture-of-experts models can run on commodity CPU hardware without dedicated accelerators, lowering the infrastructure cost for organizations that cannot access scarce GPU capacity. The 1.71 bpw Fractal-64 format and Stockfish-PVS speculative drafter provide a replicable blueprint for CPU-only inference at speeds that approach early GPU-accelerated deployments from two years ago. If the reported bit-exact lossless configuration holds under independent verification, the project establishes a new baseline for memory-constrained AI deployment in 2026.