NES Console Runs Inside Qwen3.5 Model Weights

Developer dbreunig released Nesformer on October 30, 2026, a project that compiles a complete NES console into the weights of a stock Qwen3.5 language model. The system runs Super Mario Bros. by predicting 6502 CPU execution tokens one step at a time, with every computational element of the original Nintendo hardware encoded in the model's parameter matrices. No training occurred; a compiler calculated every weight value to represent the CPU, 2 KB of work RAM and cartridge program ROM.

Architecture: Memory Lives in Gated DeltaNet State

The core innovation stores the NES machine state inside Gated DeltaNet layers. Each DeltaNet head maintains a 128 × 128 state matrix that updates at every token via the delta rule. When the model presents a one-hot key, the update overwrites exactly one row, turning each row into an addressable memory cell. The cartridge ROM, work RAM and CPU registers all map to these rows across seven stacked Gated DeltaNet layers. The checkpoint also includes one all-zero attention layer to satisfy Hugging Face cache requirements.

The final weights total 466.6 million bf16 parameters, but only 9.3 million are nonzero. The sparse structure compresses to a 3.3 MB file. A standard `Qwen3_5ForCausalLM.from_pretrained` call loads the checkpoint with no custom code. The GitHub repository states: "Nobody trained these weights; a compiler computed every one."

Validation Against Conventional Emulators

The project reports cycle-exact verification across multiple test runs. In a 600-frame run of the test ROM Thwaite, Nesformer generated 1,916,527 tokens over 10.0 seconds of emulated NES time. Every one of 300 saved frames matched a conventional 6502 emulator's output exactly. A 420-frame run of Doug Fraker's Full Game produced 2,645,885 tokens with identical frame-by-frame results. The nestest conformance suite ended in the expected HALT state after 4,586 tokens.

Only NROM cartridges (iNES mapper 0) are supported, covering titles like Super Mario Bros., Donkey Kong and Excitebike. Games that use bank-switching mappers cannot run on the current architecture.

Performance and Execution Speed

Nesformer runs between 150 and 290 times slower than real NES hardware. The fastest execution engine, a numba-compiled forward pass that skips all-zero blocks, achieves approximately 1,650 tokens per second on a single CPU core. The stock transformers path manages about 10 tokens per second. A real NES requires roughly 700,000 tokens per second to maintain frame rate, putting the compiled model far below real-time performance.

Despite the speed gap, the architecture proves that a deterministic state machine can be encoded in transformer weights and executed through token prediction. Each token represents one step in the 6502 instruction cycle, and the model's autoregressive generation reproduces the CPU's execution trace without deviation.

Release Date and Predecessor Work

Officially released on October 30, 2026, Nesformer builds on Qwenputer, an earlier project by Divyansh Agrawal that compiled CHIP-8 and RISC-V cores into Qwen3.5-4B. Both projects demonstrate that compiled, deterministic computation can live inside language model architectures without training. The weights serve as a storage medium for logic gates and state transitions rather than learned patterns from data.

What This Means

Nesformer validates a new category of model architecture where weights encode explicit computation rather than statistical approximation. The cycle-exact verification against conventional emulators proves the approach can replicate deterministic systems with no error accumulation. While execution speed remains impractical for real-time use, the technique opens questions about hybrid architectures that combine compiled logic with learned behavior, and whether future hardware accelerators could close the performance gap for state-machine transformers.