nesformer README

项目的 README,来自 github.com/dbreunig/nesformer。合并为一个 markdown 文件:README.md。

nesformer

An NES CPU compiled into the weights of a stock Qwen3.5 model.

The NES's 6502 CPU (the core of its 2A03 chip), its 2 KB of work RAM, and a cartridge's program ROM run inside a standard Qwen3_5ForCausalLM checkpoint. Nobody trained these weights; a compiler computed every one. When you play a game, the model predicts the CPU's execution log one token at a time. A small host program plays the rest of the console around it: it draws the picture, reads the controller, and fires the vblank interrupt.

nesformer builds on Qwenputer by Divyansh Agrawal, which compiled a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B. It reuses Qwenputer's compiler and its central trick: the model's Gated DeltaNet state is the machine's memory.

Super Mario Bros. (World 1-1) Full Game by Doug Fraker (title screen and first level)
Super Mario Bros., played by the weights Full Game, played by the weights

Every CPU token in these runs came from the model. For the bundled test games, every frame also matches a conventional 6502 emulator's output exactly (see How it was checked).

Quick start

1. Folder layout

Unzip nesformer.zip and nesformer-weights.zip into the same folder. The weights go next to the nesformer/ package, not inside it:

nesformer/              <- run commands from here
├── nesformer/          <- the Python package
├── chipformer/         <- Qwenputer's compiler
├── roms/               <- test ROMs (put your own here too)
├── examples/           <- sample GIFs and frame sheets
├── tests/
├── LICENSE
├── LICENSE.chipformer
├── README.md
├── pyproject.toml
└── weights/            <- from nesformer-weights.zip
    ├── config.json
    ├── generation_config.json
    ├── model.safetensors
    ├── nesformer.json
    └── SHA256SUMS

The CLI looks for weights/ beside the nesformer/ package. To keep the weights elsewhere, pass --weights /path/to/weights to run, check or trace, or give the path to info.

To confirm the weights unzipped correctly, run shasum -a 256 -c weights/SHA256SUMS from the top folder.

2. Install

You need Python 3.11 or newer.

python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install numpy scipy numba pillow

3. Play a game

python -m nesformer run roms/thwaite.nes --frames 600 --show \
  --buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593"

--show draws each frame in your terminal as the model generates it. When the run ends, the CLI saves an animated GIF named after the ROM (here thwaite.gif) in the current folder.

Expect it to be slow. The first run spends about 20 seconds compiling the engine with numba. After that, a frame takes 1–6 seconds, so 600 frames (10 seconds of NES time) take around 25 minutes.

Playing your own ROMs

Point run at any .nes file that uses the NROM board (iNES mapper 0), such as Super Mario Bros., Donkey Kong or Excitebike. You supply the ROM; nesformer only ships the three freely licensed test ROMs in roms/.

python -m nesformer run roms/smb.nes --frames 600 --show \
  --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555"

That script starts the game, then walks Mario right and jumps about once a second. The CLI loads the ROM into the model's state at startup, so the same weights run any NROM game without a rebuild. Games on other boards stop with a message: see Limitations.

Controller scripts

--buttons takes a comma-separated list of entries. Each entry names one or more buttons joined by +, then @, then one or more frame ranges joined by +:

start@60-65                 press Start during frames 60 to 65
right@200-600               hold Right from frame 200 to frame 600
a@250-265+300-315           press A twice
right+b@200-600             hold Right and B together (run, in Super Mario Bros.)

The buttons are a, b, select, start, up, down, left and right. Frames count from power-on at 60 per second, so frame 600 is 10 seconds in. Games ignore input on screens where you can't move, so holding a button a little early does no harm. In Super Mario Bros., a longer A press makes a higher jump.

Commands

Command What it does
python -m nesformer run ROM Play a ROM on the weights and save a GIF
python -m nesformer check ROM Run the weights and the reference CPU side by side; stop at the first difference
python -m nesformer trace ROM Print the model's token stream (default 40 tokens)
python -m nesformer build Compile the circuit and write weights/ (about 10 seconds)
python -m nesformer info Describe the checkpoint

Options for run:

Option Default Meaning
--frames N 120 Stop after N frames
--buttons SCRIPT none Controller input (see above)
--show off Draw frames in the terminal (needs 24-bit color, e.g. iTerm2 or Windows Terminal)
--gif PATH ROM name Where to save the animated GIF
--png DIR off Save each kept frame as a PNG in DIR. With --png alone, no GIF is saved. Add --gif to get both.
--every N 1 Keep every N-th frame
--scale N 2 Enlarge saved frames N times
--engine fast fast (numba), numpy, or hf (stock transformers; needs torch, ~10 tokens/s)
--no-skip off Run every iteration of idle wait loops (slower; identical frames)
--max-tokens N none Stop after N tokens
--trace FILE none Save the token ids

run, check and trace all accept --weights, --engine, --buttons and --no-skip. check --tokens N sets how far to compare (default 50,000). trace --tokens N sets how many tokens to print (default 40). build --out DIR writes the checkpoint somewhere other than weights/.

How it works

What's in the weights, and what isn't

In the model (weights + DeltaNet state) In the host (nesformer/host.py)
The 6502 core: all 151 official opcodes, every addressing mode, flags, stack, BRK and NMI entry The PPU: a scanline renderer and the palette
2 KB of work RAM Controllers; APU registers (written, otherwise ignored)
32 KB of PRG-ROM, stored twice: one copy for instruction fetch, one for data reads NMI timing at vblank, injected as an NMI token
OAM DMA reads (the 256 bytes appear as DM tokens) Answers to I/O reads, injected as IN tokens

This split follows Qwenputer's CHIP-8 build, whose host injects timer ticks and draws the display from the trace. The host never sees the CPU's registers or RAM except through the trace.

The memory trick works like this. Each DeltaNet head keeps a 128 × 128 state matrix that the delta rule updates at every token. With the gate held open and a one-hot key, an update overwrites exactly one row of that matrix, and a one-hot query reads it back. So every row is a memory cell, and the cartridge, RAM and registers all live in rows.

The checkpoint

weights/ is an ordinary Hugging Face folder. It loads with Qwen3_5ForCausalLM.from_pretrained and needs no custom code.

Architecture Qwen3_5ForCausalLM, with Qwen3.5-4B's DeltaNet head shapes (16 key heads, 32 value heads, dim 128) and MLP width (9,216)
Layers 7 Gated DeltaNet layers (the circuit), plus 1 all-zero attention layer that Hugging Face's cache needs
Hidden size 768
Vocabulary 263,808 tokens
Parameters 466.6M in bf16 (933 MB), of which 9.3M are nonzero, so the weights zip to 3.3 MB

nesformer.json records which layer and head holds each part of the machine. A run writes the cartridge, registers and RAM into those DeltaNet states, then decodes greedily.

Layer plan

Layer Attention (state) MLP
1 PC anchor; instruction counters; A, X, Y, SP; flags N V Z C I D; pointer latches; NMI flag counter thermometers; SP ± 1; LO + Y
2 — PC = anchor + bytes executed since; PC + 1, PC + 2
3 PRG fetch: opcode and two operand bytes at the PC opcode decode; operand bits; indexed sums; branch test and target
4 — effective address, by addressing mode and record stage; the byte a push stores
5 RAM and PRG data reads at the effective address operand; ADC/SBC adder, compares, BIT; address decode
6 — result byte, carry, overflow, flags, jump targets
7 — the next token

The trace

Each instruction becomes a record, and the record's last token commits it. As in Qwenputer's RISC-V core, the PC is implied rather than written: each final token carries its instruction's length, three DeltaNet counters tally the instructions of each length since the last jump, and the PC is the jump target plus that offset. Excerpts from Full Game:

SEI            P(000100 px0 L1)
LDX #$40       X($40 c0 v0 px0 L2)
STX $4017      IOW($4017=$40 L3)
TXS            S($FF L1)
BIT $2002      IOR($2002) IN($00) P(000110 px0 L3)       <- the host injects IN
BPL $8017      J($8017)
JSR $8279      PSH($FE) PSH($FD) J($8279)                 <- the return address is pushed in-circuit
RTS            PUL($FC) LO($7B) PUL($FD) J($827C)
STA ($72),Y    LO($08) HI($03) ST($00 a7=0 L2)            <- pointer bytes latched, then the store
STA $4014      IOW($4014=$02 L0) DM(0:$FF) … DM(255:$00) NT(L3)   <- OAM DMA

Most instructions take a single token; real game code averages 1.1–1.3 tokens per instruction. nesformer/spec.py documents every token family.

How it was checked

Speed

The default engine computes Qwen3.5's forward pass from model.safetensors, compiled with numba and skipping the all-zero blocks. It ran at about 1,650 tokens/s on one CPU core of the machine it was built on. The plain numpy engine does about 210, and stock transformers about 10. A real NES would need about 700,000 tokens per second.

Games spend most of each frame in a loop that waits for vblank. When the host sees a loop body repeat exactly, with no RAM or I/O writes, it advances the clock to just before the next PPU event, and the remaining iterations still run. That cuts the token count by 2.6× to 4.2×. Overall, nesformer runs 150–290× slower than the real console.

Limitations

Tests

pip install pytest py65
python -m pytest -q tests

The tests take a few minutes. They check the reference CPU against nestest's log, the circuit against the reference, the weights free-running against the reference, and the idle-loop skip.

Running the checkpoint in stock transformers

pip install "torch>=2.6" "transformers>=5.22"
python -m nesformer run roms/thwaite.nes --frames 3 --engine hf

nesformer/hfcheck.py shows the whole procedure: load the checkpoint, decode one padding token, write the machine's state into the cache's DeltaNet states, then feed tokens one at a time.

Files

Path Contents
nesformer/spec.py token families and records
nesformer/cpu.py the reference 6502 that writes the trace
nesformer/host.py everything around the CPU: PPU, controllers, NMI timing, cycle counting, idle skip
nesformer/core.py the circuit and the state snapshot
nesformer/weights.py the emitter that turns the circuit into a Qwen3_5ForCausalLM checkpoint
nesformer/engine.py, fastengine.py run the weights (numpy / numba)
nesformer/hfcheck.py run the weights in stock transformers
nesformer/run.py, check.py, cli.py, __main__.py drivers, the teacher-forced check, the CLI
chipformer/ Qwenputer's q35 compiler (q35/ emitter and primitives, dsl/ graph)
weights/ the compiled checkpoint and its SHA-256 checksum
roms/ test ROMs; roms/SOURCES.md lists their authors and licenses
examples/ GIFs and frame sheets from the runs; demo.gif is the Super Mario Bros. run shown at the top
tests/test_nesformer.py the test suite

Credits

License

nesformer is released under the MIT License (see LICENSE). The ROMs in roms/ keep their own licenses, listed in roms/SOURCES.md.

nesformer (dbreunig/nesformer)

正在搜索数据库…

这是什么?nesformer 把 NES 的 6502 CPU、它的 2 KB 工作内存(RAM),以及游戏卡带的程序 ROM 直接编译进标准 Qwen3.5 模型的权重中。这样大语言模型就能一次预测一个 token 的 CPU 执行日志,而一个小型宿主程序负责绘制画面并处理输入。
这是经过训练的 AI 模型,还是真正编译出来的 CPU 逻辑?两者都不是,也不是普通的模拟器:编译器手工算出了每一个权重,模型的 Gated DeltaNet 状态被当作机器的可寻址内存使用,用来存放 RAM、寄存器和 ROM,而不是学习到的参数。
我该怎么让它运行起来?将 nesformer.zip 和单独的 nesformer-weights.zip 解压到同一个文件夹中,使 weights/ 与 nesformer/ 包并列,然后运行类似 python -m nesformer run roms/game.nes --show 的命令,即可游玩并保存动态 GIF 动图。
试用前需要先安装什么?Python 3.11 或更高版本。默认的快速引擎使用 numba;可选的 numpy 引擎无需额外安装任何东西;若要通过标准的 Hugging Face transformers(即 hf 引擎)运行检查点,还需额外安装 torch。
它能运行任何 NES 游戏吗?不能。它只支持 NROM 卡带(iNES 映射器 0)和官方 6502 操作码,因此使用 MMC1 或 MMC3 等分页切换(bank-switching)映射器的大型游戏不受支持;遇到非官方操作码或跳入 RAM 都会使执行停止;此外也不支持声音和 IRQ。
nesformer 的速度够快,能实时玩游戏吗?不能。快速引擎的速度约为每秒 1,650 个 token,因此渲染一帧需要 1 到 6 秒,整个系统比真正的 NES 慢 150 到 290 倍;游戏中 10 秒的时长可能需要约 25 分钟的实际运行时间。
我能使用自己的 NES 游戏 ROM 吗,比如《超级马里奥兄弟》或《大金刚》?可以,任何使用 NROM 电路板(映射器 0)的 .nes 文件都可以运行;nesformer 只附带三个许可证自由的测试 ROM,其余的需要你自行提供。
这个系统的准确性是如何验证的?将权重在自由运行时的输出与传统的 6502 参考模拟器以及 py65 进行逐 token 比对,涵盖数百万条指令;所有随附的测试运行产生的画面都与该参考完全相同。
为什么要在 LLM 的权重中构建 CPU,而不是直接模拟 NES?这是一个研究演示,基于 Divyansh Agrawal 的 Qwenputer,旨在证明 Gated DeltaNet 模型的循环状态可以充当精确的、可寻址的机器内存,它并不是一种实用的玩游戏的方式。
作者
dbreunig
大小
3.8 MB
星标
295
更新
提交Oct 30
时长
19天
提交 · 90 天