Claude Sonnet Walks Into DOOM, One Frozen Frame at a Time

Andrea Ricci wanted a language model to do more than answer questions. He wanted it to walk a corridor, notice a door, raise a shotgun, and live long enough to tell you what just tried to kill it. He put that test on a board small enough to sit in a palm, running the 1993 shooter that still gets ported to anything with a screen. Claude Sonnet is the marine, and a custom ESP32 module is the planet.

Scintix P4, RELOC’s module, also known as RM-CMP4, has an intriguing design. It’s just a module trimmed to fit a Raspberry Pi Compute Module 4 or 5, and you can simply plop it into a Pi carrier; everything from HDMI ports to USB sockets to camera connectors works as expected. Inside, there’s a dual-core RISC-V ESP32-P4 processor running at 400 MHz, 32 MB of PSRAM to play with, and a decent 32 MB of flash memory. In addition, an ESP32-C6 manages Wi-Fi and Bluetooth over a four-bit SDIO interface to keep everything running smoothly. A 7-inch 1024 x 600 MIPI-DSI panel, controlled by an EK79007 controller, handles the display responsibilities. The audio is then processed through an ES8311 codec, and they’ve even tested it with an M5 Atomic EchoBase speaker.

You can still use a USB keyboard or a DualShock 4 controller, but Claude doesn’t require any of them. The DOOM game in question does not even run on a custom engine. Ricci, the person who worked on it, delved into an existing ESP32-P4 port created by another guy, Mazur888. It already had DOOM running on a different screen, but there were a few bugs to resolve. Ricci improved the display bring-up to work with Espressif’s standard code, ensured that the panel remained in native landscape mode, and resolved an issue with the MIPI clock source that had not functioned properly with newer silicon. One of the best features is the scaling. Originally, each frame would take roughly 24 milliseconds to enter the display buffer and then require a copy to the screen, which is an additional overhead that consumes CPU time. With the P4’s pixel processing accelerator, it can now stretch each frame into the screen buffer in around 9 milliseconds and page flip without having to copy the data to the screen, allowing the system to run much more smoothly.


Boot-up displays a nice color test screen before loading DOOM1.WAD from a 12 MB SPIFFS partition. No saved Wi-Fi settings yet? Well, an open access point named Scintix P4 -XXXXXX will emerge, and a captive portal will ask for a password. Once that is completed and you are on board, you can access settings via a browser page on the board’s IP. Any subsequent traffic directed at ws://that-ip/agent will be correctly redirected. The module is available to back on Crowd Supply, and the source is all on GitHub under the GPL v3 license, while the WAD is still subject to id’s regular shareware restrictions.


Watch the video, and the player seemingly jerks forward at random. Click a few steps forward, then pause, turn, and fire. The stuttering pace is actually intentional, and not caused by a latency bug. The firmware creates a WebSocket path called /agent, and as one action is performed, game time advances slightly; a few ticks are all that is required. Then you receive a JSON observation, and the level remains frozen until you send the next command. Ricci refers to it as lockstep, and the concept is that a fixed frame provides the model with a steady world to narrate, rather than a confused mess of motion that it cannot keep up with. There’s a small Python server, tools/doom_mcp_server.py, that wraps the socket, allowing Claude Code to treat the board like a workshop full of tools: you can observe, move the player forward or back, turn left or right, strafe, fire your gun, use an item, select a new weapon, or request a map. Adding it in is a single line that points to the board’s WebSocket, and the model is now holding the weapon.
[Source]