DoomVLM is now Open Source - VLM models playing Doom

r/LocalLLaMA
Generative AI Open Source AI

A couple days ago I posted a video of Qwen 3.5 0.8B playing Doom here - it blew up way than I expected, and a lot of people asked me to open source it. Here it is: Since then I've reworked things pretty heavily. The big addition is deathmatch - you can now pit up to 4 models against each other on the same map and see who wins. Quick reminder how it works: the notebook takes a screenshot from ViZDoom, draws a numbered column grid on top, sends it to a VLM via any OpenAI-compatible API. The model has two tools - shoot(column) and move(direction), with tool_choice: "required.