Teaching a local model Dwarf Fortress without training it

  • ai
  • agents
  • experiments

For the last two weeks my Mac Studio has been running Dwarf Fortress forts for language models.

The setup copies the Factorio Learning Environment: the model doesn’t press keys. Each turn it reads a plain-text report on its fort and writes a short Python program against a fort API (dig and build with DFHack’s quickfort blueprints, queue work orders, check farm plots and what dwarves need, then fort.wait(days)). The game stays paused between fort.wait() calls, and each one advances the requested time, cut short at the end of the year or if the fort is lost. A turn that forgets to wait gets one day automatically. Every turn is recorded: a top-down map, the code beside it, and the fort’s wealth.

Everyone plays the same six saved embarks. A year is 336 game days, and each agent plays two years on each embark, so 12 runs each, all on the same harness. Runs that crashed for infrastructure reasons were rerun and aren’t counted.

A year in a fort

For each agent, the median over 12 one-year runs of the score, the score with crafts excluded, and raw created wealth, with how many forts each lost

AgentForts lostDwarves died (12 runs)Score, medianCrafts excluded, medianRaw wealth, median
Claude Opus 5.50 of 121317,82210,88825,458
Claude Sonnet 5.51 of 123317,35412,82023,745
DeepSeek V4.1 Flash1 of 122313,9769,32424,641
GPT-6 Astra0 of 12422,70813,67044,610
Dreamfort script0 of 123611,09611,09611,942
Doing nothing4 of 1297282861

Models in alphabetical order. Dreamfort is DFHack’s well-known blueprint fort, applied in checklist order by a script.

Doing nothing loses a third of the forts. Anything that plays keeps nearly all of them: the four models lost 2 of 48, and the script lost none. What kills forts is thirst in late autumn and winter, once the starting drink is gone. That’s how all four of the do-nothing forts died, and both of the models’ lost forts. Sonnet saw “drink alcohol” at the top of its dwarves’ needs and never got brewing going again. DeepSeek’s dwarves died of thirst while my fort report said it still had 40 drinks. I think those drinks weren’t reachable and my counter overstated them, but I haven’t confirmed that in the game yet.

I’m not ranking the models. Among Opus, Sonnet and Astra, every pair splits the six embarks 3–3 or 4–2, which isn’t enough to establish an ordering. DeepSeek ran through its own API with different settings, so its numbers aren’t a clean comparison either. And even the Claude settings weren’t equal: the gateway I run Claude through gives Opus a little hidden reasoning and Sonnet none, and Sonnet’s many short turns meant its early history got trimmed far more often. A fair comparison needs matched settings first.

They do play very differently. Astra’s median is 27 turns a year, in long strides. Sonnet’s is 99. DeepSeek’s 12 years cost $1.01 in API fees; the others ran on subscriptions.

Three things the runs taught me

The game’s own score can be farmed. Dwarf Fortress tracks “created wealth”, and crafts count toward it. Some agents lean on that hard: the median craft share of a fort’s wealth is 67% for Astra and 61% for DeepSeek. So my score counts crafts only up to the value of what the fort actually built (buildings, furniture, weapons, armor) and multiplies by the share of dwarves still alive. I fixed that rule before these runs. With crafts left out entirely, the models land close to the script’s fort, which makes no crafts at all. Most of the gap on the headline score is crafts.

They found a back door, and it was my bug. In earlier rounds the Python process kept a private handle to the game. Sonnet and DeepSeek reached through it to DFHack’s Lua console and wrote game memory directly: they set farm crops and turned on labors. Mostly things a player can do, but my API didn’t offer them. I closed it: the agent’s code now runs in a sandbox with no connection to the game, and every call is checked. In the 48 model runs above, the gate refused no calls.

They asked for features. I gave the API a fort.request_feature(feature, why, example) call. The top requests were things a human player would want too: farm plot status, cancelling orders, slaughtering animals, what each dwarf needs. I built them, and the models use them a lot: in these runs Sonnet included fort.farm_plots() in 193 of its turn programs. Whether they help, I can’t tell yet. Scores before and after sit within run-to-run noise.

The model I want playing

The model I actually want running these is Qwen3.8 27B, on my own Mac. It plays a season (84 days) in about an hour. On the current harness its median score over six seasons is 672. I haven’t run it for full years, so it isn’t in the table.

The normal fix is training on the frontier models’ games, and I have their transcripts. What I don’t have is the memory. Qwen3.8 is a hybrid, and with my MLX setup, backpropagating through one of its linear-attention layers added about 12.5 MiB per token between two test sizes. One such layer at 4,096 tokens tripped my memory-pressure guard, while a full-attention layer at nearly 8,000 tokens fit. I didn’t want to rent GPUs for this.

Weight-space programming

A week ago MIT CSAIL published Sparks of Weight-Space Programming. Their claim is that frontier models can read and write network weights directly, with no gradients. GPT-6 Astra wrote small game-playing networks from scratch that beat PPO trained for 100 million steps. It also edited 0.016% of Qwen3.5-0.8B’s weights to reduce its reasoning loops, which took MATH-500 from 7.9% to 48.1%.

So the plan became: let the big model program the small one. Astra gets the 27B’s weights, a Python kernel, a budget of forward passes, and a closed-form solver from In-Context Compilation. That solver takes “how the model behaves with some text in its context” and tries to approximate it with a weight edit, without the text.

What’s happened so far:

  • One output row, 5,119 weights. Astra’s first edit to the 27B played six seasons. Its median wealth was 1,345, against 1,321 for the unedited model, both on an earlier harness than the six-season baseline above. No measurable effect either way.
  • The smallest real test. I picked one rule: start every program with fort.farm_plots(). With the rule in the latest user message, Qwen follows it on 20 of 20 held-out game states. Without it, 0 of 20. Compiled into two layers’ weights, with the rule gone: 0 of 20. That’s one configuration, on states from one held-out embark, and one base and one edited output hit the length cap. Nine unrelated control prompts came out unchanged, but the rule didn’t transfer.
  • A calibration on a small model. On Qwen3.5-0.8B, where real training fits on the CPU, a broader compile (all layers, every reply position, attention and MLP) recovered 96% of the improvement three in-context examples make to its loss on the teacher’s replies (one compile configuration, 120 rows from one held-out embark; a proxy, not gameplay). But in-context examples only get about half as far as LoRA training, so the compile got 44% of LoRA’s improvement. It also made the model worse at ordinary prose and code, and its weight change looks nothing like LoRA’s (mean cosine similarity per matrix: 0.0003).

So far the edits either do nothing or recover part of what a prompt already does. The next try ports the recipe that worked on the 0.8B, every layer and every reply position, to the 27B.

I don’t know yet if this gets Qwen anywhere near the frontier models. It might turn out that a better prompt gets most of the way there, which would be a fine answer too.