Weight-space programming labs

Big thing: These nine hands-on labs reproduce and take apart Experiment 8 of the MIT CSAIL blog post "Sparks of Weight-Space Programming" on two 8-GPU A100 machines. In that experiment, an AI agent fixed a small language model (Qwen3.5-0.8B) that got stuck repeating itself on math problems by editing 131,072 of its 873,438,784 weights with arithmetic, without training.

Why it matters: Most model fixes change the prompt or sampling settings, or run gradient descent (training that adjusts weights to reduce errors). Here you do four things:

• Inspect the 131,072 edited weights (130,188 change after bf16 rounding, which stores numbers at reduced precision).
• Rebuild the edit from 8 short texts in one minute.
• Prove that the weight edit equals an input-driven steering vector (a change to the model's internal numbers whose strength depends on the input).
• Build your own edit from the model's failures.

The story#

Weight-space programming changes what a trained neural network does by reading and editing its weight matrices with arithmetic instead of gradient training.

In Experiment 8 (2 Nov 2026), an AI agent (GPT-6 Astra) got a CPU-only Python session with Qwen3.5-0.8B, an open model with about 0.87 billion weights across 24 repeated layers (blocks). It had practice math problems with answers and a budget of 50 forward passes. A forward pass runs inputs through the network to read internal activations (numbers computed as the input passes through each layer) or generate answers. One pass could answer up to 512 problems or read activations on up to 1,024 texts, and all training was forbidden.

With thinking on, the model writes a <think>...</think> scratchpad before its final answer. In this mode, the unedited model kept falling into verbatim loops, repeating the same block of tokens (word pieces) over and over until it hit the 32,768-token cap: 85 of its first 96 practice answers hit that limit.

The agent wrote 4 productive and 4 looping scratchpads and compared the model's activations on them. That contrast gave a unit direction vector $u$: a list of 1,024 numbers whose overall length is 1, pointing from productive text toward looping text. It then applied a rank-one projection edit (one column vector times one row vector) in blocks 15 and 19. Each block's MLP (multi-layer perceptron, the block's second half) has an output matrix down_proj of shape 1,024 by 3,584. Each column is the 1,024-number vector one neuron adds to the residual stream (the running list of numbers each token carries from block to block) per unit of activation. The agent picked the 64 neurons that push hardest along $u$ during loops and updated their columns $W_S$ via $W' = W - \alpha u (u^T W_S)$ with strength $\alpha = 6$. That edit touches $2 \times 64 \times 1,024 = 131,072$ weights (p50c, from the 50-pass run folder runs/p50c).

On three unseen benchmarks, the paper reported these results for p50c:

• MATH-500: Accuracy rose from 7.9% to 48.1%. Answers cut at the cap fell from 93.0% to 33.05%, and mean tokens fell from 30,868 to 22,037.
• GSM8K: Accuracy rose from 13.84% to 67.1%. Answers cut at the cap fell from 87.5% to 7.24%, and mean tokens fell from 29,289 to 11,993.
• IFEval: Instruction-following accuracy rose from 21.0% to 62.3%. Answers cut at the cap fell from 74.9% to 2.2%, and mean tokens fell from 25,296 to 4,241.

What we found#

• The benchmark gains reproduce on our hardware (labs/lab2_reproduce.md, labs/lab1_diff.md). Diffing all 488 weight tensors (arrays of weights) confirms that only layers.15.mlp.down_proj.weight and layers.19.mlp.down_proj.weight changed, in 64 columns each, with fitted $\alpha = 6.0003$ and $6.0002$. On our GPUs, p50c raises MATH-500 from $7.3 \pm 0.9\%$ to $51.8 \pm 1.9\%$, GSM8K from $14.2 \pm 0.8\%$ to $66.9 \pm 1.2\%$, and IFEval from $21.2 \pm 1.4\%$ to $63.4 \pm 1.8\%$.
• The 5-minute quick run shows the same effect on 16 problems (QUICKSTART.md, labs/lab2_reproduce.md). On the first 16 GSM8K problems of our 256-problem practice test set (dev256), the unedited model scores $18.8 \pm 7.7\%$ and p50c scores $65.6 \pm 11.8\%$ ($+46.9 \pm 11.6$ percentage points). Answers cut at the 32,768-token cap fall from 27 of 32 (all ending in a verbatim loop) to 3 of 32 (0 in a loop).
• Rebuilding the edit from the agent's 8 texts recovers all 64 neurons in both layers (labs/lab3_rebuild.md). Teacher-forcing the agent's 4 productive and 4 looping texts (feeding fixed text through the model and recording activations with hooks, small functions attached to layers) shows that the MLPs of blocks 15 and 19 widen the looping-minus-productive gap the most. The rebuilt edit recovers directions matching the released $u$ to cosine $+0.99998$ and $-0.99998$ (nearly the same line, with opposite orientation at the second layer), picks the same 64 neurons in both layers, and scores $61.5 \pm 2.8\%$ on dev256 (matching p50c's $61.5 \pm 2.8\%$). Using only the first 2 text pairs still reaches $62.3 \pm 2.7\%$.
• Accuracy rises with $\alpha$ up to a plateau from 4 to 8, while random controls move accuracy by about +2 points (labs/lab4_dose.md, labs/lab7_side_effects.md). On dev256, accuracy goes from $12.1\%$ ($\alpha = 0$, unedited) to $22.7\%$ ($\alpha = 1$, erase), $35.2\%$ ($\alpha = 2$, mirror), $59.2\%$ ($\alpha = 4$), $61.5\%$ ($\alpha = 6$), and $60.7\%$ ($\alpha = 8$). Accuracy then drops to $51.8\%$ at $\alpha = 12$, and WikiText-2 perplexity (prediction uncertainty on ordinary text, lower is better) rises by $+14.34\%$, versus $+0.79\%$ at $\alpha = 6$. Matched-norm controls with a random orthogonal direction (randdir) or 64 random neurons (randcols) score $14.5\%$ ($+2.3 \pm 1.6$ points) and $14.6\%$ ($+2.5 \pm 1.6$ points).
• Setting $\alpha = -4$ gains +7.6 points for the wrong reason (labs/lab4_dose.md). Amplifying the 64 neurons' push toward the loop direction 5 times raises dev256 accuracy from $12.1\%$ to $19.7 \pm 1.9\%$ ($+7.6 \pm 2.2$ points) because 197 of 512 answers abruptly emit </think> out of a loop. Yet it loses 24 problems the unedited model solved, drops accuracy among finished answers from $100.0\%$ to $55.4\%$, and raises WikiText-2 perplexity by $+5.39\%$.
• Turning thinking off reaches 52.1% using 13 times fewer tokens, and the weight edit adds nothing there (labs/lab5_decoding.md). With thinking on, setting temperature to 1.0 ($27.9\%$) or repetition penalty to 1.1 ($34.4\%$) falls well short of the weight edit ($61.5\%$), and dropping the 1.5 presence penalty cuts accuracy to $4.5\%$, with $99.0\%$ of answers cut. Without thinking, the base model reaches $52.1 \pm 2.7\%$ on dev256 with 2,270 tokens per answer on average, versus 29,683 with thinking on (16,104 with the edit). Adding p50c with thinking off scores $53.5 \pm 2.8\%$ ($+1.4 \pm 1.9$ points), because answers rarely loop without thinking.
• The static weight edit is algebraically equal to token-by-token activation steering (labs/lab6_steering.md, labs/lab0_toy.md). Multiplying the edited matrix by neuron activations $a(t)$ gives $W' a(t) = W a(t) - \alpha s(t) u$, where $s(t) = u \cdot (W_S a_S(t))$ is one number per token measuring how hard the 64 chosen neurons push along $u$. In 64-bit floats on one layer, the matrix edit and a runtime steering hook match to $3.33 \times 10^{-16}$. On 48 real looping answers from the base model, $s(t)$ stays low early in the answer (0.096 at layer 15, 0.116 at layer 19) and jumps inside the loop (0.424 and 0.758), so the edit pushes back hardest inside loops.
• An edit built from the model's own loops reaches 58.2% without hand-written texts (labs/lab8_capstone.md). On 225 looping answers the unedited model wrote for 128 separate practice problems (build128), comparing loop tokens with tokens well before the loop (recipe B) matches $u$ to cosine 0.782 and 0.773 and scores $58.2 \pm 2.9\%$ on dev256 ($+46.1 \pm 2.6$ points). Comparing with the 32 tokens just before the loop starts (recipe A) reaches only $43.0 \pm 2.6\%$ because pre-loop text has already drifted along $u$.

Start here#

Start with QUICKSTART.md for a one-page, 2-hour path through the labs with exact commands and expected numbers.

The Quickstart also shows how to download the code from this site. Lab 0 runs on any laptop. The other labs need NVIDIA GPUs; we used two machines with 8 A100 80 GB GPUs each, and most steps use 1 or 2 of them.

The nine labs#

Project layout#

• configs/: JSON definitions of edited models (variants.json for Lab 4 controls and $\alpha$ sweep, rebuilt.json from Lab 3, capstone.json from Lab 8).
• data/: Prepared JSONL datasets (dev256.jsonl with 256 unseen practice problems, build128.jsonl with 128 practice problems for Lab 8, plus math500.jsonl, gsm8k.jsonl, and ifeval.jsonl).
• external/: Local checkout of the paper's log repository (lm_understand-logs, ignored by .gitignore; exp8_math/ holds the agent's scripts, edit manifest, and work notes).
• labs/: Lab scripts (lab0_toy.py through lab8_capstone.py), answer inspection tools (peek.py, lab4_endings.py), lab sheets (lab0_toy.md through lab8_capstone.md), and the agent's 8 contrastive texts (labs/data/astra_pairs.json).
• plans/: Generation job plans for scripts/run_jobs.py (quick.json, trainer.json, sampler.json, extra.json, and lab3-tries.json).
• reference/: The first-principles companion guide reference/theory-guide.md.
• results/: Saved JSON outputs for Labs 1 through 8 (including committed results/*/try-* folders to compare your runs with ours), per-run graded records in results/runs/, the master table results/summary.md (results/summary.json), and all plots in results/figures/*.png.
• scripts/: VM setup and sync scripts (setup_vm.sh, sync_vm.sh, pull_runs.sh, env.sh), dataset and model builders (prepare_data.py, build_models.py), the multi-GPU vLLM runner (run_jobs.py, gen_worker.py), and reporting scripts (summarize.py, plot_runs.py).
• wsp/: Shared Python library (hf.py for model loading and PyTorch hooks, edits.py for rank-one edits and SVD recovery, loops.py for verbatim loop detection, protocol.py for presets and sampler probabilities, grading.py, and summary.py).

Running things#

Cloning the paper's logs. .gitignore ignores external/, so the paper's log repository is not committed here. If external/lm_understand-logs is missing, clone it from the project root before syncing or building models:

git clone https://github.com/RikiyaT/lm_understand-logs external/lm_understand-logs
git -C external/lm_understand-logs log -1

We used commit 4d09ba6d (29 October 2026). scripts/prepare_data.py and scripts/build_models.py read from external/lm_understand-logs/exp8_math, and scripts/sync_vm.sh copies the parts the GPU machines need (code, practice, and runs/p50c).

The two GPU VMs and environment setup. GPU work runs on trainer and sampler, each with 8 NVIDIA A100 80 GB GPUs. All VM files live under /nvme/wsp-lab (venv/, models/, data/, runs/, logs/, and code/). Start every VM command by sourcing env.sh, which activates Python 3.12 (vLLM 0.33.0, PyTorch 2.14.0, transformers 5.20.2, flash-linear-attention 0.6.0) and keeps caches inside /nvme/wsp-lab:

. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code

Syncing code and pulling results. Files move between each VM and this machine. From the project root on this machine:

bash scripts/sync_vm.sh trainer sampler    # push wsp, scripts, configs, plans, labs, and exp8_math files
bash scripts/pull_runs.sh trainer sampler  # pull merged.jsonl, summary.json, and *.done.json into results/runs/
python3 scripts/summarize.py results/runs          # rebuild results/summary.md and results/summary.json
uv run --with matplotlib python scripts/plot_runs.py

sync_vm.sh mirrors the five code folders onto the VM with rsync --delete, so any file you create or edit only on the VM is deleted or overwritten on the next sync. Edit on this machine first, then sync. The one exception is configs/capstone.json, which Lab 8 writes on the VM: syncing never deletes it, but a local copy will overwrite the VM's copy, so pull the VM version first. pull_runs.sh copies only run records. Lab scripts write their outputs to results/labN/ and results/figures/ on the VM where they ran; copy those back by name:

rsync -az sampler:/nvme/wsp-lab/code/results/lab6/ results/lab6/ </dev/null
rsync -az sampler:/nvme/wsp-lab/code/results/figures/lab6_real_loop.png results/figures/ </dev/null

Copy figures individually instead of syncing the whole folder, since each VM holds only the figures it generated and an older copy would overwrite a newer one.

Setting up a new machine. To provision a fresh GPU machine (about 10 minutes and 15 GB under /nvme/wsp-lab, no root needed), run these commands from the project root (see scripts/setup_vm.sh):

scp scripts/setup_vm.sh <host>:/tmp/ && ssh <host> bash /tmp/setup_vm.sh
bash scripts/sync_vm.sh <host>
ssh <host> 'cd /nvme/wsp-lab/code && . scripts/env.sh &&
  python scripts/prepare_data.py --exp8 external/lm_understand-logs/exp8_math --out /nvme/wsp-lab/data &&
  python scripts/build_models.py && python scripts/build_models.py --variants configs/rebuilt.json'

Where full answer texts live. Full answer texts (runs/<name>/shard*.texts.jsonl.gz) are ignored by .gitignore and skipped by scripts/pull_runs.sh, so they stay on the VM that generated them (while results/*/try-* folders are committed to git). Because labs/peek.py and labs/lab4_endings.py read .texts.jsonl.gz files, run them on the VM that produced the run:

• On trainer: bench/*, quick/*, dev/rebuilt-pairs2, dev/rebuilt-windows36-36, and dev/rebuilt-layers15-19-21.
• On sampler: build/base and every other dev/* run (dev/base, dev/p50c, dev/rebuilt, the Lab 4 dose and control runs, the Lab 5 decoding runs, and the Lab 8 dev/cap* runs).

Rerunning a finished job. scripts/run_jobs.py skips any shard (a batch of problems) that has a shard*.done.json file, so rerunning a finished plan reports 0 shards to run. To run a finished job again, move its output folder aside on the VM first:

mv /nvme/wsp-lab/runs/quick /nvme/wsp-lab/runs/quick-ours

Then run the plan; python labs/peek.py quick/base reads the new run and python labs/peek.py quick-ours/base reads the earlier one.

How to read results/summary.md. Every generation run has one row in results/summary.md with these nine columns:

• run: Run identifier (bench/<model>/<dataset>, dev/<variant>, build/base, or quick/<model>).
• problems x samples: Distinct problems times sampled answers per problem (for example 256x2 = 512 answers on dev256).
• accuracy %: Percentage of correct answers, plus or minus one standard error (SE, typical sampling uncertainty; a gap of 2 SE or more is unlikely to be chance).
• cut at the cap %: Percentage of answers that hit the token limit (32,768 by default) without finishing.
• mean tokens: Average generated tokens per answer.
• correct when finished %: Accuracy among answers that finished before the token limit (97.5% to 100% for the unedited model on math benchmarks, showing its main failure is not finishing).
• cut answers ending in repetition %: Percentage of cap-cut answers ending in repetition. An answer counts if it ends in a word-for-word loop (a block of words repeated at least 3 times in a row) or in looser circling. Looser circling means at least half of the 4-word phrase positions in the last 200 words hold a phrase repeated 3 or more times there (see results/summary.md).
• gain vs baseline (pp): Accuracy change in percentage points versus the paired baseline on the same problems, plus or minus paired SE (dev/* against dev/base, dev/p50c+nothink against dev/base+nothink, bench/p50c/X against bench/base/X, and quick/* against quick/base).
• rescued / lost problems: rescued counts problems where the run got at least one answer right and the baseline got none; lost counts the reverse.

Sources#

• Blog post: Rikiya Takahashi, "Sparks of Weight-Space Programming" (MIT CSAIL, 2 Nov 2026).
• Experiment logs and code: https://github.com/RikiyaT/lm_understand-logs (local copy in external/lm_understand-logs/, with Experiment 8 in external/lm_understand-logs/exp8_math/).
• Released edited model: https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming.
• Interactive demo Space: https://huggingface.co/spaces/ML-Intern-lab/weight-space-programming-demo.
• First-principles theory guide: reference/theory-guide.md.