Lab 1: Find the edit by diffing the two weight files

Big thing: You will compare all 488 weight tensors (873,438,784 numbers) of base Qwen3.5-0.8B and the paper's released model bit for bit. From the weights alone, you recover the full edit: layers = [15, 19], 64 columns per layer, rank-one direction u, and strength alpha = 6.

Why it matters: When you have the weight files, you do not have to take a paper's description on faith. Tensor diffs and an SVD turn a 0.87-billion-parameter checkpoint diff into one short formula.

Time and hardware: Under a minute on CPU on sampler or trainer (no GPU needed).

Theory guide: Section 9, step 4.

Before you run: predict#

Write a short guess for each question before running the script.

1. Out of the 488 weight tensors in Qwen3.5-0.8B, how many differ between the base model and the released model qwen35-p50c?
2. Each edited down_proj.weight matrix has shape 1,024 x 3,584, and the edit touches 64 columns (64 x 1,024 = 65,536 numbers). Will the file have 65,536 changed entries or slightly fewer, and why?
3. In Lab 0, the second singular value of W' - W was 0.0000. For the real weight change, will the second singular value be exactly zero or slightly above zero?

Run it#

Connect to one of the lab VMs (sampler or trainer), activate the lab environment, and run labs/lab1_diff.py on CPU. This script does not touch a GPU.

ssh sampler
. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code
python labs/lab1_diff.py

The script compares every tensor in /nvme/wsp-lab/models/Qwen3.5-0.8B against /nvme/wsp-lab/models/qwen35-p50c, writes results/lab1/diff.json, and saves results/figures/lab1_diff.png.

What you should see#

2 of 488 tensors differ (873M numbers in all):
   model.language_model.layers.15.mlp.down_proj.weight  shape [1024, 3584]
      entries changed: 65,260 of 3,670,016, all inside 64 columns (neurons)
      singular values of the change: [1.7489, 0.001, 0.0009, 0.0009] ...  rank-one share 0.999994
      fitted alpha = 6.0003; the matrix moved by 10.6%
   model.language_model.layers.19.mlp.down_proj.weight  shape [1024, 3584]
      entries changed: 64,928 of 3,670,016, all inside 64 columns (neurons)
      singular values of the change: [1.3988, 0.0009, 0.0008, 0.0007] ...  rank-one share 0.999991
      fitted alpha = 6.0002; the matrix moved by 8.1%
|cos| between the two directions: 0.849 (1 = same direction, 0 = unrelated; the SVD sign is arbitrary, Lab 3 fixes it)
wrote /nvme/wsp-lab/code/results/lab1

results/lab1/diff.json and results/edit_info.json record the exact numbers:

In lab1_diff.png, the top row plots the Euclidean length (norm) of the change in each of the 3,584 columns for layers 15 and 19. You see 64 red spikes rising from a floor of exact zeros; in layer 19, the tallest spike near the left edge is neuron 11. The bottom row plots the top 10 singular values on a logarithmic scale. Bar 1 sits above 10^0 (1.7489 and 1.3988), while bars 2 through 10 sit more than three orders of magnitude lower around 10^-3.

Why it works#

1. The detective pipeline: from two files to four numbers#

The script inspects the two model folders in four steps, using no prior knowledge of how the paper built its edit.

First, safe_open streams all 488 tensors in model.safetensors-00001-of-00001.safetensors and checks torch.equal(tb, te). Exactly 486 are bit-for-bit identical; only model.language_model.layers.15.mlp.down_proj.weight and model.language_model.layers.19.mlp.down_proj.weight differ.

Second, the script subtracts the base weights from the edited weights in 32-bit floating point (fp32): D = W1.float() - W0.float(). Checking which columns have a non-zero sum of absolute differences (D.abs().sum(0) > 0) shows that every changed entry falls inside 64 columns in layer 15 and 64 columns in layer 19, leaving the other 3,520 columns untouched. Column 11 in layer 19's list [1, 5, 8, 11, ...] is the single neuron the agent zeroed out early in its search (+6.25 percentage points on practice problems) before moving to a 64-neuron edit.

2. Why 65,260 and 64,928 entries changed instead of 65,536#

64 columns times 1,024 rows is 65,536 entries per layer, yet only 65,260 entries change in layer 15 (276 unchanged) and 64,928 in layer 19 (608 unchanged).

The reason is number precision. The weights are stored on disk in bf16 (bfloat16, a 16-bit floating-point format with 2 to 3 significant decimal digits). Where u or u . W0[:, j] is near zero, the update -alpha * u[i] * (u . W0[:, j]) is smaller than half a bf16 step around W0[i, j]. Casting the edited tensor from fp32 back to bf16 rounds those entries back to their original bits.

3. Taking the change apart with SVD#

To check whether all 64 columns moved along one direction u or 64 separate directions, recover(W0, W1) in wsp/edits.py takes the 1,024 x 64 slice D[:, cols] and computes its Singular Value Decomposition (SVD):

SVD writes a matrix as a sum of rank-one pieces. The singular values $s_1 \ge s_2 \ge \dots \ge s_{64} \ge 0$ give the size of each piece, and the columns $u_k$ of $U$ are mutually perpendicular unit vectors.

• In layer 15, $s_1 = 1.7489$, while $s_2 = 0.0010$, $s_3 = 0.0009$, and $s_4 = 0.0009$. The first piece holds 0.999994 (99.9994%) of the squared Frobenius norm ($s_1^2 / \sum_k s_k^2$).
• In layer 19, $s_1 = 1.3988$, while $s_2 = 0.0009$, $s_3 = 0.0008$, and $s_4 = 0.0007$. The rank-one share is 0.999991 (99.9991%).

The change is a rank-one outer product; the tail of singular values near 0.001 is consistent with bf16 rounding noise. The top left singular vector u = U[:, 0] is the edit direction u up to a sign.

4. Recovering alpha and checking the bits#

With u and cols known from D, the unrounded change on the chosen columns is $-\alpha P$, where $P = u (u^T W_0[:, \text{cols}])$ is the projection of the base columns onto u. Here $P$ depends only on the line through u, since $(-u)(-u)^T = u u^T$. A least-squares fit in recover finds the scalar alpha that best matches D[:, cols] to -alpha * P:

This gives 6.0003 for layer 15 and 6.0002 for layer 19, matching the paper's alpha = 6 to four digits.

In scripts/build_models.py (logged in results/edit_info.json), we plug the recovered u, cols, and alpha into projection_edit on W0 and compare the resulting bf16 tensor against W1. Out of 3,670,016 entries per layer, all but 5,875 in layer 15 (5,924 at alpha = 6.0) and 6,158 in layer 19 (6,166 at alpha = 6.0) match bit for bit. The largest gap anywhere is 0.00048828125 (about 0.00049), one bf16 rounding step.

Comparing the unit vectors $u_{15}$ and $u_{19}$ gives $|u_{15} \cdot u_{19}| = 0.849$. An absolute cosine of 1.0 means two vectors lie on the same line, pointing the same way or opposite ways, and 0.0 means they are orthogonal (unrelated). A value of 0.849 is an angle of about 32 degrees: the loop directions at blocks 15 and 19 are closely related in the 1,024-dimensional stream, but distinct.

Check yourself#

1. Why does labs/lab1_diff.py report the absolute value |cos| = 0.849 between u1 and u2 instead of a signed cosine?

2. What causes the second singular value of W1 - W0 to be around 0.0010 in layer 15 instead of 0?

3. The edit flips each edited column's part along u and multiplies it by 5. Yet ||W1 - W0|| / ||W0|| is only 10.6% in layer 15 and 8.1% in layer 19. Is the edit really a small change?

Try this next#

1. Inspect how many neurons layers 15 and 19 share. Run a one-line check on results/lab1/diff.json from the VM shell:

   bash

   python -c 'import json; d=json.load(open("results/lab1/diff.json"))["changed"]; c15,c19=[set(v["cols"]) for v in d.values()]; print("shared column indices:", sorted(c15 & c19))'
   

   It prints [11, 34, 63]. Neuron numbers do not link layers. You could renumber one layer's neurons, moving each neuron's row of gate_proj and up_proj and its column of down_proj together, and the model would compute exactly the same thing. So neuron 11 in layer 15 has no built-in tie to neuron 11 in layer 19. Two random 64-neuron subsets of 3,584 neurons would share 64 * 64 / 3,584 = 1.1 indices on average, and would share 3 or more about 10% of the time.

   The picks are not random, though. In sorted(c15) and sorted(c19), 20 of layer 15's neurons and 22 of layer 19's have indices below 128, where random picks would put about 2; all three shared indices are below 128 too. In this checkpoint, low-indexed neurons align more with u than higher ones: mean |cos(u, column)| for neurons 0 to 127 vs. 2,560 and up is 0.036 vs. 0.021 in layer 15, and 0.051 vs. 0.022 in layer 19. Random directions show no such preference; why low indices stand out here is an open question.

2. Diff another model variant from /nvme/wsp-lab/models. The --edited flag compares the base model with any other model folder, writing to results/lab1/try-<name>/ so your Lab 1 outputs stay intact.

   bash

   python labs/lab1_diff.py --edited qwen35-all2   # about a minute and a half
   python labs/lab1_diff.py --edited qwen35-a1     # a few seconds
   

   qwen35-all2 rebuilds an earlier edit from the agent's run (strength 2 on every column along the released u). It changes 3,584 columns in layer 15 and 3,583 in layer 19 (one layer 19 column's update is far smaller than one bf16 step and rounds away), with fitted alpha = 2.0000 and 1.9999. The rank-one share dips to about 0.9995 because 3.2 million changed entries each carry rounding noise. The matrices move by only 7.1% and 6.7%, less than the released edit despite touching 56 times as many columns. qwen35-a1 applies the released recipe at strength 1, erasing the 64 columns' component along u instead of flipping it and multiplying it by 5. It touches the same 64 columns with alpha = 1.0002 and 0.9996, moving the matrices by 1.8% and 1.4% (about one sixth of the released edit's 10.6% and 8.1%). With --edited qwen35-l15only, the script reports 1 of 488 tensors differ.

3. Check the round-trip bf16 error in results/edit_info.json. Run:

   bash

   python -c 'import json; d=json.load(open("results/edit_info.json"))["layers"]; print({k: {m: v[m] for m in v if "rebuild" in m} for k, v in d.items()})'
   

   Compare rebuild_alpha_6.0003_entries_differing (5,875 in layer 15) and rebuild_alpha_6.0000_entries_differing (5,924 in layer 15), and verify that max_abs_diff is 0.00048828125 in every case.