Ranking feasibility boundary under Gaussian weight perturbation
Rankings from pythia-70m-deduped step142000 (100-sample Pile set, first rankings), checked against the unembedding W + εG, Gij ∼ 𝒩(0, 1) (one fixed draw); x-axis is the RMSE of the perturbed weights (≈ ε). Boundary = first infeasible top-k (top-k order + top>bottom constraints, MOSEK LP); k = v = 50,304 means the full ranking is feasible. Weight std is 0.136.
| ε | RMSE | n | median | range | frac full |
|---|---|---|---|---|---|
| 1e-12 | 1e-12 | 5 | 50,304 | 16,384–50,304 | 0.60 |
| 1e-11 | 1e-11 | 5 | 50,304 | 50,230–50,304 | 0.80 |
| 3e-11 | 3e-11 | 5 | 50,304 | 28,672–50,304 | 0.80 |
| 1e-10 | 1e-10 | 5 | 50,304 | 50,304–50,304 | 1.00 |
| 3e-10 | 3e-10 | 5 | 50,304 | 50,239–50,304 | 0.80 |
| 1e-09 | 1e-09 | 5 | 50,304 | 50,026–50,304 | 0.80 |
| 3e-09 | 3e-09 | 5 | 40,919 | 15,536–50,304 | 0.20 |
| 4e-09 | 4e-09 | 5 | 32,512 | 16,384–38,518 | 0.00 |
| 5e-09 | 5e-09 | 4 | 24,432 | 12,286–32,942 | 0.00 |
| 7e-09 | 7e-09 | 4 | 24,027 | 23,392–25,252 | 0.00 |
| 1e-08 | 1e-08 | 5 | 18,896 | 17,193–19,969 | 0.00 |
| 3e-08 | 3e-08 | 5 | 9,328 | 8,104–10,442 | 0.00 |
| 1e-07 | 1e-07 | 5 | 5,248 | 4,784–5,607 | 0.00 |
| 3e-07 | 3e-07 | 5 | 3,099 | 2,844–3,297 | 0.00 |
| 1e-06 | 1e-06 | 5 | 1,765 | 1,705–1,982 | 0.00 |
| 3e-06 | 3e-06 | 5 | 1,189 | 1,173–1,219 | 0.00 |
| 1e-05 | 1e-05 | 5 | 835 | 821–868 | 0.00 |
| 3e-05 | 3e-05 | 5 | 669 | 636–675 | 0.00 |
| 0.0001 | 0.0001 | 5 | 549 | 543–574 | 0.00 |
| 0.0003 | 0.0003 | 5 | 472 | 467–484 | 0.00 |
| 0.001 | 0.001 | 5 | 398 | 384–404 | 0.00 |
| 0.003 | 0.003 | 4 | 366 | 342–373 | 0.00 |
| 0.01 | 0.01 | 5 | 337 | 279–342 | 0.00 |
| 0.03 | 0.03 | 5 | 232 | 198–255 | 0.00 |
| 0.1 | 0.1 | 4 | 116 | 111–123 | 0.00 |
Data (CSV): summary, per ranking. Sweep still in progress. Updated 2026-11-05 09:01.