Skip to content

fix: Chapter 4's demos report the survivors, and five more honesty defects - #127

Merged
qmohsu merged 1 commit into
mainfrom
claude/ch4-demos-tell-the-truth
Aug 31, 2026
Merged

qmohsu merged 1 commit into
mainfrom
claude/ch4-demos-tell-the-truth

Conversation

@qmohsu

@qmohsu qmohsu commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Chapter 4's core math was verified clean by the ION GNSS+ pre-conference audit
(information equivalence 2.2e-16 at all 400 points, CRLB attained, image-source
re-derived). Every defect below is a demo, figure or README honesty defect, and
core/ is untouched.

The through-line: converged is a step-tolerance stop

In these solvers info["converged"] is set by norm(delta) < tol at
tol = 1e-6 m. The residual test above it cannot fire at all on noisy data —
four ranges over two unknowns leave a residual no solve can drive to zero. So
the honest reading of converged=False is "needed more than max_iters
steps"
, not "failed". Four demos filtered their samples on it.

Site Reported Honest Why
example_toa_positioning Ex. 6 WLS improvement 23.1% 53.4% (theory 50.1%) LS scored over 120/200 draws, WLS over 200/200. The 80 dropped were the worse half: RMSE 0.416 m vs 0.181 m for survivors, 0.298 m over all 200.
example_comparison RSS series RMSE over converged (13/50 at 6 dB) median over all 50 The console table beside it already reported a median over everything, so figure and table disagreed about the same run.
example_tdoa_positioning Demo 9 RW-LS 0.3163 m over 459 draws 0.3384 m over 500 Three sibling columns were already over 500.
Demo 4 both arms filtered 200/200 either way No effect — worth recording, because the shape is not always live.

Raising max_iters to 50 gives 200/200 (Ex. 6) and 500/500 (Demo 9) with the
answers moving by 1e-4 m: the extra iterations were polishing, not rescuing.
example_comparison now runs all four methods through core.rf.solve_batch,
so the medians are over every fix and the failures live in their own panel on
solve_batch's four conditions.

Demos contradicting their own numbers

  • TDOA Demo 2 drew independent noise per difference, deleting the shared
    reference error. err/noise ran 0.56–0.59; with e_j − e_0 it runs 0.80–0.88.
    Its "ratio constant" self-check passes either way — a statistic that tests
    only proportionality cannot see a wrong variance.
  • TDOA Demo 6 seeded both solves at the truth and printed one draw to four
    decimals. From a neutral seed the collinear array's median over 500 is
    1.2687 m, not 0.9724. Seeded on the beacon line all 500 stall at
    5.3852 m — the seed-to-truth distance, identical across draws. That case
    is now a row of its own rather than a number that looks like an error.
  • AOA Demo 7 printed "aligned anchors … cause high condition number and
    large PLE errors"
    above four rows in which the near-collinear array had the
    smallest error, the square the lowest condition number, and
    geometry_warning read "no" throughout. Two defects: the two columns drew
    independent noise, at one draw each. On 500 shared draws all four layouts
    land within 1.37×. The flag was right — it thresholds the bearing spread
    at the agent
    , which is 19.5° from (5, 7) however collinear the anchors are.
    A second table walks the agent onto the line, where the spread does collapse
    and the cost scales with range: −22.3 m of bias along the shared direction
    and 4.6 m of scatter at 30 m out, against 0.13 m across it.

Figure

ch4_rf_comparison plotted AOA's [0,1,3,5,10] degrees and RSS's
[0,2,4,6,8] dB at TOA's metre positions under an axis reading
"Measurement Noise (m)". x is a level index now, each series carries its own
schedule in the legend, and the CDF and box-plot titles name all four methods'
noise at that level. Panel 1 is labelled median (it was an unlabelled RMSE);
panel 4 is "Fixes solved".

--data defaulted to the same figure basename as the inline run, so it
overwrote the committed comparison with a different picture. It writes
ch4_rf_comparison_dataset now.

ch4_initial_guess_basin no longer reproduced from HEAD — its traced seed
had moved. Regenerated; the classification counts are unchanged
(785/341/896/82/181/522/1340) and only panel 4's trace differs. Proven
idempotent: two further full regenerations leave every byte identical.

READMEs

  • Chapter: basin trace (−4.5,−2.0) → 9.4e11 m in 18 iters → (12,12) → 4.43e12 m in 14; 305/196 → 308/182; PLE's "sensitive to aligned beacons"
    rewritten to what Demo 7 measures.
  • ch4_rf_2d_square: the stale "TDOA ~0.07 m … most accurate … GDOP lowest"
    sentence, which the same file disowns 240 lines earlier; the geometry and
    TOA-noise tables rebuilt from the shipped data (the old ones put the optimal
    array at 0.8–0.9 GDOP, below the theoretical floor of 1 for four beacons, and
    named a Clustered geometry --geometry has no choice for); "twelve
    times" → eleven (11.95/1.07); 1.75 → 1.76 (10·tan 10°); "20-50%" → the
    measured per-method 10% / −31% / 31%.
  • ch4_rf_2d_linear: 111 → 113, which its own two other lines already said.
  • ch4_rf_2d_nlos: "much larger than 0.818 m" → the measured 0.635 m,
    which is smaller, plus why (only 2 of 4 beacons biased, so LS splits the
    difference) and the shape test that says it is bias — median 0.615, mean
    0.607, RMSE 0.635 all agree, where noise spreads them.
  • example_tdoa_positioning printed "always beneficial" above a −0.3% row.
    Qualified: that row's true gain is under 1% and a ratio of two RMSEs over 200
    draws carries ~5% standard error; over eight other seeds it runs +0.15% to
    +0.94%.

Minors

--compare-geometry's DOP column gained a unit column (AOA's entry is m/rad);
the covariance-matrix figure's half-integer ticks are now one per cell named
d1,0 … d4,0; closed_form_comparison's four boxes are all over 500 draws.

Test plan

  • tests/ch4_rf_point_positioning tests/core/rf — 243 passed
  • tests/docs (minus notebooks) — 866 passed, 1 skipped
  • tests/test_datasets_reproduce_from_their_recipe.py — 43 passed, no
    dataset byte moved
  • tests/test_every_figure_has_a_demo_behind_it.py — 200 passed, 8 skipped
  • test_repo_conventions / test_lint_debt_only_shrinks /
    test_black_is_actually_run / test_mypy_debt_only_shrinks /
    test_doctests_that_pass_keep_passing / test_python_floor_matches_syntax
    / test_repo_validators / test_semantic_api_debt_only_shrinks — green
  • Zero-noise rows still exact: TOA/TDOA/AOA all 0.000 at level 1 in
    example_comparison; TDOA Demo 2's 0.00 row unchanged; TOA Examples 1–5
    byte-identical
  • Every changed PNG opened and read

🤖 Generated with Claude Code

…fects

`converged` in these solvers is a step-tolerance stop -- `norm(delta) < tol`
at tol = 1e-6 m -- not a success flag. Four demos filtered their samples on
it, so each reported the accuracy of the draws that happened to stop fastest.

- example_toa_positioning Example 6 scored LS over 120/200 draws and WLS over
  200/200, then divided. The 80 dropped were the worse half (RMSE 0.416 m vs
  0.181 m for the survivors, 0.298 m over all 200), so "WLS improvement 23.1%"
  is really 53.4% -- against a BLUE prediction of 50.1%, now printed beside it.
  At max_iters=50 both arms finish on every draw and the answers move by 1e-4 m.
- example_comparison's four methods now go through core.rf.solve_batch. The RSS
  series was an RMSE over the converged fixes, 13 of 50 at 6 dB, next to a
  console table already reporting a median over everything. Panel 1 is the
  median over all 50 and panel 4 carries the failures on solve_batch's four
  conditions.
- example_tdoa_positioning Demo 9 put RW-LS's 459 draws beside three columns of
  500, understating 0.3384 m as 0.3163 m. All four are over 500 now.

Also, each a demo contradicting its own numbers:

- TDOA Demo 2 drew independent noise per difference, deleting the shared
  reference error. err/noise 0.56-0.59 becomes 0.80-0.88 with e_j - e_0.
- TDOA Demo 6 seeded both solves at the truth and reported one draw. From a
  neutral seed the collinear array is 1.2687 m median over 500, not 0.9724;
  seeded on the beacon line all 500 stall at 5.3852 m, which is now a row of
  its own rather than a number that looks like an error.
- AOA Demo 7 printed "aligned anchors cause large PLE errors" above four rows
  in which the near-collinear array had the smallest error. Both columns drew
  independent noise, one draw each. On 500 shared draws all four layouts land
  within 1.37x, because the bearing spread at the agent is 19.5 deg however
  collinear the anchors are. A second table walks the agent onto the line,
  where the spread does collapse and the cost scales with range.
- The comparison figure plotted AOA's degrees and RSS's dB at TOA's metre
  positions under "Measurement Noise (m)". x is a level index now, with each
  series' own schedule in the legend and every panel title naming all four.
- `--data` defaulted to the same figure basename as the inline run, so it
  overwrote the committed comparison with a different picture.

READMEs: the basin trio no longer reproduced from HEAD (its traced seed is
(12,12) -> 4.43e12 m in 14 iters, not (-4.5,-2.0) -> 9.4e11 in 18; 308/182 not
305/196); square's stale "TDOA ~0.07 m, most accurate" sentence, its unmeasured
geometry and noise tables, "twelve times" -> eleven, 1.75 -> 1.76, "20-50%" ->
the measured per-method 10% / -31% / 31%; linear's 111 -> 113; NLOS's "much
larger than 0.818 m" -> the measured 0.635 m, which is smaller and says why.

No dataset byte moves; tests/test_datasets_reproduce_from_their_recipe.py
passes. Figures regenerate idempotently.
@qmohsu
qmohsu merged commit adc6ec8 into main Aug 31, 2026
1 check passed
@qmohsu
qmohsu deleted the claude/ch4-demos-tell-the-truth branch August 31, 2026 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant