Skip to content

Reduce complete research-draft creation time by 14.55% (no new κ) - #6199

Merged
Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
petalas:research/w8-1029b-original-result
Oct 30, 2026
Merged

Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
petalas:research/w8-1029b-original-result

Conversation

@petalas

@petalas petalas commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Research-draft creation reads its pinned catalogue twice. Read it once through the existing source stream, use those bytes for card selection and the source pin, then sort the inventory before rendering the same five files.

 create
-  git show: select the catalogue card
   stream pinned source blobs
+  select the card from the catalogue blob in that stream
   check live bytes and pins, sort metadata, publish five draft files

Claim type: empirical tooling improvement with a scoped source-level output-equivalence argument. No new kappa, supplier admission or selected-frontier change.

Evidence

  • Before: 67.232 ms median for complete default draft creation, range 65.786 to 73.393 ms.
  • After: 57.447 ms, range 56.164 to 61.209 ms, a 14.55% reduction. Eight explicit sources improve by 13.49%, from 68.880 to 59.587 ms.
  • Each workload has 31 alternating pairs after a warm-up pair. Every candidate wins its pair. All 128 commands succeed with identical five-file bytes and modes within each workload. Timing includes fresh Python startup, Git reads and all file writes. Comparison and moving the output to scratch occur outside the timer.
  • The untimed profile attributes about the saved 9.8 ms to the removed Git launch. Startup/imports and the remaining Git work limit the gain. Raw samples, output hashes and source hashes are in research/draft-catalogue-single-read/MEASUREMENTS.json.
  • Local warning-strict onboarding suite: 710 tests pass, 30 platform-specific skips. Normal and optimized registered checks pass in 1.062 s and 1.147 s. The package directly compares 15 generated files across three further cases and runs nine guard tests. Assertions are not required by the package verifier.
  • Committed bytes, owner-execute modes and full source ancestry pass for all 12 package/input files at a6bee4472b4df4ff17510dba2b2bb06e5af0f029. Official export integrity and normal/optimized read-only Git-free source-export replay pass; all 13 exported files retain their hashes and modes. Git is required to construct the synthetic test fixtures.

Exact local reproduction, from the repository root:

/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -W error -B research/draft-catalogue-single-read/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/draft-catalogue-single-read
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/draft-catalogue-single-read --check-source-history --check-modes
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -W error -B research/draft-catalogue-single-read/benchmark.py --scratch /absolute/new-scratch/draft-benchmark --samples 31

The last command requires a new scratch directory on the checkout's filesystem. The first three check the artifact; they do not remeasure historical timings. CI uses Python 3.12 through {python}.

Limits and credit

Measured on one macOS machine with CPython 3.12.15 and Git 2.54.0. Windows and older-Git performance are unmeasured. Catalogue validation now precedes validation of the other live inputs, so the first diagnostic can change when several inputs are invalid. Stable-checkout, source-authentication, output-alias, overwrite, early-close and final-reader-failure contracts remain explicit. The source-equivalence argument is informal; independent human and formal proof-kernel review are unclaimed.

No selected mathematical, native, caller, compiler, precision, BIT, outer-ledger, Lean or Docker replay was run. The five selected framework assumptions remain.

This contribution was produced by Nick Petalas. Credit Douglas Colkitt for the generator/framework, ywz65's #2539 for batched draft pins, Ali Romman's #5734 for output guards, and the retained immutable-object, exact-path, Git-root and process-ownership contributors. The exact baseline copy preserves its original notice because the benchmark executes it.

Base: canonical main a5bff3bf133e482541c09eba4e773884bdb2b1b7, priority revision 8, refreshed unchanged before commitment. Advance claim. This is an original replay-runtime refinement. Live claims, all open PRs in the overlap snapshot, targeted merged PRs, whole-main README terms and recent account work were checked. Sparse local notes alone were incomplete; no global novelty guarantee is claimed.

Related open work

The completion refresh checked all fifteen other account PRs opened since the advance CLAIM: #6192 through #6198 and #6200 through #6207. None studies this draft generator or computes the same timing quantity on its workload. No same-object numerical disagreement was found, and no extra predecessor computations were run.

Open next questions

Does the gain persist for an actual source-heavy package with hundreds of pins? Start with one named retained inventory and the same full-command comparison after materializing its inputs. This needs a separate measurement budget and overlap check; no follow-up is reserved.

CI notes

Run 40911920758 finished at head a6bee4472b4df4ff17510dba2b2bb06e5af0f029. plan and the four replays for draft-catalogue-single-read, publication-metadata-reuse, replay-input-head-spelling and source-comparison-memory passed. Two downstream replays failed before their first command ran:

  • replay-input-symlink-alias, job 123584224189: working source drift: scripts/new_research_package.py. Its live pin expects SHA256 7d300bf45baffced49d2bbea795ce3abe5f860f0dc1172845fc352b398331f34. The unchanged PR parent/canonical main a5bff3bf133e482541c09eba4e773884bdb2b1b7 already has a9a66cb591239c27928047146040cb9adb8d138fdc327f2fc8cb873ef9892c27, after Reject linked output paths before creating research drafts #5734's output-path guard commit 9a49794b08658da52f83b3a3b214366136808740. Its parent has the pinned hash. This PR changes the generator again, to 567b5300992b03aa685854a902445758ad8ee00f24ac1289f5ae711c7fb28542, and therefore triggers the replay; it does not convert a passing parent replay into a failing one. The package also has an already-drifted live input pin for tests/test_research_input_paths.py on main.
  • research-package-pin-batching, job 123584223957: working source drift: tests/test_research_onboarding.py. Its live pin expects 07ae0ac60d417cfe31eae528cc71aa7d64993171b339fb59ecf378b0281687a3; both the unchanged parent/main and this PR have 5e3993fd7192f900d146df5db1a04d2793377a4c935c60bfcf1a48fcb4b56b9b. This PR's diff does not touch that file or either downstream package. The package's generator pin is already drifted on main as well.

Both complete package commands were run read-only in the clean reference checkout at canonical main a5bff3bf133e482541c09eba4e773884bdb2b1b7, using /Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package <package>. Both exit 1 immediately with the exact errors above. The relevant reference-checkout files match origin/main. Three sampled recent failed workflow runs, 40728523354, 40893686492 and 40726915850, did not select either named replay; no matching cross-PR replay failure is claimed from that sample. The direct clean-main reproduction establishes that both failures predate this PR.

No frozen package, source receipt, pinned byte or credit was changed. Restoring only this PR's generator edit would still leave both replays failing on their inherited pins. Making those historical jobs green requires separate frozen-input compatibility work; the repository's docs/research/task-startup-compatibility.md distinguishes already-drifted live pins from newly broken passing packages and forbids silently rewriting source receipts. This PR retains its original scope and checks; no source guard was weakened or evidence repinned.

No follow-up commit or push was made after this diagnosis. The reviewed head remains a6bee4472b4df4ff17510dba2b2bb06e5af0f029. The inherited failures are recorded for maintainer handling and are not contribution blockers. Completion records the exact result: plan and the contribution's own replay passed, the three other named replays passed, and the two inherited source-pin replays failed reproducibly on main. The overall workflow remains red; CI is not described as fully green.

Merge Danger

Door: two-way. Blast radius: draft generation. The change is reversible; all source and output guards remain. The documented diagnostic ordering change is the observable compatibility limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants