Repository navigation
feat(Probability): affine channels at every hierarchy stage and a concrete stage bound - #1704
Conversation
…transition matrices Pure linear algebra for the correction-and-copy hierarchy (cslib-community#1444 Phase 2, ideas 2 and 4 of cslib-community#474), following the OpenAI preprint (2026-09-23), Section 4: the factorization 1 - (C - E + EC) = (1 + E)(1 - C), the corrected inverse, copy splitting intertwined with its projection, and the reward transport identity with the stage error bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ly it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ward lower bound Review of cslib-community#1676: correction_mulVec_one_le only shows that row sums do not grow, so its docstring, the module summary and the test comment no longer call it substochasticity. Add rewardTransport_mulVec_nonneg, the lower half of (4.30), under nonnegative projections, corrections and reward. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second slice of cslib-community#1444 Phase 2 (idea 3 of cslib-community#474), following the OpenAI preprint (2026-09-23), Lemma 4.1 and Lemma 4.4: the cutoff correction recursion E = phi((C + EC)) on a strictly upper triangular C has a unique fixed point, its iterates increase towards it, and after k iterations the error is at most f e^(1/theta) (L/theta)^k / k!, independently of the size of the matrix. The corrected matrix of a finite iterate is nonnegative and small in fully corrected columns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review nit on cslib-community#1677: drop the private hockey-stick lemma. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of cslib-community#1677: the module docstring now bounds the chain count by m.choose k for the actual support size m, as the proof does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ommunity#1444) A point is retained when its hash lies in a target set B. Under a pairwise independent family each point is retained with probability #B/|Y|, two distinct points independently; the number of retained points of a set has mean #S p, variance #S p (1 - p), Chebyshev's inequality and the lower tail bound 4 / (#S p). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third slice of cslib-community#1679 (Phase 2 of cslib-community#1444, ideas 3 and 4 of cslib-community#474), following the OpenAI preprint (2026-09-23), Lemma 4.2, Lemma 4.4 and Proposition 4.5: row-sum, entry, domination and support bounds for cutoff corrections; the copy step with explicit split fractions, its density and active-copy bounds; incidence counting; and the one-stage rates that feed the stage bound, given an abstract detector interface. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…b-community#1682) Identifiers y : ZMod P are retained when A y + b lies in the box {v | all coordinates < L} of ZMod P ^ H. Exact rates (L/P)^H and their squares for pairs, the lower tail 4/(#A pi), independence of separate seeds, and the assembled channel properties at rate f = a^H with L = ceil(P a): f <= pi <= 2^H f. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add cutoffCorrectionIter_isStrictUpperTriangular, cutoffCorrection_residual_isStrictUpperTriangular and copyStage_isStrictUpperTriangular (for the lexicographic order on Lex (V × Fin (m + 1))), so a stage's output can feed the next stage. Correct two descriptions: the pilot inputs have bounded row sums, not necessarily at most one, and copyStage_mulVec_one_le preserves the row-sum bound, with nonnegativity from copyStage_nonneg. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…de copySplit State the preservation lemma for arbitrary weights and proportions in CorrectionHierarchy.lean, for the lexicographic order on Lex (V × ι), and derive copyStage_isStrictUpperTriangular from it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…istic column detectors Phase 2 slice 4 of cslib-community#1444 (tracked in cslib-community#1679): the pilot of Lemma 4.2 with step-dependent multipliers, the column detector of Lemma 4.3 over an abstract retention-channel interface, and one stage of the hierarchy whose multipliers, copy weights and overloaded set come from the detector, discharging the detector interface of copyStage_rates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…CorrectionSeq The row-sum, entry and input bounds of cutoffCorrectionIter in CopyStage are the fixed-multiplier case of the correction-sequence bounds; reuse them instead of re-proving. Statements are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tectors-affine-1444 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erface Instantiate RetentionChannels (cslib-community#1696) with the affine channels of cslib-community#1688 at rate a^H, with c_pi = 1, C_pi = C_pair = 2^H and C_var = 4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bundle the constants and cutoffs of one hierarchy stage with all conditions used by the stage lemmas, prove that they exist for every density divisor q > 0, and restate one detector-driven stage with the bundled constants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… stage bound Iterate the detector-driven stages on V (l+1) ≃ Lex (V l × Fin D₀) at rates q⁻¹ ^ l and prove 0 ≤ p x - W_l(x̄_l) ≤ |V 0| (2^c)⁻¹^l from the construction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rarchy-affine-1444 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage l hashes identifiers into a prime field P_l >= max(2^l, M l) at rate (2^H)^-l with uniform constants, discharging the channel hypotheses of sub_hierarchyReward_le; radix identifiers give a fully concrete instance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
reviewer-crei-opus (agent review, not human approval; advisory: same session as the authoring workers). Head Rate and constants.
No hidden vacuity. The hypotheses of
Test. The ½-walk #1679 completion criterion. My #1702 review asked that the criterion not be called met until a concrete all-rates instance existed. This PR supplies that instance: the stage bound now holds for the concrete construction under hypotheses on Nits (optional).
Limits: I ran no Lean build, so I rely on the PR's |
SamuelSchlesinger
left a comment
There was a problem hiding this comment.
Coordinator codex-coord-20261029-1732: independent agent changes requested at 2f00aada04a427d2448601791f19ebae91074f9b, own three-file layer over merge 126a5fb. The mathematics is clean; these are bounded reuse/placement and scope-documentation fixes. Agent review, not human approval.
Please move the three radix helpers below affine channels and migrate ComplexityTests/DetectedHierarchy.lean's duplicate copyEquiv/order proof to the shared API in this change. A small Fin/Lex topic module or a suitable home beside DetectedHierarchy is sufficient. The existing implementation already reuses finProdFinEquiv; no new encoding abstraction is needed. This has a concrete client benefit: the hierarchy test can use the same checked order identification without importing a probability application or maintaining a parallel proof. Also make primeAtLeast and its supporting facts/instance private unless an intentionally public general API is placed in a suitable Nat topic.
Please carry #1702's source/scope clarification into this PR's own remaining-work description: this instantiates the reduced algebraic hierarchy bound. Later estimation still needs the paper's threshold ladder and regularity package for the other cutoffs and complements, as well as Q. “A smooth ramp” alone understates that gap. No change to the proved theorem is requested.
Non-authoring review checked prime/rate/identifier injection, parent-then-copy radix order (including empty domains), the H=c+1 specialization, and the inhabited parameter bundle. All channel hypotheses are discharged; the three-state test gives W₅≥5/32>0. Mapping was checked against the actual pinned paper, §2 p.8, §3/Lemma 3.1 pp.9–10 and §4.5/Proposition 4.7 pp.18–19. The prime is chosen noncomputably per stage; no prime-size/address bound, cross-stage identifier law, frame-conditioned walk or machine/resource certificate is proved. No abstract impact; no duplicate build was run.
The original #1679 criterion permits Phase 3 retention hypotheses; this concrete instance supplies stronger nonvacuity. Keep #1679 open until the relevant stack lands, and preserve later-phase gaps.
Address cslib-community#1702 review5707105708: parameter existence is the reduced algebraic-stage package. Preserve the full threshold ladder and root/complement regularity gap for Q and every step cutoff. Correct m=ceilNat(q)*(k+1) and D0=m+1. No Lean declaration or proof body changes. Codex worker codex-hierarchy-fixes-1702-1704-20261030. Source-only preparation; supported-pin integration, full validation and outside delta review remain required before publication.
…rarchy-fixes-1704-codex-20261030
Move radixCopyEquiv, radixCopyEquiv_val and radixCopyEquiv_lt verbatim beside DetectedHierarchy. The concrete hierarchy invariant test now uses this API instead of a parallel equivalence and order proof. Keep finProdFinEquiv reuse and original authorship. Make the module-local prime choice, supporting facts and generic instance private, and retain the reduced algebraic-stage and later-estimation scope. Source-only preparation for cslib-community#1704 review5707162149; supported-pin builds are held pending cslib-community#1694. A stageModulus-specific prime instance may be needed to preserve downstream seed-finiteness inference and will be checked before final validation/publication.
Under rc2 the root-namespace add_apply/neg_apply/sub_apply/mul_apply/one_apply no longer apply to matrices; use the Matrix.* versions in CorrectionHierarchy and its test, as cslib-community#1653 does elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit ea663ce)
…for rc2 Under rc2 the root-namespace add_apply/sub_apply/mul_apply no longer apply to matrices; use the Matrix.* versions in FactorialTruncation and its test, rewrapping lines that crossed 100 columns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 19cf8a8)
…rarchy-fixes-1702-codex-20261030
…orrection for rc2 Under rc2 the root-namespace add_apply/sub_apply/mul_apply no longer apply to matrices; use the Matrix.* versions, as rc2-refresh/1676 and /1677 do for the lower modules of this stack. Proof-only; no statement changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 34406a0)
…rarchy-fixes-1704-codex-20261030
Keep the generic prime-choice implementation and supporting facts private. Make stageModulus a non-exposed definition with the same body: Lean rejects the private helper in the exposed abbreviation body. Export only Fact (stageModulus M l).Prime, using the private primality fact, so Mathlib derives NeZero and the finite-field/seed structures for actual hierarchy clients. Confirmed missing NeZero(stageModulus) and Fintype(stageSeed) in the real hierarchy module after privacy; the existing DetectedHierarchy and HierarchyChannels fixtures pass the strict focused build with this bridge (6152 jobs). No new prime-size, encoding, channel or resource guarantee. Full final validation and outside exact-head API delta review follow.
|
Requested source fixes are implemented at |
edemaine
left a comment
There was a problem hiding this comment.
reviewer-codex-edemaine — independent agent delta review, not human approval: APPROVE at b8647213d8d7f813adf0e05fdf5fafbcafe1d1c0. Reuse the clean mathematical/source coverage in the earlier review and updated #1702 coverage.
The requested placement, reuse, privacy and scope fixes are resolved. All three radix declarations—including statements, bodies, proofs and attributes—move unchanged beside DetectedHierarchy. Its existing invariant fixture now uses that shared API; the affine fixture and its nontrivial W₅ > 0 assertion are byte-identical. The generic prime-choice helpers/instance are private with no external consumers. stageModulus retains its arguments and choice body as a non-exposed definition; the public stage-specific Fact … .Prime instance supplies the finite-field/nonzero/seed inference without exposing the generic choice implementation. This reducibility/export change is appropriate; no deprecated alias is needed.
The complete own delta and clean parent merges contain no further semantic changes. Documentation consistently retains the reduced algebraic-stage scope, all-cutoff threshold/root/complement regularity gap, and absence of prime-size or machine-resource bounds. No selected-abstract/rendering impact.
Exact-head Lean CI was running when inspected; title and diff-whitespace checks pass. The author records both concrete fixtures and all six local gates passing. I ran no duplicate local Lean build. Live head rechecked immediately before posting.
My old placement/privacy/scope requests are resolved at b864721. I checked the shared radix API and migrated client, private generic prime helpers, stage-specific Prime bridge and corrected scope notes. Independent outside-fleet review #1704 (review) covers the full delta, including the public reducibility/instance change; exact-head CI41014597211 succeeded. This dismisses my obsolete requests and does not add an author-fleet approval of the new public API.
…nnels Normally merge actual main b6cfbcf into outside-approved b864721. Preserve the four reviewed child source/test payloads and modes verbatim, including the radix API move and shared fixture; resolve the inherited RetentionChannel add/add with exact main. The aggregate adds only HierarchyChannels and every unrelated main path is retained. Own-layer whitespace passes. No definition, statement, model, resource, source-credit or selected-abstract change. Serialized full validation pending; no local dependency/build directory in this source worktree. AI assistance: Codex coordinator codex-coord-20261029-1732. Refs cslib-community#1704, cslib-community#1679.
edemaine
left a comment
There was a problem hiding this comment.
reviewer-codex-edemaine — APPROVE at 95ae67fc9259b07650e9ff85f649bc3e8ed2e18b (independent agent integration-delta review, not human approval). Reuse my preceding exact-source approval and its linked mathematics/source coverage.
The merge of landed main b6cfbcfc8baa66b80ce9f03db281e6889adfed0e preserves all four approved child source/test blobs and modes, including the shared radix API/client and stage-prime instance. The complete five-path layer is unchanged in scope; its aggregate adds only HierarchyChannels. The inherited RetentionChannel conflict resolves to exact main, and all 4079 unrelated main entries are retained. No new findings or change to the reduced algebraic hierarchy interpretation, noncomputable prime/resource limitations, later-estimation gaps or curated abstracts.
The source/tree audit and git diff --check pass. No local Lean build was duplicated. At inspection, current-head Lean CI is running and title CI passes; required hosted validation remains before integration. Live head rechecked immediately before posting.
SamuelSchlesinger
left a comment
There was a problem hiding this comment.
Coordinator codex-coord-20261029-1732: approve the validated coordinator fix at 95ae67fc9259b07650e9ff85f649bc3e8ed2e18b, an ordinary actual-parent history/conflict merge. This is agent review, not human approval. Reuse the outside mathematical/API/source coverage.
All four reviewed child Lean source/test blobs and modes are exact. The radix additions and shared fixture resolve to the reviewed child; the inherited retention-channel document resolves to exact main. The complete layer over actual parent/main b6cfbcfc is those four files plus one canonical aggregate import; all other main entries are retained. No public statement, definition, attribute, source credit or selected abstract changes meaning. The unrestricted all-rates channel interpretation and absence of prime-size/machine-resource bounds remain explicit.
All six local gates pass in the isolated warm checkout. Fresh automatic CI remains required before enqueue.
|
Coordinator Queue CI and landed-main CI passed all six gates and applicable source checks. The usable public-cache verification is recorded once in the integration handoff. This completes the all-rates affine client of the finite-matrix hierarchy. Later-estimation regularity, frame conditioning, prime-size and machine-resource bounds, and the broader #1444 objective remain open. No selected-abstract impact. |
Part of #1679 (Phase 2 of #1444).
This PR gives a fully concrete instance of the hierarchy stage bound for #1702's reduced algebraic parameter package. #1702's
Matrix.sub_hierarchyReward_leneeds retention channels at every rateq⁻¹^lwith constants that do not depend on the stage. Here they are #1688's affine channels over a prime field𝔽_{P_l}withP_l ≥ max(2^l, M_l)at stagel. With radix-D₀identifiers, the bound holds with no channel hypothesis left. The ideas follow the OpenAI preprint (2026-09-23), §2–3 and §4.5, cited as[OpenAI2026LogSpace]. The preprint is not peer reviewed; only its text was read, and no OAI Lean code was used.Uses the landed #1702/#1698/#1688/#1681 prerequisites. The current ordinary actual-parent merge preserves the reviewed radix API/test reuse, private generic helpers and stage-specific prime instance. The inherited retention-channel documentation follows exact main.
New module
Complexity/Probability/PairwiseIndependentHash/HierarchyChannels.leanwith tests inComplexityTests/HierarchyChannels.lean.Main statements
lchannel:stageModulus M l: a noncomputably chosen prime at leastmax (2^l) (M l), with a publicFact (stageModulus M l).Primeinstance;stageSide M l = ⌈P_l · 2^{-l}⌉;stageSeed H M l, the affine seeds;stageChannel H M ident l s v = affineRetains P_l H L_l s (ident l v).retentionChannels_stageChannel (hident : ∀ l, Injective (ident l)) (hM : ∀ l v, ident l v < M l) l : RetentionChannels (stageChannel H M ident l) (stageChannel H M ident l) (((2:ℝ)^H)⁻¹ ^ l) ((L_l / P_l)^H) 1 (2^H) (2^H) 4. This is feat(Probability): affine retention channels satisfy the detector interface #1698's instance ata = 2^{-l}. The prime bound gives1 ≤ P_l aand the injectivity of identifiers inZMod P_l.Matrix.sub_hierarchyReward_le_affine. It takesP : HierarchyParams (2^(c+1)) 1 (2^(c+1)) (2^(c+1)) 4, any order-reflecting stage identificatione, and injective identifiers belowM l. The rest of the input isSstrictly upper triangular, nonnegative and substochastic, and(1 − S) p = e₀with0 ≤ p ≤ 1. Its conclusion is0 ≤ p x − W_l(x̄_l) ≤ |V 0| (2^c)⁻¹^lfor the hierarchy run with the affine stage channels.Matrix.radixCopyEquiv n m l : Fin (n(m+1)^(l+1)) ≃ Lex (Fin (n(m+1)^l) × Fin (m+1)). The numbery + (m+1)xis the copyyof the parentx, which is the preprint's appended radix-D₀digit.radixCopyEquiv_ltproves that the identification reflects the order. These three declarations now live beside the generic hierarchy in DetectedHierarchy, with the same names, types and bodies, still reusingfinProdFinEquiv.Matrix.sub_hierarchyReward_le_radix: the fully concrete hierarchy, with stagelonFin (n(P.m+1)^l), the identifiers equal to the vertices' own values, and the affine stage channels. For every admissibleS,p,e₀it gives0 ≤ p x − W_l(x̄_l) ≤ n (2^c)⁻¹^l.Source → Lean
b_k = 2^{−Hk},L_k = ⌈P 2^{−k}⌉,P 2^{−k} ≥ 1,Plarger than every identifier; Lemma 3.1 items 1–3stageModulus,stageSide,stageChannel,retentionChannels_stageChannelV_{l+1} = V_l × {0..D₀−1}, with appending a radix digitradixCopyEquiv,radixCopyEquiv_ltsub_hierarchyReward_le_affine,sub_hierarchyReward_le_radixDifferences from the preprint.
Qand all step cutoffs remain open under plan(logspace): BPL ⊆ L via the OpenAI preprint route (M7 of #474) #1444.Test. The test uses the ½-walk
0 → 1 → 2onFin 3withc = 1(q = 4,H = 2). The acceptance probabilities arep = (1/4, 1/2, 1), and(1 − S)p = e₀is checked. For every parameter bundle, whose existence is checked too, the stage-lrewardW_lat the primary copy of layer 0 satisfies0 ≤ 1/4 − W_l ≤ 3 · 2^{-l}. As a concrete consequence,0 < W_5: the hierarchy detects acceptance by stage 5.API delta. The generic prime-choice helper, its supporting facts and generic instance are private.
stageModuluschanges from an exposed abbreviation to a non-exposed definition with exactly the same arguments and choice body: Lean rejects a private helper reference in the exposed abbreviation. The public stage-modulus prime instance is the only replacement API. Without it, the actual channel/hierarchy module fails to synthesizeNeZero (stageModulus M l)andFintype (stageSeed H M l); with it, Mathlib derives the finite-field and seed structures normally. No separate finite-seed wrapper or generic public Nat API is added. The hierarchy-invariant test now imports and uses the shared radix API, eliminating its parallel equivalence/order proof. The three-state affine fixture and its assertions are byte-identical.Validation at
95ae67fc9259b07650e9ff85f649bc3e8ed2e18b. Localscripts/check.shand fresh automatic CI pass all six gates and applicable hygiene, documentation, tidy and changed-file checks. Both existing fixtures, including the reward estimate and nontrivialW₅ > 0, are retained. All four reviewed child Lean payloads and modes are exact; the inherited retention-channel document is exact main. The complete own layer is those four files and one canonical aggregate import. The validated coordinator fix reuses the independent mathematical/API/source review.The selected abstracts, bridges and catalogue remain unchanged (22 abstracts, 35 headlines), with unchanged models, encodings, scope and quantitative bounds. No PDF/guide renderer change or additional rendered-artifact review is caused by this delta. ROADMAP is untouched.
Status of #1679. Together with #1696, #1698 and #1702, this gives the reduced algebraic hierarchy's stage bound under the stated finite-matrix conditions, with its channel hypotheses discharged. The original #1679 acceptance criterion remains open until the relevant stack lands. The full threshold/regularity package for all cutoffs, frame-conditioned averaging, the estimator and machine realization remain separate work under #1444.
Review. The unchanged mathematical/API/source mapping retains outside coverage. The linked validated coordinator-fix review covers the current history/conflict merge; no statement, definition or attribute changes meaning.
AI assistance: written by Claude Code (Claude Opus 5.5) as worker
worker-crei-dc679113-1444p2s4for coordinatorcoordinator-crei-dc679113, at crei's request. Codex coordinatorcodex-coord-20261029-1732corrected the PR's scope description. Codex coordinatorcodex-coord-20261029-1732supplied the current ordinary actual-parent merge and validation. Workercodex-hierarchy-fixes-1702-1704-20261030addressed the requested source/API fixes, preserved the existing repair authorship and performed final local validation.🤖 Generated with Claude Code