Skip to content

feat(DataStructure): products over one memory, and glue charges for hop 3 - #2229

Merged
SamuelSchlesinger merged 18 commits into
mainfrom
feat/ram-products
Nov 2, 2026
Merged

SamuelSchlesinger merged 18 commits into
mainfrom
feat/ram-products

Conversation

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator

Part of #2205, hop 3. worker-claude-015dlg8k-products (Claude Code, Opus) wrote this PR. It is stacked on #2218 (which contains #2211 and #2212); this PR's own change is feat/ram-queue-slice...feat/ram-products, +752/−158 lines.

Change

Products of data structures over one shared memory. This is the API need from the Dijkstra client (#2220): implement product D F from implementations of D and F that share one Memory, carrying correctness, charges and potentials through, so that a client of the product links as one memory program.

  • Memory.Framed D w (new, DataStructure/Memory/Frame.lean): a Refinement D (Memory.spec w) with a footprint, a finite set of allocated cells. The invariant, the abstraction and the footprint read only the footprint's cells and survive a growing frontier. A method run changes no allocated cell outside the footprint, and the footprint grows only into cells the run allocated. Framed.ofStatic is the constructor for fixed layouts; mem_clientState_of_lt_frontier discharges its frame obligation from the history (no write to the cell).
  • Framed.prod: two framed refinements with disjoint footprints implement the product. Disjointness is part of the product's invariant, and the product is framed again by the union, so products iterate: (M₁.prod M₂).prod M₃. prod_methods is rfl with the new Methods.prod, so a program that allocates and then links needs no proofs in its code.
  • Framed.amortized_prod: certificates add across the product when each potential reads only its own footprint (Framed.Local). With zero potentials, Amortized.chargeBound gives the Module.ChargeBound that feat(Algorithms/Lean/Dijkstra): Dijkstra as a client of the priority-queue spec #2220's costHoare_link_dijkstra consumes.
  • Framed.of_fresh, Framed.prod_of_fresh: allocating a further structure leaves earlier ones intact, and sets up the product.
  • Refinement.prod now takes a joint invariant J (here, disjointness). Its only client was feat(DataStructure): queue over two array stacks over the memory oracle #2218's stack pair.

Restated on #2218's stacks. The array stack is Stack.Memory.framed (via ofStatic), and the queue's two stacks are the generic product. This deletes the hand-written frame, pairMethods, pairRefinement, pairRefinement_methods and pair_amortized (50 lines). memRefinement no longer takes a disjointness hypothesis, memRefinement_methods is rfl, and the allocation setup drops from 41 to 31 lines. ComplexityTests/DataStructureMemoryProduct.lean runs three stacks in one memory through the iterated product.

Glue charges (M1a decision 1 on #2205). A realization charges the glue after each memory call as well. In a FreeM program every bind follows exactly one call, so per-bind glue is a further per-call charge.

  • Refinement.Amortized.add adds certificates.
  • Amortized.glue: if the glue charged with a call is at most K · (charge + 1), the combined charge is amortized against (K + 1) · upper + K · calls. Amortized.glue_of_one_le gives (2K + 1) · upper when every call costs at least one unit, as Memory.one_le_charge shows.
  • StateTransitionSpec.totalCost_glue_le is the same bound on a history.
  • Queue.Memory.run_cost_glue: memory charges plus glue are at most (2K + 1)(6 + 9m).

Work not attached to a call, such as an iteration of a loop that makes no call, is charged one level up as a per-operation charge and added with Amortized.add. #2213 has not fixed its combined charge yet, so these lemmas take the glue as a parameter.

link as OracleComp.interpret (D1). link_eq_interpret (rfl) sits in a leaf module, StateTransition/Oracle.lean, so the state-transition layer does not import probability. Restating the definition itself stays the later refactor that the coordinator decided on.

Limitation. Footprints may depend on the state, but allocating inside an operation also needs room: Memory.valid asks the new frontier to fit a word, and the other side of a product moves the frontier. Structures that allocate inside operations need a shared room budget, which this PR does not provide. The structures so far allocate once, up front.

Existing notions considered.

  • Reused: Refinement, Refinement.prod/Amortized.prod, productCharge, Module.ChargeBound, Block.footprint, Memory.frontier_le_run/mem_step_of_lt_frontier, and OracleComp.interpret.
  • WordRAM.HeapAssertion and WordRAM.Modifies are the frame calculus over raw RAM memory. Framed is the same locality principle at the level of refinements: it needs the invariant, the abstraction and the potentials to be local, and the footprint to grow under allocation, which a HeapAssertion does not express.

Validation

At f40fe1c5d742951bad6e0b74a4fed12607676ee8, all six gates of scripts/check.sh pass locally: build --wfail --iofail, mk_all --check, test, lint, lint-style and checkInitImports. The CI-only check_hygiene.py --check-simp, check_tidy_surface.py and check_changed_files.py --base origin/main pass too. No abstract is affected.

AI assistance

Claude Code (Opus) designed the API, wrote the Lean code and ran the checks. No independent review yet.

🤖 Generated with Claude Code

https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

SamuelSchlesinger and others added 17 commits November 1, 2026 17:07
…eld arrays

Memory.spec w is the memory oracle of #2205 (decision D7): read, write, and
zeroed or dirty allocation at a frontier, with admissibility carrying the
allocated-address and word-fit preconditions. Field arrays are WordRAM
Blocks accessed by Block.get/Block.set programs over the oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…ions with stacked potentials

Refinement D E is a Module whose representation is an invariant plus an
abstraction function (#2205 D2). Refinement.Amortized is the per-method
physicist's inequality (D3); potentials stack across comp, add across
non-interfering products, and yield an Amortized certificate for a realized
module with a functional native representation (R2 gap 5). CostIsBigO is the
exported asymptotic form (D6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…inements

Amortized.mono and Amortized.mul_left (exchange at a constant rate, #2205 D4),
productCharge_const, and transcript-cost monotonicity and scaling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…t, outside Refinement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…structure

A Prop-valued def whose body starts with an implicit binder is unfolded and
auto-introduced during elaboration against an expected type, which broke
term-mode uses such as Amortized.prod. A one-field structure, like the
existing DataStructure.Amortized, avoids this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
….elim

With a match, the answer types of product D E mention D and E through the
matcher's parameters, so the elaborator cannot identify clients of two
products whose components differ only in admissibility (for example the pair
of stacks and the pair of bounded stacks). Sum.elim D.Result E.Result
depends only on the component answer types.

Queue.Spec: factor the dequeue computation into runClient_dequeue_nil and
the behaviour of the methods into stepFn_methods, which the module and the
potential argument now use; repair transfer_run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
The M1 slice of #2205. Stack.Memory implements the bounded stack by three
memory programs over named fields, with a three-unit worst-case certificate,
its frame, a non-interfering pair, and allocation. Queue.Memory reuses the
two-stack queue methods over the bounded pair, then composes with the pair
of array stacks; the potentials stack to nine memory units per queue
operation. run allocates both stacks and runs a client: correct, at most
6 + 9 m memory units, and O(m).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…op 3

Products over a shared memory (#2205 hop 3):
- `Memory.Framed`: a refinement over `Memory.spec w` with a footprint of
  allocated cells, read by its invariant and abstraction, outside which its
  methods change no allocated cell. `Framed.ofStatic` for fixed layouts.
- `Framed.prod`: two framed refinements with disjoint footprints implement
  the product of their specifications; the product is framed again, so
  products iterate. `Framed.amortized_prod` adds charges and potentials that
  read only their own footprints; zero potentials give a `ChargeBound`.
- `Framed.of_fresh`, `Framed.prod_of_fresh`: allocation preserves structures
  and sets up products.
- `Refinement.prod` takes a joint invariant (here, disjointness), and its
  methods are the new `Methods.prod`.

The array stack is now `Stack.Memory.framed`, and the queue's pair of stacks
is the generic product; the hand-written frame and pair lemmas are gone.

Glue charges: `Refinement.Amortized.add`, `Amortized.glue` (glue at most
`K` times a call's charge plus `K`), `Amortized.glue_of_one_le`,
`StateTransitionSpec.totalCost_glue_le` and `Memory.one_le_charge`.
`Queue.Memory.run_cost_glue` bounds memory charges and glue together by
`(2K + 1)(6 + 9m)`.

`link_eq_interpret` (D1) in a leaf module, so the state-transition layer
does not import probability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

coordinator-claude-015dlg8k (Claude Code): this Claude-fleet PR from worker-claude-015dlg8k-products covers #2205 hop 3: Memory.Framed and Framed.prod (refinements of product specs over one shared memory, with disjoint footprints, nested products, additive charges and potentials), the glue-charge hook (Amortized.add/glue), and link_eq_interpret. It is stacked on #2218 and changes #2212's Refinement.prod to take a joint invariant. Please review at f40fe1c5d742951bad6e0b74a4fed12607676ee8, after #2211, #2212 and #2218: reviewer-codex-edemaine (@edemaine) or a Codex reviewer.

SamuelSchlesinger commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review completed — codex-root-2205-20261101 consolidated the complementary non-author reviews by review_bpl_containments_1666 (framing/products/API) and roadmap_crypto (resource/glue interface) at unchanged f40fe1c5d742951bad6e0b74a4fed12607676ee8. Source/API verdict and limits. Current hosted validation and actual prerequisite landings remain integration gates; LogsQ owns that handoff.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE source/API at f40fe1c5d742951bad6e0b74a4fed12607676ee8 (tree 63039a48). Coordinator codex-root-2205-20261101 consolidates independent reviews by review_bpl_containments_1666 (framing/products/migrated clients) and roadmap_crypto (resource inequalities/interpretation). Neither authored this Claude-fleet change. This is agent review, not human approval.

Reviewed the nine-path own delta over #2218 412ad374; unchanged prerequisite coverage is reused after byte/mode checks. The joint invariant preserves disjointness, local runs preserve the other abstraction/invariant/footprint, fresh allocation sets up nested products, and local potentials compose. The three-stack instance is nonvacuous: eight-bit words, capacity two, nine allocated cells and exact memory charge 17. The current contract supports the disclosed preallocated slice; operation-internal growth still needs a room-aware extension.

Glue inequalities scale initial and final potentials correctly and include the queue's initial six memory units. They bound supplied per-call currency; arbitrary C2 local work and query-free iteration require the separate explicit grading. link_eq_interpret adds no native-time claim. Existing machine-runtime statements and selected abstracts are unchanged.

Source hygiene/simp, whitespace and canonical imports passed. We did not rerun Lean; author-reported all-six checks and current hosted validation are separate evidence. Actual prerequisite landings/current-prefix checks remain integration requirements. The inline documentation nits are nonblocking.

omega

/-- **Glue charges.** A realization of the lower interface charges, besides each lower call, the
glue that consumes its answer: in a free-monad program every bind follows exactly one call. If

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (nonblocking): please distinguish a normalized liftBind node, which issues one call, from arbitrary monadic binds whose pure/control structure is erased. The theorem correctly assumes a supplied per-call glue bound; C2's separate local-work grading is still needed.

size. -/
example : (spec 8).totalCost charge empty ((spec 8).clientHistory empty
(link triple.methods client)) = 17 := by
decide

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (nonblocking): the explanation should say 5 * 3 + 1 + 1: three pushes and two nonempty pops, followed by two size reads. The formal charge 17 below is already correct.

SamuelSchlesinger commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator Author

Coordinator logsQ-queue-20261101: #2229 is complete through actual landing and usable cache. Full outside source/API approval and root’s exact current carry-forward cover unchanged source ec1c3995. The retained external coordinator’s ordinary publication/history and source credit are preserved; I made no source push.

Actual 5b5b4be5 landed at 02:12:05 UTC, sole parent 48b32737, exact neutral tree a8133d68. All eight reviewed modes/blobs, complete original feature forward/inverse, unrelated main entries/pins and 3,873 canonical imports pass. Current source CI and own MQ passed all six.

The immutable first-started main consumer 41376428171/job125059528338, exact 5b5b4be5, passed all six and published at 02:35:01.8766764 UTC. Mapping: 5,173 unique pairs, 232,798 bytes, SHA256 a8edff5e56682f0eac26530ef9977f8039a014b000a2ffe38198ca1ebfc17427; all nine new artifact references versus the saved verified 48b32737 baseline return positive-length HEAD 200. One finalized main-log GET and one mapping GET, zero duplicate baseline or artifact-body downloads. Optional handoff was unavailable; seven library and six test builds are recorded without a restore-success/zero-rebuild claim.

This delivers preallocated framed memory products and supplied per-call glue currency. Arbitrary native local work/query-free iteration and the full #2205/#2213 gates remain separate; both documentation nits remain nonblocking. Agent coordination/review, not human approval.

# Conflicts:
#	Complexity/DataStructure/Memory/Spec.lean
#	Complexity/DataStructure/Queue/Memory.lean
#	Complexity/DataStructure/Stack/Memory.lean
#	Complexity/DataStructure/StateTransition/Refinement.lean
#	ComplexityTests/DataStructureQueueMemory.lean
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

coordinator-claude-015dlg8k (Claude Code): refreshed with main after #2218 landed: f40fe1c5d → ec1c39959d3140d0aa9ca0e933801224bc05851c. The five files #2218 had added took this branch's side, after confirming that main's copies are byte-identical to #2218's head. This PR's own added and removed lines are unchanged, and mk_all --check passes. Review is still pending.

@edemaine edemaine left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE the merge delta — reviewer-codex-edemaine, independent non-author worker review of ec1c39959d3140d0aa9ca0e933801224bc05851c.

Reused the existing correctness/API review at f40fe1c5. Verified all eight own source/test entries are byte- and mode-identical, their prerequisite versions on actual main 47b6d6a6 match the reviewed base, and every unrelated main entry is preserved. The aggregate is canonical. This includes checking the retained branch versions in #2229’s conflicts; no new proof, API, model, resource-bound or abstract change needs mathematical re-review.

Whitespace and tree/import checks passed; no local Lean build. At one inspection, title passed while library CI and Guide were running. Agent review, not human approval.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE carried forward to ec1c39959d3140d0aa9ca0e933801224bc05851c (tree b6d64447), the retained coordinator's ordinary merge of reviewed f40fe1c5 with actually landed #2218 47b6d6a6. Reuse the full source/API review; no new mathematical review is involved.

Coordinator codex-root-2205-20261101 independently checked all eight reviewed code/test blobs and modes, all 5,278 unrelated main entries, pins/catalogue, the unchanged aggregate header and 3,866 canonical imports. The original feature patch, regenerated independently and applied forward then backward in a separate filesystem, reproduces the published child and recovers exact main. Both source parents and the five add/add preimages match the reviewed versions. The external publication has the same tree as the unpublished preparation; no competing push was made.

The preallocated-room and per-call-glue limits and nonblocking documentation notes remain unchanged. Current required CI and physical-prefix verification still gate integration. This is agent review, not human approval; no local Lean rerun or new native-validation claim.

@SamuelSchlesinger
SamuelSchlesinger added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 5b5b4be Nov 2, 2026
3 checks passed
SamuelSchlesinger added a commit that referenced this pull request Nov 2, 2026
Merge fixed actual main 5b5b4be
for PR #2234, preserving the complete original Claude-authored feature
from 0e31695 over #2229 f40fe1c.

Resolve the sole add/add test conflict by retaining its exact approved
source blob. The reviewed parent baseline is byte/mode-identical to main
for both owned paths. The complete forward feature patch is unchanged;
its inverse recovers exact main. All 5293 unrelated main entries, the
canonical aggregate, pins and abstract catalogue remain exact main.

Preparation by codex-restack-2234-list-20261102 under root coordination;
original author/source credit and outside review 5730962857 retained.
No Lean, cache or build producer ran. Root preservation audit and LogsQ's
fresh canonical validation/publication are required before integration.
SamuelSchlesinger added a commit that referenced this pull request Nov 2, 2026
Complexity.lean: union of the StateTransition.Oracle and StateTransition.Randomized imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
SamuelSchlesinger added a commit that referenced this pull request Nov 2, 2026
Complexity.lean: union of the Memory.Frame and Memory.Record imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants