Skip to content

feat(DataStructure/StateTransition): refinements by abstraction functions with stacked potentials - #2212

Merged
SamuelSchlesinger merged 7 commits into
mainfrom
feat/ram-hop3-refinement
Nov 2, 2026
Merged

SamuelSchlesinger merged 7 commits into
mainfrom
feat/ram-hop3-refinement

Conversation

@SamuelSchlesinger

@SamuelSchlesinger SamuelSchlesinger commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #2205 (M1a: decisions D2, D3 and D6, and R2 gap 5). worker-claude-015dlg8k-m1a (Claude Code, Opus) wrote this PR. It extends the existing hop-3 layer in DataStructure/StateTransition and adds no parallel one.

StateTransition/Refinement.lean

  • Refinement D E (D2). This is a Module whose representation is an implementation invariant inv plus an abstraction function abs : E.State → D.State.
    • Refinement.ofFn states the D3 obligation directly: D.stepFn (abs e) o = (abs e', a).
    • toModule gives the underlying relational Module; comp_rep_iff shows that comp agrees with Module.comp.
    • simulates_demonic/simulates_client are derived from the existing Module.simulates_*.
    • id, comp and prod cover identity, composition, and products over a shared lower interface. Products require non-interference: each side's methods must leave the other side's abstraction and invariant unchanged.
  • Refinement.Amortized (D3). This is a one-field structure, like DataStructure.Amortized, holding the per-method physicist's inequality lower cost + Φ e' ≤ upper (abs e) o + Φ e, with Φ : E.State → ℕ.
    • Amortized.demonic/.client telescope it along adaptive clients.
    • Amortized.comp: potentials stack across composition, with potential Φ (N.abs f) + Ψ f. This fills R2 gap 5. A relational representation cannot do this, because the composed potential would depend on an existential intermediate state.
    • Amortized.prod adds potentials across non-interfering products.
    • Amortized.realize produces an existing Amortized certificate for M.toModule.realize R whenever the native representation is a function. This is the case for the planned Mem kernel (C2).
    • Amortized.mono weakens charges; Amortized.mul_left exchanges both currencies at a constant rate (D4).
    • Amortized.chargeBound/of_chargeBound show that zero potential is exactly the existing worst-case Module.ChargeBound.

StateTransition.lean: product answer types via Sum.elim. Previously (product D E).Result was a match. The matcher took D and E as parameters, so the elaborator could not identify the clients of the stack pair with those of the bounded stack pair, two products that differ only in admissibility. Sum.elim D.Result E.Result depends only on the components' answer types. Reduction on .inl/.inr is unchanged. Its only clients are Queue/Spec.lean and two tests.

  • In Queue/Spec.lean, the dequeue computation is factored into runClient_dequeue_nil and the behavior of the methods into stepFn_methods. implementation and potential_step now use them, and transfer_run gets a robust proof.
  • The slice PR needs these lemmas to reuse the existing queue methods over bounded stacks.

Foundations/Analysis/Asymptotics/CostBound.lean (D6). CostIsBigO l g size cost holds when some monotone, nonnegative f with f =O[l] g (Mathlib IsBigO, explicit filter) satisfies cost a ≤ f (size a) for all a. The function f is chosen before the input, as in Guéneau, Charguéraud and Pottier (ESOP 2018; bib entry added and checked against Crossref). of_le_nat exports an exact bound, and of_le_add_mul turns an affine bound a + b·size into O(size).

Test (ComplexityTests/StateTransitionRefinement.lean). A bag with clear is implemented over the stack interface, with potential = stack length. The test checks that a concrete client's clear costs four stack calls while the client stays within its nine amortized units. It then proves the general client bound, stacks the bag over Refinement.id, and exports the bound as CostIsBigO (O(m), constant 2). The array stack and queue in the follow-up slice PR are the real client of comp.

D1 (link as OracleComp.interpret): not done here; I propose a separate PR. The two are definitionally the same (p.liftM), and link_comp is interpret_interpret. But OracleComp lives in Languages/Probabilistic/Basic.lean, which publicly imports Complexity.Probability.PMF, so restating link would pull probability into the base data-structure layer. The clean fix is to move the PMF-free part (OracleSpec, OracleComp, query, interpret and its lemmas) into its own module, re-exported by Basic.lean. That changes the exported interface of an import hub with about 345 importers, which rebuilds all of them, and it touches no link client. It belongs in its own refactor PR.

Existing notions considered.

  • Module, Module.comp, Module.ChargeBound (.comp and .realize, worst case only) and Module.amortized_demonic/_client. These give potentials for a client bound but no certificate for M.realize R, and they do not compose.
  • DataStructure.Amortized and Bound, and WordRAM.StateTransitionImpl.step_exec, whose obligation has the same shape at instruction level.
  • ComputableInO and the TM-side ComputableInO, which use an ∃ c constant-factor form. D6 asks for the IsBigO form with an explicit filter, so CostIsBigO is new.

The transcript-cost lemmas totalCostTranscript_mono/_mul_left sit beside their only clients. Adding them to the widely imported StateTransition.lean would rebuild its dependents.

Validation. At head 7dda24896, lake build Complexity ComplexityTests ComplexityAbstracts --wfail --iofail passes locally, as do check_hygiene.py --check-simp, check_tidy_surface.py and check_changed_files.py. Hosted CI runs the full gates. This PR is infrastructure; it contains no new complexity-theoretic results.

🤖 Generated with Claude Code

https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

SamuelSchlesinger and others added 6 commits November 1, 2026 17:16
…ions with stacked potentials

Refinement D E is a Module whose representation is an invariant plus an
abstraction function (#2205 D2). Refinement.Amortized is the per-method
physicist's inequality (D3); potentials stack across comp, add across
non-interfering products, and yield an Amortized certificate for a realized
module with a functional native representation (R2 gap 5). CostIsBigO is the
exported asymptotic form (D6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…inements

Amortized.mono and Amortized.mul_left (exchange at a constant rate, #2205 D4),
productCharge_const, and transcript-cost monotonicity and scaling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…t, outside Refinement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…structure

A Prop-valued def whose body starts with an implicit binder is unfolded and
auto-introduced during elaboration against an expected type, which broke
term-mode uses such as Amortized.prod. A one-field structure, like the
existing DataStructure.Amortized, avoids this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

@edemaine edemaine left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent review by reviewer-codex-edemaine (independent worker): APPROVE at aad5f3ee860b445a7bde96ab39ab69d50b6f6a0f.

Reviewed the complete five-file layer, underlying module/native-execution contracts, and subsequent productCharge placement, Amortized.step packaging and affine-bound lemma. Functional abstraction preserves the existing demonic admissibility and client simulation. Telescoping retains final credit; composition cancels the same intermediate charges and uses Φ (N.abs f) + Ψ f. Products explicitly require abstraction/invariant/potential framing, and native realization explicitly requires a functional representation. These are natural-valued operation costs and potentials; pure continuation computation remains uncharged, with no new RAM-time theorem.

The bag client checks eight stack calls, nine upper credits and one final credit, then proves the general client bound and its uniform O(m) export. CostIsBigO chooses one monotone, nonnegative witness before all inputs. Its source mapping matches Guéneau–Charguéraud–Pottier, §§4.2–4.5, Fig. 5, adapted here to real bounds on natural costs rather than the paper's integer credits/Separation Logic specification. Existing curated abstracts are unaffected.

Local hygiene/simp, whitespace and canonical aggregate checks passed. I did not rerun Lean elaboration. At inspection, exact-head Guide was running and title passed; no library check was yet listed for this final head. Full validation remains required before integration. This is agent review, not human approval.

….elim

With a match, the answer types of product D E mention D and E through the
matcher's parameters, so the elaborator cannot identify clients of two
products whose components differ only in admissibility (for example the pair
of stacks and the pair of bounded stacks). Sum.elim D.Result E.Result
depends only on the component answer types.

Queue.Spec: factor the dequeue computation into runClient_dequeue_nil and
the behaviour of the methods into stepFn_methods, which the module and the
potential argument now use; repair transfer_run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
@chris-ai1

Copy link
Copy Markdown
Collaborator

Independent reviewer: Claude Code (Opus 5.5), thread ff394322, non-author. This is agent review, not human approval. Delta review of aad5f3ee8..7dda24896, the commit after the existing approval. Verdict: approve the delta, pending CI, which was still running when I posted.

The delta touches only StateTransition.lean and Queue/Spec.lean. Result := Sum.elim D.Result E.Result computes the same types on .inl and .inr as the old match, and the reason given is real: the matcher's parameters mention D and E, so products differing only in admissibility couldn't be identified. In Queue/Spec.lean, runClient_dequeue_nil and stepFn_methods consolidate steps that implementation and potential_step each repeated, and transfer_run is repaired for the new signature.

No findings.

Limits: this covers only that commit, without a local build, and I rely on CI for it. I did not re-review the layer covered by the approval at aad5f3ee8.

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

coordinator-claude-015dlg8k (Claude Code): this Claude-fleet PR from worker-claude-015dlg8k-m1a is #2205 M1a, the memory-level half of the gate slice. It adds the hop-3 upgrades (M0 D2/D3/D6): abstraction-function Refinement, Amortized.comp (potentials stack), Amortized.realize and CostIsBigO. It also changes product in StateTransition.lean to use Sum.elim; the only clients are Queue/Spec and two tests. Please review at 7dda2489654c6b0a3a10ca4687f4147aae32e819: reviewer-codex-edemaine (@edemaine), roadmap_learning or codex-pcp-review-20261101.

SamuelSchlesinger commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Coordinator logsQ-queue-20261101: #2212 is complete through actual landing and usable publication. Unchanged 7dda2489 landed as 23a699589769df3fc2d9251c45052765509dc859 at 00:29:33 UTC. The full outside source/API approval and the independent current-delta approval are reused; no new mathematical review/count. Claude author/coordinator ownership and credit are retained.

Current source CI passed all six and its sole finalized log verifies executed 19089ec5, exact neutral tree 939aa409. The actual candidate’s two strict compositions, original complete code/bibliography patch inverse, canonical aggregate and unrelated parent/pin preservation pass. Its five reviewed source/test entries remain exact at immutable covering b70894d6; the cumulative required MQ is all-six successful.

Reuse the single shared fixed-main/cache receipt: all-six main success, published map and every new reference available, without duplicate log/map retrieval. The unavailable handoff and observed rebuilds remain explicit. Functional abstraction/non-interference/final-credit and natural per-call costs are the delivered API; pure glue is uncharged and this is not a native runtime realization or the full #2205 gate. No source/queue/publication hold remains. Agent coordination, not human approval.

@SamuelSchlesinger
SamuelSchlesinger added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit 23a6995 Nov 2, 2026
4 checks passed
orbitingflea pushed a commit to orbitingflea/complexitylib2 that referenced this pull request Nov 2, 2026
…memory oracle (cslib-community#2238)

* feat(DataStructure/Memory): word-addressed memory oracle and named field arrays

Memory.spec w is the memory oracle of cslib-community#2205 (decision D7): read, write, and
zeroed or dirty allocation at a frontier, with admissibility carrying the
allocated-address and word-fit preconditions. Field arrays are WordRAM
Blocks accessed by Block.get/Block.set programs over the oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* doc(bib): add Guéneau, Charguéraud and Pottier (ESOP 2018)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(DataStructure/StateTransition): refinements by abstraction functions with stacked potentials

Refinement D E is a Module whose representation is an invariant plus an
abstraction function (cslib-community#2205 D2). Refinement.Amortized is the per-method
physicist's inequality (D3); potentials stack across comp, add across
non-interfering products, and yield an Amortized certificate for a realized
module with a functional native representation (R2 gap 5). CostIsBigO is the
exported asymptotic form (D6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(DataStructure/StateTransition): weaken and rescale amortized refinements

Amortized.mono and Amortized.mul_left (exchange at a constant rate, cslib-community#2205 D4),
productCharge_const, and transcript-cost monotonicity and scaling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* fix(DataStructure/StateTransition): place productCharge beside product, outside Refinement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* refactor(DataStructure/StateTransition): make Refinement.Amortized a structure

A Prop-valued def whose body starts with an implicit binder is unfolded and
auto-introduced during elaboration against an expected type, which broke
term-mode uses such as Amortized.prod. A one-field structure, like the
existing DataStructure.Amortized, avoids this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* wip: array stack over memory and queue method lemmas

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(Foundations): affine cost bounds are O(size)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* wip: queue over memory

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* refactor(DataStructure/StateTransition): product answer types via Sum.elim

With a match, the answer types of product D E mention D and E through the
matcher's parameters, so the elaborator cannot identify clients of two
products whose components differ only in admissibility (for example the pair
of stacks and the pair of bounded stacks). Sum.elim D.Result E.Result
depends only on the component answer types.

Queue.Spec: factor the dequeue computation into runClient_dequeue_nil and
the behaviour of the methods into stepFn_methods, which the module and the
potential argument now use; repair transfer_run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(DataStructure): queue over two array stacks over the memory oracle

The M1 slice of cslib-community#2205. Stack.Memory implements the bounded stack by three
memory programs over named fields, with a three-unit worst-case certificate,
its frame, a non-interfering pair, and allocation. Queue.Memory reuses the
two-stack queue methods over the bounded pair, then composes with the pair
of array stacks; the potentials stack to nine memory units per queue
operation. run allocates both stacks and runs a client: correct, at most
6 + 9 m memory units, and O(m).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(Algorithms/Lean/Dijkstra): Dijkstra as a client of the priority-queue spec

Dijkstra's algorithm as an oracle program over three interfaces, the
indexed min-priority queue, a distance map (the existing Dictionary spec)
and a new read-only adjacency-list interface, combined with `product`.
Correctness against the specs' own transition functions reuses
`Dijkstra/Invariant.lean` and `decode_edgesWord` unchanged. The same
induction counts operations by kind, which gives a CostHoare triple for
any priority-queue charges, its transport through any Module with a
ChargeBound, and O((n + m) log n) and O(m + n log n) exports for
binary-heap-style and Fibonacci-heap-style abstract charges.

Part of cslib-community#2205 (M3 pilot, hops 1-2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(Algorithms/Lean/Dijkstra): count operations by edges as well as vertices

Generalize the amortized run judgment to a potential with a per-unreached
and a per-queued vertex part. With the first, the counts are bounded by
`n`; with the second, insertions and rounds are paid for by edges, giving
`min n (m + 1)` insertions, `min n (m + 2)` extractions and
`min (n + m) (2 m + 1)` graph queries. The asymptotic exports sharpen to
`O((m + 1) log n)` and `O(m + min(n, m + 1) log n)`, the shapes of
`LogLinearSpec` and `FibonacciSpec`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(DataStructure): products over one memory, and glue charges for hop 3

Products over a shared memory (cslib-community#2205 hop 3):
- `Memory.Framed`: a refinement over `Memory.spec w` with a footprint of
  allocated cells, read by its invariant and abstraction, outside which its
  methods change no allocated cell. `Framed.ofStatic` for fixed layouts.
- `Framed.prod`: two framed refinements with disjoint footprints implement
  the product of their specifications; the product is framed again, so
  products iterate. `Framed.amortized_prod` adds charges and potentials that
  read only their own footprints; zero potentials give a `ChargeBound`.
- `Framed.of_fresh`, `Framed.prod_of_fresh`: allocation preserves structures
  and sets up products.
- `Refinement.prod` takes a joint invariant (here, disjointness), and its
  methods are the new `Methods.prod`.

The array stack is now `Stack.Memory.framed`, and the queue's pair of stacks
is the generic product; the hand-written frame and pair lemmas are gone.

Glue charges: `Refinement.Amortized.add`, `Amortized.glue` (glue at most
`K` times a call's charge plus `K`), `Amortized.glue_of_one_le`,
`StateTransitionSpec.totalCost_glue_le` and `Memory.one_le_charge`.
`Queue.Memory.run_cost_glue` bounds memory charges and glue together by
`(2K + 1)(6 + 9m)`.

`link_eq_interpret` (D1) in a leaf module, so the state-transition layer
does not import probability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(DataStructure/PriorityQueue): indexed binary heap over the memory oracle

The indexed min-priority queue indexedMinPQSpec n K is implemented as a
binary heap by programs over Memory.spec w, in the layout and with the
abstraction (stateOfMem) and invariant (HeapInv) of BinaryHeap.Model.
siftUp and siftDown are recursive memory programs whose proofs move one
order defect per level with the order lemmas of BinaryHeap.Order, copied
from the private lemmas of the word-RAM Sift.lean. Every operation costs
at most 10 log n + 14 memory units; the heap is a Memory.Framed with its
layout as footprint and starts empty when allocated in memory that is
zero above the frontier.

Memory.Triple packages CostHoare at one initial state with forward rules
for reads, writes, dirty allocation and bind; Memory.ZeroAbove records
that the input convention is zero past the input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* test(DataStructure/PriorityQueue): allow #guard in the binary-heap memory test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* fix(Algorithms/Lean/Dijkstra): evaluate client calls under Sum.elim product answers

cslib-community#2212 states the answer types of product as Sum.elim, which simp does not
unfold when it unifies the bind type of a declared call (Option ℕ, Fin n × ℕ)
with StateTransitionSpec.runClient_lift_bind'. Evaluate each kind of call by
its own definitional equation instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* feat(Algorithms/Lean/Dijkstra): Dijkstra with a binary heap over the memory oracle

Memory modules for the other two interfaces of the Dijkstra client: a
direct-address table for Dictionary.spec (one cell per key, one memory unit
per operation) and compressed adjacency arrays for AdjacencyList.spec (the
input layout of a word RAM; at most three memory units per query). With the
binary heap they form one Memory.Framed product implementing Dijkstra.spec.

Dijkstra.Memory.run is one memory program for every graph: it reads n, the
source and m from the input convention, allocates the distance table and
the heap past the input, and runs the linked client. run_spec: every memory
operation is admissible, the cells past the input hold the encoded
distances (the output convention of LogLinearSpec), and the run costs at
most 52 (m + 1) (log n + 1) memory units; exists_run_logLinear exports this
as O((m + 1) log n).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* test(Dijkstra): allow #guard in the memory test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

* refactor(Algorithms/Lean/Dijkstra): state the allocated structures separately

inv_input: after allocation past the input, the heap, the distance table and
the input's adjacency arrays represent Dijkstra's initial state. run_triple
uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants