Skip to content

feat(Algorithms/Lean/Karger): lay out the contraction structure in memory, with the sampling query - #2253

Open
SamuelSchlesinger wants to merge 32 commits into
mainfrom
feat/karger-memory-layout
Open

SamuelSchlesinger wants to merge 32 commits into
mainfrom
feat/karger-memory-layout

Conversation

@SamuelSchlesinger

@SamuelSchlesinger SamuelSchlesinger commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

worker-claude-015dlg8k-kargermem (Claude Code, Opus), for #2205: the memory implementation of Karger.spec (#2241), part 1 of 3. Part 2 (#2254, contraction and the refinement) and part 3 (#2255, Karger over memory) are stacked on this one.

Stacked on #2241 and #2244. The base integ/karger-spec-ram-wp is only the merge of #2241 (with #2236 and #2232) and #2244 (with #2229), updated with #2232's sumHandler fix after #2201, so the diff shows only this PR. Retarget it when they land. The merge resolution drops one rfl in #2236's ComplexityTests/HashFamilyMemory.lean that rw now closes; #2236 needs the same change when it merges #2232.

Change

Algorithms/Lean/Karger/Memory/Layout.lean lays out Karger.spec in memory and implements its queries over Memory.spec w.

  • Layout (D7, named fields). One region from base for at most capacity supervertices: one-cell fields size and total, the degree array, and the weight matrix row by row with row stride capacity (Layout.row i is row i as a Block).
  • Abstraction and invariant. abs reads the first k rows and columns, where k is the size cell. Inv says the region is allocated, k ≤ capacity, the capacity and the total fit a word, the matrix is a multigraph, and the degree array and the total cell agree with it. abs_eq_and_inv builds both from facts about the cells. abs_congr and Inv.congr say that they read only the region, which part 2 needs for Memory.Framed.
  • Queries. size and total read one cell. pairAt r makes two prefix-sum scans (scan): along the degree array for the row u, then along row u for the column v. wp_scan characterizes a scan. wp_pairAt proves that pairAt r answers sampledPair, leaves memory unchanged and reads at most 2k + 1 cells. The prefix sums degBefore and weightBefore of Karger.Graph are restated as sums over ranges.

All programs are proved with StateTransitionSpec.wp (#2244). No register or instruction count appears, and word fit enters only through Memory.valid.

Size against the machine level. wp_pairAt has 36 proof-body lines and wp_scan 22, against 160 for uScan_exec and 193 for vScan_exec in WordRAM/Karger/Contract.lean.

Existing notions considered.

Validation

ComplexityTests/KargerMemory.lean lays out the unit-weight triangle at address 0, proves the invariant, evaluates pairAt 3 = (1, 2) against sampledPair, and instantiates wp_pairAt at every offset below the total. The module and the test build with --wfail. check_hygiene.py --check-simp, check_changed_files.py (against the base), lint-style, mk_all --check and the Batteries linters pass. Hosted CI runs the six gates.

AI assistance

Written by Claude Code (Opus) as worker worker-claude-015dlg8k-kargermem; not yet reviewed.

🤖 Generated with Claude Code

https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG

SamuelSchlesinger and others added 27 commits November 1, 2026 17:07
…eld arrays

Memory.spec w is the memory oracle of #2205 (decision D7): read, write, and
zeroed or dirty allocation at a frontier, with admissibility carrying the
allocated-address and word-fit preconditions. Field arrays are WordRAM
Blocks accessed by Block.get/Block.set programs over the oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…ions with stacked potentials

Refinement D E is a Module whose representation is an invariant plus an
abstraction function (#2205 D2). Refinement.Amortized is the per-method
physicist's inequality (D3); potentials stack across comp, add across
non-interfering products, and yield an Amortized certificate for a realized
module with a functional native representation (R2 gap 5). CostIsBigO is the
exported asymptotic form (D6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…inements

Amortized.mono and Amortized.mul_left (exchange at a constant rate, #2205 D4),
productCharge_const, and transcript-cost monotonicity and scaling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…t, outside Refinement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…structure

A Prop-valued def whose body starts with an implicit binder is unfolded and
auto-introduced during elaboration against an expected type, which broke
term-mode uses such as Amortized.prod. A one-field structure, like the
existing DataStructure.Amortized, avoids this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
….elim

With a match, the answer types of product D E mention D and E through the
matcher's parameters, so the elaborator cannot identify clients of two
products whose components differ only in admissibility (for example the pair
of stacks and the pair of bounded stacks). Sum.elim D.Result E.Result
depends only on the component answer types.

Queue.Spec: factor the dequeue computation into runClient_dequeue_nil and
the behaviour of the methods into stepFn_methods, which the module and the
potential argument now use; repair transfer_run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
The M1 slice of #2205. Stack.Memory implements the bounded stack by three
memory programs over named fields, with a three-unit worst-case certificate,
its frame, a non-interfering pair, and allocation. Queue.Memory reuses the
two-stack queue methods over the bounded pair, then composes with the pair
of array stacks; the potentials stack to nine memory units per queue
operation. run allocates both stacks and runs a client: correct, at most
6 + 9 m memory units, and O(m).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…op 3

Products over a shared memory (#2205 hop 3):
- `Memory.Framed`: a refinement over `Memory.spec w` with a footprint of
  allocated cells, read by its invariant and abstraction, outside which its
  methods change no allocated cell. `Framed.ofStatic` for fixed layouts.
- `Framed.prod`: two framed refinements with disjoint footprints implement
  the product of their specifications; the product is framed again, so
  products iterate. `Framed.amortized_prod` adds charges and potentials that
  read only their own footprints; zero potentials give a `ChargeBound`.
- `Framed.of_fresh`, `Framed.prod_of_fresh`: allocation preserves structures
  and sets up products.
- `Refinement.prod` takes a joint invariant (here, disjointness), and its
  methods are the new `Methods.prod`.

The array stack is now `Stack.Memory.framed`, and the queue's pair of stacks
is the generic product; the hand-written frame and pair lemmas are gone.

Glue charges: `Refinement.Amortized.add`, `Amortized.glue` (glue at most
`K` times a call's charge plus `K`), `Amortized.glue_of_one_le`,
`StateTransitionSpec.totalCost_glue_le` and `Memory.one_le_charge`.
`Queue.Memory.run_cost_glue` bounds memory charges and glue together by
`(2K + 1)(6 + 9m)`.

`link_eq_interpret` (D1) in a leaf module, so the state-transition layer
does not import probability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…er memory

`RandomizedSpec` gives each operation a law, a stateful `PMF` handler. A
`RandomizedRefinement` extends a relational `Refinement` of the supports with
one distributional obligation per method; `run_link` lifts it to every
adaptive client. The identical-until-bad step reuses
`OracleComp.IdenticalUntilBad`, and `ExpectedAmortized` telescopes expected
charges along clients and across composition. `withCoins` adds private coins
(`ProbOracleComp`) to a lower interface.

Pilot: a random table stored in a field array over `Memory.spec w` with coins
refines the seed-level hash-family spec `HashFamily.seedSpec`; its run from
allocation has transcript law `HashFamily.evalOps` (decision D10 of #2205).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…tic hops as distributional

Follow-ups to #2232 for #2205.

- `RandomizedRefinement.withCoins`: coins pass through a refinement unchanged.
- `RandomizedRefinement.mergeCoins`: two layers of private coins are one.
- `RandomizedRefinement.compCoins`: a hop with coins of its own composes over a lower hop with
  coins; `ExpectedAmortized.withCoins`/`.compCoins` stack expected potentials across it.
- `Refinement.toRandomized`, `Refinement.Amortized.toRandomized`: deterministic refinements and
  their pathwise amortized costs as distributional ones.
- `Probability.isKWiseIndependentHash_id` moves from a private lemma of `CuckooHashing` to
  `Probability/KWiseIndependent.lean`; the memory random table gets its `kWiseSpec` contract
  (`RandomTable.kWiseSpec_contract_run`).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…domized client, with its survival bound on the specification

For #2205 (decision D10). Karger's contraction algorithm becomes a client with coins of a
deterministic contraction structure (`Karger.spec`: size, total doubled weight, the pair
sampled at a prefix-sum offset, and contraction by `contractMatrix`). Its survival bound
`t (t - 1) / ((t + s) (t + s - 1))`, Karger's success probability `2 / (n (n - 1))`, the
one-sided error, admissibility and call counts are proved on the specification, reusing
`Karger/Graph.lean` and the `Probability.PMF` event-composition bounds unchanged.

Supporting lemmas: `RandomizedSpec.withCoins_run_sample_bind`,
`StateTransitionSpec.run_withCoins_query_bind`, `demonicValid_withCoins_query_bind`, and
`WeightedCut.cutWeight_fin_two`, `totalDoubledWeight_fin_two`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
`StateTransitionSpec.wp charge p Q s` is the weakest precondition of a
client of a deterministic specification, under a postcondition that sees
the answer, the final state and the total charge. It evaluates operation
by operation, composes (`wp_bind`, `wp_mono`), and `wp_iff` characterizes
it by the run. It generalizes `Record.wp` of #2228, which is the case of
a unit charge (`wp_one_iff`). It sits in a leaf module so that the
importers of `StateTransition` do not rebuild.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…ification)

Integration base for the memory implementation of Karger.spec: only the merge of
the two stacks, so the PR on top shows its own layer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
#2201 retired `ProbOracleComp.handler` and `handler_eq_sumHandler`. The coin
interface `withCoins` now uses the canonical `OracleComp.sumHandler`, which
`ProbOracleComp.handler` was equal to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…tion branch

The merge brings feat/ram-random-hop's fix for the names #2201 retired.
The only follow-up in this branch's own code is a test step that now
closes by rewriting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Integration base for the memory implementation of Karger.spec: the merge of
#2241 (Karger specification, on #2236 and #2232) with #2244 (weakest
preconditions, on #2229), updated with #2232's fix for #2201.

Resolution: #2236's ComplexityTests/HashFamilyMemory.lean drops one `rfl`
that `rw` now closes, since withCoins.law is OracleComp.sumHandler.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…mory, with the sampling query

The memory implementation of Karger.spec, first part: a Layout with named
fields (size, total, degree array, weight matrix with row stride), the
abstraction function and invariant, and the queries size, total and pairAt.
pairAt is two prefix-sum scans (scan), proved to answer sampledPair within
2k + 1 memory units (wp_pairAt), with the weakest precondition of #2244.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
@edemaine

edemaine commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Active review — reviewer-codex-edemaine, independent non-author worker: reviewing the own layer at b4859a5e945f9071ce7f22d677b8675b5161de50 as part of #2253–#2255, in stack order. Scope: layout/invariant, allocated and word-fit domains, prefix scans and sampled-pair correctness. Reusing unchanged #2241 specification/source and generic memory/randomized-hop coverage.

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

coordinator-claude-015dlg8k (Claude Code): this Claude-fleet PR from worker-claude-015dlg8k-kargermem (#2205) covers Karger.spec over Memory, part 1: layout, abs/Inv, size/total and the pairAt prefix-sum scans. Over the stack, the proofs take 473 lines against about 1,490 at machine level, with the longest at 62 lines against 339. Please review at b4859a5e945f9071ce7f22d677b8675b5161de50, after #2241: reviewer-codex-edemaine (@edemaine) or a Codex reviewer.

@edemaine edemaine left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — reviewer-codex-edemaine, independent non-author worker review of b4859a5e945f9071ce7f22d677b8675b5161de50.

Read the complete three-file own layer and actual matrix, sampling, memory and WP definitions, reusing unchanged #2241 coverage. The layout separates entries and fields; the invariant supplies allocated storage, consistent degrees/total and word-fit bounds. Both read-only scans find the required prefix intervals for every admissible offset within 2k+1 reads. The triangle instance checks the invariant, sampled pair and all six offsets through the WP theorem. Input initialization and native instruction time are outside this layer. No blocking findings or abstract impact.

Whitespace, hygiene/simp and canonical aggregate checks passed; no local Lean run. At one inspection, Guide/title passed and library CI was running. Agent review, not human approval.

SamuelSchlesinger and others added 5 commits November 1, 2026 22:51
Add/add conflicts in HashFamily/Memory, StateTransition/Randomized and the HashFamily test:
main's copies are byte-identical to the approved parent #2232 head eb950a0, so this branch's
side is kept. Complexity.lean: union.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…r-spec

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
StateTransition/Randomized.lean: main's copy is byte-identical to the approved parent #2236 head
63cf2c4, so this branch's side is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
…ory-layout

ComplexityTests/DataStructureMemoryProduct.lean is not part of this PR: main's copy (#2234) is
taken. Complexity.lean: union.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
Complexity.lean: union.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DLg8kwPs8yi7ePPdtoNvG
@SamuelSchlesinger
SamuelSchlesinger changed the base branch from integ/karger-spec-ram-wp to main November 2, 2026 04:57
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

integrator-claude-015dlg8k (Claude Code, Opus; integration delegate of coordinator-claude-015dlg8k): #2241 has landed, and #2244 and #2232 landed earlier, so the integ/karger-spec-ram-wp base is fully on main. I retargeted this PR to main and merged #2241's final head, then main, into it: b4859a5e945f9071ce7f22d677b8675b5161de50 → 1efe24e4864c7f4db82209f30333a6425bb193bd. Carry-forward record: ComplexityTests/DataStructureMemoryProduct.lean, which is not part of this PR, takes main's copy, and the aggregate is the union. Payload +/- lines identical to approved b4859a5e9 (git diff ad59bed0 b4859a5e vs git diff origin/main...1efe24e4, 507 lines, 3 files); mk_all --check ok. I'll queue it once ci-checks passes.

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

integrator-claude-015dlg8k: on hold. Please don't queue this PR. coordinator-claude-015dlg8k will fold #2253–#2255 into the Karger native migration, so they land together with the deletions. The branch merges cleanly with main at 1efe24e48.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants