From 874872f77d1d5c4cb709661a259dedb0637d338b Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 15:59:05 +0000 Subject: [PATCH 01/18] feat(stateful,word-ram): add relational specifications and subroutine certificates --- Complexity.lean | 2 + .../Computability/WordRAM/RelSubroutine.lean | 1008 ++++++++++++++ .../Semantics/Stateful/Relational.lean | 1181 +++++++++++++++++ ComplexityTests/StatefulRelational.lean | 241 ++++ ComplexityTests/WordRAMRelSubroutine.lean | 351 +++++ 5 files changed, 2783 insertions(+) create mode 100644 Complexity/Computability/WordRAM/RelSubroutine.lean create mode 100644 Complexity/Foundations/Semantics/Stateful/Relational.lean create mode 100644 ComplexityTests/StatefulRelational.lean create mode 100644 ComplexityTests/WordRAMRelSubroutine.lean diff --git a/Complexity.lean b/Complexity.lean index a53d95e8e..5c5dbb976 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -2014,6 +2014,7 @@ public import Complexity.Computability.WordRAM.Randomized.Expected public import Complexity.Computability.WordRAM.Randomized.LoopSegment public import Complexity.Computability.WordRAM.Randomized.Structured public import Complexity.Computability.WordRAM.RegisterFrame +public import Complexity.Computability.WordRAM.RelSubroutine public import Complexity.Computability.WordRAM.Sequence public import Complexity.Computability.WordRAM.Signed.Arithmetic public import Complexity.Computability.WordRAM.Signed.Comparison @@ -2634,6 +2635,7 @@ public import Complexity.Foundations.Semantics.Stateful.CostHoare public import Complexity.Foundations.Semantics.Stateful.Hoare public import Complexity.Foundations.Semantics.Stateful.Module public import Complexity.Foundations.Semantics.Stateful.Realization +public import Complexity.Foundations.Semantics.Stateful.Relational public import Complexity.Foundations.Semantics.Stateful.Resource public import Complexity.Init public import Complexity.Languages.Probabilistic.Basic diff --git a/Complexity/Computability/WordRAM/RelSubroutine.lean b/Complexity/Computability/WordRAM/RelSubroutine.lean new file mode 100644 index 000000000..9b92e4dee --- /dev/null +++ b/Complexity/Computability/WordRAM/RelSubroutine.lean @@ -0,0 +1,1008 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Computability.WordRAM.Subroutine +public import Complexity.Foundations.Semantics.Stateful.Relational + +/-! +# Relational Word-RAM dynamic programs and composable data-structure subroutines + +This file extends `Complexity.Computability.WordRAM.Dynamic` and +`Complexity.Computability.WordRAM.Subroutine` from deterministic specifications (`Stateful.Spec`) +to relational specifications (`Stateful.RelSpec`): + +1. **Standalone relational fixed-program implementations (`RelDynamicProgram`, + `RelDynamicCertificate`):** Generalizes `DynamicProgram` and `DynamicCertificate` to + `D : Stateful.RelSpec`, where `execute` witnesses an angelic outcome `(s', a)` satisfying + `D.step s o s' a`. Because the Word-RAM structured semantics `Exec` is deterministic, + `RelDynamicCertificate.realization` establishes `Stateful.RelRealization` directly via + `Exec.unique`. +2. **Unified standalone and inline relational subroutine certificates (`RelSubroutineDS`):** + Mirrors `SubroutineDS` over `D : Stateful.RelSpec`, omitting `rep_unique` (since concrete + potential `potential : Data r_ds → ℕ` depends only on machine data) and existentially + quantifying over `(s', a)` with `D.step s o s' a` in `step_exec`. +3. **Zero-cost embedding of deterministic subroutines (`SubroutineDS.toRelSubroutineDS`):** + Every `SubroutineDS P D w base params s₀` embeds into + `RelSubroutineDS P D.toRelSpec w base params s₀`. +4. **Basis lifting, standalone `Stateful` projections, inline subroutine calls, and flat-program + operation sequences:** Provides `mapSimulates`, `toStandard`, `toRelDynamicProgram`, + `toRelCertificate`, `toRelAmortized`, `toRelBound`, `run_client_amortized`, `run_client_bound`, + `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, + `initClear_rename`, `code_rename`, `call_rename`, `frame_caller`, and flat-program sequence + execution (`RunsRelOps`, `seq_runsRelOps`, `run_ops_exact_fresh`, `run_ops_exact_reusable`). + +## Main definitions + +* `Cslib.WordRAM.RelDynamicProgram`, `Cslib.WordRAM.RelDynamicCertificate`: standalone fixed-program + realization and certificate for a relational specification `D : Stateful.RelSpec`. +* `Cslib.WordRAM.RelSubroutineDS`: unified standalone and inline subroutine specification and + amortized resource certificate for `P : SubroutineCode B r_ds k_init` realizing + `D : Stateful.RelSpec` (without `rep_unique`). +* `Cslib.WordRAM.SubroutineDS.toRelSubroutineDS`: embedding of a deterministic `SubroutineDS` into + `RelSubroutineDS`. +* `Cslib.WordRAM.RelSubroutineDS.RunsRelOps`: sequential flat-program execution of an operation list + recording the resulting abstract transcript `D.Transcript`. + +## Main results + +* `Cslib.WordRAM.RelDynamicCertificate.realization`, + `Cslib.WordRAM.RelDynamicCertificate.instruction_bound`: standalone `Stateful.RelRealization` + and `Stateful.RelBound` certificates. +* `Cslib.WordRAM.RelSubroutineDS.toRelAmortized`, + `Cslib.WordRAM.RelSubroutineDS.run_client_amortized`: standalone amortized potential certificate + and adaptive-client bound over `Basis.standard`. +* `Cslib.WordRAM.RelSubroutineDS.initFresh_rename`, + `Cslib.WordRAM.RelSubroutineDS.initClear_rename`, + `Cslib.WordRAM.RelSubroutineDS.call_rename`, + `Cslib.WordRAM.RelSubroutineDS.frame_caller`: relocated inline initialization, inline operation + calls, and bidirectional framing. +* `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_fresh`, + `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_reusable`: end-to-end flat-program execution of any + demonically admissible operation sequence `D.ValidOpSeq s₀ ops`. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Chapter 17][CLRS2009] +* [J. C. Reynolds, *Separation Logic: A Logic for Shared Mutable Data Structures*][Reynolds2002] +-/ + +@[expose] public section + +namespace Cslib.WordRAM + +universe u v z + +/-! ## Standalone relational dynamic programs (`RelDynamicProgram`, `RelDynamicCertificate`) -/ + +variable {r : ℕ} {D : Stateful.RelSpec.{u, v, z}} + +/-- A single `.standard`-basis Word-RAM program serving all operations of a relational +specification `D : Stateful.RelSpec` through a fixed register calling convention. -/ +structure RelDynamicProgram (D : Stateful.RelSpec.{u, v, z}) (r : ℕ) where + /-- The same structured program is executed for every operation. -/ + code : Code .standard r + /-- Register returned by the final `halt` instruction. -/ + output : Fin r + /-- Registers overwritten by externally supplied operation arguments. -/ + input : Fin r → Bool + /-- Encoded arguments, independent of the maintained machine state. -/ + encode : D.Op → Fin r → ℕ + /-- Interpretation of the final register tuple as the operation's typed observation. -/ + decode : (o : D.Op) → (Fin r → ℕ) → Option (D.Result o) + +/-- Embed a deterministic `DynamicProgram D r` as a `RelDynamicProgram D.toRelSpec r`. -/ +abbrev DynamicProgram.toRelDynamicProgram {D : Stateful.Spec.{u, v, z}} {r : ℕ} + (P : DynamicProgram D r) : RelDynamicProgram D.toRelSpec r where + code := P.code + output := P.output + input := P.input + encode := P.encode + decode := P.decode + +namespace RelDynamicProgram + +variable (P : RelDynamicProgram D r) + +/-- Deliver operation arguments without changing maintained memory or non-input registers. -/ +def prepare (o : D.Op) (x : Data r) : Data r where + regs i := if P.input i then P.encode o i else x.regs i + mem := x.mem + +/-- Delivering operation arguments preserves the machine memory unchanged. -/ +@[simp] +theorem prepare_mem (o : D.Op) (x : Data r) : (P.prepare o x).mem = x.mem := rfl + +/-- Registers outside the calling convention retain their contents. -/ +theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : D.Op) (x : Data r) : + (P.prepare o x).regs i = x.regs i := by + simp [prepare, hi] + +/-- Argument delivery preserves bounded words when the arguments fit. -/ +theorem prepare_bounded {w : ℕ} {o : D.Op} {x : Data r} (hx : x.Bounded w) + (ho : ∀ i, P.input i = true → Fits w (P.encode o i)) : (P.prepare o x).Bounded w := by + refine ⟨fun i => ?_, hx.2⟩ + change Fits w (if P.input i then P.encode o i else x.regs i) + split + · exact ho i (by assumption) + · exact hx.1 i + +/-- Evaluate one typed call, delivering its arguments and decoding its observation. The returned +cost includes the final `halt`. -/ +def eval (w fuel : ℕ) (o : D.Op) (x : Data r) : Option (Data r × D.Result o × ℕ) := + match P.code.eval w fuel (P.prepare o x) with + | none => none + | some (y, n) => (P.decode o y.regs).map fun a => (y, a, n + 1) + +/-- A bounded machine call with native execution evidence (`DynamicProgram.OperationTrace`). -/ +def ExecOp (w : ℕ) : Stateful.RelExecution D (Data r) DynamicProgram.OperationTrace := + fun x o a y trace => + (P.prepare o x).Bounded w ∧ ∃ n, TraceExec w P.code (P.prepare o x) y n trace.probes ∧ + P.decode o y.regs = some a ∧ trace.instructions = n + 1 + +/-- Successful typed evaluation supplies an exact operation trace when the delivered state is +bounded. -/ +theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result o} + (hb : (P.prepare o x).Bounded w) (h : P.eval w fuel o x = some (y, a, k)) : + ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := by + cases he : P.code.eval w fuel (P.prepare o x) with + | none => simp [eval, he] at h + | some yn => + obtain ⟨z, n⟩ := yn + cases hd : P.decode o z.regs with + | none => simp [eval, he, hd] at h + | some b => + simp only [eval, he, hd, Option.map_some, Option.some.injEq, Prod.mk.injEq] at h + obtain ⟨rfl, rfl, rfl⟩ := h + obtain ⟨probes, hp⟩ := (Exec.of_eval he).exists_trace + exact ⟨probes, hb, n, hp, hd, rfl⟩ + +/-- Every call is an execution of one fixed flat machine program. -/ +theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : + Executes w (P.code.program P.output) ⟨0, P.prepare o x⟩ (y.regs P.output) + trace.instructions := by + obtain ⟨_, n, he, _, ht⟩ := h + rw [ht] + exact he.exec.executes P.output + +/-- A call preserves bounded words. -/ +theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : y.Bounded w := by + obtain ⟨hb, _, he, _, _⟩ := h + exact he.exec.bounded hb + +/-- Calls are deterministic in their final representation, observation, and instruction count. -/ +theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} + {trace trace' : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) + (h' : P.ExecOp w x o a' y' trace') : + y = y' ∧ a = a' ∧ trace.instructions = trace'.instructions := by + obtain ⟨_, _, he, ha, ht⟩ := h + obtain ⟨_, _, he', ha', ht'⟩ := h' + obtain ⟨rfl, rfl⟩ := he.exec.unique he'.exec + exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ + +/-- A call's final state and instruction count agree with any structured execution of the code. -/ +theorem ExecOp.eq_of_exec {P : RelDynamicProgram D r} {w : ℕ} {x y z : Data r} {o : D.Op} + {a : D.Result o} {trace : DynamicProgram.OperationTrace} {m : ℕ} + (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : + y = z ∧ trace.instructions = m + 1 := by + obtain ⟨_, _, ht, _, hn⟩ := he + obtain ⟨rfl, rfl⟩ := ht.exec.unique h + exact ⟨rfl, hn⟩ + +/-- The write transcript supplies a frame rule for a call, including register-only delivery. -/ +theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) {Q : HeapAssertion} + (hQ : Q.holds x.mem) (hd : Disjoint Q.footprint (Probe.writes trace.probes)) : + Q.holds y.mem := by + obtain ⟨_, _, he, _, _⟩ := h + exact he.frame hQ hd + +end RelDynamicProgram + +/-- Per-operation angelic machine proof obligations for a `RelDynamicProgram`. -/ +structure RelDynamicCertificate (P : RelDynamicProgram D r) (w : ℕ) where + /-- Abstract state represented by the persistent machine data. -/ + rep : D.State → Data r → Prop + /-- Represented configurations contain only machine words. -/ + bounded : ∀ {s x}, rep s x → x.Bounded w + /-- Admissible arguments fit the machine's word size. -/ + arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (P.encode o i) + /-- A machine execution implements an admissible operation by realizing some permitted + `(s', a)` with `D.step s o s' a`. -/ + execute : ∀ {s x} (o), rep s x → D.valid s o → + ∃ y s' a n, Exec w P.code (P.prepare o x) y n ∧ + D.step s o s' a ∧ P.decode o y.regs = some a ∧ rep s' y + +/-- Embed a deterministic `DynamicCertificate` as a `RelDynamicCertificate`. -/ +def DynamicCertificate.toRelDynamicCertificate {D : Stateful.Spec.{u, v, z}} {r w : ℕ} + {P : DynamicProgram D r} (C : DynamicCertificate P w) : + RelDynamicCertificate P.toRelDynamicProgram w where + rep := C.rep + bounded := C.bounded + arguments_fit := C.arguments_fit + execute o hr hv := by + obtain ⟨y, n, he, hdec, hrep⟩ := C.execute o hr hv + exact ⟨y, (D.step _ o).1, (D.step _ o).2, n, he, rfl, hdec, hrep⟩ + +namespace RelDynamicCertificate + +variable {P : RelDynamicProgram D r} {w : ℕ} (C : RelDynamicCertificate P w) + +/-- Turn a successful typed evaluation on a represented state and admissible operation into +certified `ExecOp` evidence. -/ +theorem execOp_of_eval {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} {fuel k : ℕ} + (hr : C.rep s x) (hv : D.valid s o) (h : P.eval w fuel o x = some (y, a, k)) : + ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := + RelDynamicProgram.ExecOp.of_eval P + (P.prepare_bounded (C.bounded hr) (C.arguments_fit o hv)) h + +/-- Fixed-program relational certificates realize `D` via `Stateful.RelRealization`. -/ +def realization : Stateful.RelRealization D (P.ExecOp w) where + rep := C.rep + total o hr hv := by + obtain ⟨y, _, a, n, he, _, ha, _⟩ := C.execute o hr hv + obtain ⟨ts, hts⟩ := he.exists_trace + exact ⟨a, y, ⟨n + 1, ts⟩, + P.prepare_bounded (C.bounded hr) (C.arguments_fit o hv), n, hts, ha, rfl⟩ + correct hr hv he := by + obtain ⟨y, s', a, n, he', hstep, ha, hr'⟩ := C.execute _ hr hv + obtain ⟨_, _, ht, ha', _⟩ := he + obtain ⟨rfl, rfl⟩ := ht.exec.unique he' + have h_eq : _ = a := Option.some.inj (ha'.symm.trans ha) + subst h_eq + exact ⟨s', hstep, hr'⟩ + +/-- A separate per-operation instruction estimate supplies a reusable `Stateful.RelBound`. -/ +theorem instruction_bound (charge : D.State → D.Op → ℕ) + (hbound : ∀ {s x} (o) {y n}, C.rep s x → D.valid s o → + Exec w P.code (P.prepare o x) y n → n + 1 ≤ charge s o) : + Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions charge := by + intro s x o a y trace hr hv he + obtain ⟨_, n, ht, _, hn⟩ := he + rw [hn] + exact hbound o hr hv ht.exec + +/-- A uniform constant instruction bound follows from an existential per-operation execution. -/ +theorem instruction_bound_const (K : ℕ) + (hbound : ∀ {s x} (o), C.rep s x → D.valid s o → + ∃ y n, Exec w P.code (P.prepare o x) y n ∧ n + 1 ≤ K) : + Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions + (fun _ _ => K) := by + apply C.instruction_bound + intro s x o y n hr hv he + obtain ⟨_, _, he', hn⟩ := hbound o hr hv + obtain ⟨_, rfl⟩ := he.unique he' + exact hn + +/-- Adaptive client execution, final representation, and linear total instruction bound for a +uniform constant per-operation instruction certificate. -/ +theorem run_client_const {K : ℕ} + (B : Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions + (fun _ _ => K)) + {α : Type*} (p : D.Client α) {s : D.State} {x : Data r} + (hr : C.rep s x) (hv : D.DemonicValid s p) : + ∃ a y traces s' tr, + Stateful.RelExecutes (P.ExecOp w) p x a y traces ∧ + D.ClientRun s p a s' tr ∧ + C.rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ tr.length * K := by + obtain ⟨a, y, traces, s', tr, he, hrun, hr', hlen, hcost⟩ := B.exists_executes p hr hv + rw [D.totalCost_const] at hcost + exact ⟨a, y, traces, s', tr, he, hrun, hr', hlen, hcost⟩ + +/-- Lift a per-operation pointwise memory frame from `Exec` to `ExecOp`. -/ +theorem frame_of_execute {j : ℕ} + (hframe : ∀ {s x} (o), C.rep s x → D.valid s o → + ∃ z n, Exec w P.code (P.prepare o x) z n ∧ z.mem j = x.mem j) + {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.valid s o) (he : P.ExecOp w x o a y trace) : + y.mem j = x.mem j := by + obtain ⟨z, n, hz, hf⟩ := hframe o hr hv + exact (he.eq_of_exec hz).1 ▸ hf + +/-- Lift a per-operation `Modifies` bound from `Exec` to `ExecOp`. -/ +theorem modifies_of_execute {F : Finset ℕ} + (hmod : ∀ {s x} (o), C.rep s x → D.valid s o → + ∃ z n, Exec w P.code (P.prepare o x) z n ∧ Modifies F x.mem z.mem) + {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.valid s o) (he : P.ExecOp w x o a y trace) : + Modifies F x.mem y.mem := + fun j hj => C.frame_of_execute + (fun o hr' hv' => let ⟨z, n, hz, hf⟩ := hmod o hr' hv'; ⟨z, n, hz, hf j hj⟩) + hr hv he + +/-- Lift a per-operation pointwise memory frame across a demonically admissible adaptive client +execution. -/ +theorem executes_client_frame {j : ℕ} + (hframe : ∀ {s x y o a trace}, C.rep s x → D.valid s o → + P.ExecOp w x o a y trace → y.mem j = x.mem j) + {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} + {traces : List DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.DemonicValid s p) + (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + y.mem j = x.mem j := by + induction he generalizing s with + | pure _ _ => rfl + | @call o k c a c' trace _ _ _ hstep _ ih => + obtain ⟨hv_o, hv_k⟩ := (D.demonicValid_liftBind_iff s o k).mp hv + obtain ⟨s', hstep_d, hr'⟩ := C.realization.correct hr hv_o hstep + exact (ih hr' (hv_k s' a hstep_d)).trans (hframe hr hv_o hstep) + +/-- Lift a per-operation `Modifies` bound across a demonically admissible adaptive client +execution. -/ +theorem executes_client_modifies {F : Finset ℕ} + (hmod : ∀ {s x y o a trace}, C.rep s x → D.valid s o → + P.ExecOp w x o a y trace → Modifies F x.mem y.mem) + {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} + {traces : List DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.DemonicValid s p) + (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + Modifies F x.mem y.mem := + fun j hj => C.executes_client_frame + (fun hr' hv' he' => hmod hr' hv' he' j hj) hr hv he + +/-- Memory probes are bounded by the instruction charge. -/ +theorem probe_bound (charge : D.State → D.Op → ℕ) + (B : Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions charge) : + Stateful.RelBound C.realization (fun t => t.probes.length) charge := by + intro s x o a y trace hr hv he + have hb := B hr hv he + obtain ⟨_, n, ht, _, hn⟩ := he + exact ht.length_le.trans ((Nat.le_add_right n 1).trans (hn ▸ hb)) + +end RelDynamicCertificate + +/-! ## `RelSubroutineDS` and `SubroutineDS.toRelSubroutineDS` -/ + +namespace SubroutineCode + +variable {B : Basis} {r_ds k_init : ℕ} + +/-- Compiled flat `Program B r_ds` for `O(1)` fresh zeroed-memory initialization. -/ +def initFreshProg (P : SubroutineCode B r_ds k_init) : Program B r_ds := + P.initFresh.program P.output + +/-- Compiled flat `Program B r_ds` for `O(space)` dirty-memory reusable initialization. -/ +def initClearProg (P : SubroutineCode B r_ds k_init) : Program B r_ds := + P.initClear.program P.output + +/-- Compiled flat `Program B r_ds` serving all operations of the data structure. -/ +def opProg (P : SubroutineCode B r_ds k_init) : Program B r_ds := + P.code.program P.output + +end SubroutineCode + +/-- Unified standalone and subroutine specification/resource certificate for +`P : SubroutineCode B r_ds k_init` realizing a relational specification `D : Stateful.RelSpec` +at word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. + +In contrast to `SubroutineDS`, `rep_unique` is omitted (because concrete potential +`potential : Data r_ds → ℕ` depends only on machine data) and `step_exec` existentially +quantifies over a permitted outcome `(s', a)` satisfying `D.step s o s' a`. -/ +structure RelSubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) + (D : Stateful.RelSpec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) where + /-- Contiguous memory footprint length `[base, base + space)`. -/ + space : ℕ + /-- Instruction bound for `P.allocSpan`. -/ + spanTime : ℕ + /-- Instruction bound for `P.initSetup`. -/ + setupTime : ℕ + /-- Upper bound on initial concrete potential after initialization. -/ + initPotential : ℕ + /-- Amortized instruction charge per operation (excluding trailing standalone `halt`). -/ + charge : D.State → D.Op → ℕ + /-- Representation invariant linking mathematical state `s` to concrete `Data r_ds`. -/ + rep : D.State → Data r_ds → Prop + /-- Concrete integer potential depending only on `Data r_ds`. -/ + potential : Data r_ds → ℕ + /-- Standalone register encoding for `RelDynamicProgram.prepare`. -/ + encode : D.Op → Fin r_ds → ℕ + /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ + argMatch : D.Op → (Fin r_ds → ℕ) → Prop + /-- Output decoder reading typed operation observations from final registers. -/ + decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) + /-- Represented configurations are word-bounded. -/ + bounded : ∀ {s x}, rep s x → x.Bounded w + /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ + arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) + /-- Full standalone `prepare` delivery satisfies `argMatch`. -/ + encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), + argMatch o (fun i => if P.input i then encode o i else regs i) + /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. + -/ + rep_supported : ∀ {s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + x'.Bounded w → rep s x → rep s x' + /-- `potential` depends only on persistent registers and `[base, base + space)` under `rep`. -/ + potential_supported : ∀ {s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + rep s x → potential x' = potential x + /-- Distinctness of registers used by `zeroFillCode` relative to `baseReg` and `frontierReg`. -/ + clear_nodup : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup + /-- Initialization parameter registers are not overwritten by `zeroFillCode`. -/ + params_not_clear : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg] + /-- `P.incCur` increments `P.curReg` by `1` when `P.oneReg = 1` and the sum fits in `w` bits. -/ + incCur_eval : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → + P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1)) + /-- Execution of `P.allocSpan`: records `baseReg = base` and advances `frontierReg`. -/ + span_exec : ∀ {x : Data r_ds}, + x.Bounded w → x.regs P.frontierReg = base → (∀ j, x.regs (P.initParams j) = params j) → + ∃ x₁, ExecWithin w P.allocSpan x x₁ spanTime ∧ x₁.mem = x.mem ∧ + x₁.regs P.baseReg = base ∧ x₁.regs P.frontierReg = base + space ∧ + (∀ j, x₁.regs (P.initParams j) = params j) + /-- Execution of `P.initSetup` once `[base, base + space)` holds zeros. -/ + setup_exec : ∀ {x₁ : Data r_ds}, + x₁.Bounded w → x₁.regs P.baseReg = base → x₁.regs P.frontierReg = base + space → + (∀ j, x₁.regs (P.initParams j) = params j) → + (∀ a ∈ (Block.mk base space).footprint, x₁.mem a = 0) → + ∃ y, ExecWithin w P.initSetup x₁ y setupTime ∧ y.regs P.frontierReg = base + space ∧ + rep s₀ y ∧ potential y ≤ initPotential ∧ + Modifies (Block.mk base space).footprint x₁.mem y.mem + /-- Angelic execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`, + witnessing some permitted outcome `(s', a)` with `D.step s o s' a`. -/ + step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), + rep s x → D.valid s o → argMatch o x.regs → + ∃ y s' a n, Exec w P.code x y n ∧ + D.step s o s' a ∧ + decode o y.regs = some a ∧ + rep s' y ∧ + Modifies (Block.mk base space).footprint x.mem y.mem ∧ + n + potential y ≤ charge s o + potential x + +/-- Embed any deterministic `SubroutineDS P D w base params s₀` as a relational +`RelSubroutineDS P D.toRelSpec w base params s₀`. -/ +def SubroutineDS.toRelSubroutineDS {B : Basis} {r_ds k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + (C : SubroutineDS P D w base params s₀) : + RelSubroutineDS P D.toRelSpec w base params s₀ where + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep := C.rep + potential := C.potential + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec := C.setup_exec + step_exec o hr hv harg := by + obtain ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + exact ⟨y, (D.step _ o).1, (D.step _ o).2, n, he, rfl, hdec, hrep, hmod, hpot⟩ + +namespace RelSubroutineDS + +variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.RelSpec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-- Total instruction bound for `P.initFresh`. -/ +def initTime (C : RelSubroutineDS P D w base params s₀) : ℕ := + C.spanTime + C.setupTime + +/-! ### Projection 1: Basis lifting and standalone `RelDynamicProgram` / `Stateful` theorems -/ + +/-- Lift a `RelSubroutineDS` certificate along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : RelSubroutineDS P D w base params s₀) : + RelSubroutineDS (P.mapSimulates φ) D w base params s₀ where + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep := C.rep + potential := C.potential + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval h1 hfit := Op.eval_mapSimulates_of_some φ (C.incCur_eval h1 hfit) + span_exec hx hf hp := + let ⟨x₁, he, hm, hb, hfr, hpar⟩ := C.span_exec hx hf hp + ⟨x₁, he.mapSimulates φ, hm, hb, hfr, hpar⟩ + setup_exec hx₁ hb hfr hpar hz := + let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz + ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ + step_exec o hr hv harg := + let ⟨y, s', a, n, he, hstep, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + ⟨y, s', a, n, he.mapSimulates φ, hstep, hdec, hrep, hmod, hpot⟩ + +/-- Lift an exact-basis `RelSubroutineDS` certificate to `Basis.standard`. -/ +abbrev toStandard {P : SubroutineCode .exact r_ds k_init} + (C : RelSubroutineDS P D w base params s₀) : + RelSubroutineDS P.toStandard D w base params s₀ := + C.mapSimulates .exactStandard + +section Standard + +variable {P_std : SubroutineCode .standard r_ds k_init} + (C : RelSubroutineDS P_std D w base params s₀) + +/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicProgram`. -/ +def toRelDynamicProgram : RelDynamicProgram D r_ds where + code := P_std.code + output := P_std.output + input := P_std.input + encode := C.encode + decode := C.decode + +/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicCertificate`. -/ +def toRelCertificate : RelDynamicCertificate C.toRelDynamicProgram w where + rep := C.rep + bounded := C.bounded + arguments_fit := C.arguments_fit + execute {s x} o hr hv := by + have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := + C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨y, s', a, n, he, hstep, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg + exact ⟨y, s', a, n, he, hstep, hdec, hrep⟩ + +/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `Stateful.RelAmortized` +certificate, charging `+ 1` per operation for `RelDynamicProgram`'s trailing `halt` instruction. -/ +theorem toRelAmortized : + Stateful.RelAmortized C.toRelCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun _ x => C.potential x) + (fun s o => C.charge s o + 1) where + step {s x o a y trace} hr hv he := by + have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := + C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have hp_eq : C.potential (C.toRelDynamicProgram.prepare o x) = C.potential x := + C.potential_supported + (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hr + have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨z, s', a', m, hz, hstep, hdec, hrep, _, hpot⟩ := C.step_exec o hr' hv harg + obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz + obtain ⟨_, _, _, ha_trace, _⟩ := he + have ha_eq : a = a' := Option.some.inj (ha_trace.symm.trans hdec) + subst ha_eq + refine ⟨s', hstep, hrep, ?_⟩ + rw [hinst] + rw [hp_eq] at hpot + omega + +/-- Alias for `toRelAmortized` matching `SubroutineDS.toAmortized`. -/ +abbrev toAmortized := @toRelAmortized + +/-- When `C.potential` is identically zero, `RelSubroutineDS` projects directly to a worst-case +`Stateful.RelBound` certificate. -/ +theorem toRelBound (hpot : ∀ x, C.potential x = 0) : + Stateful.RelBound C.toRelCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun s o => C.charge s o + 1) := by + intro s x o a y trace hr hv he + obtain ⟨_, _, _, h⟩ := C.toRelAmortized.step hr hv he + simp only [hpot] at h + omega + +/-- Alias for `toRelBound` matching `SubroutineDS.toBound`. -/ +abbrev toBound := @toRelBound + +/-- Standalone adaptive-client amortized bound for any demonically admissible client `p`. -/ +theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} + (hr : C.rep s x) (hv : D.DemonicValid s p) : + ∃ a y traces s' tr, + Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + D.ClientRun s p a s' tr ∧ + C.rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces + C.potential y ≤ + D.totalCost (fun s o => C.charge s o + 1) s tr + C.potential x := + C.toRelAmortized.run_client_amortized p hr hv + +/-- Standalone adaptive-client worst-case bound when `C.potential` is identically zero. -/ +theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) + {s : D.State} {x : Data r_ds} (hr : C.rep s x) (hv : D.DemonicValid s p) : + ∃ a y traces s' tr, + Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + D.ClientRun s p a s' tr ∧ + C.rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ + D.totalCost (fun s o => C.charge s o + 1) s tr := + Stateful.RelBound.exists_executes (C.toRelBound hpot) p hr hv + +end Standard + +/-! ### Projection 2: Fresh `O(1)` initialization and register relocation -/ + +/-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ +theorem initFresh_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ y, ExecWithin w P.initFresh x y C.initTime ∧ + y.regs P.frontierReg = base + C.space ∧ + C.rep s₀ y ∧ + C.potential y ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem y.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + have hx₁ : x₁.Bounded w := he₁.bounded hx + have hz₁ : ∀ a ∈ (Block.mk base C.space).footprint, x₁.mem a = 0 := by + intro a ha + rw [hm₁] + exact hz a ha + obtain ⟨y, he₂, hfr₂, hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb₁ hfr₁ hpar₁ hz₁ + rw [hm₁] at hmod + exact ⟨y, he₁.seq he₂, hfr₂, hrep, hpot, hmod⟩ + +/-- Relocated `O(1)` fresh initialization along any injective register embedding `f`. -/ +theorem initFresh_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ z, ExecWithin w (P.initFresh.rename f) x z C.initTime ∧ + z.regs (f P.frontierReg) = base + C.space ∧ + C.rep s₀ (z.comap f) ∧ + C.potential (z.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x.regs j) := by + obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initFresh_exec hx hfr hpar hz + refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, + fun j hj => Data.extend_regs_of_notMem y x hj⟩ + · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`initClear`) -/ + +/-- Auxiliary deterministic `SubroutineDS` sharing `C`'s initialization fields so +`zeroFillCode_exec` is reused without duplicating its loop proof. -/ +private def toInitSubroutineDS (C : RelSubroutineDS P D w base params s₀) : + SubroutineDS P + { State := PUnit.{1}, Op := PUnit.{1}, Result := fun _ => PUnit.{1}, + step := fun _ _ => (⟨⟩, ⟨⟩), valid := fun _ _ => False } + w base params ⟨⟩ where + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge _ _ := 0 + rep _ x := C.rep s₀ x + potential := C.potential + encode _ _ := 0 + argMatch _ _ := True + decode _ _ := some ⟨⟩ + bounded := C.bounded + rep_unique _ _ := rfl + arguments_fit _ hv := False.elim hv + encode_matches _ _ := trivial + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec := C.setup_exec + step_exec _ _ hv := False.elim hv + +/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` +instructions over any basis `B`. -/ +theorem zeroFillCode_exec (C : RelSubroutineDS P D w base params s₀) {x₁ : Data r_ds} + (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) + (hfr₁ : x₁.regs P.frontierReg = base + C.space) + (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : + ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ + x₂.Bounded w ∧ + x₂.regs P.baseReg = base ∧ + x₂.regs P.frontierReg = base + C.space ∧ + (∀ j, x₂.regs (P.initParams j) = params j) ∧ + (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ + Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := + C.toInitSubroutineDS.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ + +/-- Execute `P.initClear` on arbitrary dirty memory in `C.initTime + 4 * C.space + 6` +instructions. -/ +theorem initClear_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) : + ∃ y, ExecWithin w P.initClear x y (C.initTime + 4 * C.space + 6) ∧ + y.regs P.frontierReg = base + C.space ∧ + C.rep s₀ y ∧ + C.potential y ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem y.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := + C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + obtain ⟨y, he₃, hfr₃, hrep, hpot, hmod₃⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ + rw [hm₁] at hmod₂ + have hmod : Modifies (Block.mk base C.space).footprint x.mem y.mem := by + simpa only [Finset.union_idempotent] using hmod₂.trans hmod₃ + exact ⟨y, (he₁.seq (he₂.seq he₃)).mono (by unfold initTime; omega), hfr₃, hrep, hpot, hmod⟩ + +/-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ +theorem initClear_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : + ∃ z, ExecWithin w (P.initClear.rename f) x z (C.initTime + 4 * C.space + 6) ∧ + z.regs (f P.frontierReg) = base + C.space ∧ + C.rep s₀ (z.comap f) ∧ + C.potential (z.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x.regs j) := by + obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initClear_exec hx hfr hpar + refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, + fun j hj => Data.extend_regs_of_notMem y x hj⟩ + · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Projection 4: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ + +/-- Execute `P.code.rename f` inline when arguments already match `(x.comap f).regs`, +witnessing a permitted outcome `(s', a)` with `D.step s o s' a`. -/ +theorem code_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} {x : Data r_caller} (o : D.Op) + (hr : C.rep s (x.comap f)) (hv : D.valid s o) (harg : C.argMatch o (x.comap f).regs) : + ∃ z s' a t, ExecWithin w (P.code.rename f) x z t ∧ + D.step s o s' a ∧ + C.decode o (z.comap f).regs = some a ∧ + C.rep s' (z.comap f) ∧ + Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x.regs j) ∧ + t + C.potential (z.comap f) ≤ C.charge s o + C.potential (x.comap f) := by + obtain ⟨y, s', a, n, he, hstep, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + refine ⟨y.extend f x, s', a, n, (he.rename hf).within, hstep, ?_, ?_, hmod, + fun j hj => Data.extend_regs_of_notMem y x hj, ?_⟩ + · rw [Data.comap_extend hf] + exact hdec + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-- Execute `.seq prep (P.code.rename f)` inline after a caller argument-preparation block `prep` +that preserves persistent DS registers and memory in `[base, base + C.space)`. -/ +theorem call_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} {x x' : Data r_caller} {prep : Code B r_caller} {t_prep : ℕ} (o : D.Op) + (hr : C.rep s (x.comap f)) (hv : D.valid s o) + (hprep : ExecWithin w prep x x' t_prep) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hmem : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a) + (harg : C.argMatch o (x'.comap f).regs) : + ∃ z s' a t, ExecWithin w (.seq prep (P.code.rename f)) x z t ∧ + D.step s o s' a ∧ + C.decode o (z.comap f).regs = some a ∧ + C.rep s' (z.comap f) ∧ + Modifies (Block.mk base C.space).footprint x'.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x'.regs j) ∧ + t + C.potential (z.comap f) ≤ t_prep + C.charge s o + C.potential (x.comap f) := by + have hr' : C.rep s (x'.comap f) := + C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hmem hb' hr + have hp_eq : C.potential (x'.comap f) = C.potential (x.comap f) := + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hmem hr + obtain ⟨z, s', a, t_op, he_op, hstep, hdec, hrep_z, hmod_z, hregs_z, hpot_z⟩ := + C.code_rename hf o hr' hv harg + exact ⟨z, s', a, t_prep + t_op, hprep.seq he_op, hstep, hdec, hrep_z, hmod_z, hregs_z, + by rw [hp_eq] at hpot_z; omega⟩ + +/-! ### Projection 5: Caller-to-DS bidirectional framing (`frame_caller`) -/ + +/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS +registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from +`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ +theorem frame_caller (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} {s : D.State} {x x' : Data r_caller} {F : Finset ℕ} + (hr : C.rep s (x.comap f)) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hmem : Modifies F x.mem x'.mem) : + C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by + have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => + hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) + exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ + +/-! ### Projection 6: Flat-program operation-sequence execution (`RunsRelOps`, `run_ops_exact_*`) -/ + +/-- Default register-preparation function overwriting `P.input` registers with `C.encode o`. -/ +def prepareRegs (C : RelSubroutineDS P D w base params s₀) (o : D.Op) + (regs : Fin r_ds → ℕ) : Fin r_ds → ℕ := + fun i => if P.input i then C.encode o i else regs i + +/-- Sequential flat-program execution of `ops : List D.Op` by repeated calls to `P.opProg`: +each call prepares input registers via `prepRegs`, runs `P.opProg` from `pc = 0` to `halt`, +decodes a permitted outcome `(s₁, a)` with `D.step s o s₁ a`, modifies memory only within +`[base, base + C.space)`, records the resulting transcript `tr : D.Transcript`, and accumulates +total instructions executed (including each trailing `halt`). -/ +inductive RunsRelOps (C : RelSubroutineDS P D w base params s₀) + (prepRegs : D.Op → (Fin r_ds → ℕ) → (Fin r_ds → ℕ) := C.prepareRegs) : + D.State → Data r_ds → List D.Op → D.Transcript → D.State → Data r_ds → ℕ → Prop + | nil {s x} : RunsRelOps C prepRegs s x [] [] s x 0 + | cons {s s₁ s₂ x o a ops tr pc y z steps rest} : + Runs w P.opProg ⟨0, ⟨prepRegs o x.regs, x.mem⟩⟩ ⟨pc, y⟩ steps → + (∃ out, step w P.opProg ⟨pc, y⟩ = .halt out) → + D.step s o s₁ a → + C.decode o y.regs = some a → + Modifies (Block.mk base C.space).footprint x.mem y.mem → + RunsRelOps C prepRegs s₁ y ops tr s₂ z rest → + RunsRelOps C prepRegs s x (o :: ops) (⟨o, a, s₁⟩ :: tr) s₂ z ((steps + 1) + rest) + +private theorem step_program_halt (c : Code B r_ds) (out : Fin r_ds) (y : Data r_ds) : + step w (c.program out) ⟨0 + c.size, y⟩ = .halt (y.regs out) := + step_halt (by + simp only [Code.program, Nat.zero_add, + List.getElem?_append_right (Code.length_assemble c 0).le, + Code.length_assemble, Nat.sub_self, List.getElem?_cons_zero]) + +/-- Flat-program sequence execution under a custom register-preparation function `prepRegs`. -/ +theorem seq_runsRelOps_of_prep (C : RelSubroutineDS P D w base params s₀) + (prepRegs : D.Op → (Fin r_ds → ℕ) → (Fin r_ds → ℕ)) + (hfit : ∀ {s} (o) (regs : Fin r_ds → ℕ), D.valid s o → (∀ i, Fits w (regs i)) → + ∀ i, Fits w (prepRegs o regs i)) + (hnot_in : ∀ (o : D.Op) (regs : Fin r_ds → ℕ) (i : Fin r_ds), + P.input i = false → prepRegs o regs i = regs i) + (harg : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), C.argMatch o (prepRegs o regs)) : + ∀ (ops : List D.Op) {s : D.State} {x : Data r_ds}, + C.rep s x → D.ValidOpSeq s ops → + ∃ tr s_fin z t_ops, + C.RunsRelOps prepRegs s x ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s tr s_fin ∧ + C.rep s_fin z ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s tr + C.potential x + | [], s, x, hr, _ => + ⟨[], s, x, 0, .nil, rfl, .nil s, hr, Nat.zero_add (C.potential x) ▸ le_rfl⟩ + | o :: ops, s, x, hr, ⟨hv_o, hv_rest⟩ => by + let x' : Data r_ds := ⟨prepRegs o x.regs, x.mem⟩ + have hbx := C.bounded hr + have hb' : x'.Bounded w := ⟨hfit o x.regs hv_o hbx.1, hbx.2⟩ + have hreg' : ∀ i, P.input i = false → x'.regs i = x.regs i := + fun i hi => hnot_in o x.regs i hi + have hr' : C.rep s x' := C.rep_supported hreg' (fun _ _ => rfl) hb' hr + have hpot' : C.potential x' = C.potential x := + C.potential_supported hreg' (fun _ _ => rfl) hr + obtain ⟨y, s₁, a, m, he_op, hstep, hdec, hr_y, hmod, hpot_m⟩ := + C.step_exec o hr' hv_o (harg o x.regs) + rw [hpot'] at hpot_m + have hruns : Runs w P.opProg ⟨0, x'⟩ ⟨0 + P.code.size, y⟩ m := + he_op.runs (pc := 0) (p := P.opProg) (Program.Contains.self _).append_left + have hhalt : step w P.opProg ⟨0 + P.code.size, y⟩ = .halt (y.regs P.output) := + step_program_halt P.code P.output y + obtain ⟨tr, s_fin, z, rest, hrest, hops, htr, hr_z, hpot_rest⟩ := + seq_runsRelOps_of_prep C prepRegs hfit hnot_in harg ops hr_y (hv_rest s₁ a hstep) + refine ⟨⟨o, a, s₁⟩ :: tr, s_fin, z, (m + 1) + rest, + .cons hruns ⟨_, hhalt⟩ hstep hdec hmod hrest, + congrArg (o :: ·) hops, .cons _ hv_o hstep htr, hr_z, ?_⟩ + simp only [Stateful.RelSpec.totalCost_cons] + omega + +/-- Flat-program sequence execution using `C.prepareRegs`. -/ +theorem seq_runsRelOps (C : RelSubroutineDS P D w base params s₀) + (ops : List D.Op) {s : D.State} {x : Data r_ds} + (hr : C.rep s x) (hv : D.ValidOpSeq s ops) : + ∃ tr s_fin z t_ops, + C.RunsRelOps C.prepareRegs s x ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s tr s_fin ∧ + C.rep s_fin z ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s tr + C.potential x := by + refine C.seq_runsRelOps_of_prep C.prepareRegs ?_ ?_ C.encode_matches ops hr hv + · intro s' o regs hv_o hregs i + simp only [prepareRegs] + split_ifs with hi + · exact C.arguments_fit o hv_o i hi + · exact hregs i + · intro o regs i hi + simp [prepareRegs, hi] + +/-- Standalone flat-program operation-sequence theorem with `O(1)` fresh initialization on zeroed +`[base, base + C.space)`. -/ +theorem run_ops_exact_fresh (C : RelSubroutineDS P D w base params s₀) + {d₀ : Data r_ds} (hd₀ : d₀.Bounded w) + (hfr : d₀.regs P.frontierReg = base) + (hpar : ∀ j, d₀.regs (P.initParams j) = params j) + (hzero : ∀ a ∈ (Block.mk base C.space).footprint, d₀.mem a = 0) + (ops : List D.Op) (hv : D.ValidOpSeq s₀ ops) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w P.initFreshProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w P.initFreshProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base C.space).footprint d₀.mem x₀.mem ∧ + C.RunsRelOps C.prepareRegs s₀ x₀ ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s₀ tr s_fin ∧ + C.rep s_fin z ∧ + t_init ≤ C.initTime ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s₀ tr + C.initPotential := by + obtain ⟨x₀, ⟨t_init, ht_init, he_init⟩, _, hr₀, hpot₀, hmod₀⟩ := + C.initFresh_exec hd₀ hfr hpar hzero + have hruns₀ : Runs w P.initFreshProg ⟨0, d₀⟩ ⟨0 + P.initFresh.size, x₀⟩ t_init := + he_init.runs (pc := 0) (p := P.initFreshProg) (Program.Contains.self _).append_left + have hhalt₀ : step w P.initFreshProg ⟨0 + P.initFresh.size, x₀⟩ = .halt (x₀.regs P.output) := + step_program_halt P.initFresh P.output x₀ + obtain ⟨tr, s_fin, z, t_ops, hruns_ops, hops, htr, hr_z, hbound⟩ := + C.seq_runsRelOps ops hr₀ hv + exact ⟨_, x₀, t_init, tr, s_fin, z, t_ops, hruns₀, ⟨_, hhalt₀⟩, hmod₀, hruns_ops, + hops, htr, hr_z, ht_init, by omega⟩ + +/-- Standalone flat-program operation-sequence theorem with `O(space)` reusable initialization on +arbitrary `w`-bounded memory. -/ +theorem run_ops_exact_reusable (C : RelSubroutineDS P D w base params s₀) + {d₀ : Data r_ds} (hd₀ : d₀.Bounded w) + (hfr : d₀.regs P.frontierReg = base) + (hpar : ∀ j, d₀.regs (P.initParams j) = params j) + (ops : List D.Op) (hv : D.ValidOpSeq s₀ ops) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w P.initClearProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w P.initClearProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base C.space).footprint d₀.mem x₀.mem ∧ + C.RunsRelOps C.prepareRegs s₀ x₀ ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s₀ tr s_fin ∧ + C.rep s_fin z ∧ + t_init ≤ C.initTime + 4 * C.space + 6 ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s₀ tr + C.initPotential := by + obtain ⟨x₀, ⟨t_init, ht_init, he_init⟩, _, hr₀, hpot₀, hmod₀⟩ := + C.initClear_exec hd₀ hfr hpar + have hruns₀ : Runs w P.initClearProg ⟨0, d₀⟩ ⟨0 + P.initClear.size, x₀⟩ t_init := + he_init.runs (pc := 0) (p := P.initClearProg) (Program.Contains.self _).append_left + have hhalt₀ : step w P.initClearProg ⟨0 + P.initClear.size, x₀⟩ = .halt (x₀.regs P.output) := + step_program_halt P.initClear P.output x₀ + obtain ⟨tr, s_fin, z, t_ops, hruns_ops, hops, htr, hr_z, hbound⟩ := + C.seq_runsRelOps ops hr₀ hv + exact ⟨_, x₀, t_init, tr, s_fin, z, t_ops, hruns₀, ⟨_, hhalt₀⟩, hmod₀, hruns_ops, + hops, htr, hr_z, ht_init, by omega⟩ + +end RelSubroutineDS + +end Cslib.WordRAM diff --git a/Complexity/Foundations/Semantics/Stateful/Relational.lean b/Complexity/Foundations/Semantics/Stateful/Relational.lean new file mode 100644 index 000000000..40d7761e7 --- /dev/null +++ b/Complexity/Foundations/Semantics/Stateful/Relational.lean @@ -0,0 +1,1181 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Foundations.Semantics.Stateful.Amortized +public import Complexity.Foundations.Semantics.Stateful.CostHoare +public import Cslib.Foundations.Semantics.LTS.Basic + +/-! +# Relational stateful specifications, demonic adaptive clients, and angelic realizations + +A relational stateful specification (`Cslib.Stateful.RelSpec`) generalizes `Cslib.Stateful.Spec` +from a deterministic transition function `step : State → (o : Op) → State × Result o` to a +transition relation `step : State → (o : Op) → State → Result o → Prop`. This allows data-structure +specifications to leave implementation choices (such as which free slot or handle is allocated, or +how equal-key ties are broken) underspecified in the abstract interface while keeping internal +allocator state hidden inside concrete representation invariants. + +The semantics is **demonic** for callers and **angelic** for implementations: +* An adaptive client `p : D.Client α` is demonically admissible (`D.DemonicValid s p`) when each + issued operation is admissible (`D.valid`) and the continuation remains admissible under *every* + outcome `(s', a)` permitted by `D.step`. Caller postconditions (`D.Hoare`) and abstract cost + budgets (`D.CostHoare`) hold across *all* permitted client runs (`D.ClientRun`). +* A concrete backend realization (`RelRealization`, `RelAmortized`) is angelic in the abstract + post-state: each concrete operation execution witnesses *some* permitted abstract transition + `D.step s o s' a` maintaining the representation invariant and potential inequality. + +Every deterministic `D : Spec` embeds into `RelSpec` with zero overhead via `Spec.toRelSpec` +(preserving `State`, `Op`, `Result`, `valid`, `signature`, and `Client α` definitionally), and +`DemonicValid`, `ClientRun`, `Hoare`, and `CostHoare` on `D.toRelSpec` are equivalent to their +deterministic counterparts on `D`. + +## Main definitions + +* `Cslib.Stateful.RelSpec`: relational stateful specification with `State`, `Op`, `Result`, + transition relation `step`, and input admissibility `valid`. +* `Cslib.Stateful.RelSpec.Total`: typeclass asserting that every admissible `(s, o)` has at least + one permitted outcome `(s', a)`. +* `Cslib.Stateful.Spec.toRelSpec`: zero-cost embedding of a deterministic `Spec` into `RelSpec`. +* `Cslib.Stateful.RelSpec.toLTS`: projection of a `RelSpec` to a `Cslib.LTS` labelled by + input-output pairs `Σ o : D.Op, D.Result o`. +* `Cslib.Stateful.RelSpec.Step`, `Cslib.Stateful.RelSpec.Transcript`, + `Cslib.Stateful.RelSpec.ValidTranscript`, `Cslib.Stateful.RelSpec.ValidOpSeq`, + `Cslib.Stateful.RelSpec.totalCost`, `Cslib.Stateful.RelSpec.totalCostOutcome`: execution + transcripts, non-adaptive operation-sequence admissibility, and abstract cost accounting. +* `Cslib.Stateful.RelSpec.ClientRun`, `Cslib.Stateful.RelSpec.DemonicValid`, + `Cslib.Stateful.RelSpec.Hoare`, `Cslib.Stateful.RelSpec.CostHoare`: adaptive client runs, + demonic admissibility, and compositional Hoare / CostHoare triples. +* `Cslib.Stateful.RelExecution`, `Cslib.Stateful.RelExecutes`, + `Cslib.Stateful.RelRealization`, `Cslib.Stateful.RelBound`, `Cslib.Stateful.RelAmortized`: + native execution traces, angelic representation simulation, and potential-method certificates. + +## Main results + +* `Cslib.Stateful.Spec.toRelSpec_demonicValid_iff`, + `Cslib.Stateful.Spec.toRelSpec_clientRun_iff`, + `Cslib.Stateful.Spec.toRelSpec_hoare_iff`, + `Cslib.Stateful.Spec.toRelSpec_costHoare_iff`: exact collapse of relational client semantics to + deterministic `Spec` semantics on `D.toRelSpec`. +* `Cslib.Stateful.RelRealization.correct_client`, + `Cslib.Stateful.RelRealization.exists_executes`: soundness and existence of native client + executions under `DemonicValid`. +* `Cslib.Stateful.RelAmortized.executes_le`, `Cslib.Stateful.RelAmortized.run_client_amortized`: + amortized potential telescoping along adaptive client executions. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Chapter 17][CLRS2009] +* [D. Sangiorgi, *Introduction to Bisimulation and Coinduction*][Sangiorgi2011] +-/ + +@[expose] public section + +namespace Cslib.Stateful + +universe u v w x y + +/-! ## Core relational specification and totality -/ + +/-- A relational stateful specification with state type `State`, operation type `Op`, +operation-dependent observation family `Result`, transition relation `step`, and input +admissibility predicate `valid`. -/ +structure RelSpec where + /-- Mathematical states, independent of a concrete storage representation. -/ + State : Type u + /-- Operations available to clients. -/ + Op : Type v + /-- The observation type returned by each operation. -/ + Result : Op → Type w + /-- Relational step specification: `step s o s' a` holds when transitioning from `s` to `s'` + and returning `a : Result o` is a permitted outcome of `o`. -/ + step : State → (o : Op) → State → Result o → Prop + /-- Preconditions under which an operation's implementation must satisfy its contract. -/ + valid : State → Op → Prop := fun _ _ => True + +namespace RelSpec + +variable (D : RelSpec.{u, v, w}) + +/-- The set of permitted `(nextState, result)` outcomes of operation `o` from state `s`. -/ +def outcomes (s : D.State) (o : D.Op) : Set (D.State × D.Result o) := + {p | D.step s o p.1 p.2} + +/-- Membership in `D.outcomes s o` unfolds to `D.step s o`. -/ +@[simp] +theorem mem_outcomes_iff (s : D.State) (o : D.Op) (p : D.State × D.Result o) : + p ∈ D.outcomes s o ↔ D.step s o p.1 p.2 := Iff.rfl + +/-- A relational specification is total on admissible inputs if every valid `(s, o)` admits at +least one permitted outcome `(s', a)`. -/ +class Total : Prop where + /-- Every admissible operation call has at least one permitted post-state and result. -/ + step_total : ∀ s o, D.valid s o → ∃ s' a, D.step s o s' a + +/-- Obtain a permitted outcome `(s', a)` for an admissible operation under `[D.Total]`. -/ +theorem exists_step [D.Total] {s : D.State} {o : D.Op} (hv : D.valid s o) : + ∃ s' a, D.step s o s' a := + Total.step_total s o hv + +/-- The polynomial operation signature `⟨D.Op, D.Result⟩`. -/ +abbrev signature : PFunctor.{v, w} := ⟨D.Op, D.Result⟩ + +/-- An adaptive client over `D`: each continuation may branch on the observed operation result. -/ +abbrev Client (α : Type y) : Type (max v w y) := D.signature.FreeM α + +/-- Invoke a single operation `o : D.Op` in an adaptive `D.Client`. -/ +abbrev call (o : D.Op) : D.Client (D.Result o) := .liftBind o .pure + +/-- An input-output transition label pairing an operation `o : D.Op` with its result `a`. -/ +abbrev Label : Type (max v w) := Σ o : D.Op, D.Result o + +/-- View a relational stateful specification as a labelled transition system (`Cslib.LTS`) +labelled by input-output pairs `⟨o, a⟩ : Σ o : D.Op, D.Result o`. -/ +def toLTS : LTS D.State D.Label where + Tr s lbl s' := D.step s lbl.1 s' lbl.2 + +/-- A transition in `D.toLTS` is a step of `D`. -/ +@[simp] +theorem toLTS_tr (s s' : D.State) (lbl : D.Label) : + D.toLTS.Tr s lbl s' ↔ D.step s lbl.1 s' lbl.2 := Iff.rfl + +end RelSpec + +/-! ## Zero-cost embedding of deterministic specifications (`Spec.toRelSpec`) -/ + +/-- Embed a deterministic `Stateful.Spec` as a relational `RelSpec` whose transition relation +holds if and only if `(s', a) = D.step s o`. -/ +abbrev Spec.toRelSpec (D : Spec.{u, v, w}) : RelSpec.{u, v, w} where + State := D.State + Op := D.Op + Result := D.Result + step s o s' a := (s', a) = D.step s o + valid := D.valid + +instance : Coe Spec.{u, v, w} RelSpec.{u, v, w} := ⟨Spec.toRelSpec⟩ + +namespace Spec + +variable (D : Spec.{u, v, w}) + +/-- `toRelSpec` preserves the state type definitionally. -/ +@[simp] +theorem toRelSpec_state : D.toRelSpec.State = D.State := rfl + +/-- `toRelSpec` preserves the operation type definitionally. -/ +@[simp] +theorem toRelSpec_op : D.toRelSpec.Op = D.Op := rfl + +/-- `toRelSpec` preserves the result family definitionally. -/ +@[simp] +theorem toRelSpec_result : D.toRelSpec.Result = D.Result := rfl + +/-- A step in `D.toRelSpec` holds iff `(s', a) = D.step s o`. -/ +@[simp] +theorem toRelSpec_step (s : D.State) (o : D.Op) (s' : D.State) (a : D.Result o) : + D.toRelSpec.step s o s' a ↔ (s', a) = D.step s o := Iff.rfl + +/-- `toRelSpec` preserves the admissibility predicate definitionally. -/ +@[simp] +theorem toRelSpec_valid (s : D.State) (o : D.Op) : + D.toRelSpec.valid s o ↔ D.valid s o := Iff.rfl + +/-- `toRelSpec` preserves the polynomial signature definitionally. -/ +@[simp] +theorem toRelSpec_signature : D.toRelSpec.signature = D.signature := rfl + +/-- `toRelSpec` preserves the adaptive client type definitionally. -/ +@[simp] +theorem toRelSpec_client (α : Type y) : D.toRelSpec.Client α = D.Client α := rfl + +/-- Every embedded deterministic specification is automatically total. -/ +instance : D.toRelSpec.Total where + step_total s o _ := ⟨(D.step s o).1, (D.step s o).2, Prod.mk.eta⟩ + +/-- Transitions in `D.toRelSpec.toLTS` agree with `D.toFLTS` on states and `(D.step s o).2` on +observations. -/ +theorem toRelSpec_toLTS_tr (s s' : D.State) (o : D.Op) (a : D.Result o) : + D.toRelSpec.toLTS.Tr s ⟨o, a⟩ s' ↔ s' = D.toFLTS.tr s o ∧ a = (D.step s o).2 := by + simp [RelSpec.toLTS, Spec.toRelSpec, Spec.toFLTS, Prod.ext_iff] + +/-- The labelled transition system of an embedded deterministic specification is deterministic. -/ +instance : D.toRelSpec.toLTS.Deterministic where + deterministic s lbl s₁ s₂ h₁ h₂ := by + have h₁' := congrArg Prod.fst h₁ + have h₂' := congrArg Prod.fst h₂ + exact h₁'.trans h₂'.symm + +/-- Multi-step execution `D.run` produces a valid multi-step trace in `D.toRelSpec.toLTS` ending +at `D.toFLTS.mtr s os`. -/ +theorem toRelSpec_toLTS_mtr_run (s : D.State) (os : List D.Op) : + D.toRelSpec.toLTS.MTr s (D.run s os).2 (D.toFLTS.mtr s os) := by + induction os generalizing s with + | nil => exact .refl + | cons o os ih => + exact .stepL Prod.mk.eta (ih (D.step s o).1) + +end Spec + +/-! ## Transcripts, non-adaptive operation sequences, and cost accounting -/ + +namespace RelSpec + +variable (D : RelSpec.{u, v, w}) + +/-- One recorded step of a relational specification: the issued operation `op`, its returned +observation `result`, and the resulting post-state `nextState`. -/ +structure Step : Type (max u v w) where + /-- The operation invoked at this step. -/ + op : D.Op + /-- The observation returned by `op`. -/ + result : D.Result op + /-- The abstract post-state reached after `op`. -/ + nextState : D.State + +/-- A finite execution transcript is a list of recorded steps. -/ +abbrev Transcript : Type (max u v w) := List D.Step + +/-- Extract the underlying sequence of operations from a transcript. -/ +def Transcript.ops {D : RelSpec.{u, v, w}} (tr : D.Transcript) : List D.Op := + tr.map Step.op + +/-- Extract the `Cslib.LTS` input-output labels from a transcript. -/ +def Transcript.labels {D : RelSpec.{u, v, w}} (tr : D.Transcript) : List D.Label := + tr.map fun st => ⟨st.op, st.result⟩ + +@[simp] +theorem Transcript.ops_nil {D : RelSpec.{u, v, w}} : + Transcript.ops ([] : D.Transcript) = [] := rfl + +@[simp] +theorem Transcript.ops_cons {D : RelSpec.{u, v, w}} (st : D.Step) (tr : D.Transcript) : + Transcript.ops (st :: tr) = st.op :: Transcript.ops tr := rfl + +@[simp] +theorem Transcript.ops_append {D : RelSpec.{u, v, w}} (tr₁ tr₂ : D.Transcript) : + Transcript.ops (tr₁ ++ tr₂) = Transcript.ops tr₁ ++ Transcript.ops tr₂ := + List.map_append + +@[simp] +theorem Transcript.length_ops {D : RelSpec.{u, v, w}} (tr : D.Transcript) : + (Transcript.ops tr).length = tr.length := + List.length_map _ + +@[simp] +theorem Transcript.labels_nil {D : RelSpec.{u, v, w}} : + Transcript.labels ([] : D.Transcript) = [] := rfl + +@[simp] +theorem Transcript.labels_cons {D : RelSpec.{u, v, w}} (st : D.Step) (tr : D.Transcript) : + Transcript.labels (st :: tr) = ⟨st.op, st.result⟩ :: Transcript.labels tr := rfl + +@[simp] +theorem Transcript.labels_append {D : RelSpec.{u, v, w}} (tr₁ tr₂ : D.Transcript) : + Transcript.labels (tr₁ ++ tr₂) = Transcript.labels tr₁ ++ Transcript.labels tr₂ := + List.map_append + +@[simp] +theorem Transcript.length_labels {D : RelSpec.{u, v, w}} (tr : D.Transcript) : + (Transcript.labels tr).length = tr.length := + List.length_map _ + +/-- `D.ValidTranscript s tr s'` holds when `tr` is a valid step-by-step execution starting at `s` +and ending at `s'`, where each step is admissible (`D.valid`) and permitted by `D.step`. -/ +inductive ValidTranscript : D.State → D.Transcript → D.State → Prop + | nil (s : D.State) : ValidTranscript s [] s + | cons {s : D.State} (st : D.Step) {tr : D.Transcript} {s_fin : D.State} : + D.valid s st.op → + D.step s st.op st.nextState st.result → + ValidTranscript st.nextState tr s_fin → + ValidTranscript s (st :: tr) s_fin + +/-- The empty transcript is valid iff the initial and final states coincide. -/ +@[simp] +theorem validTranscript_nil_iff {D : RelSpec.{u, v, w}} (s s' : D.State) : + D.ValidTranscript s [] s' ↔ s = s' := by + constructor + · intro h; cases h; rfl + · rintro rfl; exact .nil s + +/-- A nonempty transcript is valid iff its head step is admissible and permitted and its tail is +valid from `st.nextState`. -/ +@[simp] +theorem validTranscript_cons_iff {D : RelSpec.{u, v, w}} (s s' : D.State) (st : D.Step) + (tr : D.Transcript) : + D.ValidTranscript s (st :: tr) s' ↔ + D.valid s st.op ∧ D.step s st.op st.nextState st.result ∧ + D.ValidTranscript st.nextState tr s' := by + constructor + · intro h + cases h with + | cons _ hv hstep htr => exact ⟨hv, hstep, htr⟩ + · rintro ⟨hv, hstep, htr⟩ + exact .cons st hv hstep htr + +/-- Valid transcripts compose across concatenation. -/ +theorem ValidTranscript.append {D : RelSpec.{u, v, w}} {s s₁ s₂ : D.State} + {tr₁ tr₂ : D.Transcript} (h₁ : D.ValidTranscript s tr₁ s₁) + (h₂ : D.ValidTranscript s₁ tr₂ s₂) : + D.ValidTranscript s (tr₁ ++ tr₂) s₂ := by + induction h₁ with + | nil _ => exact h₂ + | cons st hv hstep _ ih => exact .cons st hv hstep (ih h₂) + +/-- Every valid transcript projects to a multi-step transition in `D.toLTS`. -/ +theorem ValidTranscript.toLTS_mtr {D : RelSpec.{u, v, w}} {s s' : D.State} + {tr : D.Transcript} (h : D.ValidTranscript s tr s') : + D.toLTS.MTr s tr.labels s' := by + induction h with + | nil _ => exact .refl + | cons _ _ hstep _ ih => exact .stepL hstep ih + +/-- Demonic admissibility of a non-adaptive operation sequence `ops : List D.Op` from `s`: +the head operation is admissible at `s`, and the tail sequence remains admissible from every +permitted post-state `s'` of the head operation. -/ +def ValidOpSeq (s : D.State) : List D.Op → Prop + | [] => True + | o :: ops => D.valid s o ∧ ∀ s' a, D.step s o s' a → ValidOpSeq s' ops + +@[simp] +theorem validOpSeq_nil (s : D.State) : D.ValidOpSeq s [] ↔ True := Iff.rfl + +@[simp] +theorem validOpSeq_cons (s : D.State) (o : D.Op) (ops : List D.Op) : + D.ValidOpSeq s (o :: ops) ↔ D.valid s o ∧ ∀ s' a, D.step s o s' a → D.ValidOpSeq s' ops := + Iff.rfl + +/-- Total interfaces have no operation-sequence admissibility obligations. -/ +theorem validOpSeq_of_forall_valid (h : ∀ s o, D.valid s o) (s : D.State) (ops : List D.Op) : + D.ValidOpSeq s ops := by + induction ops generalizing s with + | nil => trivial + | cons o ops ih => exact ⟨h s o, fun s' _ _ => ih s'⟩ + +/-- Under `[D.Total]`, every demonically admissible operation sequence admits at least one valid +transcript. -/ +theorem exists_validTranscript_of_validOpSeq [D.Total] {s : D.State} {ops : List D.Op} + (hv : D.ValidOpSeq s ops) : + ∃ tr s', tr.ops = ops ∧ D.ValidTranscript s tr s' := by + induction ops generalizing s with + | nil => exact ⟨[], s, rfl, .nil s⟩ + | cons o ops ih => + obtain ⟨hv_o, hv_rest⟩ := hv + obtain ⟨s₁, a, hstep⟩ := D.exists_step hv_o + obtain ⟨tr, s₂, hops, htr⟩ := ih (hv_rest s₁ a hstep) + exact ⟨⟨o, a, s₁⟩ :: tr, s₂, congrArg (o :: ·) hops, .cons _ hv_o hstep htr⟩ + +/-- State-dependent charges summed along a transcript. -/ +def totalCost (charge : D.State → D.Op → ℕ) : D.State → D.Transcript → ℕ + | _, [] => 0 + | s, st :: tr => charge s st.op + totalCost charge st.nextState tr + +/-- Outcome-dependent charges summed along a transcript, where each step charge may also inspect +the post-state `st.nextState` and result `st.result`. -/ +def totalCostOutcome (charge : D.State → (o : D.Op) → D.State → D.Result o → ℕ) : + D.State → D.Transcript → ℕ + | _, [] => 0 + | s, st :: tr => + charge s st.op st.nextState st.result + totalCostOutcome charge st.nextState tr + +@[simp] +theorem totalCost_nil (charge : D.State → D.Op → ℕ) (s : D.State) : + D.totalCost charge s [] = 0 := rfl + +@[simp] +theorem totalCost_cons (charge : D.State → D.Op → ℕ) (s : D.State) (st : D.Step) + (tr : D.Transcript) : + D.totalCost charge s (st :: tr) = charge s st.op + D.totalCost charge st.nextState tr := rfl + +@[simp] +theorem totalCostOutcome_nil (charge : D.State → (o : D.Op) → D.State → D.Result o → ℕ) + (s : D.State) : + D.totalCostOutcome charge s [] = 0 := rfl + +@[simp] +theorem totalCostOutcome_cons (charge : D.State → (o : D.Op) → D.State → D.Result o → ℕ) + (s : D.State) (st : D.Step) (tr : D.Transcript) : + D.totalCostOutcome charge s (st :: tr) = + charge s st.op st.nextState st.result + D.totalCostOutcome charge st.nextState tr := rfl + +/-- `totalCost` is the special case of `totalCostOutcome` that ignores the post-state and +result. -/ +theorem totalCost_eq_totalCostOutcome (charge : D.State → D.Op → ℕ) (s : D.State) + (tr : D.Transcript) : + D.totalCost charge s tr = D.totalCostOutcome (fun s o _ _ => charge s o) s tr := by + induction tr generalizing s with + | nil => rfl + | cons st tr ih => simp [ih] + +/-- Along a valid transcript prefix `D.ValidTranscript s tr₁ s₁`, `totalCost` splits additively at +the intermediate state `s₁`. -/ +theorem totalCost_append {D : RelSpec.{u, v, w}} (charge : D.State → D.Op → ℕ) + {s s₁ : D.State} {tr₁ : D.Transcript} (h₁ : D.ValidTranscript s tr₁ s₁) + (tr₂ : D.Transcript) : + D.totalCost charge s (tr₁ ++ tr₂) = + D.totalCost charge s tr₁ + D.totalCost charge s₁ tr₂ := by + induction h₁ with + | nil _ => simp + | cons st _ _ _ ih => simp [ih, Nat.add_assoc] + +/-- Uniform per-operation bounds scale by the transcript length on any valid transcript. -/ +theorem totalCost_le_length_mul {D : RelSpec.{u, v, w}} (charge : D.State → D.Op → ℕ) (T : ℕ) + (h : ∀ s o, D.valid s o → charge s o ≤ T) {s s' : D.State} {tr : D.Transcript} + (hv : D.ValidTranscript s tr s') : + D.totalCost charge s tr ≤ tr.length * T := by + induction hv with + | nil _ => simp + | cons st hv_o _ _ ih => + have hstep := h _ st.op hv_o + simp only [totalCost_cons, List.length_cons, Nat.add_mul, Nat.one_mul] + omega + +/-- A constant charge `K` per operation totals `tr.length * K`. -/ +theorem totalCost_const (K : ℕ) (s : D.State) (tr : D.Transcript) : + D.totalCost (fun _ _ => K) s tr = tr.length * K := by + induction tr generalizing s with + | nil => simp + | cons st tr ih => simp [ih, Nat.add_mul, Nat.add_comm] + +/-- A unit charge per operation totals the transcript length. -/ +theorem totalCost_one (s : D.State) (tr : D.Transcript) : + D.totalCost (fun _ _ => 1) s tr = tr.length := by + simpa using D.totalCost_const 1 s tr + +end RelSpec + +/-! ## Adaptive clients (`ClientRun`, `DemonicValid`) and demonic `Hoare` / `CostHoare` -/ + +namespace RelSpec + +variable (D : RelSpec.{u, v, w}) {α β : Type y} + +/-- `D.ClientRun s p v s' tr` holds when running the adaptive client `p : D.Client α` from initial +state `s` can produce return value `v : α`, final state `s' : D.State`, and transcript `tr` by +making admissible and permitted transitions at each operation call. -/ +inductive ClientRun : D.State → D.Client α → α → D.State → D.Transcript → Prop + | pure (s : D.State) (a : α) : + ClientRun s (.pure a) a s [] + | liftBind {s s₁ s₂ : D.State} (o : D.Op) (k : D.Result o → D.Client α) + (a : D.Result o) {v : α} {tr : D.Transcript} : + D.valid s o → + D.step s o s₁ a → + ClientRun s₁ (k a) v s₂ tr → + ClientRun s (.liftBind o k) v s₂ (⟨o, a, s₁⟩ :: tr) + +/-- Every `ClientRun` produces a valid transcript. -/ +theorem ClientRun.validTranscript {D : RelSpec.{u, v, w}} {s s' : D.State} {p : D.Client α} + {v : α} {tr : D.Transcript} (h : D.ClientRun s p v s' tr) : + D.ValidTranscript s tr s' := by + induction h with + | pure s _ => exact .nil s + | liftBind _ _ _ hv hstep _ ih => exact .cons _ hv hstep ih + +theorem clientRun_pure_iff {D : RelSpec.{u, v, w}} (s : D.State) (a v : α) (s' : D.State) + (tr : D.Transcript) : + D.ClientRun s (.pure a) v s' tr ↔ v = a ∧ s' = s ∧ tr = [] := by + constructor + · intro h + cases h + exact ⟨rfl, rfl, rfl⟩ + · rintro ⟨rfl, rfl, rfl⟩ + exact .pure s' v + +theorem clientRun_liftBind_iff {D : RelSpec.{u, v, w}} (s : D.State) (o : D.Op) + (k : D.Result o → D.Client α) (v : α) (s₂ : D.State) (tr : D.Transcript) : + D.ClientRun s (.liftBind o k) v s₂ tr ↔ + D.valid s o ∧ ∃ a s₁ tr', D.step s o s₁ a ∧ + D.ClientRun s₁ (k a) v s₂ tr' ∧ tr = ⟨o, a, s₁⟩ :: tr' := by + constructor + · intro h + cases h with + | liftBind _ _ a hv hstep hrest => + exact ⟨hv, a, _, _, hstep, hrest, rfl⟩ + · rintro ⟨hv, a, s₁, tr', hstep, hrest, rfl⟩ + exact .liftBind o k a hv hstep hrest + +/-- Sequential client runs compose by concatenating their transcripts. -/ +theorem ClientRun.bind {D : RelSpec.{u, v, w}} {s s₁ s₂ : D.State} {p : D.Client α} + {k : α → D.Client β} {a : α} {b : β} {tr₁ tr₂ : D.Transcript} + (h₁ : D.ClientRun s p a s₁ tr₁) (h₂ : D.ClientRun s₁ (k a) b s₂ tr₂) : + D.ClientRun s (p.bind k) b s₂ (tr₁ ++ tr₂) := by + induction h₁ with + | pure _ _ => exact h₂ + | liftBind o cont res hv hstep _ ih => + exact .liftBind o (fun r => (cont r).bind k) res hv hstep (ih h₂) + +/-- A client run of `p.bind k` splits into a run of `p` and a run of `k` from the intermediate +result and state. -/ +theorem clientRun_bind_iff {D : RelSpec.{u, v, w}} {s s₂ : D.State} {p : D.Client α} + {k : α → D.Client β} {b : β} {tr : D.Transcript} : + D.ClientRun s (p.bind k) b s₂ tr ↔ + ∃ a s₁ tr₁ tr₂, D.ClientRun s p a s₁ tr₁ ∧ + D.ClientRun s₁ (k a) b s₂ tr₂ ∧ tr = tr₁ ++ tr₂ := by + constructor + · intro h + induction p generalizing s tr with + | pure a => + exact ⟨a, s, [], tr, .pure s a, h, rfl⟩ + | liftBind o cont ih => + change D.ClientRun s (.liftBind o fun r => (cont r).bind k) b s₂ tr at h + cases h with + | liftBind _ _ res hv hstep hrest => + obtain ⟨a, s₁, tr₁, tr₂, hp, hk, rfl⟩ := ih res hrest + exact ⟨a, s₁, ⟨o, res, _⟩ :: tr₁, tr₂, .liftBind o cont res hv hstep hp, hk, rfl⟩ + · rintro ⟨a, s₁, tr₁, tr₂, hp, hk, rfl⟩ + exact hp.bind hk + +/-- Demonic admissibility of an adaptive client `p` from state `s`: +* `.pure a` is always admissible; +* `.liftBind o k` requires `D.valid s o` now, and for *every* permitted outcome `(s', a)` of `o` + from `s`, the continuation `k a` must be demonically admissible from `s'`. -/ +inductive DemonicValid : D.State → D.Client α → Prop + | pure (s : D.State) (a : α) : + DemonicValid s (.pure a) + | liftBind {s : D.State} (o : D.Op) (k : D.Result o → D.Client α) : + D.valid s o → + (∀ s' a, D.step s o s' a → DemonicValid s' (k a)) → + DemonicValid s (.liftBind o k) + +theorem demonicValid_pure_iff {D : RelSpec.{u, v, w}} (s : D.State) (a : α) : + D.DemonicValid s (.pure a) ↔ True := + ⟨fun _ => trivial, fun _ => .pure s a⟩ + +theorem demonicValid_liftBind_iff {D : RelSpec.{u, v, w}} (s : D.State) (o : D.Op) + (k : D.Result o → D.Client α) : + D.DemonicValid s (.liftBind o k) ↔ + D.valid s o ∧ ∀ s' a, D.step s o s' a → D.DemonicValid s' (k a) := by + constructor + · intro h + cases h with + | liftBind _ _ hv hk => exact ⟨hv, hk⟩ + · rintro ⟨hv, hk⟩; exact .liftBind o k hv hk + +/-- On a specification where every operation is valid in every state, every client is demonically +admissible. -/ +theorem demonicValid_of_forall_valid (h : ∀ s o, D.valid s o) (s : D.State) (p : D.Client α) : + D.DemonicValid s p := by + induction p generalizing s with + | pure a => exact .pure s a + | liftBind o k ih => exact .liftBind o k (h s o) fun s' a _ => ih a s' + +/-- Under `[D.Total]`, every demonically admissible client admits at least one valid `ClientRun`. -/ +theorem DemonicValid.exists_clientRun {D : RelSpec.{u, v, w}} [D.Total] {s : D.State} + {p : D.Client α} (hv : D.DemonicValid s p) : + ∃ v s' tr, D.ClientRun s p v s' tr := by + induction hv with + | pure s a => exact ⟨a, s, [], .pure s a⟩ + | liftBind o k hv_o _ ih => + obtain ⟨s₁, a, hstep⟩ := D.exists_step hv_o + obtain ⟨v, s₂, tr, hrun⟩ := ih s₁ a hstep + exact ⟨v, s₂, ⟨o, a, s₁⟩ :: tr, .liftBind o k a hv_o hstep hrun⟩ + +/-- Demonic admissibility composes across `bind` when the continuation is demonically admissible +after every valid run of the first client. -/ +theorem DemonicValid.bind {D : RelSpec.{u, v, w}} {s : D.State} {p : D.Client α} + {k : α → D.Client β} (hp : D.DemonicValid s p) + (hk : ∀ a s' tr, D.ClientRun s p a s' tr → D.DemonicValid s' (k a)) : + D.DemonicValid s (p.bind k) := by + induction hp with + | pure s a => exact hk a s [] (.pure s a) + | liftBind o cont hv_o _ ih => + refine .liftBind o (fun r => (cont r).bind k) hv_o fun s₁ r hstep => ?_ + exact ih s₁ r hstep fun a s₂ tr hrun => hk a s₂ (⟨o, r, s₁⟩ :: tr) + (.liftBind o cont r hv_o hstep hrun) + +/-- Demonic Hoare triple for a relational specification: from any initial state satisfying `pre`, +the client `p` is demonically admissible and *every* valid `ClientRun` satisfies `post`. -/ +def Hoare (pre : D.State → Prop) (p : D.Client α) (post : α → D.State → Prop) : Prop := + ∀ s, pre s → D.DemonicValid s p ∧ ∀ v s' tr, D.ClientRun s p v s' tr → post v s' + +namespace Hoare + +variable {D : RelSpec.{u, v, w}} + +/-- Returning a pure value leaves the state unchanged. -/ +theorem pure (pre : D.State → Prop) (a : α) (post : α → D.State → Prop) + (h : ∀ s, pre s → post a s) : D.Hoare pre (.pure a) post := by + intro s hs + refine ⟨.pure s a, ?_⟩ + intro v s' tr hrun + cases hrun + exact h s hs + +/-- Calling an operation requires `D.valid s o` and establishes `post a s'` for every permitted +outcome `D.step s o s' a`. -/ +theorem call (pre : D.State → Prop) (o : D.Op) (post : D.Result o → D.State → Prop) + (h : ∀ s, pre s → D.valid s o ∧ ∀ s' a, D.step s o s' a → post a s') : + D.Hoare pre (.liftBind o fun a => .pure a) post := by + intro s hs + obtain ⟨hv, hpost⟩ := h s hs + refine ⟨.liftBind o (fun a => .pure a) hv fun s' a _ => .pure s' a, ?_⟩ + intro v s' tr hrun + cases hrun with + | liftBind _ _ a _ hstep hrest => + cases hrest + exact hpost s' v hstep + +/-- Sequential composition of demonic Hoare triples. -/ +theorem bind {pre : D.State → Prop} {mid : α → D.State → Prop} + {post : β → D.State → Prop} {p : D.Client α} {k : α → D.Client β} + (hp : D.Hoare pre p mid) (hk : ∀ a, D.Hoare (mid a) (k a) post) : + D.Hoare pre (p.bind k) post := by + intro s hs + obtain ⟨hvp, hpost_p⟩ := hp s hs + refine ⟨hvp.bind fun a s₁ tr₁ hrun₁ => (hk a s₁ (hpost_p a s₁ tr₁ hrun₁)).1, ?_⟩ + intro b s₂ tr hrun + obtain ⟨a, s₁, tr₁, tr₂, hrun₁, hrun₂, _⟩ := clientRun_bind_iff.mp hrun + exact (hk a s₁ (hpost_p a s₁ tr₁ hrun₁)).2 b s₂ tr₂ hrun₂ + +/-- Consequence rule for demonic Hoare triples. -/ +theorem consequence {pre pre' : D.State → Prop} {post post' : α → D.State → Prop} + {p : D.Client α} (h : D.Hoare pre p post) (hp : ∀ s, pre' s → pre s) + (hq : ∀ a s, post a s → post' a s) : D.Hoare pre' p post' := by + intro s hs + obtain ⟨hv, hpost⟩ := h s (hp s hs) + exact ⟨hv, fun v s' tr hrun => hq v s' (hpost v s' tr hrun)⟩ + +end Hoare + +/-- Demonic `CostHoare` triple: strengthens `D.Hoare` with a worst-case abstract charge bound +`D.totalCost charge s tr ≤ budget s` across *every* valid `ClientRun`. -/ +def CostHoare (charge : D.State → D.Op → ℕ) (pre : D.State → Prop) + (p : D.Client α) (post : α → D.State → Prop) (budget : D.State → ℕ) : Prop := + D.Hoare pre p post ∧ + ∀ s, pre s → ∀ v s' tr, D.ClientRun s p v s' tr → D.totalCost charge s tr ≤ budget s + +namespace CostHoare + +variable {D : RelSpec.{u, v, w}} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} + {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} + +/-- Project a `CostHoare` triple to its underlying `Hoare` triple. -/ +theorem hoare (h : D.CostHoare charge pre p post budget) : D.Hoare pre p post := h.1 + +/-- Returning a pure value incurs zero operation cost. -/ +theorem pure (a : α) (h : ∀ s, pre s → post a s) : + D.CostHoare charge pre (.pure a) post (fun _ => 0) := by + refine ⟨Hoare.pure pre a post h, ?_⟩ + intro s _ v s' tr hrun + cases hrun + exact Nat.le_refl 0 + +/-- Calling a single operation charges `charge s o` at the pre-state `s`. -/ +theorem call (o : D.Op) {post : D.Result o → D.State → Prop} + (h : ∀ s, pre s → D.valid s o ∧ ∀ s' a, D.step s o s' a → post a s') : + D.CostHoare charge pre (.liftBind o fun a => .pure a) post (fun s => charge s o) := by + refine ⟨Hoare.call pre o post h, ?_⟩ + intro s _ v s' tr hrun + cases hrun with + | liftBind _ _ a _ _ hrest => + cases hrest + simp + +/-- Compose `CostHoare` budgets at every intermediate state satisfying `mid`. -/ +theorem bind {mid : α → D.State → Prop} {post : β → D.State → Prop} + {k : α → D.Client β} {first : D.State → ℕ} {next : α → D.State → ℕ} + (hp : D.CostHoare charge pre p mid first) + (hk : ∀ a, D.CostHoare charge (mid a) (k a) post (next a)) + (hb : ∀ s, pre s → ∀ a s', mid a s' → first s + next a s' ≤ budget s) : + D.CostHoare charge pre (p.bind k) post budget := by + refine ⟨Hoare.bind hp.hoare (fun a => (hk a).hoare), ?_⟩ + intro s hs b s₂ tr hrun + obtain ⟨a, s₁, tr₁, tr₂, hrun₁, hrun₂, rfl⟩ := clientRun_bind_iff.mp hrun + have hmid : mid a s₁ := (hp.hoare s hs).2 a s₁ tr₁ hrun₁ + have hf := hp.2 s hs a s₁ tr₁ hrun₁ + have hn := (hk a).2 s₁ hmid b s₂ tr₂ hrun₂ + rw [D.totalCost_append charge hrun₁.validTranscript tr₂] + exact (Nat.add_le_add hf hn).trans (hb s hs a s₁ hmid) + +/-- Consequence rule for demonic `CostHoare` triples. -/ +theorem consequence {pre' : D.State → Prop} {post' : α → D.State → Prop} + {budget' : D.State → ℕ} (h : D.CostHoare charge pre p post budget) + (hp : ∀ s, pre' s → pre s) (hq : ∀ a s, post a s → post' a s) + (hb : ∀ s, pre' s → budget s ≤ budget' s) : + D.CostHoare charge pre' p post' budget' := + ⟨h.hoare.consequence hp hq, + fun s hs v s' tr hrun => (h.2 s (hp s hs) v s' tr hrun).trans (hb s hs)⟩ + +end CostHoare + +end RelSpec + +/-! ## Equivalence of relational and deterministic client semantics on `D.toRelSpec` -/ + +namespace Spec + +variable (D : Spec.{u, v, w}) {α : Type y} + +/-- On `D.toRelSpec`, non-adaptive operation-sequence admissibility `ValidOpSeq` coincides with +`D.ValidHistory`. -/ +theorem toRelSpec_validOpSeq_iff (s : D.State) (ops : List D.Op) : + D.toRelSpec.ValidOpSeq s ops ↔ D.ValidHistory s ops := by + induction ops generalizing s with + | nil => rfl + | cons o ops ih => + change (D.valid s o ∧ ∀ s' a, (s', a) = D.step s o → D.toRelSpec.ValidOpSeq s' ops) ↔ + D.valid s o ∧ D.ValidHistory (D.step s o).1 ops + refine and_congr_right fun _ => ?_ + constructor + · intro h + exact (ih (D.step s o).1).mp (h (D.step s o).1 (D.step s o).2 Prod.mk.eta) + · intro h s' a heq + have hs' : s' = (D.step s o).1 := congrArg Prod.fst heq + subst hs' + exact (ih (D.step s o).1).mpr h + +/-- The unique canonical transcript determined by a deterministic specification `D` on an +operation list `os` from state `s`. -/ +def canonicalTranscript (s : D.State) : List D.Op → D.toRelSpec.Transcript + | [] => [] + | o :: os => + let next := D.step s o + ⟨o, next.2, next.1⟩ :: canonicalTranscript next.1 os + +@[simp] +theorem canonicalTranscript_nil (s : D.State) : D.canonicalTranscript s [] = [] := rfl + +@[simp] +theorem canonicalTranscript_cons (s : D.State) (o : D.Op) (os : List D.Op) : + D.canonicalTranscript s (o :: os) = + ⟨o, (D.step s o).2, (D.step s o).1⟩ :: D.canonicalTranscript (D.step s o).1 os := rfl + +/-- The underlying operation list of `D.canonicalTranscript s os` is `os`. -/ +@[simp] +theorem canonicalTranscript_ops (s : D.State) (os : List D.Op) : + (D.canonicalTranscript s os).ops = os := by + induction os generalizing s with + | nil => rfl + | cons o os ih => simp [ih] + +/-- `totalCost` along `D.canonicalTranscript s os` equals `D.totalCost charge s os`. -/ +@[simp] +theorem toRelSpec_totalCost_canonicalTranscript (charge : D.State → D.Op → ℕ) (s : D.State) + (os : List D.Op) : + D.toRelSpec.totalCost charge s (D.canonicalTranscript s os) = D.totalCost charge s os := by + induction os generalizing s with + | nil => rfl + | cons o os ih => + change charge s o + + D.toRelSpec.totalCost charge (D.step s o).1 (D.canonicalTranscript (D.step s o).1 os) = + charge s o + D.totalCost charge (D.step s o).1 os + rw [ih] + +/-- On `D.toRelSpec`, demonic client admissibility `DemonicValid s p` is equivalent to +`D.ValidHistory s (D.runClient s p).2.2`. -/ +theorem toRelSpec_demonicValid_iff (s : D.State) (p : D.Client α) : + D.toRelSpec.DemonicValid s p ↔ D.ValidHistory s (D.runClient s p).2.2 := by + induction p generalizing s with + | pure a => + exact ⟨fun _ => trivial, fun _ => RelSpec.DemonicValid.pure (D := D.toRelSpec) s a⟩ + | liftBind o k ih => + rw [RelSpec.demonicValid_liftBind_iff (D := D.toRelSpec), runClient_liftBind, validHistory_cons] + refine and_congr_right fun _ => ?_ + constructor + · intro h + exact (ih (D.step s o).2 (D.step s o).1).mp + (h (D.step s o).1 (D.step s o).2 Prod.mk.eta) + · intro h s' a heq + have hs' : s' = (D.step s o).1 := congrArg Prod.fst heq + have ha : a = (D.step s o).2 := congrArg Prod.snd heq + subst hs'; subst ha + exact (ih (D.step s o).2 (D.step s o).1).mpr h + +/-- Every client with an admissible deterministic history executes canonically in `D.toRelSpec`. -/ +theorem toRelSpec_clientRun_canonical (s : D.State) (p : D.Client α) + (hv : D.ValidHistory s (D.runClient s p).2.2) : + D.toRelSpec.ClientRun s p (D.runClient s p).1 (D.runClient s p).2.1 + (D.canonicalTranscript s (D.runClient s p).2.2) := by + induction p generalizing s with + | pure a => + exact RelSpec.ClientRun.pure (D := D.toRelSpec) s a + | liftBind o k ih => + exact RelSpec.ClientRun.liftBind (D := D.toRelSpec) o k (D.step s o).2 hv.1 Prod.mk.eta + (ih (D.step s o).2 (D.step s o).1 hv.2) + +private theorem toRelSpec_clientRun_imp {s : D.toRelSpec.State} {p : D.toRelSpec.Client α} + {v : α} {s' : D.toRelSpec.State} {tr : D.toRelSpec.Transcript} + (h : D.toRelSpec.ClientRun s p v s' tr) : + D.ValidHistory s (D.runClient s p).2.2 ∧ + v = (D.runClient s p).1 ∧ + s' = (D.runClient s p).2.1 ∧ + tr = D.canonicalTranscript s (D.runClient s p).2.2 := by + induction h with + | pure s a => + exact ⟨trivial, rfl, rfl, rfl⟩ + | @liftBind s s₁ s₂ o k res v tr hv hstep _ ih => + have hs₁ : s₁ = (D.step s o).1 := congrArg Prod.fst hstep + have hres : res = (D.step s o).2 := congrArg Prod.snd hstep + subst hs₁; subst hres + obtain ⟨hv_rest, rfl, rfl, rfl⟩ := ih + exact ⟨⟨hv, hv_rest⟩, rfl, rfl, rfl⟩ + +/-- On `D.toRelSpec`, `ClientRun` collapses uniquely to `D.runClient` and `D.ValidHistory`. -/ +theorem toRelSpec_clientRun_iff (s : D.State) (p : D.Client α) (v : α) (s' : D.State) + (tr : D.toRelSpec.Transcript) : + D.toRelSpec.ClientRun s p v s' tr ↔ + D.ValidHistory s (D.runClient s p).2.2 ∧ + v = (D.runClient s p).1 ∧ + s' = (D.runClient s p).2.1 ∧ + tr = D.canonicalTranscript s (D.runClient s p).2.2 := + ⟨D.toRelSpec_clientRun_imp, fun ⟨hv, h₁, h₂, h₃⟩ => + h₁ ▸ h₂ ▸ h₃ ▸ D.toRelSpec_clientRun_canonical s p hv⟩ + +/-- On `D.toRelSpec`, `RelSpec.Hoare` is equivalent to `Spec.Hoare`. -/ +theorem toRelSpec_hoare_iff (pre : D.State → Prop) (p : D.Client α) (post : α → D.State → Prop) : + D.toRelSpec.Hoare pre p post ↔ D.Hoare pre p post := by + refine forall_congr' fun s => imp_congr_right fun _ => ?_ + rw [toRelSpec_demonicValid_iff] + constructor + · rintro ⟨hv, hpost⟩ + exact ⟨hv, hpost _ _ _ (D.toRelSpec_clientRun_canonical s p hv)⟩ + · rintro ⟨hv, hpost⟩ + refine ⟨hv, fun v s' tr hrun => ?_⟩ + obtain ⟨_, rfl, rfl, _⟩ := (D.toRelSpec_clientRun_iff s p v s' tr).mp hrun + exact hpost + +/-- On `D.toRelSpec`, `RelSpec.CostHoare` is equivalent to `Spec.CostHoare`. -/ +theorem toRelSpec_costHoare_iff (charge : D.State → D.Op → ℕ) (pre : D.State → Prop) + (p : D.Client α) (post : α → D.State → Prop) (budget : D.State → ℕ) : + D.toRelSpec.CostHoare charge pre p post budget ↔ D.CostHoare charge pre p post budget := by + unfold RelSpec.CostHoare Spec.CostHoare + rw [toRelSpec_hoare_iff] + refine and_congr_right fun hhoare => forall_congr' fun s => imp_congr_right fun hs => ?_ + have hv := (hhoare s hs).1 + constructor + · intro h + have hb := h _ _ _ (D.toRelSpec_clientRun_canonical s p hv) + rwa [D.toRelSpec_totalCost_canonicalTranscript] at hb + · intro h v s' tr hrun + obtain ⟨_, _, _, rfl⟩ := (D.toRelSpec_clientRun_iff s p v s' tr).mp hrun + rwa [D.toRelSpec_totalCost_canonicalTranscript] + +end Spec + +/-! ## Native backend executions (`RelExecution`, `RelExecutes`), `RelRealization`, `RelAmortized` +-/ + +/-- A concrete backend execution relation for a relational specification `D : RelSpec`. -/ +abbrev RelExecution (D : RelSpec.{u, v, w}) (C Trace : Type x) := + C → (o : D.Op) → D.Result o → C → Trace → Prop + +/-- Adaptive client execution collecting native execution evidence in call order over a +relational specification `D : RelSpec`. -/ +inductive RelExecutes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + (exec : RelExecution D C τ) : D.Client α → C → α → C → List τ → Prop + | pure (a c) : RelExecutes exec (.pure a) c a c [] + | call {o k c a c' trace b c'' traces} : + exec c o a c' trace → RelExecutes exec (k a) c' b c'' traces → + RelExecutes exec (.liftBind o k) c b c'' (trace :: traces) + +/-- Sequential client execution splits its native trace at the call boundary. -/ +theorem relExecutes_bind_iff {D : RelSpec.{u, v, w}} {C τ : Type x} {α β : Type y} + (exec : RelExecution D C τ) (p : D.Client α) (k : α → D.Client β) + {c c' : C} {b : β} {ts : List τ} : + RelExecutes exec (p.bind k) c b c' ts ↔ + ∃ a mid front back, RelExecutes exec p c a mid front ∧ + RelExecutes exec (k a) mid b c' back ∧ ts = front ++ back := by + induction p generalizing c ts with + | pure a => + constructor + · intro h; exact ⟨a, c, [], ts, .pure a c, h, rfl⟩ + · rintro ⟨a', mid, front, back, hp, hk, ht⟩ + cases hp + simpa only [ht, List.nil_append, PFunctor.FreeM.bind] using hk + | liftBind o cont ih => + change RelExecutes exec (.liftBind o fun a => (cont a).bind k) c b c' ts ↔ _ + constructor + · intro h + cases h with + | call ho rest => + obtain ⟨a, mid, front, back, hp, hk, ht⟩ := (ih _).mp rest + exact ⟨a, mid, _, back, .call ho hp, hk, by simp [ht]⟩ + · rintro ⟨a, mid, front, back, hp, hk, ht⟩ + cases hp with + | call ho rest => + have hrest := (ih _).mpr ⟨a, mid, _, back, rest, hk, rfl⟩ + have h := RelExecutes.call (k := fun a => (cont a).bind k) ho hrest + simpa only [ht, List.cons_append] using h + +private theorem Spec.toRelSpec_relExecutes_imp {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : Execution D C τ} {p : D.toRelSpec.Client α} {c : C} {a : α} {c' : C} + {traces : List τ} (h : RelExecutes (D := D.toRelSpec) exec p c a c' traces) : + Executes exec p c a c' traces := by + induction h with + | pure a c => exact .pure a c + | call ho _ ih => exact .call ho ih + +/-- On `D.toRelSpec`, `RelExecutes` coincides with `Executes`. -/ +theorem Spec.toRelSpec_relExecutes_iff {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} + (exec : Execution D C τ) (p : D.Client α) (c : C) (a : α) (c' : C) (traces : List τ) : + RelExecutes (D := D.toRelSpec) exec p c a c' traces ↔ Executes exec p c a c' traces := by + refine ⟨Spec.toRelSpec_relExecutes_imp, fun h => ?_⟩ + induction h with + | pure a c => exact RelExecutes.pure (D := D.toRelSpec) a c + | call ho _ ih => exact RelExecutes.call (D := D.toRelSpec) ho ih + +/-- Angelic semantic realization of a relational specification `D : RelSpec`: +* `total`: every represented state and admissible operation admits a native execution; +* `correct`: every native execution from a represented state on an admissible operation realizes + *some* permitted abstract post-state `s'` with `D.step s o s' a ∧ rep s' c'`. -/ +structure RelRealization (D : RelSpec.{u, v, w}) {C τ : Type x} + (exec : RelExecution D C τ) where + /-- Representation invariant linking abstract states to concrete backend states. -/ + rep : D.State → C → Prop + /-- Existence of a native execution for every represented state and admissible operation. -/ + total : ∀ {s c} o, rep s c → D.valid s o → ∃ a c' trace, exec c o a c' trace + /-- Angelic step correctness: every native execution realizes some permitted abstract step. -/ + correct : ∀ {s c o a c' trace}, rep s c → D.valid s o → exec c o a c' trace → + ∃ s', D.step s o s' a ∧ rep s' c' + +namespace RelRealization + +variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} (R : RelRealization D exec) + +/-- From related states, every native execution of a demonically admissible client realizes a +valid `ClientRun` ending in a related state. -/ +theorem correct_client {p : D.Client α} {s : D.State} {c c' : C} {a : α} + {traces : List τ} (hr : R.rep s c) (hv : D.DemonicValid s p) + (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by + induction he generalizing s with + | pure a c => + exact ⟨s, [], .pure s a, hr, rfl⟩ + | @call o k c res c₁ trace b c₂ traces ho hes ih => + obtain ⟨hv_o, hv_k⟩ := (D.demonicValid_liftBind_iff s o k).mp hv + obtain ⟨s₁, hstep, hr₁⟩ := R.correct hr hv_o ho + obtain ⟨s₂, tr, hrun, hr₂, hlen⟩ := ih hr₁ (hv_k s₁ res hstep) + exact ⟨s₂, ⟨o, res, s₁⟩ :: tr, .liftBind o k res hv_o hstep hrun, hr₂, + congrArg (· + 1) hlen⟩ + +/-- From related states, every demonically admissible client admits a native execution realizing a +valid `ClientRun` ending in a related state. -/ +theorem exists_executes (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) + (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by + induction hv generalizing c with + | pure s a => + exact ⟨a, c, [], s, [], .pure a c, .pure s a, hr, rfl⟩ + | liftBind o k hv_o _ ih => + obtain ⟨res, c₁, trace, ho⟩ := R.total o hr hv_o + obtain ⟨s₁, hstep, hr₁⟩ := R.correct hr hv_o ho + obtain ⟨b, c₂, traces, s₂, tr, hes, hrun, hr₂, hlen⟩ := ih s₁ res hstep hr₁ + exact ⟨b, c₂, trace :: traces, s₂, ⟨o, res, s₁⟩ :: tr, + .call ho hes, .liftBind o k res hv_o hstep hrun, hr₂, congrArg (· + 1) hlen⟩ + +end RelRealization + +/-- Lift a deterministic `Realization D exec` to a relational `RelRealization D.toRelSpec exec`. -/ +def Realization.toRelRealization {D : Spec.{u, v, w}} {C τ : Type x} + {exec : Execution D C τ} (R : Realization D exec) : + RelRealization D.toRelSpec exec where + rep := R.rep + total o hr hv := R.total o hr hv + correct {s c o a c' trace} hr hv he := by + obtain ⟨ha, hr'⟩ := R.correct hr hv he + exact ⟨(D.step s o).1, Prod.ext rfl ha, hr'⟩ + +/-- Every native execution of a verified demonic `Hoare` client satisfies its abstract +postcondition in a related final state. -/ +theorem RelSpec.Hoare.realize {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} + {exec : RelExecution D C Trace} (R : RelRealization D exec) + {pre : D.State → Prop} {post : α → D.State → Prop} {p : D.Client α} + (h : D.Hoare pre p post) {s : D.State} {c c' : C} {a : α} {traces : List Trace} + (hp : pre s) (hr : R.rep s c) (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' := by + obtain ⟨hv, hpost⟩ := h s hp + obtain ⟨s', tr, hrun, hr', _⟩ := R.correct_client hr hv he + exact ⟨s', tr, hrun, hr', hpost a s' tr hrun⟩ + +/-- Per-operation worst-case resource certificate over a relational realization `R`. -/ +def RelBound {D : RelSpec.{u, v, w}} {C τ : Type x} {exec : RelExecution D C τ} + (R : RelRealization D exec) (weight : τ → ℕ) (charge : D.State → D.Op → ℕ) : Prop := + ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → + weight trace ≤ charge s o + +/-- Lift a deterministic `Bound R weight charge` to `RelBound R.toRelRealization weight charge`. -/ +theorem Bound.toRelBound {D : Spec.{u, v, w}} {C τ : Type x} {exec : Execution D C τ} + {R : Realization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + (B : Bound R weight charge) : + RelBound R.toRelRealization weight charge := + fun hr hv he => B hr hv he + +/-- Potential-method amortized resource certificate over a relational realization `R`: +for every native operation execution from `R.rep s c` on an admissible `o`, there exists a +permitted abstract post-state `s'` satisfying `D.step s o s' a`, `R.rep s' c'`, and the +single-step potential inequality `weight trace + potential s' c' ≤ charge s o + potential s c`. -/ +structure RelAmortized {D : RelSpec.{u, v, w}} {C Trace : Type x} + {exec : RelExecution D C Trace} (R : RelRealization D exec) (weight : Trace → ℕ) + (potential : D.State → C → ℕ) (charge : D.State → D.Op → ℕ) : Prop where + /-- Angelic single-step potential inequality witnessing a permitted post-state `s'`. -/ + step : ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → + ∃ s', D.step s o s' a ∧ R.rep s' c' ∧ + weight trace + potential s' c' ≤ charge s o + potential s c + +namespace RelAmortized + +variable {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} + {exec : RelExecution D C Trace} {R : RelRealization D exec} + {weight : Trace → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} + (A : RelAmortized R weight potential charge) + +include A in +/-- **Amortized telescoping along adaptive clients (`executes_le`).** +For any demonically admissible client `p` and native execution `RelExecutes exec p c a c' traces`, +there exist a final abstract state `s'` and transcript `tr` such that `D.ClientRun s p a s' tr`, +`R.rep s' c'`, and +`total weight traces + potential s' c' ≤ D.totalCost charge s tr + potential s c`. -/ +theorem executes_le {p : D.Client α} {s : D.State} {c c' : C} {a : α} + {traces : List Trace} (hr : R.rep s c) (hv : D.DemonicValid s p) + (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces + potential s' c' ≤ + D.totalCost charge s tr + potential s c := by + induction he generalizing s with + | pure a c => + exact ⟨s, [], .pure s a, hr, rfl, Nat.le_refl _⟩ + | @call o k c res c₁ trace b c₂ traces ho hes ih => + obtain ⟨hv_o, hv_k⟩ := (D.demonicValid_liftBind_iff s o k).mp hv + obtain ⟨s₁, hstep, hr₁, hpot₁⟩ := A.step hr hv_o ho + obtain ⟨s₂, tr, hrun, hr₂, hlen, hpot₂⟩ := ih hr₁ (hv_k s₁ res hstep) + refine ⟨s₂, ⟨o, res, s₁⟩ :: tr, .liftBind o k res hv_o hstep hrun, hr₂, + congrArg (· + 1) hlen, ?_⟩ + simp only [total_cons, RelSpec.totalCost_cons] + omega + +/-- Alias for `RelAmortized.executes_le` matching `Amortized.client`. -/ +abbrev client := @executes_le + +include A in +/-- **Existence, correctness, and amortized bound (`run_client_amortized`).** +From related initial states, every demonically admissible client `p` admits a native execution +satisfying `D.ClientRun`, `R.rep`, and the telescoping potential bound. -/ +theorem run_client_amortized (p : D.Client α) {s : D.State} {c : C} + (hr : R.rep s c) (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces + potential s' c' ≤ + D.totalCost charge s tr + potential s c := by + obtain ⟨a, c', traces, _, _, he, _, _, _⟩ := R.exists_executes p hr hv + obtain ⟨s', tr, hrun, hr', hlen, hpot⟩ := A.executes_le hr hv he + exact ⟨a, c', traces, s', tr, he, hrun, hr', hlen, hpot⟩ + +include A in +/-- Alias for `RelAmortized.run_client_amortized` matching `Amortized.exists_executes`. -/ +theorem exists_executes (p : D.Client α) {s : D.State} {c : C} + (hr : R.rep s c) (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces + potential s' c' ≤ + D.totalCost charge s tr + potential s c := + A.run_client_amortized p hr hv + +/-- A worst-case `RelBound` is a zero-potential `RelAmortized` certificate. -/ +theorem of_relBound (B : RelBound R weight charge) : + RelAmortized R weight (fun _ _ => 0) charge where + step hr hv he := by + obtain ⟨s', hstep, hr'⟩ := R.correct hr hv he + exact ⟨s', hstep, hr', by simpa using B hr hv he⟩ + +end RelAmortized + +/-- Lift a deterministic `Amortized R weight potential charge` certificate to +`RelAmortized R.toRelRealization weight potential charge`. -/ +theorem Amortized.toRelAmortized {D : Spec.{u, v, w}} {C Trace : Type x} + {exec : Execution D C Trace} {R : Realization D exec} + {weight : Trace → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} + (A : Amortized R weight potential charge) : + RelAmortized R.toRelRealization weight potential charge where + step {s c o a c' trace} hr hv he := by + obtain ⟨ha, hr'⟩ := R.correct hr hv he + exact ⟨(D.step s o).1, Prod.ext rfl ha, hr', A.step hr hv he⟩ + +namespace RelBound + +variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + (B : RelBound R weight charge) + +include B in +/-- Under a worst-case `RelBound`, every native execution of a demonically admissible client has +total weight at most `D.totalCost charge s tr` for some valid `ClientRun` transcript `tr`. -/ +theorem client {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List τ} + (hr : R.rep s c) (hv : D.DemonicValid s p) (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces ≤ D.totalCost charge s tr := by + obtain ⟨s', tr, hrun, hr', hlen, hpot⟩ := (RelAmortized.of_relBound B).executes_le hr hv he + exact ⟨s', tr, hrun, hr', hlen, by simpa using hpot⟩ + +include B in +/-- Under a worst-case `RelBound`, every demonically admissible client admits a native execution +meeting the total charge bound. -/ +theorem exists_executes (p : D.Client α) {s : D.State} {c : C} + (hr : R.rep s c) (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces ≤ D.totalCost charge s tr := by + obtain ⟨a, c', traces, s', tr, he, hrun, hr', hlen, hpot⟩ := + (RelAmortized.of_relBound B).run_client_amortized p hr hv + exact ⟨a, c', traces, s', tr, he, hrun, hr', hlen, by simpa using hpot⟩ + +end RelBound + +/-- Transport a demonic `CostHoare` triple through a `RelAmortized` certificate to bound native +execution weight plus final potential by `budget s + potential s c`. -/ +theorem RelSpec.CostHoare.realize_amortized {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} + {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} + {p : D.Client α} (A : RelAmortized R weight potential charge) + (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} + {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) + (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' ∧ + total weight traces + potential s' c' ≤ budget s + potential s c := by + obtain ⟨hv, hpost⟩ := h.1 s hs + obtain ⟨s', tr, hrun, hr', _, hpot⟩ := A.executes_le hr hv he + have hbud := h.2 s hs a s' tr hrun + exact ⟨s', tr, hrun, hr', hpost a s' tr hrun, by omega⟩ + +/-- Transport a demonic `CostHoare` triple through a worst-case `RelBound` certificate. -/ +theorem RelSpec.CostHoare.realize {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} + {p : D.Client α} (B : RelBound R weight charge) + (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} + {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) + (he : RelExecutes exec p c a c' traces) : + (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by + obtain ⟨s', _, _, hr', hpost, hpot⟩ := + h.realize_amortized (RelAmortized.of_relBound B) hs hr he + exact ⟨⟨s', hr', hpost⟩, by simpa using hpot⟩ + +/-- Existence of a native execution satisfying a demonic `CostHoare` contract under `RelBound`. -/ +theorem RelSpec.CostHoare.exists_executes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} + {p : D.Client α} (B : RelBound R weight charge) + (h : D.CostHoare charge pre p post budget) {s : D.State} {c : C} + (hs : pre s) (hr : R.rep s c) : + ∃ a c' traces, RelExecutes exec p c a c' traces ∧ + (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by + obtain ⟨a, c', traces, _, _, he, _, _, _⟩ := R.exists_executes p hr (h.1 s hs).1 + exact ⟨a, c', traces, he, h.realize B hs hr he⟩ + +end Cslib.Stateful diff --git a/ComplexityTests/StatefulRelational.lean b/ComplexityTests/StatefulRelational.lean new file mode 100644 index 000000000..8d3373331 --- /dev/null +++ b/ComplexityTests/StatefulRelational.lean @@ -0,0 +1,241 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: local-only (test driver) + +public import Complexity.Init +public import Complexity.Foundations.Semantics.Stateful.Relational + +/-! +# Tests for Relational Stateful Specifications + +Concrete non-vacuous tests for `Cslib.Stateful.RelSpec`, its totality class `RelSpec.Total`, +projection `RelSpec.toLTS`, zero-cost deterministic embedding `Spec.toRelSpec`, demonic adaptive +client semantics (`ClientRun`, `DemonicValid`, `RelSpec.Hoare`, `RelSpec.CostHoare`), and angelic +backend realization and amortized analysis (`RelRealization`, `RelBound`, `RelAmortized`). +-/ + +@[expose] public section + +namespace Cslib.Stateful.Tests + +open Cslib.Stateful + +/-! ## 1. A genuinely nondeterministic relational specification (two-slot allocator) -/ + +/-- Operations of a two-slot index allocator: `alloc` nondeterministically returns any free +slot in `Fin 2`; `free i` releases slot `i`. -/ +inductive TwoSlotOp : Type + | alloc : TwoSlotOp + | free : Fin 2 → TwoSlotOp + deriving DecidableEq + +/-- Relational specification of a two-slot allocator whose state `(b₀, b₁) : Bool × Bool` +records whether slots `0` and `1` are currently occupied. When both slots are free, `alloc` +nondeterministically returns either `0` or `1`. -/ +abbrev TwoSlotAlloc : RelSpec where + State := Bool × Bool + Op := TwoSlotOp + Result + | .alloc => Fin 2 + | .free _ => Unit + valid s + | .alloc => s.1 = false ∨ s.2 = false + | .free _ => True + step s o s' a := + match o, a with + | .alloc, i => + (i = 0 ∧ s.1 = false ∧ s' = (true, s.2)) ∨ + (i = 1 ∧ s.2 = false ∧ s' = (s.1, true)) + | .free i, () => + s' = if i = 0 then (false, s.2) else (s.1, false) + +instance : TwoSlotAlloc.Total where + step_total s o hv := by + cases o with + | alloc => + rcases hv with h0 | h1 + · exact ⟨(true, s.2), 0, Or.inl ⟨rfl, h0, rfl⟩⟩ + · exact ⟨(s.1, true), 1, Or.inr ⟨rfl, h1, rfl⟩⟩ + | free i => + exact ⟨if i = 0 then (false, s.2) else (s.1, false), (), rfl⟩ + +/-- Both slot `0` and slot `1` are valid outcomes of `alloc` from `(false, false)`. -/ +theorem twoSlotAlloc_nondeterministic : + ((true, false), (0 : Fin 2)) ∈ TwoSlotAlloc.outcomes (false, false) .alloc ∧ + ((false, true), (1 : Fin 2)) ∈ TwoSlotAlloc.outcomes (false, false) .alloc := + ⟨Or.inl ⟨rfl, rfl, rfl⟩, Or.inr ⟨rfl, rfl, rfl⟩⟩ + +/-- Sample LTS transition via `RelSpec.toLTS`. -/ +theorem twoSlotAlloc_toLTS_tr : + TwoSlotAlloc.toLTS.Tr (false, false) ⟨.alloc, (1 : Fin 2)⟩ (false, true) := + Or.inr ⟨rfl, rfl, rfl⟩ + +/-! ## 2. Adaptive client, demonic admissibility, and Hoare / CostHoare verification -/ + +/-- Adaptive client that allocates two slots sequentially, frees the first slot returned, and +returns the pair `(i, j)` of allocated slots. -/ +abbrev allocTwoAndFreeFirst : TwoSlotAlloc.Client (Fin 2 × Fin 2) := + .liftBind TwoSlotOp.alloc fun i => + .liftBind TwoSlotOp.alloc fun j => + .liftBind (TwoSlotOp.free i) fun _ => + .pure (i, j) + +/-- From `(false, false)`, `allocTwoAndFreeFirst` is demonically admissible across all +nondeterministic choices of the first and second allocations. -/ +theorem allocTwoAndFreeFirst_demonicValid : + TwoSlotAlloc.DemonicValid (false, false) allocTwoAndFreeFirst := by + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) TwoSlotOp.alloc _ (Or.inl rfl) + fun s₁ i h₁ => ?_ + have hv₂ : TwoSlotAlloc.valid s₁ TwoSlotOp.alloc := by + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · exact Or.inr rfl + · exact Or.inl rfl + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) TwoSlotOp.alloc _ hv₂ + fun s₂ j _ => ?_ + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) (TwoSlotOp.free i) _ trivial + fun s₃ _ _ => .pure s₃ (i, j) + +/-- Demonic `CostHoare` specification: under unit per-operation charge, every relational execution +of `allocTwoAndFreeFirst` from `(false, false)` returns distinct slots `i ≠ j`, leaves `i` free +and `j` occupied, and incurs abstract cost `3`. -/ +theorem allocTwoAndFreeFirst_costHoare : + TwoSlotAlloc.CostHoare (fun _ _ => 1) + (fun s => s = (false, false)) + allocTwoAndFreeFirst + (fun p s' => p.1 ≠ p.2 ∧ (if p.1 = 0 then s'.1 = false else s'.2 = false)) + (fun _ => 3) := by + refine ⟨fun s hs => ?_, fun s hs v s' tr hrun => ?_⟩ + · subst hs + refine ⟨allocTwoAndFreeFirst_demonicValid, fun v s' tr hrun => ?_⟩ + obtain ⟨_, i, s₁, tr₁, h₁, hrun₁, _⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun + obtain ⟨_, j, s₂, tr₂, h₂, hrun₂, _⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₁ + obtain ⟨_, _, s₃, tr₃, h₃, hrun₃, _⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₂ + obtain ⟨rfl, rfl, rfl⟩ := (RelSpec.clientRun_pure_iff ..).mp hrun₃ + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · rcases h₂ with ⟨_, hfalse, _⟩ | ⟨rfl, _, rfl⟩ + · cases hfalse + · subst h₃ + exact ⟨by decide, by decide⟩ + · rcases h₂ with ⟨rfl, _, rfl⟩ | ⟨_, hfalse, _⟩ + · subst h₃ + exact ⟨by decide, by decide⟩ + · cases hfalse + · subst hs + obtain ⟨_, i, s₁, tr₁, _, hrun₁, rfl⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun + obtain ⟨_, j, s₂, tr₂, _, hrun₂, rfl⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₁ + obtain ⟨_, _, s₃, tr₃, _, hrun₃, rfl⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₂ + obtain ⟨rfl, rfl, rfl⟩ := (RelSpec.clientRun_pure_iff ..).mp hrun₃ + simp + +/-- Consequence to plain demonic `Hoare`. -/ +theorem allocTwoAndFreeFirst_hoare : + TwoSlotAlloc.Hoare + (fun s => s = (false, false)) + allocTwoAndFreeFirst + (fun p s' => p.1 ≠ p.2 ∧ (if p.1 = 0 then s'.1 = false else s'.2 = false)) := + allocTwoAndFreeFirst_costHoare.hoare + +/-! ## 3. Concrete first-fit backend realization and amortized certificate -/ + +/-- Concrete first-fit implementation of `TwoSlotAlloc`: `alloc` chooses slot `0` when free and +slot `1` otherwise, recording `1` unit of execution trace cost per operation. -/ +def firstFitExec : RelExecution TwoSlotAlloc (Bool × Bool) ℕ + | c, .alloc, i, c', n => + n = 1 ∧ + if c.1 = false then i = 0 ∧ c' = (true, c.2) + else i = 1 ∧ c' = (c.1, true) + | c, .free i, (), c', n => + n = 1 ∧ c' = (if i = 0 then (false, c.2) else (c.1, false)) + +/-- Angelic realization of `TwoSlotAlloc` by `firstFitExec`. -/ +def firstFitRealization : RelRealization TwoSlotAlloc firstFitExec where + rep s c := s = c + total {s c} o hr _ := by + subst hr + cases o with + | alloc => + rcases Bool.dichotomy s.1 with h0 | h0 + · exact ⟨0, (true, s.2), 1, rfl, by simp [h0]⟩ + · exact ⟨1, (s.1, true), 1, rfl, by simp [h0]⟩ + | free i => + exact ⟨(), if i = 0 then (false, s.2) else (s.1, false), 1, rfl, rfl⟩ + correct {s c o a c' n} hr hv he := by + subst hr + cases o with + | alloc => + obtain ⟨_, hbranch⟩ := he + rcases Bool.dichotomy s.1 with h0 | h0 + · simp only [h0, ↓reduceIte] at hbranch + obtain ⟨rfl, rfl⟩ := hbranch + exact ⟨(true, s.2), Or.inl ⟨rfl, h0, rfl⟩, rfl⟩ + · simp only [h0, Bool.true_eq_false, ↓reduceIte] at hbranch + obtain ⟨rfl, rfl⟩ := hbranch + have h1 : s.2 = false := hv.resolve_left (by simp [h0]) + exact ⟨(s.1, true), Or.inr ⟨rfl, h1, rfl⟩, by simp [h0]⟩ + | free i => + obtain ⟨_, rfl⟩ := he + cases a + exact ⟨if i = 0 then (false, s.2) else (s.1, false), rfl, rfl⟩ + +/-- Per-operation worst-case bound for `firstFitRealization`. -/ +theorem firstFitBound : + RelBound firstFitRealization id (fun _ _ => 1) := by + intro s c o a c' n _ _ he + cases o with + | alloc => exact Nat.le_of_eq he.1 + | free _ => exact Nat.le_of_eq he.1 + +/-- Amortized certificate for `firstFitRealization` with zero potential. -/ +theorem firstFitAmortized : + RelAmortized firstFitRealization id (fun _ _ => 0) (fun _ _ => 1) := + RelAmortized.of_relBound firstFitBound + +/-- End-to-end execution of `allocTwoAndFreeFirst` on `firstFitRealization` via +`RelAmortized.run_client_amortized` and `RelSpec.CostHoare.realize_amortized`. -/ +theorem firstFit_allocTwoAndFreeFirst_executes : + ∃ v c' traces s' tr, + RelExecutes firstFitExec allocTwoAndFreeFirst (false, false) v c' traces ∧ + TwoSlotAlloc.ClientRun (false, false) allocTwoAndFreeFirst v s' tr ∧ + firstFitRealization.rep s' c' ∧ + (v.1 ≠ v.2 ∧ (if v.1 = 0 then s'.1 = false else s'.2 = false)) ∧ + total id traces ≤ 3 := by + obtain ⟨v, c', traces, _, _, he, _, _, _, _⟩ := + firstFitAmortized.run_client_amortized allocTwoAndFreeFirst + (s := (false, false)) (c := (false, false)) rfl allocTwoAndFreeFirst_demonicValid + obtain ⟨s', tr, hrun, hr, hpost, hcost⟩ := + allocTwoAndFreeFirst_costHoare.realize_amortized firstFitAmortized rfl rfl he + exact ⟨v, c', traces, s', tr, he, hrun, hr, hpost, by omega⟩ + +/-! ## 4. Layer 1 (`Spec.toRelSpec`) embedding and collapse equivalences -/ + +/-- Simple deterministic counter specification to exercise `Spec.toRelSpec`. -/ +abbrev CounterSpec : Spec where + State := ℕ + Op := Unit + Result _ := ℕ + step n _ := (n + 1, n) + +/-- Adaptive client on `CounterSpec` that increments twice and returns the sum of results. -/ +def counterTwice : CounterSpec.Client ℕ := + .liftBind () fun a => .liftBind () fun b => .pure (a + b) + +/-- `DemonicValid` on `CounterSpec.toRelSpec` collapses to `ValidHistory` on `CounterSpec`. -/ +theorem counterTwice_demonicValid (n : ℕ) : + CounterSpec.toRelSpec.DemonicValid n counterTwice := by + rw [CounterSpec.toRelSpec_demonicValid_iff] + exact ⟨trivial, trivial, trivial⟩ + +/-- `RelSpec.Hoare` on `CounterSpec.toRelSpec` is equivalent to `Spec.Hoare`. -/ +theorem counterTwice_relHoare : + CounterSpec.toRelSpec.Hoare (fun n => n = 5) counterTwice (fun v s' => v = 11 ∧ s' = 7) := by + rw [CounterSpec.toRelSpec_hoare_iff] + intro s hs + subst hs + exact ⟨⟨trivial, trivial, trivial⟩, rfl, rfl⟩ + +end Cslib.Stateful.Tests diff --git a/ComplexityTests/WordRAMRelSubroutine.lean b/ComplexityTests/WordRAMRelSubroutine.lean new file mode 100644 index 000000000..1eecc6baa --- /dev/null +++ b/ComplexityTests/WordRAMRelSubroutine.lean @@ -0,0 +1,351 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: local-only (test driver) + +public import Complexity.Init +public import Complexity.Computability.WordRAM.RelSubroutine + +/-! +# Tests for Relational Word-RAM Subroutine Realizations + +Concrete non-vacuous tests for `Cslib.WordRAM.RelSubroutineDS`, `RelDynamicProgram`, +`RelDynamicCertificate`, `SubroutineDS.toRelSubroutineDS`, `RelSubroutineDS.toStandard`, +`RelSubroutineDS.call_rename`, `RelSubroutineDS.run_client_amortized`, +`RelSubroutineDS.run_ops_exact_fresh`, and `RelSubroutineDS.run_ops_exact_reusable` on a +genuinely nondeterministic two-slot allocator specification. +-/ + +@[expose] public section + +namespace Cslib.WordRAM.Tests + +open Cslib Cslib.Stateful Cslib.WordRAM + +/-! ## 1. Nondeterministic two-slot allocator specification -/ + +/-- Relational specification of a two-slot allocator: from state `(b₀, b₁) : Bool × Bool`, an +allocation operation `()` nondeterministically returns any free slot `i : Fin 2` and marks slot +`i` occupied. When both slots are free (`(false, false)`), both `0` and `1` are permitted. -/ +abbrev TwoSlotAlloc : RelSpec where + State := Bool × Bool + Op := Unit + Result _ := Fin 2 + valid s _ := s.1 = false ∨ s.2 = false + step s _ s' i := + (i = 0 ∧ s.1 = false ∧ s' = (true, s.2)) ∨ + (i = 1 ∧ s.2 = false ∧ s' = (s.1, true)) + +instance : TwoSlotAlloc.Total where + step_total s _ hv := by + rcases hv with h0 | h1 + · exact ⟨(true, s.2), 0, Or.inl ⟨rfl, h0, rfl⟩⟩ + · exact ⟨(s.1, true), 1, Or.inr ⟨rfl, h1, rfl⟩⟩ + +/-! ## 2. Concrete `SubroutineCode .exact` implementation and `RelSubroutineDS` certificate -/ + +/-- Six-register `.exact` Word-RAM subroutine implementing `TwoSlotAlloc` by sequential +allocation of slot `0` followed by slot `1`: +* register `0`: `baseReg` (persistent) +* register `1`: `frontierReg` (persistent) +* register `2`: `curReg` (scratch) +* register `3`: `zeroReg` (scratch) +* register `4`: `oneReg` / `output` (scratch) +* register `5`: persistent next-free-slot counter (`0`, `1`, or `2`). -/ +abbrev twoSlotCode : SubroutineCode .exact 6 0 where + allocSpan := .op (.copy 0 1) + initSetup := .op (.const 5 0) + code := .seq (.op (.copy 4 5)) (.seq (.op (.const 3 1)) (.op (.exactBin .add 5 5 3))) + frontierReg := 1 + baseReg := 0 + initParams := Fin.elim0 + curReg := 2 + zeroReg := 3 + oneReg := 4 + incCur := .exactBin .add 2 2 4 + input i := i == 2 || i == 3 || i == 4 + output := 4 + +/-- Representation invariant for `twoSlotCode` at word width `w` and base address `base`. -/ +abbrev twoSlotRep (w base : ℕ) (s : Bool × Bool) (x : Data 6) : Prop := + x.Bounded w ∧ x.regs 0 = base ∧ + ((s = (false, false) ∧ x.regs 5 = 0) ∨ + (s = (true, false) ∧ x.regs 5 = 1) ∨ + (s = (true, true) ∧ x.regs 5 = 2)) + +/-- `RelSubroutineDS` certificate showing that `twoSlotCode` realizes the nondeterministic +`TwoSlotAlloc` specification whenever `Fits w 2`. -/ +abbrev twoSlotSubroutine (w base : ℕ) (h2 : Fits w 2) : + RelSubroutineDS twoSlotCode TwoSlotAlloc w base Fin.elim0 (false, false) where + space := 0 + spanTime := 1 + setupTime := 1 + initPotential := 0 + charge _ _ := 3 + rep := twoSlotRep w base + potential _ := 0 + encode _ _ := 0 + argMatch _ _ := True + decode _ regs := some ⟨regs 4 % 2, Nat.mod_lt _ (by decide)⟩ + bounded hr := hr.1 + arguments_fit _ _ _ _ := fits_zero w + encode_matches _ _ := trivial + rep_supported {s x x'} hregs _ hx' hr := by + refine ⟨hx', ?_, ?_⟩ + · rw [hregs 0 rfl]; exact hr.2.1 + · rw [hregs 5 rfl]; exact hr.2.2 + potential_supported _ _ _ := rfl + clear_nodup := by decide + params_not_clear j := j.elim0 + incCur_eval {x} hone hfit := by + simp only [Op.eval_exactBin, BinOp.evalExact, hone, Option.bind_some, Data.put_of_fits hfit] + span_exec {x} _ hfr _ := by + refine ⟨x.set 0 (x.regs 1), ⟨1, Nat.le_refl 1, Exec.copy⟩, rfl, ?_, ?_, fun j => j.elim0⟩ + · simp [Data.set, hfr] + · simp [Data.set, hfr] + setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by + refine ⟨x₁.set 5 0, ⟨1, Nat.le_refl 1, Exec.const (fits_zero w)⟩, ?_, ?_, Nat.le_refl 0, + fun _ _ => rfl⟩ + · simp [Data.set, hfr₁] + · refine ⟨hx₁.set (fits_zero w), ?_, Or.inl ⟨rfl, by simp [Data.set]⟩⟩ + simp [Data.set, hb₁] + step_exec {s x} _ hr hv _ := by + have h1 : Fits w 1 := h2.of_le (by decide) + rcases hr with ⟨hx, hb, ⟨rfl, h5⟩ | ⟨rfl, h5⟩ | ⟨rfl, _⟩⟩ + · let x₁ := x.set 4 (x.regs 5) + let x₂ := x₁.set 3 1 + let y := x₂.set 5 (x₂.regs 5 + x₂.regs 3) + have hfit : Fits w (x₂.regs 5 + x₂.regs 3) := by + simpa [x₂, x₁, Data.set, h5] using h1 + have heval : (Op.exactBin .add (5 : Fin 6) 5 3).eval w x₂ = some y := by + simp only [Op.eval_exactBin, BinOp.evalExact, Option.bind_some, Data.put_of_fits hfit, y] + have hex : Exec w twoSlotCode.code x y 3 := + Exec.copy.seq ((Exec.const h1).seq (Exec.op heval)) + have hy_b : y.Bounded w := hex.bounded hx + refine ⟨y, (true, false), 0, 3, hex, Or.inl ⟨rfl, rfl, rfl⟩, ?_, ?_, + fun _ _ => rfl, Nat.le_refl 3⟩ + · simp [y, x₂, x₁, Data.set, h5] + · refine ⟨hy_b, ?_, Or.inr (Or.inl ⟨rfl, ?_⟩)⟩ + · simp [y, x₂, x₁, Data.set, hb] + · simp [y, x₂, x₁, Data.set, h5] + · let x₁ := x.set 4 (x.regs 5) + let x₂ := x₁.set 3 1 + let y := x₂.set 5 (x₂.regs 5 + x₂.regs 3) + have hfit : Fits w (x₂.regs 5 + x₂.regs 3) := by + simpa [x₂, x₁, Data.set, h5] using h2 + have heval : (Op.exactBin .add (5 : Fin 6) 5 3).eval w x₂ = some y := by + simp only [Op.eval_exactBin, BinOp.evalExact, Option.bind_some, Data.put_of_fits hfit, y] + have hex : Exec w twoSlotCode.code x y 3 := + Exec.copy.seq ((Exec.const h1).seq (Exec.op heval)) + have hy_b : y.Bounded w := hex.bounded hx + refine ⟨y, (true, true), 1, 3, hex, Or.inr ⟨rfl, rfl, rfl⟩, ?_, ?_, + fun _ _ => rfl, Nat.le_refl 3⟩ + · simp [y, x₂, x₁, Data.set, h5] + · refine ⟨hy_b, ?_, Or.inr (Or.inr ⟨rfl, ?_⟩)⟩ + · simp [y, x₂, x₁, Data.set, hb] + · simp [y, x₂, x₁, Data.set, h5] + · cases hv with + | inl h => cases h + | inr h => cases h + +/-! ## 3. Exercising `call_rename`, `run_client_amortized`, and flat-program execution -/ + +/-- Adaptive client allocating two slots sequentially and returning both indices. -/ +abbrev allocPair : TwoSlotAlloc.Client (Fin 2 × Fin 2) := + .liftBind () fun i => + .liftBind () fun j => + .pure (i, j) + +/-- `allocPair` is demonically admissible from `(false, false)` across all permitted outcomes. -/ +theorem allocPair_demonicValid : + TwoSlotAlloc.DemonicValid (false, false) allocPair := by + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) () _ (Or.inl rfl) fun s₁ i h₁ => ?_ + have hv₂ : TwoSlotAlloc.valid s₁ () := by + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · exact Or.inr rfl + · exact Or.inl rfl + exact RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) () _ hv₂ + fun s₂ j _ => .pure s₂ (i, j) + +/-- Standalone adaptive-client amortized theorem via `RelSubroutineDS.toStandard` and +`RelSubroutineDS.run_client_amortized`. -/ +theorem twoSlot_run_client_amortized (w base : ℕ) (h2 : Fits w 2) + {x : Data 6} (hr : (twoSlotSubroutine w base h2).rep (false, false) x) : + ∃ a y traces s' tr, + Stateful.RelExecutes + ((twoSlotSubroutine w base h2).toStandard.toRelDynamicProgram.ExecOp w) + allocPair x a y traces ∧ + TwoSlotAlloc.ClientRun (false, false) allocPair a s' tr ∧ + (twoSlotSubroutine w base h2).rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ + TwoSlotAlloc.totalCost (fun _ _ => 4) (false, false) tr := + (twoSlotSubroutine w base h2).toStandard.run_client_bound (fun _ => rfl) + allocPair hr allocPair_demonicValid + +/-- Canonical injective register embedding `Fin 6 → Fin 8`. -/ +def embed6to8 : Fin 6 → Fin 8 := Fin.castLE (by decide) + +theorem embed6to8_injective : Function.Injective embed6to8 := + (Fin.castLEOrderEmb (by decide : 6 ≤ 8)).injective + +/-- Relocated inline call via `RelSubroutineDS.call_rename` inside an 8-register caller. -/ +theorem twoSlot_call_rename (w base : ℕ) (h2 : Fits w 2) + {x : Data 8} (hx : (x.comap embed6to8).Bounded w) + (hr : (twoSlotSubroutine w base h2).rep (false, false) (x.comap embed6to8)) : + ∃ z s' a t, + ExecWithin w (.seq (.op (.const 2 0)) (twoSlotCode.code.rename embed6to8)) x z t ∧ + TwoSlotAlloc.step (false, false) () s' a ∧ + (twoSlotSubroutine w base h2).decode () + (z.comap embed6to8).regs = some a ∧ + (twoSlotSubroutine w base h2).rep s' (z.comap embed6to8) ∧ + Modifies (Block.mk base 0).footprint (x.set 2 0).mem z.mem ∧ + (∀ j ∉ Set.range embed6to8, z.regs j = (x.set 2 0).regs j) ∧ + t ≤ 4 := by + have hprep : ExecWithin (B := .exact) w (.op (.const (2 : Fin 8) 0)) x (x.set 2 0) 1 := + ⟨1, Nat.le_refl 1, Exec.const (fits_zero w)⟩ + have hb' : ((x.set 2 0).comap embed6to8).Bounded w := by + refine ⟨fun i => ?_, hx.2⟩ + by_cases hi : embed6to8 i = 2 + · simp [Data.comap, Data.set, hi, fits_zero w] + · simp only [Fin.isValue, Data.comap_regs, Data.set_regs, Function.update_of_ne hi] + exact hx.1 i + have hreg : ∀ i : Fin 6, twoSlotCode.input i = false → + (x.set 2 0).regs (embed6to8 i) = x.regs (embed6to8 i) := by + intro i hi + have hne : embed6to8 i ≠ 2 := by + fin_cases i <;> revert hi <;> decide + simp [Data.set, Function.update_of_ne hne] + obtain ⟨z, s', a, t, he, hstep, hdec, hrep, hmod, hframe, ht⟩ := + (twoSlotSubroutine w base h2).call_rename + embed6to8_injective () hr (Or.inl rfl) hprep hb' hreg (fun _ _ => rfl) trivial + exact ⟨z, s', a, t, he, hstep, hdec, hrep, hmod, hframe, by omega⟩ + +/-- Two-operation sequence `[(), ()]` is demonically valid from `(false, false)`. -/ +theorem twoSlot_validOpSeq : + TwoSlotAlloc.ValidOpSeq (false, false) [(), ()] := by + refine ⟨Or.inl rfl, fun s₁ a₁ h₁ => ?_⟩ + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · exact ⟨Or.inr rfl, fun _ _ _ => trivial⟩ + · exact ⟨Or.inl rfl, fun _ _ _ => trivial⟩ + +/-- Flat-program execution on fresh zeroed memory via `RelSubroutineDS.run_ops_exact_fresh`. -/ +theorem twoSlot_run_ops_fresh (w base : ℕ) (h2 : Fits w 2) + {d₀ : Data 6} (hd₀ : d₀.Bounded w) (hfr : d₀.regs 1 = base) + (hz : ∀ a ∈ (Block.mk base 0).footprint, d₀.mem a = 0) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w twoSlotCode.initFreshProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w twoSlotCode.initFreshProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base 0).footprint d₀.mem x₀.mem ∧ + (twoSlotSubroutine w base h2).RunsRelOps + (twoSlotSubroutine w base h2).prepareRegs (false, false) x₀ [(), ()] tr s_fin z + t_ops ∧ + tr.ops = [(), ()] ∧ + TwoSlotAlloc.ValidTranscript (false, false) tr s_fin ∧ + (twoSlotSubroutine w base h2).rep s_fin z ∧ + t_init ≤ 2 ∧ + t_ops ≤ TwoSlotAlloc.totalCost (fun _ _ => 4) (false, false) tr := by + obtain ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, ht_ops⟩ := + (twoSlotSubroutine w base h2).run_ops_exact_fresh + hd₀ hfr (fun j => j.elim0) hz [(), ()] twoSlot_validOpSeq + exact ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, by simpa using ht_ops⟩ + +/-- Flat-program execution on reusable dirty memory via `RelSubroutineDS.run_ops_exact_reusable`. -/ +theorem twoSlot_run_ops_reusable (w base : ℕ) (h2 : Fits w 2) + {d₀ : Data 6} (hd₀ : d₀.Bounded w) (hfr : d₀.regs 1 = base) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w twoSlotCode.initClearProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w twoSlotCode.initClearProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base 0).footprint d₀.mem x₀.mem ∧ + (twoSlotSubroutine w base h2).RunsRelOps + (twoSlotSubroutine w base h2).prepareRegs (false, false) x₀ [(), ()] tr s_fin z + t_ops ∧ + tr.ops = [(), ()] ∧ + TwoSlotAlloc.ValidTranscript (false, false) tr s_fin ∧ + (twoSlotSubroutine w base h2).rep s_fin z ∧ + t_init ≤ 8 ∧ + t_ops ≤ TwoSlotAlloc.totalCost (fun _ _ => 4) (false, false) tr := by + obtain ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, ht_ops⟩ := + (twoSlotSubroutine w base h2).run_ops_exact_reusable + hd₀ hfr (fun j => j.elim0) [(), ()] twoSlot_validOpSeq + exact ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, by simpa using ht_ops⟩ + +/-! ## 4. Lifting a deterministic `SubroutineDS` via `SubroutineDS.toRelSubroutineDS` -/ + +/-- Deterministic specification of a bounded two-step counter (`0 → 1 → 2`). -/ +abbrev DetCounter2 : Spec where + State := ℕ + Op := Unit + Result _ := Fin 2 + valid n _ := n < 2 + step n _ := (n + 1, ⟨n % 2, Nat.mod_lt _ (by decide)⟩) + +/-- Deterministic `SubroutineDS` certificate for `twoSlotCode` over `DetCounter2`. -/ +abbrev detCounter2Subroutine (w base : ℕ) (h2 : Fits w 2) : + SubroutineDS twoSlotCode DetCounter2 w base Fin.elim0 0 where + space := 0 + spanTime := 1 + setupTime := 1 + initPotential := 0 + charge _ _ := 3 + rep n x := x.Bounded w ∧ x.regs 0 = base ∧ x.regs 5 = n ∧ n ≤ 2 + potential _ := 0 + encode _ _ := 0 + argMatch _ _ := True + decode _ regs := some ⟨regs 4 % 2, Nat.mod_lt _ (by decide)⟩ + bounded hr := hr.1 + rep_unique hr₁ hr₂ := hr₁.2.2.1.symm.trans hr₂.2.2.1 + arguments_fit _ _ _ _ := fits_zero w + encode_matches _ _ := trivial + rep_supported {s x x'} hregs _ hx' hr := by + refine ⟨hx', ?_, ?_, hr.2.2.2⟩ + · rw [hregs 0 rfl]; exact hr.2.1 + · rw [hregs 5 rfl]; exact hr.2.2.1 + potential_supported _ _ _ := rfl + clear_nodup := by decide + params_not_clear j := j.elim0 + incCur_eval {x} hone hfit := by + simp only [Op.eval_exactBin, BinOp.evalExact, hone, Option.bind_some, Data.put_of_fits hfit] + span_exec {x} _ hfr _ := by + refine ⟨x.set 0 (x.regs 1), ⟨1, Nat.le_refl 1, Exec.copy⟩, rfl, ?_, ?_, fun j => j.elim0⟩ + · simp [Data.set, hfr] + · simp [Data.set, hfr] + setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by + refine ⟨x₁.set 5 0, ⟨1, Nat.le_refl 1, Exec.const (fits_zero w)⟩, ?_, ?_, Nat.le_refl 0, + fun _ _ => rfl⟩ + · simp [Data.set, hfr₁] + · refine ⟨hx₁.set (fits_zero w), ?_, ?_, Nat.zero_le 2⟩ + · simp [Data.set, hb₁] + · simp [Data.set] + step_exec {s x} _ hr hv _ := by + have h1 : Fits w 1 := h2.of_le (by decide) + obtain ⟨hx, hb, h5, _⟩ := hr + let x₁ := x.set 4 (x.regs 5) + let x₂ := x₁.set 3 1 + let y := x₂.set 5 (x₂.regs 5 + x₂.regs 3) + have hfit : Fits w (x₂.regs 5 + x₂.regs 3) := by + have hfit2 : Fits w (s + 1) := h2.of_le hv + simpa [x₂, x₁, Data.set, h5] using hfit2 + have heval : (Op.exactBin .add (5 : Fin 6) 5 3).eval w x₂ = some y := by + simp only [Op.eval_exactBin, BinOp.evalExact, Option.bind_some, Data.put_of_fits hfit, y] + have hex : Exec w twoSlotCode.code x y 3 := + Exec.copy.seq ((Exec.const h1).seq (Exec.op heval)) + refine ⟨y, 3, hex, ?_, ⟨hex.bounded hx, ?_, ?_, hv⟩, fun _ _ => rfl, Nat.le_refl 3⟩ + · simp [y, x₂, x₁, Data.set, h5] + · simp [y, x₂, x₁, Data.set, hb] + · simp [y, x₂, x₁, Data.set, h5] + +/-- Lifting `detCounter2Subroutine` to `RelSubroutineDS` via `SubroutineDS.toRelSubroutineDS`. -/ +def detCounter2RelSubroutine (w base : ℕ) (h2 : Fits w 2) : + RelSubroutineDS twoSlotCode DetCounter2.toRelSpec w base Fin.elim0 0 := + (detCounter2Subroutine w base h2).toRelSubroutineDS + +end Cslib.WordRAM.Tests From 012511e6d019284380f7e13d76dcc4cf2cb5a0d9 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 16:33:45 +0000 Subject: [PATCH 02/18] feat(wordram): geometric Las Vegas loop expected-time bound and randomized SubroutineDS (#1634) --- Complexity.lean | 1 + .../WordRAM/Randomized/Expected.lean | 1738 ++++++++++++++++- .../WordRAM/Randomized/Subroutine.lean | 1046 ++++++++++ .../WordRAMRandomizedSubroutine.lean | 350 ++++ 4 files changed, 3133 insertions(+), 2 deletions(-) create mode 100644 Complexity/Computability/WordRAM/Randomized/Subroutine.lean create mode 100644 ComplexityTests/WordRAMRandomizedSubroutine.lean diff --git a/Complexity.lean b/Complexity.lean index 227ff8d31..b339fe51f 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -2009,6 +2009,7 @@ public import Complexity.Computability.WordRAM.Randomized.Defs public import Complexity.Computability.WordRAM.Randomized.Expected public import Complexity.Computability.WordRAM.Randomized.LoopSegment public import Complexity.Computability.WordRAM.Randomized.Structured +public import Complexity.Computability.WordRAM.Randomized.Subroutine public import Complexity.Computability.WordRAM.RegisterFrame public import Complexity.Computability.WordRAM.Sequence public import Complexity.Computability.WordRAM.Signed.Arithmetic diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index 7d9e1844a..acfe26e2d 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -47,6 +47,9 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, remaining instructions from a `RandCfg r` or initial `State r`. * `Cslib.WordRAM.expectedCodeTime`: `ℝ≥0∞`-valued expected instruction count of a structured `Code B r`. +* `Cslib.WordRAM.ExecCodeDist`: compositional structured execution certificate pairing an output + distribution `μ : PMF (Data r)` with an expected step bound `T : ℝ≥0∞`, closed under unbounded + almost-surely terminating geometric `while` loops (`ExecCodeDist.loop_geometric`). ## Main results @@ -63,6 +66,10 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, * `Cslib.WordRAM.expectedCodeTime_seq`, `Cslib.WordRAM.expectedCodeTime_branch_le`, `Cslib.WordRAM.expectedCodeTime_loop_le_of_drift`: compositional expected-time rules for structured `Code`. +* `Cslib.WordRAM.ExecCodeDist.loop_geometric`, + `Cslib.WordRAM.expectedStepsFrom_loop_le_of_geometric`, + `Cslib.WordRAM.ExecCodeDist.program_halts`: expected-time and almost-sure halting rules for + geometric Las Vegas `while` loops and structured programs. Discrete and conditional expectation principles are classical background; see [MitzenmacherUpfal2017], Sections 2.1 and 2.3. The particular native tail-sum/Bellman definitions, @@ -636,6 +643,1733 @@ theorem expectedCodeTime_loop_le_of_drift {w : ℕ} {cond : Cond r} {body : Code · exact ⟨Exists.intro _ (ExecRand.loop_exit hc), by rw [expectedCodeTime_loop_exit hc]; exact le_add_self⟩ -end WordRAM +/-! ## Distributional structured execution (`ExecCodeDist`) and geometric Las Vegas loops -/ -end Cslib +/-- Non-faulting structured randomized executions transfer instruction-for-instruction along any +defined-evaluation basis simulation `ψ : B.Simulates B'`. -/ +theorem ExecRand.mapSimulates {B' : Basis} (ψ : B.Simulates B') {w : ℕ} {c : Code B r} + {x : Data r} {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) : + ExecRand w (c.mapSimulates ψ) x m := by + induction h with + | op ho => exact .op (Op.eval_mapSimulates_of_some ψ ho) + | rand hfit => exact .rand hfit + | seq _ _ ih₁ ih₂ => exact .seq ih₁ ih₂ + | branch_pos hc _ ih => exact .branch_pos hc ih + | branch_neg hc _ ih => exact .branch_neg hc ih + | loop_exit hc => exact .loop_exit hc + | loop_iter hc _ _ ihb ihl => exact .loop_iter hc ihb ihl + +/-- Pointwise halting probability at `.halted v d` is monotone in the horizon `T`. -/ +theorem evalRand_halted_mono (w : ℕ) (p : Program B r) (s : State r) (v : ℕ) (d : Data r) : + Monotone fun T => evalRand w p T s (.halted v d) := by + refine monotone_nat_of_le_succ fun T => ?_ + have hstep : evalRand w p (T + 1) s = + (evalRand w p T s).bind (fun cfg => (stepRand w p cfg).eval) := by + rw [evalRand, runRand_add w p T 1, ProbComp.eval_bind] + simp only [evalRand, runRand_succ, runRand_zero, bind_pure] + rw [hstep, PMF.bind_apply] + calc + evalRand w p T s (.halted v d) + = evalRand w p T s (.halted v d) * (stepRand w p (.halted v d)).eval (.halted v d) := by + simp [stepRand_halted] + _ ≤ ∑' cfg, evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d) := + ENNReal.le_tsum + (f := fun cfg => evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d)) + (RandCfg.halted v d) + +private theorem finsetSum_iSup_of_monotone {α : Type*} (s : Finset α) {f : ℕ → α → ℝ≥0∞} + (hf : ∀ a, Monotone fun n => f n a) : + (∑ a ∈ s, ⨆ n, f n a) = ⨆ n, ∑ a ∈ s, f n a := by + classical + induction s using Finset.induction_on with + | empty => simp + | @insert a₀ s ha₀ ih => + rw [Finset.sum_insert ha₀, ih] + refine le_antisymm ?_ (iSup_le fun n => ?_) + · rw [ENNReal.add_iSup] + simp_rw [ENNReal.iSup_add] + refine iSup_le fun j => iSup_le fun i => le_iSup_of_le (max i j) ?_ + rw [Finset.sum_insert ha₀] + exact add_le_add (hf a₀ (le_max_left i j)) + (Finset.sum_le_sum fun a _ => hf a (le_max_right i j)) + · rw [Finset.sum_insert ha₀] + exact add_le_add (le_iSup (fun k => f k a₀) n) + (le_iSup (fun k => ∑ a ∈ s, f k a) n) + +private theorem tsum_iSup_of_monotone {α : Type*} {f : ℕ → α → ℝ≥0∞} + (hf : ∀ a, Monotone fun n => f n a) : + (∑' a, ⨆ n, f n a) = ⨆ n, ∑' a, f n a := by + rw [ENNReal.tsum_eq_iSup_sum] + simp_rw [finsetSum_iSup_of_monotone _ hf] + rw [iSup_comm] + simp_rw [← ENNReal.tsum_eq_iSup_sum] + +private theorem tsum_finsetSum {α β : Type*} (s : Finset β) (f : β → α → ℝ≥0∞) : + (∑' a, ∑ b ∈ s, f b a) = ∑ b ∈ s, ∑' a, f b a := by + classical + induction s using Finset.induction_on with + | empty => simp + | @insert b₀ s hb₀ ih => + simp only [Finset.sum_insert hb₀, ENNReal.tsum_add, ih] + +/-- Finite-horizon approximation of `expectedStepsFrom`: expected number of active (`.running`) +steps within the first `N` steps starting from configuration `cfg`. -/ +noncomputable def expectedStepsWithin (w : ℕ) (p : Program B r) (N : ℕ) (cfg : RandCfg r) : + ℝ≥0∞ := + ∑ t ∈ Finset.range N, (runRand w p t cfg).eval.toOuterMeasure {cfg' | ∃ u, cfg' = .running u} + +@[simp] +theorem expectedStepsWithin_zero (w : ℕ) (p : Program B r) (cfg : RandCfg r) : + expectedStepsWithin w p 0 cfg = 0 := by + simp [expectedStepsWithin] + +@[simp] +theorem expectedStepsWithin_halted (w : ℕ) (p : Program B r) (N v : ℕ) (d : Data r) : + expectedStepsWithin w p N (.halted v d) = 0 := by + simp [expectedStepsWithin, PMF.toOuterMeasure_pure_apply] + +@[simp] +theorem expectedStepsWithin_faulted (w : ℕ) (p : Program B r) (N : ℕ) : + expectedStepsWithin w p N .faulted = 0 := by + simp [expectedStepsWithin, PMF.toOuterMeasure_pure_apply] + +theorem expectedStepsWithin_mono (w : ℕ) (p : Program B r) (cfg : RandCfg r) : + Monotone fun N => expectedStepsWithin w p N cfg := by + intro N₁ N₂ hN + exact Finset.sum_le_sum_of_subset (Finset.range_mono hN) + +theorem iSup_expectedStepsWithin (w : ℕ) (p : Program B r) (cfg : RandCfg r) : + (⨆ N : ℕ, expectedStepsWithin w p N cfg) = expectedStepsFrom w p cfg := + (ENNReal.tsum_eq_iSup_nat).symm + +theorem expectedStepsWithin_le_expectedStepsFrom (w : ℕ) (p : Program B r) (N : ℕ) + (cfg : RandCfg r) : + expectedStepsWithin w p N cfg ≤ expectedStepsFrom w p cfg := by + rw [← iSup_expectedStepsWithin] + exact le_iSup (fun K => expectedStepsWithin w p K cfg) N + +/-- 1-step horizon recursion for `expectedStepsWithin`: at horizon `N + 1`, an active state `s` +incurs `1` step plus the `N`-step continuation expectation. -/ +theorem expectedStepsWithin_running_succ (w : ℕ) (p : Program B r) (N : ℕ) (s : State r) : + expectedStepsWithin w p (N + 1) (.running s) = + 1 + ProbComp.expect (stepRand w p (.running s)) (expectedStepsWithin w p N) := by + rw [expectedStepsWithin, Finset.sum_range_succ'] + have hzero : (runRand w p 0 (.running s)).eval.toOuterMeasure + {cfg' | ∃ u, cfg' = .running u} = 1 := by + simp [PMF.toOuterMeasure_pure_apply] + rw [hzero, add_comm] + congr 1 + have hsucc : ∀ t : ℕ, (runRand w p (t + 1) (.running s)).eval.toOuterMeasure + {cfg' | ∃ u, cfg' = .running u} = + ∑' cfg, (stepRand w p (.running s)).eval cfg * + (runRand w p t cfg).eval.toOuterMeasure {cfg' | ∃ u, cfg' = .running u} := by + intro t + rw [runRand_succ, ProbComp.eval_bind, PMF.toOuterMeasure_bind_apply] + simp only [hsucc] + rw [← tsum_finsetSum] + simp only [ProbComp.expect, expectedStepsWithin, Finset.mul_sum] + +theorem expectedStepsWithin_running_le (w : ℕ) (p : Program B r) (N : ℕ) (s : State r) : + expectedStepsWithin w p N (.running s) ≤ + 1 + ProbComp.expect (stepRand w p (.running s)) (expectedStepsWithin w p N) := by + cases N with + | zero => simp + | succ N => + rw [expectedStepsWithin_running_succ] + gcongr + exact ProbComp.expect_mono fun cfg => expectedStepsWithin_mono w p cfg (Nat.le_succ N) + +theorem ExecRand.expectedStepsWithin_assemble {w : ℕ} {c : Code B r} {x : Data r} + {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (N : ℕ) : + expectedStepsWithin w p N (.running ⟨pc, x⟩) ≤ + ProbComp.expect m (fun z => + (z.2 : ℝ≥0∞) + expectedStepsWithin w p N (.running ⟨pc + c.size, z.1⟩)) := by + induction h generalizing pc with + | @op o x y ho => + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_op (s := ⟨pc, x⟩) hp.head ho, ProbComp.expect_pure, ProbComp.expect_pure] + simp [Code.size] + | @rand dst src x hfit => + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, ProbComp.expect_bind, + ProbComp.expect_bind] + simp only [ProbComp.expect_pure, Code.size, Nat.cast_one] + rw [ProbComp.expect_add, ProbComp.expect_const] + | @seq c₁ c₂ x m₁ m₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine (ih₁ hp.append_left).trans ?_ + rw [ProbComp.expect_bind] + refine ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_ + refine (add_le_add le_rfl (ih₂ z₁ hz₁ hc₂)).trans (le_of_eq ?_) + rw [ProbComp.expect_bind, + ← ProbComp.expect_const (m₂ z₁.1) (z₁.2 : ℝ≥0∞), ← ProbComp.expect_add] + refine ProbComp.expect_congr_on_support fun z₂ _ => ?_ + simp only [ProbComp.expect_pure, Code.size, Nat.cast_add, add_assoc] + | @branch_pos cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + refine (add_le_add le_rfl (ih hc')).trans (le_of_eq ?_) + rw [ProbComp.expect_map, ← ProbComp.expect_const m 1, ← ProbComp.expect_add] + refine ProbComp.expect_congr_on_support fun z _ => ?_ + simp only [Code.size, Nat.cast_add, Nat.cast_one] + rw [show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + ac_rfl + | @branch_neg cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ih hp.cons_tail.append_left)).trans ?_ + rw [ProbComp.expect_map, ← ProbComp.expect_const m 1, ← ProbComp.expect_add] + refine ProbComp.expect_mono_on_support fun z _ => ?_ + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + refine (add_le_add le_rfl + (add_le_add le_rfl (expectedStepsWithin_running_le w p N ⟨pc + 1 + neg.size, z.1⟩))).trans + (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 1 + neg.size, z.1⟩) hj, ProbComp.expect_pure] + simp only [Code.size, Nat.cast_add, Nat.cast_ofNat] + rw [show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + set E := expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), z.1⟩) + calc + 1 + ((z.2 : ℝ≥0∞) + (1 + E)) = (z.2 : ℝ≥0∞) + (1 + 1) + E := by ac_rfl + _ = (z.2 : ℝ≥0∞) + 2 + E := by rw [one_add_one_eq_two] + | @loop_exit cond body x hc => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hj : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (expectedStepsWithin_running_le w p N ⟨pc + 1, x⟩)).trans + (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 1, x⟩) hj, ProbComp.expect_pure] + simp only [Code.size, Nat.cast_ofNat] + rw [← add_assoc, one_add_one_eq_two, show pc + body.size + 3 = pc + (body.size + 3) by omega] + | @loop_iter cond body x mb ml hc _ _ ihb ihl => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ihb hbody_contains)).trans ?_ + rw [ProbComp.expect_bind, ← ProbComp.expect_const mb 1, ← ProbComp.expect_add] + refine ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_ + have hjump_le : expectedStepsWithin w p N (.running ⟨pc + 2 + body.size, z₁.1⟩) ≤ + 1 + ProbComp.expect (ml z₁.1) (fun z => + (z.2 : ℝ≥0∞) + + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, z.1⟩)) := by + refine (expectedStepsWithin_running_le w p N ⟨pc + 2 + body.size, z₁.1⟩).trans ?_ + rw [stepRand_jump (s := ⟨pc + 2 + body.size, z₁.1⟩) hj, ProbComp.expect_pure] + exact add_le_add le_rfl (ihl z₁ hz₁ hp) + refine (add_le_add le_rfl (add_le_add le_rfl hjump_le)).trans (le_of_eq ?_) + rw [ProbComp.expect_bind] + have hconst : (1 : ℝ≥0∞) + ((z₁.2 : ℝ≥0∞) + + (1 + ProbComp.expect (ml z₁.1) fun z => + (z.2 : ℝ≥0∞) + + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, z.1⟩))) = + ProbComp.expect (ml z₁.1) (fun _ => (z₁.2 : ℝ≥0∞) + 2) + + ProbComp.expect (ml z₁.1) (fun z => + (z.2 : ℝ≥0∞) + + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, z.1⟩)) := by + rw [ProbComp.expect_const, ← one_add_one_eq_two] + ac_rfl + rw [hconst, ← ProbComp.expect_add] + refine ProbComp.expect_congr_on_support fun z₂ _ => ?_ + simp only [ProbComp.expect_pure, Nat.cast_add, Nat.cast_ofNat] + rw [← add_assoc ((z₁.2 : ℝ≥0∞) + 2) (z₂.2 : ℝ≥0∞), add_right_comm (z₁.2 : ℝ≥0∞) 2] + +/-- Along `ExecRand w c x m`, if no continuation from `⟨pc + c.size, y⟩` faults within `T` steps +for any supported outcome `y`, then running `p` from `⟨pc, x⟩` does not fault within `T` steps. -/ +theorem ExecRand.evalRand_faulted_eq_zero {w : ℕ} {c : Code B r} {x : Data r} + {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) {p : Program B r} {pc T : ℕ} + (hp : p.Contains pc (c.assemble pc)) + (hcont : ∀ z ∈ m.support, ∀ t ≤ T, evalRand w p t ⟨pc + c.size, z.1⟩ .faulted = 0) : + ∀ t ≤ T, evalRand w p t ⟨pc, x⟩ .faulted = 0 := by + induction h generalizing pc T with + | @op o x y ho => + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_op (s := ⟨pc, x⟩) hp.head ho, pure_bind] + simpa [evalRand, Code.size] using hcont (y, 1) rfl k (by omega) + | @rand dst src x hfit => + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, bind_assoc, + ProbComp.eval_bind, PMF.bind_apply, ENNReal.tsum_eq_zero] + intro sample + have hmem : (x.set dst sample.val, 1) ∈ + (ProbComp.unifFin (x.regs src) >>= fun s => pure (x.set dst s.val, 1)).support := + OracleComp.mem_supportWith_bind_iff.mpr ⟨sample, ProbComp.mem_support_unifFin _ sample, rfl⟩ + have hzero := hcont _ hmem k (by omega) + simp only [pure_bind] + exact mul_eq_zero_of_right _ hzero + | @seq c₁ c₂ x m₁ m₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine ih₁ hp.append_left fun z₁ hz₁ t ht => ?_ + refine ih₂ z₁ hz₁ hc₂ (T := t) (fun z₂ hz₂ t' ht' => ?_) t le_rfl + have hmem : (z₂.1, z₁.2 + z₂.2) ∈ + (m₁ >>= fun z₁ => m₂ z₁.1 >>= fun z₂ => pure (z₂.1, z₁.2 + z₂.2)).support := + OracleComp.mem_supportWith_bind_iff.mpr + ⟨z₁, hz₁, OracleComp.mem_supportWith_bind_iff.mpr ⟨z₂, hz₂, rfl⟩⟩ + simpa [Code.size, Nat.add_assoc] using hcont _ hmem t' (ht'.trans ht) + | @branch_pos cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hc' (T := k) (fun z hz t' ht' => ?_) k le_rfl + have hmem : (z.1, z.2 + 1) ∈ ((fun z : Data r × ℕ => (z.1, z.2 + 1)) <$> m).support := + OracleComp.mem_supportWith_map_iff.mpr ⟨z, hz, rfl⟩ + simpa [Code.size, show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + using hcont _ hmem t' (by omega) + | @branch_neg cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hp.cons_tail.append_left (T := k) (fun z hz t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 1 + neg.size, z.1⟩) hj, pure_bind] + have hmem : (z.1, z.2 + 2) ∈ ((fun z : Data r × ℕ => (z.1, z.2 + 2)) <$> m).support := + OracleComp.mem_supportWith_map_iff.mpr ⟨z, hz, rfl⟩ + simpa [evalRand, Code.size, + show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using hcont _ hmem k' (by omega) + | @loop_exit cond body x hc => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hj : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + cases k with + | zero => simp + | succ k' => + rw [runRand_succ, stepRand_jump (s := ⟨pc + 1, x⟩) hj, pure_bind] + simpa [evalRand, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using hcont (x, 2) rfl k' (by omega) + | @loop_iter cond body x mb ml hc _ _ ihb ihl => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ihb hbody_contains (T := k) (fun z₁ hz₁ t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 2 + body.size, z₁.1⟩) hj, pure_bind] + refine ihl z₁ hz₁ hp (T := k') (fun z₂ hz₂ t'' ht'' => ?_) k' le_rfl + have hmem : (z₂.1, z₁.2 + z₂.2 + 2) ∈ + (mb >>= fun z₁ => ml z₁.1 >>= fun z₂ => pure (z₂.1, z₁.2 + z₂.2 + 2)).support := + OracleComp.mem_supportWith_bind_iff.mpr + ⟨z₁, hz₁, OracleComp.mem_supportWith_bind_iff.mpr ⟨z₂, hz₂, rfl⟩⟩ + exact hcont _ hmem t'' (by omega) + +private theorem iSup_evalRand_halted_le_of_step {w : ℕ} {p : Program B r} {s s' : State r} + (hstep : stepRand w p (.running s) = pure (.running s')) (v : ℕ) (d : Data r) : + (⨆ T : ℕ, evalRand w p T s' (.halted v d)) ≤ ⨆ T : ℕ, evalRand w p T s (.halted v d) := by + refine iSup_le fun T => le_iSup_of_le (T + 1) ?_ + simp only [evalRand, runRand_succ, hstep, pure_bind, le_rfl] + +/-- Along `ExecRand w c x m`, the expected supremum halting probability from `⟨pc + c.size, y⟩` is a +lower bound on the supremum halting probability from `⟨pc, x⟩`. -/ +theorem ExecRand.le_iSup_evalRand_halted {w : ℕ} {c : Code B r} {x : Data r} + {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (v : ℕ) (d : Data r) : + ProbComp.expect m (fun z => ⨆ T : ℕ, evalRand w p T ⟨pc + c.size, z.1⟩ (.halted v d)) ≤ + ⨆ T : ℕ, evalRand w p T ⟨pc, x⟩ (.halted v d) := by + induction h generalizing pc with + | @op o x y ho => + rw [ProbComp.expect_pure] + simpa [Code.size] using + iSup_evalRand_halted_le_of_step (stepRand_op (s := ⟨pc, x⟩) hp.head ho) v d + | @rand dst src x hfit => + rw [ProbComp.expect_bind] + simp only [ProbComp.expect_pure, Code.size] + change (∑' sample : Fin (x.regs src + 1), + (ProbComp.unifFin (x.regs src)).eval sample * + ⨆ T : ℕ, evalRand w p T ⟨pc + 1, x.set dst sample.val⟩ (.halted v d)) ≤ + ⨆ T : ℕ, evalRand w p T ⟨pc, x⟩ (.halted v d) + simp_rw [ENNReal.mul_iSup] + rw [tsum_iSup_of_monotone fun sample T₁ T₂ hT => + mul_le_mul_right (evalRand_halted_mono w p ⟨pc + 1, x.set dst sample.val⟩ v d hT) _] + refine iSup_le fun T => le_iSup_of_le (T + 1) ?_ + rw [evalRand, runRand_succ, stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, bind_assoc, + ProbComp.eval_bind, PMF.bind_apply] + refine le_of_eq (tsum_congr fun sample => ?_) + simp [evalRand] + | @seq c₁ c₂ x m₁ m₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_).trans (ih₁ hp.append_left) + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₂ _ => ?_).trans (ih₂ z₁ hz₁ hc₂) + simp only [ProbComp.expect_pure, Code.size, Nat.add_assoc, le_rfl] + | @branch_pos cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + neg.size + 2, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_map] + refine (ProbComp.expect_mono_on_support fun z _ => ?_).trans + ((ih hc').trans (iSup_evalRand_halted_le_of_step hstep v d)) + simp only [Code.size, + show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega, le_rfl] + | @branch_neg cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_map] + refine (ProbComp.expect_mono_on_support fun z _ => ?_).trans + ((ih hp.cons_tail.append_left).trans (iSup_evalRand_halted_le_of_step hstep v d)) + simpa [Code.size, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 1 + neg.size, z.1⟩) hj) v d + | @loop_exit cond body x hc => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hj : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_pure] + simpa [Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] using + (iSup_evalRand_halted_le_of_step (stepRand_jump (s := ⟨pc + 1, x⟩) hj) v d).trans + (iSup_evalRand_halted_le_of_step hstep v d) + | @loop_iter cond body x mb ml hc _ _ ihb ihl => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 2, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_).trans + ((ihb hbody_contains).trans (iSup_evalRand_halted_le_of_step hstep v d)) + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₂ _ => ?_).trans + ((ihl z₁ hz₁ hp).trans + (iSup_evalRand_halted_le_of_step (stepRand_jump (s := ⟨pc + 2 + body.size, z₁.1⟩) hj) v d)) + simp only [ProbComp.expect_pure, le_rfl] + +/-! ### Absorbing Markov chain for geometric Las Vegas loops -/ + +/-- One step of the stopped loop Markov kernel on `Data r`: active states (`cond.Holds x`) step +according to `μ_body x`, while exiting states (`¬cond.Holds x`) are absorbing. -/ +noncomputable def loopStepPMF (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r) : + PMF (Data r) := + open Classical in + if cond.Holds x then μ_body x else PMF.pure x + +theorem loopStepPMF_of_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : cond.Holds x) : + loopStepPMF cond μ_body x = μ_body x := by + classical + simp [loopStepPMF, hc] + +theorem loopStepPMF_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : ¬cond.Holds x) : + loopStepPMF cond μ_body x = PMF.pure x := by + classical + simp [loopStepPMF, hc] + +/-- `K`-step iterate of `loopStepPMF`. -/ +noncomputable def loopIterPMF (cond : Cond r) (μ_body : Data r → PMF (Data r)) : + ℕ → Data r → PMF (Data r) + | 0, x => PMF.pure x + | K + 1, x => (loopStepPMF cond μ_body x).bind (loopIterPMF cond μ_body K) + +@[simp] +theorem loopIterPMF_zero (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r) : + loopIterPMF cond μ_body 0 x = PMF.pure x := rfl + +theorem loopIterPMF_succ (cond : Cond r) (μ_body : Data r → PMF (Data r)) (K : ℕ) (x : Data r) : + loopIterPMF cond μ_body (K + 1) x = + (loopStepPMF cond μ_body x).bind (loopIterPMF cond μ_body K) := rfl + +theorem loopIterPMF_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : ¬cond.Holds x) (K : ℕ) : + loopIterPMF cond μ_body K x = PMF.pure x := by + induction K with + | zero => rfl + | succ K ih => rw [loopIterPMF_succ, loopStepPMF_of_not_holds hc, PMF.pure_bind, ih] + +theorem loopIterPMF_succ_right (cond : Cond r) (μ_body : Data r → PMF (Data r)) (K : ℕ) + (x : Data r) : + loopIterPMF cond μ_body (K + 1) x = + (loopIterPMF cond μ_body K x).bind (loopStepPMF cond μ_body) := by + induction K generalizing x with + | zero => simp [loopIterPMF] + | succ K ih => + rw [loopIterPMF_succ (K := K), PMF.bind_bind] + exact congrArg ((loopStepPMF cond μ_body x).bind ·) (funext ih) + +theorem loopIterPMF_mono_of_not_holds (cond : Cond r) (μ_body : Data r → PMF (Data r)) + (x : Data r) {y : Data r} (hy : ¬cond.Holds y) : + Monotone fun K => loopIterPMF cond μ_body K x y := by + refine monotone_nat_of_le_succ fun K => ?_ + rw [loopIterPMF_succ_right, PMF.bind_apply] + calc + loopIterPMF cond μ_body K x y + = loopIterPMF cond μ_body K x y * loopStepPMF cond μ_body y y := by + rw [loopStepPMF_of_not_holds hy, PMF.pure_apply_self, mul_one] + _ ≤ ∑' z, loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y := + ENNReal.le_tsum + (f := fun z => loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y) y + +theorem loopIterPMF_indicator_mono (cond : Cond r) (μ_body : Data r → PMF (Data r)) + (x y : Data r) : + Monotone fun K => Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y := by + intro K₁ K₂ hK + dsimp only + by_cases hy : y ∈ {z | ¬cond.Holds z} + · rw [Set.indicator_of_mem hy, Set.indicator_of_mem hy] + exact loopIterPMF_mono_of_not_holds cond μ_body x hy hK + · rw [Set.indicator_of_notMem hy, Set.indicator_of_notMem hy] + +/-- Combined loop state invariant: either active (`cond.Holds x ∧ I x`) or exited +(`¬cond.Holds x ∧ Q x`). -/ +def LoopStateInv (cond : Cond r) (I Q : Data r → Prop) (x : Data r) : Prop := + (cond.Holds x ∧ I x) ∨ (¬cond.Holds x ∧ Q x) + +theorem loopIterPMF_support_inv {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (K : ℕ) {x : Data r} (hx : LoopStateInv cond I Q x) : + ∀ y ∈ (loopIterPMF cond μ_body K x).support, LoopStateInv cond I Q y := by + induction K generalizing x with + | zero => + intro y hy + rw [loopIterPMF_zero, PMF.support_pure, Set.mem_singleton_iff] at hy + rwa [hy] + | succ K ih => + intro y hy + rw [loopIterPMF_succ, PMF.mem_support_bind_iff] at hy + obtain ⟨z, hz, hyz⟩ := hy + rcases hx with ⟨hcx, hIx⟩ | ⟨hcx, hQx⟩ + · rw [loopStepPMF_of_holds hcx] at hz + exact ih (hinv x hIx hcx z hz) y hyz + · rw [loopStepPMF_of_not_holds hcx, PMF.support_pure, Set.mem_singleton_iff] at hz + subst hz + exact ih (Or.inr ⟨hcx, hQx⟩) y hyz + +private theorem pmf_toOuterMeasure_compl_add (μ : PMF (Data r)) (cond : Cond r) : + μ.toOuterMeasure {y | ¬cond.Holds y} + μ.toOuterMeasure {y | cond.Holds y} = 1 := by + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + convert μ.tsum_coe using 1 + congr 1 + funext y + by_cases hc : cond.Holds y <;> simp [hc] + +private theorem pmf_step_drift_bound (μ : PMF (Data r)) (cond : Cond r) + {C p_exit : ℝ≥0∞} (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hp_exit : p_exit ≤ μ.toOuterMeasure {y | ¬cond.Holds y}) : + C + ∑' y, μ y * (if cond.Holds y then C / p_exit else 0) ≤ C / p_exit := by + set M := C / p_exit + have hC : C = p_exit * M := by + rw [mul_comm, ENNReal.div_mul_cancel hp_pos.ne' hp_top] + have hact : (∑' y, μ y * (if cond.Holds y then M else 0)) = + μ.toOuterMeasure {y | cond.Holds y} * M := by + rw [PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] + refine tsum_congr fun y => ?_ + by_cases hc : cond.Holds y <;> simp [hc] + calc + C + ∑' y, μ y * (if cond.Holds y then M else 0) + = p_exit * M + μ.toOuterMeasure {y | cond.Holds y} * M := by rw [hC, hact] + _ ≤ μ.toOuterMeasure {y | ¬cond.Holds y} * M + + μ.toOuterMeasure {y | cond.Holds y} * M := by gcongr + _ = (μ.toOuterMeasure {y | ¬cond.Holds y} + + μ.toOuterMeasure {y | cond.Holds y}) * M := by rw [← add_mul] + _ = M := by rw [pmf_toOuterMeasure_compl_add, one_mul] + +/-- Expected number of active loop iterations within the first `N` steps of `loopIterPMF`. -/ +private noncomputable def loopActiveSteps (cond : Cond r) (μ_body : Data r → PMF (Data r)) + (N : ℕ) (x : Data r) : ℝ≥0∞ := + ∑ K ∈ Finset.range N, (loopIterPMF cond μ_body K x).toOuterMeasure {y | cond.Holds y} + +private theorem loopActiveSteps_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} + {x : Data r} (hc : ¬cond.Holds x) (N : ℕ) : + loopActiveSteps cond μ_body N x = 0 := by + unfold loopActiveSteps + refine Finset.sum_eq_zero fun K _ => ?_ + rw [loopIterPMF_of_not_holds hc, PMF.toOuterMeasure_pure_apply] + simp [hc] + +private theorem loopActiveSteps_succ_of_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} + {x : Data r} (hc : cond.Holds x) (N : ℕ) : + loopActiveSteps cond μ_body (N + 1) x = + 1 + ∑' z, μ_body x z * loopActiveSteps cond μ_body N z := by + rw [loopActiveSteps, Finset.sum_range_succ'] + have hzero : (loopIterPMF cond μ_body 0 x).toOuterMeasure {y | cond.Holds y} = 1 := by + simp [PMF.toOuterMeasure_pure_apply, hc] + rw [hzero, add_comm] + congr 1 + have hsucc : ∀ K : ℕ, (loopIterPMF cond μ_body (K + 1) x).toOuterMeasure {y | cond.Holds y} = + ∑' z, μ_body x z * (loopIterPMF cond μ_body K z).toOuterMeasure {y | cond.Holds y} := by + intro K + rw [loopIterPMF_succ, loopStepPMF_of_holds hc, PMF.toOuterMeasure_bind_apply] + simp only [hsucc] + rw [← tsum_finsetSum] + simp only [loopActiveSteps, Finset.mul_sum] + +private theorem loopActiveSteps_le {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + (N : ℕ) {x : Data r} (hx : LoopStateInv cond I Q x) : + loopActiveSteps cond μ_body N x ≤ if cond.Holds x then 1 / p_exit else 0 := by + induction N generalizing x with + | zero => simp [loopActiveSteps] + | succ N ih => + rcases hx with ⟨hcx, hIx⟩ | ⟨hcx, _⟩ + · simp only [hcx, ↓reduceIte] + rw [loopActiveSteps_succ_of_holds hcx] + refine (add_le_add le_rfl (ENNReal.tsum_le_tsum fun z => ?_)).trans + (pmf_step_drift_bound (μ_body x) cond hp_pos hp_top (hp_exit x hIx hcx)) + by_cases hz : z ∈ (μ_body x).support + · exact mul_le_mul_right (ih (hinv x hIx hcx z hz)) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hz, zero_mul, zero_mul] + · simp [loopActiveSteps_of_not_holds hcx, hcx] + +theorem hasSum_loopExitDensity {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + {x : Data r} (hx : LoopStateInv cond I Q x) : + HasSum (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1 := by + rw [ENNReal.summable.hasSum_iff, tsum_iSup_of_monotone (loopIterPMF_indicator_mono cond μ_body x)] + simp_rw [← PMF.toOuterMeasure_apply] + set b := ⨆ K : ℕ, (loopIterPMF cond μ_body K x).toOuterMeasure {z | ¬cond.Holds z} + have hb_le : b ≤ 1 := iSup_le fun K => Cslib.Probability.PMF.toOuterMeasure_le_one _ _ + have hsub_zero : 1 - b = 0 := by + by_contra hne + have hN_bound : ∀ N : ℕ, (N : ℝ≥0∞) * (1 - b) ≤ 1 / p_exit := by + intro N + have hterm : ∀ K : ℕ, 1 - b ≤ + (loopIterPMF cond μ_body K x).toOuterMeasure {z | cond.Holds z} := by + intro K + refine tsub_le_iff_left.mpr ?_ + rw [← pmf_toOuterMeasure_compl_add (loopIterPMF cond μ_body K x) cond] + exact add_le_add + (le_iSup (fun J => (loopIterPMF cond μ_body J x).toOuterMeasure {z | ¬cond.Holds z}) K) + le_rfl + calc + (N : ℝ≥0∞) * (1 - b) + = ∑ _K ∈ Finset.range N, (1 - b) := by simp + _ ≤ ∑ K ∈ Finset.range N, + (loopIterPMF cond μ_body K x).toOuterMeasure {z | cond.Holds z} := + Finset.sum_le_sum fun K _ => hterm K + _ ≤ if cond.Holds x then 1 / p_exit else 0 := + loopActiveSteps_le hp_pos hp_top hinv hp_exit N hx + _ ≤ 1 / p_exit := by split_ifs <;> simp + have htop : (⊤ : ℝ≥0∞) ≤ 1 / p_exit := by + rw [← ENNReal.top_mul hne, ← ENNReal.iSup_natCast, ENNReal.iSup_mul] + exact iSup_le hN_bound + exact ENNReal.div_ne_top ENNReal.one_ne_top hp_pos.ne' (top_le_iff.mp htop) + exact le_antisymm hb_le (tsub_eq_zero_iff_le.mp hsub_zero) + +/-- Limiting exit distribution of `Code.loop cond body` starting from `x` under one-step body +distribution family `μ_body`. -/ +noncomputable def loopExitPMF (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r) : + PMF (Data r) := + open Classical in + if h : HasSum (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1 then + ⟨_, h⟩ + else + PMF.pure x + +theorem loopExitPMF_apply_of_hasSum {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (h : HasSum (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1) + (y : Data r) : + loopExitPMF cond μ_body x y = + ⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y := by + classical + simp only [loopExitPMF, h, ↓reduceDIte] + rfl + +theorem loopExitPMF_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : ¬cond.Holds x) : + loopExitPMF cond μ_body x = PMF.pure x := by + have hfun : (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) = PMF.pure x := by + funext y + have hK : ∀ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y = PMF.pure x y := by + intro K + rw [loopIterPMF_of_not_holds hc] + by_cases hyx : y = x + · subst hyx + simp [hc] + · simp [PMF.pure_apply, hyx] + simp [hK] + ext y + rw [loopExitPMF_apply_of_hasSum (hfun ▸ (PMF.pure x).2), congrFun hfun y] + +theorem loopExitPMF_of_holds {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + {x : Data r} (hIx : I x) (hcx : cond.Holds x) : + loopExitPMF cond μ_body x = (μ_body x).bind (loopExitPMF cond μ_body) := by + ext y + have hx : LoopStateInv cond I Q x := Or.inl ⟨hcx, hIx⟩ + rw [loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx), + PMF.bind_apply] + have hzero : Set.indicator {z | ¬cond.Holds z} (PMF.pure x) y = 0 := by + by_cases hy : ¬cond.Holds y + · have hyx : y ≠ x := fun h => hy (h ▸ hcx) + simp [hy, PMF.pure_apply, hyx] + · simp [Set.indicator_of_notMem, hy] + have hiSup_shift : + (⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) = + ⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body (K + 1) x) y := by + refine le_antisymm (iSup_le fun K => ?_) (iSup_le fun K => + le_iSup (fun J => Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body J x) y) (K + 1)) + cases K with + | zero => + rw [loopIterPMF_zero, hzero] + exact zero_le + | succ K => + exact le_iSup + (fun J => Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body (J + 1) x) y) K + rw [hiSup_shift] + have hstep : ∀ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body (K + 1) x) y = + ∑' z, μ_body x z * + Set.indicator {w' | ¬cond.Holds w'} (loopIterPMF cond μ_body K z) y := by + intro K + rw [loopIterPMF_succ, loopStepPMF_of_holds hcx] + by_cases hy : ¬cond.Holds y + · simp [Set.indicator_of_mem, hy, PMF.bind_apply] + · simp [Set.indicator_of_notMem, hy] + simp_rw [hstep] + rw [← tsum_iSup_of_monotone + (f := fun K z => μ_body x z * + Set.indicator {w' | ¬cond.Holds w'} (loopIterPMF cond μ_body K z) y) + (fun z _ _ hK => mul_le_mul_right (loopIterPMF_indicator_mono cond μ_body z y hK) _)] + refine tsum_congr fun z => ?_ + by_cases hz : z ∈ (μ_body x).support + · rw [← ENNReal.mul_iSup, + loopExitPMF_apply_of_hasSum + (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit (hinv x hIx hcx z hz))] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hz, zero_mul] + simp + +theorem mem_support_loopExitPMF {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + {x : Data r} (hx : LoopStateInv cond I Q x) + {y : Data r} (hy : y ∈ (loopExitPMF cond μ_body x).support) : + (¬cond.Holds y ∧ Q y) ∧ ∃ K, y ∈ (loopIterPMF cond μ_body K x).support := by + rw [PMF.mem_support_iff, + loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx)] at hy + have ⟨K, hK⟩ : ∃ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y ≠ 0 := by + by_contra! h + exact hy (by simp [h]) + have hnc : ¬cond.Holds y := by + by_contra hc + exact hK (by simp [Set.indicator_of_notMem, hc]) + rw [Set.indicator_of_mem hnc, ← PMF.mem_support_iff] at hK + rcases loopIterPMF_support_inv hinv K hx y hK with ⟨hcy, _⟩ | ⟨_, hQy⟩ + · exact absurd hcy hnc + · exact ⟨⟨hnc, hQy⟩, K, hK⟩ + +/-! ### Compositional distributional execution (`ExecCodeDist`) -/ + +/-- `ExecCodeDist w c x μ T`: at word size `w`, running structured program `c` from data `x` +terminates almost surely without faulting, realizes exit distribution `μ : PMF (Data r)` at the +fallthrough address `pc + c.size` of any containing flat program, and uses at most `T : ℝ≥0∞` +expected instructions. + +Unlike `ExecRand`, which requires a finite-depth computation tree, `ExecCodeDist` includes +unbounded Las Vegas `while` loops whose each active iteration exits with probability bounded below +by `p_exit > 0` (`ExecCodeDist.loop_geometric`). -/ +inductive ExecCodeDist {B : Basis} {r : ℕ} (w : ℕ) : Code B r → Data r → PMF (Data r) → + ℝ≥0∞ → Prop where + | of_execRand {c : Code B r} {x : Data r} {m : ProbComp (Data r × ℕ)} : + ExecRand w c x m → + ExecCodeDist w c x (Prod.fst <$> m).eval (ProbComp.expect m fun z => (z.2 : ℝ≥0∞)) + | mono {c : Code B r} {x : Data r} {μ : PMF (Data r)} {T T' : ℝ≥0∞} : + ExecCodeDist w c x μ T → T ≤ T' → ExecCodeDist w c x μ T' + | seq {c₁ c₂ : Code B r} {x : Data r} {μ₁ : PMF (Data r)} + {μ₂ : Data r → PMF (Data r)} {T₁ : ℝ≥0∞} {T₂ : Data r → ℝ≥0∞} : + ExecCodeDist w c₁ x μ₁ T₁ → + (∀ y ∈ μ₁.support, ExecCodeDist w c₂ y (μ₂ y) (T₂ y)) → + ExecCodeDist w (.seq c₁ c₂) x (μ₁.bind μ₂) (T₁ + ∑' y, μ₁ y * T₂ y) + | branch_pos {cond : Cond r} {pos neg : Code B r} {x : Data r} + {μ : PMF (Data r)} {T : ℝ≥0∞} : + cond.Holds x → ExecCodeDist w pos x μ T → + ExecCodeDist w (.branch cond pos neg) x μ (T + 1) + | branch_neg {cond : Cond r} {pos neg : Code B r} {x : Data r} + {μ : PMF (Data r)} {T : ℝ≥0∞} : + ¬cond.Holds x → ExecCodeDist w neg x μ T → + ExecCodeDist w (.branch cond pos neg) x μ (T + 2) + | loop_geometric {cond : Cond r} {body : Code B r} + (I Q : Data r → Prop) {μ_body : Data r → PMF (Data r)} {T_body : Data r → ℝ≥0∞} + {B_body p_exit : ℝ≥0∞} {x : Data r} : + 0 < p_exit → p_exit ≠ ⊤ → + (∀ z, I z → cond.Holds z → ExecCodeDist w body z (μ_body z) (T_body z)) → + (∀ z, I z → cond.Holds z → T_body z ≤ B_body) → + (∀ z, I z → cond.Holds z → + ∀ y ∈ (μ_body z).support, LoopStateInv cond I Q y) → + (∀ z, I z → cond.Holds z → + p_exit ≤ (μ_body z).toOuterMeasure {y | ¬cond.Holds y}) → + LoopStateInv cond I Q x → + ExecCodeDist w (.loop cond body) x (loopExitPMF cond μ_body x) + ((B_body + 2) / p_exit + 2) + +namespace ExecCodeDist + +variable {w : ℕ} {c c₁ c₂ pos neg body : Code B r} {cond : Cond r} {x y : Data r} + {μ μ₁ μ' : PMF (Data r)} {T T' T₁ T₂ : ℝ≥0∞} + +theorem congr_dist (h : ExecCodeDist w c x μ T) (hμ : μ = μ') (hT : T ≤ T') : + ExecCodeDist w c x μ' T' := by + subst hμ + exact h.mono hT + +/-- Every deterministic `Exec w c x y n` gives `ExecCodeDist w c x (PMF.pure y) n`. -/ +theorem of_exec {n : ℕ} (h : Exec w c x y n) : + ExecCodeDist w c x (PMF.pure y) n := by + simpa [ProbComp.expect_pure] using of_execRand h.execRand + +/-- Every deterministic `ExecWithin w c x y n` gives `ExecCodeDist w c x (PMF.pure y) n`. -/ +theorem of_execWithin {n : ℕ} (h : ExecWithin w c x y n) : + ExecCodeDist w c x (PMF.pure y) n := by + obtain ⟨m, hm, he⟩ := h + exact (of_exec he).mono (Nat.cast_le.mpr hm) + +/-- Single deterministic operation instruction. -/ +theorem op {o : Op B r} (ho : o.eval w x = some y) : + ExecCodeDist w (.op o) x (PMF.pure y) 1 := by + simpa using of_exec (Exec.op ho) + +/-- Single `rand dst src` instruction. -/ +theorem rand {dst src : Fin r} (hfit : Fits w (x.regs src)) : + ExecCodeDist (B := B) w (.rand dst src) x + ((ProbComp.unifFin (x.regs src)).eval.map fun k => x.set dst k.val) 1 := by + have h := of_execRand (ExecRand.rand (B := B) (dst := dst) (src := src) hfit) + refine h.congr_dist ?_ ?_ + · ext z + simp [ProbComp.eval_map, PMF.map_apply] + · simp [ProbComp.expect_map, ProbComp.expect_const] + +/-- Exiting a loop immediately when `¬cond.Holds x` takes `2` steps and leaves `x` unchanged. -/ +theorem loop_exit (hc : ¬cond.Holds x) : + ExecCodeDist w (.loop cond body) x (PMF.pure x) 2 := by + simpa using of_exec (Exec.loop_exit hc) + +/-- Sequencing with a uniform continuation step bound `T₂`. -/ +theorem seq_const {μ₂ : Data r → PMF (Data r)} {T₂ : Data r → ℝ≥0∞} {B₂ : ℝ≥0∞} + (h₁ : ExecCodeDist w c₁ x μ₁ T₁) + (h₂ : ∀ y ∈ μ₁.support, ExecCodeDist w c₂ y (μ₂ y) (T₂ y)) + (hB₂ : ∀ y ∈ μ₁.support, T₂ y ≤ B₂) : + ExecCodeDist w (.seq c₁ c₂) x (μ₁.bind μ₂) (T₁ + B₂) := by + refine (h₁.seq h₂).mono (add_le_add le_rfl ?_) + calc + (∑' y, μ₁ y * T₂ y) + ≤ ∑' y, μ₁ y * B₂ := by + refine ENNReal.tsum_le_tsum fun y => ?_ + by_cases hy : y ∈ μ₁.support + · exact mul_le_mul_right (hB₂ y hy) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] + _ = B₂ := by rw [ENNReal.tsum_mul_right, μ₁.tsum_coe, one_mul] + +/-- Sequencing after a deterministic first stage (`PMF.pure y`). -/ +theorem seq_pure (h₁ : ExecCodeDist w c₁ x (PMF.pure y) T₁) + (h₂ : ExecCodeDist w c₂ y μ T₂) : + ExecCodeDist w (.seq c₁ c₂) x μ (T₁ + T₂) := by + classical + have h := h₁.seq (c₂ := c₂) (μ₂ := fun z => if z = y then μ else PMF.pure z) + (T₂ := fun z => if z = y then T₂ else 0) fun z hz => by + rw [PMF.support_pure, Set.mem_singleton_iff] at hz + subst hz + simpa only [↓reduceIte] using h₂ + refine h.congr_dist ?_ ?_ + · simp + · rw [tsum_eq_single y] + · simp + · intro z hz + simp [PMF.pure_apply, hz] + +/-- Every state in the support of `μ` preserves `Data.Bounded w` when `x.Bounded w`. -/ +theorem bounded (h : ExecCodeDist w c x μ T) (hx : x.Bounded w) : + ∀ y ∈ μ.support, y.Bounded w := by + induction h with + | @of_execRand c x m hm => + intro y hy + rw [ProbComp.eval_map, PMF.mem_support_map_iff] at hy + obtain ⟨z, hz, rfl⟩ := hy + rw [ProbComp.support_eval] at hz + exact hm.bounded hx hz + | mono _ _ ih => exact ih hx + | seq _ _ ih₁ ih₂ => + intro y hy + rw [PMF.mem_support_bind_iff] at hy + obtain ⟨z, hz, hyz⟩ := hy + exact ih₂ z hz (ih₁ hx z hz) y hyz + | branch_pos _ _ ih => exact ih hx + | branch_neg _ _ ih => exact ih hx + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + intro y hy + obtain ⟨_, K, hK⟩ := mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx_inv hy + have hK_bdd : ∀ K : ℕ, ∀ x₀ : Data r, LoopStateInv cond I Q x₀ → x₀.Bounded w → + ∀ z ∈ (loopIterPMF cond μ_body K x₀).support, z.Bounded w := by + intro K + induction K with + | zero => + intro x₀ _ hx₀ z hz + rw [loopIterPMF_zero, PMF.support_pure, Set.mem_singleton_iff] at hz + rwa [hz] + | succ K ihK => + intro x₀ hx₀_inv hx₀ z hz + rw [loopIterPMF_succ, PMF.mem_support_bind_iff] at hz + obtain ⟨u, hu, hzu⟩ := hz + rcases hx₀_inv with ⟨hcx₀, hIx₀⟩ | ⟨hcx₀, hQx₀⟩ + · rw [loopStepPMF_of_holds hcx₀] at hu + exact ihK u (hinv x₀ hIx₀ hcx₀ u hu) (ih_body x₀ hIx₀ hcx₀ hx₀ u hu) z hzu + · rw [loopStepPMF_of_not_holds hcx₀, PMF.support_pure, Set.mem_singleton_iff] at hu + rw [hu] at hzu + exact ihK x₀ (Or.inr ⟨hcx₀, hQx₀⟩) hx₀ z hzu + exact hK_bdd K x hx_inv hx y hK + +/-- Any register `i` not written by `c` (`c.Preserves i`) is preserved across the support of `μ`. -/ +theorem regs_eq {i : Fin r} (h : ExecCodeDist w c x μ T) (hi : c.Preserves i) : + ∀ y ∈ μ.support, y.regs i = x.regs i := by + induction h with + | @of_execRand c x m hm => + intro y hy + rw [ProbComp.eval_map, PMF.mem_support_map_iff] at hy + obtain ⟨z, hz, rfl⟩ := hy + rw [ProbComp.support_eval] at hz + exact hm.regs_eq hi hz + | mono _ _ ih => exact ih hi + | seq _ _ ih₁ ih₂ => + intro y hy + rw [PMF.mem_support_bind_iff] at hy + obtain ⟨z, hz, hyz⟩ := hy + exact (ih₂ z hz hi.2 y hyz).trans (ih₁ hi.1 z hz) + | branch_pos _ _ ih => exact ih hi.1 + | branch_neg _ _ ih => exact ih hi.2 + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + intro y hy + obtain ⟨_, K, hK⟩ := mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx_inv hy + have hK_eq : ∀ K : ℕ, ∀ x₀ : Data r, LoopStateInv cond I Q x₀ → + ∀ z ∈ (loopIterPMF cond μ_body K x₀).support, z.regs i = x₀.regs i := by + intro K + induction K with + | zero => + intro x₀ _ z hz + rw [loopIterPMF_zero, PMF.support_pure, Set.mem_singleton_iff] at hz + rw [hz] + | succ K ihK => + intro x₀ hx₀_inv z hz + rw [loopIterPMF_succ, PMF.mem_support_bind_iff] at hz + obtain ⟨u, hu, hzu⟩ := hz + rcases hx₀_inv with ⟨hcx₀, hIx₀⟩ | ⟨hcx₀, hQx₀⟩ + · rw [loopStepPMF_of_holds hcx₀] at hu + exact (ihK u (hinv x₀ hIx₀ hcx₀ u hu) z hzu).trans (ih_body x₀ hIx₀ hcx₀ hi u hu) + · rw [loopStepPMF_of_not_holds hcx₀, PMF.support_pure, Set.mem_singleton_iff] at hu + rw [hu] at hzu + exact ihK x₀ (Or.inr ⟨hcx₀, hQx₀⟩) z hzu + exact hK_eq K x hx_inv y hK + +/-- Distributional executions transfer along any defined-evaluation basis simulation +`ψ : B.Simulates B'`. -/ +theorem mapSimulates {B' : Basis} (ψ : B.Simulates B') (h : ExecCodeDist w c x μ T) : + ExecCodeDist w (c.mapSimulates ψ) x μ T := by + induction h with + | of_execRand hm => exact of_execRand (hm.mapSimulates ψ) + | mono _ hle ih => exact ih.mono hle + | seq _ _ ih₁ ih₂ => exact ih₁.seq ih₂ + | branch_pos hc _ ih => exact branch_pos hc ih + | branch_neg hc _ ih => exact branch_neg hc ih + | loop_geometric I Q hp_pos hp_top _ hB hinv hp_exit hx ih_body => + exact loop_geometric I Q hp_pos hp_top ih_body hB hinv hp_exit hx + +private theorem extend_comap_eq {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (x : Data r') : + (x.comap f).extend f x = x := by + ext j + · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j + by_cases hj : ∃ i, f i = j + · obtain ⟨i, rfl⟩ := hj + exact hf.extend_apply _ _ i + · exact Function.extend_apply' _ _ _ hj + · rfl + +private theorem extend_left_injective {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) + (x : Data r') : Function.Injective (fun y : Data r => y.extend f x) := by + intro y₁ y₂ h + have h' := congrArg (Data.comap f) h + simpa [Data.comap_extend hf] using h' + +private theorem pmf_map_toOuterMeasure {α β : Type*} (p : PMF α) (g : α → β) (S : Set β) : + (p.map g).toOuterMeasure S = p.toOuterMeasure (g ⁻¹' S) := by + rw [PMF.map, PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] + refine tsum_congr fun a => ?_ + rw [Function.comp_apply, PMF.toOuterMeasure_pure_apply] + by_cases ha : g a ∈ S <;> simp [ha] + +private theorem pmf_map_apply_injective {α β : Type*} (p : PMF α) {g : α → β} + (hg : g.Injective) (a : α) : + p.map g (g a) = p a := by + rw [PMF.map_apply, tsum_eq_single a] + · simp + · intro a' ha' + have hne : g a ≠ g a' := hg.ne ha'.symm + simp [hne] + +private theorem pmf_map_apply_not_mem_range {α β : Type*} (p : PMF α) (g : α → β) {b : β} + (hb : b ∉ Set.range g) : + p.map g b = 0 := by + rw [PMF.map_apply, ENNReal.tsum_eq_zero] + intro a + have hne : b ≠ g a := fun h => hb ⟨a, h.symm⟩ + simp [hne] + +private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [ENNReal.tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +private theorem loopIterPMF_rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) + (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r') (K : ℕ) (u : Data r) : + loopIterPMF (cond.rename f) (fun x' => (μ_body (x'.comap f)).map (fun y => y.extend f x)) + K (u.extend f x) = + (loopIterPMF cond μ_body K u).map (fun y => y.extend f x) := by + induction K generalizing u with + | zero => simp [loopIterPMF] + | succ K ih => + rw [loopIterPMF_succ, loopIterPMF_succ] + by_cases hc : cond.Holds u + · have hc' : (cond.rename f).Holds (u.extend f x) := by + rw [Cond.holds_rename, Data.comap_extend hf] + exact hc + rw [loopStepPMF_of_holds hc', loopStepPMF_of_holds hc, Data.comap_extend hf, + PMF.bind_map, PMF.map_bind] + refine congrArg ((μ_body u).bind ·) (funext fun z => ih z) + · have hc' : ¬(cond.rename f).Holds (u.extend f x) := by + rw [Cond.holds_rename, Data.comap_extend hf] + exact hc + rw [loopStepPMF_of_not_holds hc', loopStepPMF_of_not_holds hc, PMF.pure_bind, + PMF.pure_bind, ih] + +/-- Renaming a distributional execution along an injective register map `f : Fin r → Fin r'` +transforms the output distribution via `Data.extend f x` and preserves the expected step bound. -/ +theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r'} + (h : ExecCodeDist w c (x.comap f) μ T) : + ExecCodeDist w (c.rename f) x (μ.map fun y => y.extend f x) T := by + generalize hx₀ : x.comap f = x₀ at h + induction h generalizing x with + | @of_execRand c _ m hm => + subst hx₀ + have hm' := of_execRand (hm.rename hf) + refine hm'.congr_dist ?_ ?_ + · rw [Functor.map_map, ProbComp.eval_map, ProbComp.eval_map, PMF.map_comp] + rfl + · rw [ProbComp.expect_map] + | mono _ hle ih => + exact (ih hx₀).mono hle + | @seq c₁ c₂ _ μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + subst hx₀ + have h₁' := ih₁ rfl + have hseq := h₁'.seq + (μ₂ := fun y' => (μ₂ (y'.comap f)).map fun z => z.extend f x) + (T₂ := fun y' => T₂ (y'.comap f)) + fun y' hy' => by + rw [PMF.mem_support_map_iff] at hy' + obtain ⟨y₁, hy₁, rfl⟩ := hy' + have h₂' := ih₂ y₁ hy₁ (x := y₁.extend f x) (Data.comap_extend hf y₁ x) + simpa only [Data.comap_extend hf, Data.extend_extend hf] using h₂' + refine hseq.congr_dist ?_ ?_ + · rw [PMF.map_bind, PMF.bind_map] + refine congrArg (μ₁.bind ·) (funext fun y₁ => ?_) + simp [Data.comap_extend hf] + · refine add_le_add le_rfl ?_ + rw [tsum_pmf_map] + simp [Data.comap_extend hf] + | branch_pos hc _ ih => + subst hx₀ + exact branch_pos ((Cond.holds_rename _ _).2 hc) (ih rfl) + | branch_neg hc _ ih => + subst hx₀ + exact branch_neg (fun h => hc ((Cond.holds_rename _ _).1 h)) (ih rfl) + | @loop_geometric cond body I Q μ_body T_body B_body p_exit _ + hp_pos hp_top _ hB hinv hp_exit hx_inv ih_body => + subst hx₀ + let g : Data r → Data r' := fun y => y.extend f x + have hg : g.Injective := extend_left_injective hf x + let I' : Data r' → Prop := fun x' => ∃ u, I u ∧ x' = g u + let Q' : Data r' → Prop := fun x' => ∃ u, Q u ∧ x' = g u + let μ_body' : Data r' → PMF (Data r') := fun x' => (μ_body (x'.comap f)).map g + let T_body' : Data r' → ℝ≥0∞ := fun x' => T_body (x'.comap f) + have hinv' : ∀ z', I' z' → (cond.rename f).Holds z' → + ∀ y' ∈ (μ_body' z').support, LoopStateInv (cond.rename f) I' Q' y' := by + rintro z' ⟨u, hIu, rfl⟩ hcz' y' hy' + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + simp only [μ_body', g, Data.comap_extend hf, PMF.mem_support_map_iff] at hy' + obtain ⟨y, hy, rfl⟩ := hy' + rcases hinv u hIu hcu y hy with ⟨hcy, hIy⟩ | ⟨hcy, hQy⟩ + · exact Or.inl ⟨by simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy, + ⟨y, hIy, rfl⟩⟩ + · exact Or.inr ⟨by simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy, + ⟨y, hQy, rfl⟩⟩ + have hp_exit' : ∀ z', I' z' → (cond.rename f).Holds z' → + p_exit ≤ (μ_body' z').toOuterMeasure {y' | ¬(cond.rename f).Holds y'} := by + rintro z' ⟨u, hIu, rfl⟩ hcz' + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + simp only [μ_body', g, Data.comap_extend hf, pmf_map_toOuterMeasure] + have hset : ((fun y : Data r => y.extend f x) ⁻¹' {y' | ¬(cond.rename f).Holds y'}) = + {y | ¬cond.Holds y} := by + ext y + simp [Cond.holds_rename, Data.comap_extend hf] + rw [hset] + exact hp_exit u hIu hcu + have hx_inv' : LoopStateInv (cond.rename f) I' Q' x := by + rcases hx_inv with ⟨hcx, hIx⟩ | ⟨hcx, hQx⟩ + · exact Or.inl ⟨(Cond.holds_rename _ _).2 hcx, ⟨x.comap f, hIx, (extend_comap_eq hf x).symm⟩⟩ + · exact Or.inr ⟨fun h => hcx ((Cond.holds_rename _ _).1 h), + ⟨x.comap f, hQx, (extend_comap_eq hf x).symm⟩⟩ + have hloop := loop_geometric (cond := cond.rename f) (body := body.rename f) + I' Q' hp_pos hp_top + (μ_body := μ_body') (T_body := T_body') (B_body := B_body) (x := x) + (fun z' ⟨u, hIu, hzu⟩ hcz' => by + subst hzu + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + have hb' := ih_body u hIu hcu (x := g u) (Data.comap_extend hf u x) + simpa only [μ_body', T_body', g, Data.comap_extend hf, Data.extend_extend hf] using hb') + (fun z' ⟨u, hIu, hzu⟩ hcz' => by + subst hzu + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + simpa only [T_body', g, Data.comap_extend hf] using hB u hIu hcu) + hinv' hp_exit' hx_inv' + refine hloop.congr_dist ?_ le_rfl + ext y' + rw [loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv' hp_exit' hx_inv')] + have hx_eq : x = g (x.comap f) := (extend_comap_eq hf x).symm + by_cases hy' : y' ∈ Set.range g + · obtain ⟨y, rfl⟩ := hy' + rw [pmf_map_apply_injective _ hg, + loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx_inv)] + refine iSup_congr fun K => ?_ + have hiter : loopIterPMF (cond.rename f) μ_body' K x = + (loopIterPMF cond μ_body K (x.comap f)).map g := by + conv_lhs => rw [hx_eq] + exact loopIterPMF_rename hf cond μ_body x K (x.comap f) + rw [hiter] + by_cases hcy : ¬cond.Holds y + · have hcy' : ¬(cond.rename f).Holds (g y) := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy + simp [Set.indicator_of_mem, hcy, hcy', pmf_map_apply_injective _ hg] + · have hcy' : ¬¬(cond.rename f).Holds (g y) := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy + simp [Set.indicator_of_notMem, hcy, hcy'] + · rw [pmf_map_apply_not_mem_range _ g hy'] + refine le_antisymm (iSup_le fun K => ?_) zero_le + have hiter : loopIterPMF (cond.rename f) μ_body' K x = + (loopIterPMF cond μ_body K (x.comap f)).map g := by + conv_lhs => rw [hx_eq] + exact loopIterPMF_rename hf cond μ_body x K (x.comap f) + rw [hiter] + by_cases hcy' : ¬(cond.rename f).Holds y' + · simp [Set.indicator_of_mem, hcy', pmf_map_apply_not_mem_range _ g hy'] + · simp [Set.indicator_of_notMem, hcy'] + +/-! ### Compilation of `ExecCodeDist` to flat `Program` semantics -/ + +/-- Finite-horizon Bellman inequality for `ExecCodeDist` inside any containing flat program `p`. -/ +theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (N : ℕ) : + expectedStepsWithin w p N (.running ⟨pc, x⟩) ≤ + T + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + c.size, y⟩) := by + induction h generalizing pc N with + | @of_execRand c x m hm => + refine (hm.expectedStepsWithin_assemble hp N).trans (le_of_eq ?_) + rw [ProbComp.expect_add, + ← ProbComp.expect_map Prod.fst m (fun y => + expectedStepsWithin w p N (.running ⟨pc + c.size, y⟩))] + rfl + | mono _ hle ih => + exact (ih hp N).trans (add_le_add hle le_rfl) + | @seq c₁ c₂ x μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine (ih₁ hp.append_left N).trans ?_ + have hstep : (∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩)) ≤ + (∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by + calc + (∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩)) + ≤ ∑' y, μ₁ y * (T₂ y + + ∑' z, (μ₂ y) z * + expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩)) := by + refine ENNReal.tsum_le_tsum fun y => ?_ + by_cases hy : y ∈ μ₁.support + · exact mul_le_mul_right (ih₂ y hy hc₂ N) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] + _ = (∑' y, μ₁ y * T₂ y) + + ∑' y, μ₁ y * ∑' z, (μ₂ y) z * + expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩) := by + simp_rw [mul_add, ENNReal.tsum_add] + _ = (∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by + congr 1 + simp only [PMF.bind_apply, Code.size, Nat.add_assoc] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + calc + T₁ + ∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩) + ≤ T₁ + ((∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩)) := + add_le_add le_rfl hstep + _ = (T₁ + ∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by + rw [add_assoc] + | @branch_pos cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ih hc' N)).trans (le_of_eq ?_) + simp only [Code.size, + show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + ac_rfl + | @branch_neg cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ih hp.cons_tail.append_left N)).trans ?_ + have hjump : (∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) ≤ + 1 + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩) := by + calc + (∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) + ≤ ∑' y, μ y * (1 + + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩)) := by + refine ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _ + refine (expectedStepsWithin_running_le w p N ⟨pc + 1 + neg.size, y⟩).trans (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj, ProbComp.expect_pure, + show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + _ = 1 + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩) := by + simp_rw [mul_add, ENNReal.tsum_add, mul_one, μ.tsum_coe] + calc + 1 + (T + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) + ≤ 1 + (T + (1 + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩))) := + add_le_add le_rfl (add_le_add le_rfl hjump) + _ = (T + (1 + 1)) + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (Code.branch cond pos neg).size, y⟩) := by + simp only [Code.size] + ac_rfl + _ = (T + 2) + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (Code.branch cond pos neg).size, y⟩) := by + rw [one_add_one_eq_two] + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ hB hinv hp_exit hx_inv ih_body => + have hj_exit : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj_back : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + set K_cont : Data r → ℝ≥0∞ := fun y => + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, y⟩) + set M := (B_body + 2) / p_exit + have hmain : ∀ n ≤ N, ∀ z : Data r, LoopStateInv cond I Q z → + expectedStepsWithin w p n (.running ⟨pc, z⟩) ≤ + (if cond.Holds z then M else 0) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + intro n + induction n with + | zero => simp + | succ n ihn => + intro hn z hz_inv + rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ + · simp only [hcz, ↓reduceIte] + rw [expectedStepsWithin_running_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, ProbComp.expect_pure] + have hbody_step := (ih_body z hIz hcz hbody_contains n).trans + (add_le_add (hB z hIz hcz) le_rfl) + have hback_step : + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc + 2 + body.size, u⟩)) ≤ + 1 + ∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩) := by + calc + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc + 2 + body.size, u⟩)) + ≤ ∑' u, μ_body z u * (1 + expectedStepsWithin w p n (.running ⟨pc, u⟩)) := by + refine ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _ + refine (expectedStepsWithin_running_le w p n ⟨pc + 2 + body.size, u⟩).trans + (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back, ProbComp.expect_pure] + _ = 1 + ∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩) := by + simp_rw [mul_add, ENNReal.tsum_add, mul_one, (μ_body z).tsum_coe] + have hbind_k : + (∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y) = + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz] + simp only [PMF.bind_apply] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + have hih_step : + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ + (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + calc + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) + ≤ ∑' u, μ_body z u * ((if cond.Holds u then M else 0) + 2 + + ∑' y, loopExitPMF cond μ_body u y * K_cont y) := by + refine ENNReal.tsum_le_tsum fun u => ?_ + by_cases hu : u ∈ (μ_body z).support + · exact mul_le_mul_right + (ihn (Nat.le_of_succ_le hn) u (hinv z hIz hcz u hu)) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] + _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y := by + simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, (μ_body z).tsum_coe, + one_mul] + _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + rw [hbind_k] + have hdrift := pmf_step_drift_bound (μ_body z) cond (C := B_body + 2) + hp_pos hp_top (hp_exit z hIz hcz) + calc + 1 + expectedStepsWithin w p n (.running ⟨pc + 2, z⟩) + ≤ 1 + (B_body + (1 + ∑' u, μ_body z u * + expectedStepsWithin w p n (.running ⟨pc, u⟩))) := + add_le_add le_rfl (hbody_step.trans (add_le_add le_rfl hback_step)) + _ = (B_body + (1 + 1)) + ∑' u, μ_body z u * + expectedStepsWithin w p n (.running ⟨pc, u⟩) := by ac_rfl + _ = (B_body + 2) + ∑' u, μ_body z u * + expectedStepsWithin w p n (.running ⟨pc, u⟩) := by rw [one_add_one_eq_two] + _ ≤ (B_body + 2) + ((∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y) := + add_le_add le_rfl hih_step + _ = ((B_body + 2) + ∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by ac_rfl + _ ≤ M + 2 + ∑' y, loopExitPMF cond μ_body z y * K_cont y := + add_le_add (add_le_add hdrift le_rfl) le_rfl + · simp only [hcz, ↓reduceIte, zero_add] + rw [loopExitPMF_of_not_holds hcz, tsum_eq_single z + (fun y hy => by simp [PMF.pure_apply, hy])] + simp only [PMF.pure_apply_self, one_mul] + rw [expectedStepsWithin_running_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (expectedStepsWithin_running_le w p n ⟨pc + 1, z⟩)).trans ?_ + rw [stepRand_jump (s := ⟨pc + 1, z⟩) hj_exit, ProbComp.expect_pure, ← add_assoc, + one_add_one_eq_two] + refine add_le_add le_rfl ?_ + simpa [K_cont, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using expectedStepsWithin_mono w p + (.running ⟨pc + (body.size + 3), z⟩) (Nat.le_of_succ_le hn) + refine (hmain N le_rfl x hx_inv).trans ?_ + refine add_le_add (add_le_add ?_ le_rfl) le_rfl + split_ifs <;> simp + +/-- Full-horizon Bellman inequality for `ExecCodeDist` inside any containing flat program `p`. -/ +theorem expectedStepsFrom_assemble (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) : + expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ + T + ∑' y, μ y * expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) := by + rw [← iSup_expectedStepsWithin w p (.running ⟨pc, x⟩)] + refine iSup_le fun N => (h.expectedStepsWithin_assemble hp N).trans ?_ + refine add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _) + exact expectedStepsWithin_le_expectedStepsFrom w p N (.running ⟨pc + c.size, y⟩) + +/-- Continuation upper-bound form of `expectedStepsFrom_assemble`. -/ +theorem expectedStepsFrom_le (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) {k : Data r → ℝ≥0∞} + (hk : ∀ y ∈ μ.support, expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) ≤ k y) : + expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ T + ∑' y, μ y * k y := by + refine (h.expectedStepsFrom_assemble hp).trans + (add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => ?_)) + by_cases hy : y ∈ μ.support + · exact mul_le_mul_right (hk y hy) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] + +/-- Along `ExecCodeDist w c x μ T`, if no continuation from `⟨pc + c.size, y⟩` faults within +`T_hor` steps for any supported `y ∈ μ.support`, then running `p` from `⟨pc, x⟩` does not fault +within `T_hor` steps. -/ +theorem evalRand_faulted_eq_zero (h : ExecCodeDist w c x μ T) {p : Program B r} {pc T_hor : ℕ} + (hp : p.Contains pc (c.assemble pc)) + (hcont : ∀ y ∈ μ.support, ∀ t ≤ T_hor, evalRand w p t ⟨pc + c.size, y⟩ .faulted = 0) : + ∀ t ≤ T_hor, evalRand w p t ⟨pc, x⟩ .faulted = 0 := by + induction h generalizing pc T_hor with + | @of_execRand c x m hm => + refine hm.evalRand_faulted_eq_zero hp fun z hz t ht => ?_ + apply hcont z.1 + · rw [ProbComp.eval_map, PMF.mem_support_map_iff] + refine ⟨z, ?_, rfl⟩ + rwa [ProbComp.support_eval] + · exact ht + | mono _ _ ih => exact ih hp hcont + | @seq c₁ c₂ x μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine ih₁ hp.append_left fun y₁ hy₁ => ih₂ y₁ hy₁ hc₂ fun y₂ hy₂ t ht => ?_ + have hmem : y₂ ∈ (μ₁.bind μ₂).support := + (PMF.mem_support_bind_iff _ _ _).mpr ⟨y₁, hy₁, hy₂⟩ + simpa [Code.size, Nat.add_assoc] using hcont y₂ hmem t ht + | @branch_pos cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hc' (T_hor := k) (fun y hy t' ht' => ?_) k le_rfl + simpa [Code.size, show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + using hcont y hy t' (by omega) + | @branch_neg cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hp.cons_tail.append_left (T_hor := k) (fun y hy t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj, pure_bind] + simpa [evalRand, Code.size, + show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using hcont y hy k' (by omega) + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + have hj_exit : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj_back : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + have hmain : ∀ n ≤ T_hor, ∀ z : Data r, LoopStateInv cond I Q z → + (∀ y ∈ (loopExitPMF cond μ_body z).support, ∀ t ≤ T_hor, + evalRand w p t ⟨pc + (Code.loop cond body).size, y⟩ .faulted = 0) → + evalRand w p n ⟨pc, z⟩ .faulted = 0 := by + intro n + induction n using Nat.strong_induction_on with + | _ n ihn => + intro hn z hz_inv hcont_z + cases n with + | zero => simp [evalRand] + | succ k => + rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ + · rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, pure_bind] + refine ih_body z hIz hcz hbody_contains (T_hor := k) (fun u hu t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back, + pure_bind] + refine ihn k' (by omega) (by omega) u (hinv z hIz hcz u hu) fun y hy t'' ht'' => ?_ + apply hcont_z y _ t'' ht'' + rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz, PMF.mem_support_bind_iff] + exact ⟨u, hu, hy⟩ + · rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, pure_bind] + cases k with + | zero => simp + | succ k' => + rw [runRand_succ, stepRand_jump (s := ⟨pc + 1, z⟩) hj_exit, pure_bind] + have hz_mem : z ∈ (loopExitPMF cond μ_body z).support := by + simp [loopExitPMF_of_not_holds hcz] + simpa [evalRand, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using hcont_z z hz_mem k' (by omega) + intro t ht + exact hmain t ht x hx_inv hcont + +/-- If every continuation from `⟨pc + c.size, y⟩` (`y ∈ μ.support`) is fault-free at all horizons, +then running `p` from `⟨pc, x⟩` is fault-free at all horizons. -/ +theorem randFaultFree_assemble (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) + (hcont : ∀ y ∈ μ.support, RandFaultFree w p ⟨pc + c.size, y⟩) : + RandFaultFree w p ⟨pc, x⟩ := by + intro T_hor + rw [← support_evalRand, ← PMF.apply_eq_zero_iff] + exact h.evalRand_faulted_eq_zero hp + (fun y hy t _ => (hcont y hy).evalRand_faulted_eq_zero t) + T_hor le_rfl + +/-- Along `ExecCodeDist w c x μ T`, the `μ`-weighted supremum halting probability from +`⟨pc + c.size, y⟩` is a lower bound on the supremum halting probability from `⟨pc, x⟩`. -/ +theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (v : ℕ) (d : Data r) : + (∑' y, μ y * ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c.size, y⟩ (.halted v d)) ≤ + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc, x⟩ (.halted v d) := by + induction h generalizing pc with + | @of_execRand c x m hm => + refine (le_of_eq ?_).trans (hm.le_iSup_evalRand_halted hp v d) + exact ProbComp.expect_map Prod.fst m + (fun y => ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c.size, y⟩ (.halted v d)) + | mono _ _ ih => exact ih hp + | @seq c₁ c₂ x μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + have hbind : + (∑' y, (μ₁.bind μ₂) y * + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + (c₁.seq c₂).size, y⟩ (.halted v d)) = + ∑' y₁, μ₁ y₁ * ∑' y, (μ₂ y₁) y * + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c₁.size + c₂.size, y⟩ (.halted v d) := by + simp only [PMF.bind_apply, Code.size, Nat.add_assoc] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + rw [hbind] + refine (ENNReal.tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) + by_cases hy₁ : y₁ ∈ μ₁.support + · exact mul_le_mul_right (ih₂ y₁ hy₁ hc₂) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy₁, zero_mul, zero_mul] + | @branch_pos cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + neg.size + 2, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + refine (le_of_eq ?_).trans ((ih hc').trans (iSup_evalRand_halted_le_of_step hstep v d)) + refine tsum_congr fun y => ?_ + simp [Code.size, show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + | @branch_neg cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + refine (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _).trans + ((ih hp.cons_tail.append_left).trans (iSup_evalRand_halted_le_of_step hstep v d)) + simpa [Code.size, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj) v d + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + have hj_exit : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj_back : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + set H_loop : Data r → ℝ≥0∞ := fun z => ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc, z⟩ (.halted v d) + set H_exit : Data r → ℝ≥0∞ := fun z => + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + (Code.loop cond body).size, z⟩ (.halted v d) + have hexit_le : ∀ z : Data r, ¬cond.Holds z → H_exit z ≤ H_loop z := by + intro z hcz + have hbr : stepRand w p (.running ⟨pc, z⟩) = pure (.running ⟨pc + 1, z⟩) := by + rw [stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp [hcz] + simpa [H_exit, H_loop, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using (iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 1, z⟩) hj_exit) v d).trans + (iSup_evalRand_halted_le_of_step hbr v d) + have hstep_le : ∀ z : Data r, I z → cond.Holds z → + (∑' u, μ_body z u * H_loop u) ≤ H_loop z := by + intro z hIz hcz + have hbr : stepRand w p (.running ⟨pc, z⟩) = pure (.running ⟨pc + 2, z⟩) := by + rw [stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp [hcz] + refine (ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _).trans + ((ih_body z hIz hcz hbody_contains).trans (iSup_evalRand_halted_le_of_step hbr v d)) + exact iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back) v d + have hiter_le : ∀ K : ℕ, ∀ z : Data r, LoopStateInv cond I Q z → + (∑' y, loopIterPMF cond μ_body K z y * H_loop y) ≤ H_loop z := by + intro K + induction K with + | zero => + intro z _ + rw [tsum_eq_single z (fun y hy => by simp [loopIterPMF_zero, PMF.pure_apply, hy])] + simp [H_loop] + | succ K ihK => + intro z hz_inv + rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ + · rw [loopIterPMF_succ, loopStepPMF_of_holds hcz] + have hbind : (∑' y, ((μ_body z).bind (loopIterPMF cond μ_body K)) y * H_loop y) = + ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := by + simp only [PMF.bind_apply] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + rw [hbind] + refine (ENNReal.tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) + by_cases hu : u ∈ (μ_body z).support + · exact mul_le_mul_right (ihK u (hinv z hIz hcz u hu)) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] + · rw [loopIterPMF_of_not_holds hcz, + tsum_eq_single z (fun y hy => by simp [PMF.pure_apply, hy])] + simp + have hrewrite_sum : + (∑' y, loopExitPMF cond μ_body x y * H_exit y) = + ⨆ K : ℕ, ∑' y, + Set.indicator {w' | ¬cond.Holds w'} (loopIterPMF cond μ_body K x) y * H_exit y := by + simp_rw [loopExitPMF_apply_of_hasSum + (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx_inv), ENNReal.iSup_mul] + exact tsum_iSup_of_monotone fun y K₁ K₂ hK => + mul_le_mul_left (loopIterPMF_indicator_mono cond μ_body x y hK) (H_exit y) + rw [hrewrite_sum] + refine iSup_le fun K => (ENNReal.tsum_le_tsum fun y => ?_).trans (hiter_le K x hx_inv) + by_cases hcy : ¬cond.Holds y + · rw [Set.indicator_of_mem hcy] + exact mul_le_mul_right (hexit_le y hcy) _ + · simp [Set.indicator_of_notMem, hcy] + +/-- Compiling `ExecCodeDist w c x μ T` with `c.program out` yields: +1. `expectedTime w (c.program out) ⟨0, x⟩ ≤ T + 1`, +2. `RandFaultFree w (c.program out) ⟨0, x⟩`, +3. `AlmostSurelyHalts w (c.program out) ⟨0, x⟩` whenever `T ≠ ⊤`, and +4. for every `y : Data r`, + `μ y ≤ ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0, x⟩ (.halted (y.regs out) y)`. -/ +theorem program_halts (h : ExecCodeDist w c x μ T) (out : Fin r) : + expectedTime w (c.program out) ⟨0, x⟩ ≤ T + 1 ∧ + RandFaultFree w (c.program out) ⟨0, x⟩ ∧ + (T ≠ ⊤ → AlmostSurelyHalts w (c.program out) ⟨0, x⟩) ∧ + ∀ y : Data r, + μ y ≤ ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0, x⟩ (.halted (y.regs out) y) := by + have hp : (c.program out).Contains 0 (c.assemble 0) := + (Program.Contains.self (c.program out)).append_left + have hhalt : (c.program out)[0 + c.size]? = some (.halt out) := by + simp [Code.program] + have hexp : expectedTime w (c.program out) ⟨0, x⟩ ≤ T + 1 := by + rw [expectedTime_eq_expectedStepsFrom] + refine (h.expectedStepsFrom_le (k := fun _ => 1) hp fun y _ => ?_).trans ?_ + · rw [expectedStepsFrom_running, stepRand_halt (s := ⟨0 + c.size, y⟩) hhalt, + ProbComp.expect_pure, expectedStepsFrom_halted, add_zero] + · simp [μ.tsum_coe] + have hff : RandFaultFree w (c.program out) ⟨0, x⟩ := by + refine h.randFaultFree_assemble hp fun y _ t => ?_ + rw [← support_evalRand, ← PMF.apply_eq_zero_iff] + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_halt (s := ⟨0 + c.size, y⟩) hhalt, pure_bind, + runRand_halted] + simp [PMF.pure_apply] + refine ⟨hexp, hff, fun hT => ?_, fun y => ?_⟩ + · refine AlmostSurelyHalts.of_expectedTime_ne_top hff ?_ + exact ne_top_of_le_ne_top (ENNReal.add_ne_top.mpr ⟨hT, ENNReal.one_ne_top⟩) hexp + · have hle := h.le_iSup_evalRand_halted hp (y.regs out) y + refine (le_trans ?_ + (ENNReal.le_tsum + (f := fun z => μ z * + ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0 + c.size, z⟩ + (.halted (y.regs out) y)) y)).trans hle + have h1 : (1 : ℝ≥0∞) ≤ + ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0 + c.size, y⟩ (.halted (y.regs out) y) := by + refine le_iSup_of_le 1 ?_ + rw [evalRand, runRand_succ, stepRand_halt (s := ⟨0 + c.size, y⟩) hhalt, pure_bind, + runRand_zero, ProbComp.eval_pure, PMF.pure_apply_self] + simpa using mul_le_mul_right h1 (μ y) + +end ExecCodeDist + +/-- Geometric Las Vegas loop expected-step bound inside any flat program `p` containing +`(Code.loop cond body).assemble pc`. -/ +theorem expectedStepsFrom_loop_le_of_geometric {w : ℕ} {cond : Cond r} {body : Code B r} + (I Q : Data r → Prop) {μ_body : Data r → PMF (Data r)} {T_body : Data r → ℝ≥0∞} + {B_body p_exit : ℝ≥0∞} {x : Data r} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hbody : ∀ z, I z → cond.Holds z → ExecCodeDist w body z (μ_body z) (T_body z)) + (hB : ∀ z, I z → cond.Holds z → T_body z ≤ B_body) + (hinv : ∀ z, I z → cond.Holds z → + ∀ y ∈ (μ_body z).support, LoopStateInv cond I Q y) + (hp_exit : ∀ z, I z → cond.Holds z → + p_exit ≤ (μ_body z).toOuterMeasure {y | ¬cond.Holds y}) + (hx_inv : LoopStateInv cond I Q x) + {p : Program B r} {pc : ℕ} (hp : p.Contains pc ((Code.loop cond body).assemble pc)) + {k : Data r → ℝ≥0∞} + (hk : ∀ y, ¬cond.Holds y → Q y → + expectedStepsFrom w p (.running ⟨pc + (Code.loop cond body).size, y⟩) ≤ k y) : + expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ + ((B_body + 2) / p_exit + 2) + ∑' y, loopExitPMF cond μ_body x y * k y := by + have hloop := ExecCodeDist.loop_geometric I Q hp_pos hp_top hbody hB hinv hp_exit hx_inv + refine hloop.expectedStepsFrom_le hp fun y hy => ?_ + obtain ⟨⟨hcy, hQy⟩, _⟩ := mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx_inv hy + exact hk y hcy hQy + +end Cslib.WordRAM + +end diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean new file mode 100644 index 000000000..f89efc199 --- /dev/null +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -0,0 +1,1046 @@ +/- +Copyright (c) 2026 Chaeyoung Lim. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Chaeyoung Lim +-/ + +module +-- upstream: candidate + +public import Complexity.Computability.WordRAM.Subroutine +public import Complexity.Computability.WordRAM.Randomized.Expected + +/-! +# Composable Randomized Word-RAM Data-Structure Subroutines + +This file extends the deterministic `Cslib.WordRAM.SubroutineDS` interface to **randomized data +structures and Las Vegas subroutines** on the Word-RAM — covering both structures initialized from a +random seed (such as universal hash tables, 2-choice hash tables, and cuckoo hash tables) and +structures whose operations make random choices via `Instr.rand` (such as treaps, skip lists, +reservoir samplers, or randomized rehashing) `[MitzenmacherUpfal2017]`. + +## Core design + +A `Cslib.WordRAM.Randomized.SubroutineDS P D w base params s₀` couples machine execution with a +latent seed type `Seed : Type` and reference distribution `seedLaw : PMF Seed` up to a bounded +density factor `seedSlack : ℝ≥0∞`: + +1. **Seed-coupled representation (`rep θ s x`) and potential (`potential θ x`):** + - Data structures without a persistent random seed set `Seed := Unit`, `seedLaw := PMF.pure ()`, + and `seedSlack := 1` (in particular, `Cslib.WordRAM.SubroutineDS.toRandomized` embeds every + deterministic `Cslib.WordRAM.SubroutineDS` directly). + - Hash tables that resample their hash family on rare table-full events maintain a joint + distribution `ν : PMF (Seed × Data r_ds)` whose marginal on `Seed` satisfies + `(ν.map Prod.fst) θ ≤ seedSlack * seedLaw θ`. + +2. **Distributional execution (`ExecCodeDist`):** + - `setup_exec` and `step_exec` use `ExecCodeDist` from + `Complexity.Computability.WordRAM.Randomized.Expected`, which supports both bounded + randomized code (`ExecRand`) and unbounded almost-surely terminating Las Vegas `while` loops + (`ExecCodeDist.loop_geometric`). + +3. **Addition-only `ℝ≥0∞` amortized-in-expectation bound:** + - Every admissible operation `o : D.Op` from abstract state `s : D.State` and coupled state + distribution `ν : PMF (Seed × Data r_ds)` produces a transition kernel + `μ_step : Seed × Data r_ds → PMF (Seed × Data r_ds)` and expected step count + `T_step : Seed × Data r_ds → ℝ≥0∞` satisfying + `∑' p, ν p * (T_step p + ∑' q, μ_step p q * Φ(q)) ≤ charge s o + ∑' p, ν p * Φ(p)`. + - Across any admissible finite operation sequence `ops : List D.Op`, `seq_expected_amortized` + telescopes these one-step bounds without `ℝ≥0∞` subtraction: + total expected step cost is at most `initTime + initPotential + D.totalCost charge s₀ ops`. + +## Main definitions + +* `Cslib.WordRAM.Randomized.SubroutineDS`: reusable randomized Word-RAM subroutine certificate for + a deterministic specification `D : Stateful.Spec`. +* `Cslib.WordRAM.SubroutineDS.toRandomized`: canonical embedding of any deterministic + `Cslib.WordRAM.SubroutineDS` into `Cslib.WordRAM.Randomized.SubroutineDS`. +* `Cslib.WordRAM.Randomized.SubroutineDS.mapSimulates`, `toStandard`: basis transport from + `Basis.exact` to `Basis.standard`. + +## Main results + +* `Cslib.WordRAM.Randomized.SubroutineDS.initFresh_exec`, `initFresh_rename`: expected-time + initialization on fresh zeroed memory. +* `Cslib.WordRAM.Randomized.SubroutineDS.zeroFillCode_exec`, `initClear_exec`, `initClear_rename`: + expected-time initialization on arbitrary dirty memory. +* `Cslib.WordRAM.Randomized.SubroutineDS.code_rename`, `call_rename`: inline subroutine invocation + on caller state space `Data r_caller` via register injection `f : Fin r_ds → Fin r_caller`. +* `Cslib.WordRAM.Randomized.SubroutineDS.frame_caller`, `frame_caller_dist`: preservation of `rep`, + `potential`, and the coupled `(Seed × Data r_caller)` distribution across arbitrary deterministic + or randomized caller steps that do not modify persistent DS registers (`P.input i = false`) or + `[base, base + space)`. +* `Cslib.WordRAM.Randomized.SubroutineDS.steps_expected_amortized`, `seq_expected_amortized`: + end-to-end telescope theorems bounding total expected time across any admissible operation + sequence `ops : List D.Op`. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Chapter 17][CLRS2009] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing: Randomization and Probabilistic + Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017] +-/ + +@[expose] public section + +namespace Cslib.WordRAM.Randomized + +open Cslib.WordRAM +open scoped ENNReal + +universe u v z + +/-- Unified standalone and inline subroutine specification/resource certificate for a randomized +Word-RAM data structure `P` realizing `D` at word width `w`, base address `base`, runtime +parameters `params`, and initial abstract state `s₀`. -/ +structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} + (P : SubroutineCode B r_ds k_init) (D : Stateful.Spec.{u, v, z}) + (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) : Type (max u v z 1) where + /-- Latent random seed type (use `Unit` when no persistent seed is needed). -/ + Seed : Type + /-- Reference distribution over latent seeds. -/ + seedLaw : PMF Seed + /-- Marginal density slack factor (`1` when the seed distribution is exact). -/ + seedSlack : ℝ≥0∞ := 1 + /-- The marginal density slack factor is finite. -/ + seedSlack_ne_top : seedSlack ≠ ⊤ := by simp + /-- Contiguous memory footprint length `[base, base + space)`. -/ + space : ℕ + /-- Worst-case instruction bound for `P.allocSpan`. -/ + spanTime : ℕ + /-- Expected instruction bound for `P.initSetup`. -/ + setupTime : ℝ≥0∞ + /-- Upper bound on initial concrete potential after initialization. -/ + initPotential : ℕ + /-- Amortized instruction charge per operation. -/ + charge : D.State → D.Op → ℕ + /-- Seed-indexed representation invariant linking `Seed`, `D.State`, and `Data r_ds`. -/ + rep : Seed → D.State → Data r_ds → Prop + /-- Seed-indexed nonnegative integer potential. -/ + potential : Seed → Data r_ds → ℕ + /-- Standalone register encoding for argument delivery. -/ + encode : D.Op → Fin r_ds → ℕ + /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ + argMatch : D.Op → (Fin r_ds → ℕ) → Prop + /-- Output decoder reading typed operation observations from final registers. -/ + decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) + /-- Represented configurations are word-bounded. -/ + bounded : ∀ {θ s x}, rep θ s x → x.Bounded w + /-- For a fixed seed `θ`, concrete machine data determines at most one abstract state. -/ + rep_unique : ∀ {θ s₁ s₂ x}, rep θ s₁ x → rep θ s₂ x → s₁ = s₂ + /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ + arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) + /-- Full standalone argument delivery satisfies `argMatch`. -/ + encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), + argMatch o (fun i => if P.input i then encode o i else regs i) + /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. + -/ + rep_supported : ∀ {θ s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + x'.Bounded w → rep θ s x → rep θ s x' + /-- `potential` depends only on persistent registers and `[base, base + space)` under `rep`. -/ + potential_supported : ∀ {θ s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + rep θ s x → potential θ x' = potential θ x + /-- Distinctness of registers used by `zeroFillCode` relative to `baseReg` and `frontierReg`. -/ + clear_nodup : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup + /-- Initialization parameter registers are not overwritten by `zeroFillCode`. -/ + params_not_clear : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg] + /-- `P.incCur` increments `P.curReg` by `1` when `P.oneReg = 1` and the sum fits in `w` bits. -/ + incCur_eval : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → + P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1)) + /-- Execution of `P.allocSpan`: records `baseReg = base` and advances `frontierReg`. -/ + span_exec : ∀ {x : Data r_ds}, + x.Bounded w → x.regs P.frontierReg = base → (∀ j, x.regs (P.initParams j) = params j) → + ∃ x₁, ExecWithin w P.allocSpan x x₁ spanTime ∧ x₁.mem = x.mem ∧ + x₁.regs P.baseReg = base ∧ x₁.regs P.frontierReg = base + space ∧ + (∀ j, x₁.regs (P.initParams j) = params j) + /-- Execution of `P.initSetup` once `[base, base + space)` holds zeros, producing a coupled + distribution `ν₀ : PMF (Seed × Data r_ds)` whose seed marginal is bounded by + `seedSlack * seedLaw`. -/ + setup_exec : ∀ {x₁ : Data r_ds}, + x₁.Bounded w → x₁.regs P.baseReg = base → x₁.regs P.frontierReg = base + space → + (∀ j, x₁.regs (P.initParams j) = params j) → + (∀ a ∈ (Block.mk base space).footprint, x₁.mem a = 0) → + ∃ ν₀ : PMF (Seed × Data r_ds), + ExecCodeDist w P.initSetup x₁ (ν₀.map Prod.snd) setupTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ seedSlack * seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + space ∧ + rep p.1 s₀ p.2 ∧ + potential p.1 p.2 ≤ initPotential ∧ + Modifies (Block.mk base space).footprint x₁.mem p.2.mem + /-- Coupled one-step execution of `P.code` for any admissible operation `o` from abstract state + `s` and coupled state distribution `ν : PMF (Seed × Data r_ds)` with seed marginal bounded by + `seedSlack * seedLaw`: produces a transition kernel `μ_step` and expected step count `T_step` + preserving the seed-marginal bound, realizing `D.step s o`, modifying only `[base, base + space)`, + and satisfying the addition-only `ℝ≥0∞` amortized-in-expectation inequality. -/ + step_exec : ∀ {s : D.State} (o : D.Op) (ν : PMF (Seed × Data r_ds)), + D.valid s o → + (∀ θ, (ν.map Prod.fst) θ ≤ seedSlack * seedLaw θ) → + (∀ p ∈ ν.support, rep p.1 s p.2 ∧ argMatch o p.2.regs) → + ∃ (μ_step : Seed × Data r_ds → PMF (Seed × Data r_ds)) + (T_step : Seed × Data r_ds → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w P.code p.2 ((μ_step p).map Prod.snd) (T_step p) ∧ + ∀ q ∈ (μ_step p).support, + decode o q.2.regs = some (D.step s o).2 ∧ + rep q.1 (D.step s o).1 q.2 ∧ + Modifies (Block.mk base space).footprint p.2.mem q.2.mem) ∧ + (∀ θ', ((ν.bind μ_step).map Prod.fst) θ' ≤ seedSlack * seedLaw θ') ∧ + (∑' p, ν p * (T_step p + ∑' q, μ_step p q * (potential q.1 q.2 : ℝ≥0∞))) ≤ + (charge s o : ℝ≥0∞) + ∑' p, ν p * (potential p.1 p.2 : ℝ≥0∞) + +end Cslib.WordRAM.Randomized + +namespace Cslib.WordRAM.SubroutineDS + +open scoped ENNReal + +universe u v z + +variable {B : Basis} {r_ds k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-- Canonical embedding of any deterministic `Cslib.WordRAM.SubroutineDS` certificate into +`Cslib.WordRAM.Randomized.SubroutineDS` (`Seed := Unit`, `seedLaw := PMF.pure ()`, +`seedSlack := 1`). -/ +noncomputable def toRandomized (C : SubroutineDS P D w base params s₀) : + Randomized.SubroutineDS P D w base params s₀ where + Seed := Unit + seedLaw := PMF.pure () + seedSlack := 1 + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep _ s x := C.rep s x + potential _ x := C.potential x + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + rep_unique := C.rep_unique + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec hx₁ hb hfr hpar hz := by + obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz + refine ⟨PMF.pure ((), y), ?_, ?_, ?_⟩ + · simpa using ExecCodeDist.of_execWithin he + · intro θ + cases θ + simp + · intro p hp + rw [PMF.support_pure, Set.mem_singleton_iff] at hp + subst hp + exact ⟨hfr', hrep, hpot, hmod⟩ + step_exec {s} o ν hv _ hsupp := by + classical + let y : Unit × Data r_ds → Data r_ds := fun p => + if hp : C.rep s p.2 ∧ C.argMatch o p.2.regs then + Classical.choose (C.step_exec o hp.1 hv hp.2) + else p.2 + let n : Unit × Data r_ds → ℕ := fun p => + if hp : C.rep s p.2 ∧ C.argMatch o p.2.regs then + Classical.choose (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) + else 0 + refine ⟨fun p => PMF.pure ((), y p), fun p => (n p : ℝ≥0∞), ?_, ?_, ?_⟩ + · intro p hp_mem + have hp : C.rep s p.2 ∧ C.argMatch o p.2.regs := hsupp p hp_mem + have hspec := Classical.choose_spec (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) + rcases hspec with ⟨he, hdec, hrep', hmod, _⟩ + have hy_eq : y p = Classical.choose (C.step_exec o hp.1 hv hp.2) := by simp [y, hp] + have hn_eq : n p = + Classical.choose (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) := by simp [n, hp] + refine ⟨?_, ?_⟩ + · simpa [hy_eq, hn_eq] using ExecCodeDist.of_exec he + · intro q hq + rw [PMF.support_pure, Set.mem_singleton_iff] at hq + subst hq + simpa only [hy_eq] using ⟨hdec, hrep', hmod⟩ + · intro θ' + cases θ' + have hconst : ((ν.bind fun p => PMF.pure ((), y p)).map Prod.fst) () = 1 := by + rw [PMF.map_bind] + simp + simp [hconst] + · calc + (∑' p, ν p * ((n p : ℝ≥0∞) + ∑' q, PMF.pure ((), y p) q * (C.potential q.2 : ℝ≥0∞))) + ≤ ∑' p, ν p * ((C.charge s o : ℝ≥0∞) + (C.potential p.2 : ℝ≥0∞)) := by + refine ENNReal.tsum_le_tsum fun p => ?_ + by_cases hp_mem : p ∈ ν.support + · refine mul_le_mul_right ?_ _ + have hp : C.rep s p.2 ∧ C.argMatch o p.2.regs := hsupp p hp_mem + have hspec := + Classical.choose_spec (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) + rcases hspec with ⟨_, _, _, _, hpot⟩ + have hy_eq : y p = Classical.choose (C.step_exec o hp.1 hv hp.2) := by simp [y, hp] + have hn_eq : n p = + Classical.choose (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) := by + simp [n, hp] + rw [tsum_eq_single ((), y p) (fun q hq => by simp [PMF.pure_apply, hq])] + simp only [PMF.pure_apply_self, one_mul, hy_eq, hn_eq] + exact_mod_cast hpot + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp_mem, zero_mul, zero_mul] + _ = (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.2 : ℝ≥0∞) := by + simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + +end Cslib.WordRAM.SubroutineDS + +namespace Cslib.WordRAM.Randomized.SubroutineDS + +open Cslib.WordRAM +open scoped ENNReal + +universe u v z + +variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-- Total expected instruction bound for `P.initFresh`. -/ +def initTime (C : SubroutineDS P D w base params s₀) : ℝ≥0∞ := + (C.spanTime : ℝ≥0∞) + C.setupTime + +/-! ### Projection 1: Basis lifting (`mapSimulates`, `toStandard`) -/ + +/-- Lift a randomized `SubroutineDS` certificate along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) : + SubroutineDS (P.mapSimulates φ) D w base params s₀ where + Seed := C.Seed + seedLaw := C.seedLaw + seedSlack := C.seedSlack + seedSlack_ne_top := C.seedSlack_ne_top + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep := C.rep + potential := C.potential + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + rep_unique := C.rep_unique + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval h1 hfit := Op.eval_mapSimulates_of_some φ (C.incCur_eval h1 hfit) + span_exec hx hf hp := + let ⟨x₁, he, hm, hb, hfr, hpar⟩ := C.span_exec hx hf hp + ⟨x₁, he.mapSimulates φ, hm, hb, hfr, hpar⟩ + setup_exec hx₁ hb hfr hpar hz := + let ⟨ν₀, he, hseed, hsupp⟩ := C.setup_exec hx₁ hb hfr hpar hz + ⟨ν₀, he.mapSimulates φ, hseed, hsupp⟩ + step_exec o ν hv hseed hsupp := + let ⟨μ_step, T_step, hstep, hseed', hamort⟩ := C.step_exec o ν hv hseed hsupp + ⟨μ_step, T_step, + fun p hp => ⟨(hstep p hp).1.mapSimulates φ, (hstep p hp).2⟩, + hseed', hamort⟩ + +/-- Lift an exact-basis randomized `SubroutineDS` certificate to `Basis.standard`. -/ +abbrev toStandard {P : SubroutineCode .exact r_ds k_init} + (C : SubroutineDS P D w base params s₀) : + SubroutineDS P.toStandard D w base params s₀ := + C.mapSimulates .exactStandard + +/-! ### Projection 2: Fresh initialization and register relocation (`initFresh_exec`, +`initFresh_rename`) -/ + +/-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` expected +instructions. -/ +theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ ν₀ : PMF (C.Seed × Data r_ds), + ExecCodeDist w P.initFresh x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + C.space ∧ + C.rep p.1 s₀ p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + have hx₁ : x₁.Bounded w := he₁.bounded hx + have hz₁ : ∀ a ∈ (Block.mk base C.space).footprint, x₁.mem a = 0 := by + intro a ha + rw [hm₁] + exact hz a ha + obtain ⟨ν₀, he₂, hseed, hsupp⟩ := C.setup_exec hx₁ hb₁ hfr₁ hpar₁ hz₁ + rw [hm₁] at hsupp + exact ⟨ν₀, (ExecCodeDist.of_execWithin he₁).seq_pure he₂, hseed, hsupp⟩ + +/-- Relocated fresh initialization along any injective register embedding `f`. -/ +theorem initFresh_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ ν₀ : PMF (C.Seed × Data r_caller), + ExecCodeDist w (P.initFresh.rename f) x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs (f P.frontierReg) = base + C.space ∧ + C.rep p.1 s₀ (p.2.comap f) ∧ + C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by + obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initFresh_exec hx hfr hpar hz + refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ + · have he' := he.rename hf + refine he'.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro θ + rw [PMF.map_comp] + exact hseed θ + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨p, hp, rfl⟩ := hq + obtain ⟨hfr', hrep, hpot, hmod⟩ := hsupp p hp + refine ⟨?_, ?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem p.2 x hj⟩ + · change ((p.2.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`zeroFillCode_exec`, +`initClear_exec`, `initClear_rename`) -/ + +private def zeroFillSpec : Stateful.Spec.{0, 0, 0} where + State := PUnit + Op := PEmpty + Result o := PEmpty.elim o + step _ o := PEmpty.elim o + +private noncomputable def toZeroFillDet (C : SubroutineDS P D w base params s₀) : + Cslib.WordRAM.SubroutineDS + { P with initSetup := .op (.copy P.zeroReg P.zeroReg) } + zeroFillSpec w base params PUnit.unit where + space := C.space + spanTime := C.spanTime + setupTime := 1 + initPotential := 0 + charge _ o := PEmpty.elim o + rep _ x := x.Bounded w + potential _ := 0 + encode o := PEmpty.elim o + argMatch o := PEmpty.elim o + decode o := PEmpty.elim o + bounded h := h + rep_unique _ _ := rfl + arguments_fit o := PEmpty.elim o + encode_matches o := PEmpty.elim o + rep_supported _ _ hb _ := hb + potential_supported _ _ _ := rfl + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec {x₁} hx₁ _ hfr _ _ := by + refine ⟨x₁.set P.zeroReg (x₁.regs P.zeroReg), ExecWithin.copy, ?_, + hx₁.set (hx₁.1 P.zeroReg), Nat.le_refl 0, Modifies.refl _ _⟩ + simp [Data.set, hfr] + step_exec o := PEmpty.elim o + +/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` +instructions over any basis `B`. -/ +theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data r_ds} + (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) + (hfr₁ : x₁.regs P.frontierReg = base + C.space) + (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : + ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ + x₂.Bounded w ∧ + x₂.regs P.baseReg = base ∧ + x₂.regs P.frontierReg = base + C.space ∧ + (∀ j, x₂.regs (P.initParams j) = params j) ∧ + (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ + Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := + C.toZeroFillDet.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ + +/-- Execute `P.initClear` on **arbitrary dirty memory** in +`C.initTime + (4 * C.space + 6 : ℕ)` expected instructions. -/ +theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) : + ∃ ν₀ : PMF (C.Seed × Data r_ds), + ExecCodeDist w P.initClear x (ν₀.map Prod.snd) + (C.initTime + (4 * C.space + 6 : ℕ)) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + C.space ∧ + C.rep p.1 s₀ p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := + C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + obtain ⟨ν₀, he₃, hseed, hsupp⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ + rw [hm₁] at hmod₂ + have he_clear : ExecCodeDist w P.initClear x (ν₀.map Prod.snd) + ((C.spanTime : ℝ≥0∞) + ((4 * C.space + 6 : ℕ) + C.setupTime)) := + (ExecCodeDist.of_execWithin he₁).seq_pure ((ExecCodeDist.of_execWithin he₂).seq_pure he₃) + refine ⟨ν₀, he_clear.mono ?_, hseed, fun p hp => ?_⟩ + · unfold initTime + exact le_of_eq (by ac_rfl) + · obtain ⟨hfr₃, hrep, hpot, hmod₃⟩ := hsupp p hp + have hmod : Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by + simpa only [Finset.union_idempotent] using hmod₂.trans hmod₃ + exact ⟨hfr₃, hrep, hpot, hmod⟩ + +/-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ +theorem initClear_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : + ∃ ν₀ : PMF (C.Seed × Data r_caller), + ExecCodeDist w (P.initClear.rename f) x (ν₀.map Prod.snd) + (C.initTime + (4 * C.space + 6 : ℕ)) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs (f P.frontierReg) = base + C.space ∧ + C.rep p.1 s₀ (p.2.comap f) ∧ + C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by + obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initClear_exec hx hfr hpar + refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ + · have he' := he.rename hf + refine he'.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro θ + rw [PMF.map_comp] + exact hseed θ + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨p, hp, rfl⟩ := hq + obtain ⟨hfr', hrep, hpot, hmod⟩ := hsupp p hp + refine ⟨?_, ?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem p.2 x hj⟩ + · change ((p.2.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Helper lemmas for coupled PMF expectations -/ + +private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [ENNReal.tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +private theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : + (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by + simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] + exact ENNReal.tsum_comm + +/-! ### Projection 4: Caller-to-DS bidirectional framing (`frame_caller`, `frame_caller_dist`) -/ + +/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS +registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from +`[base, base + C.space)` preserves both `C.rep θ s (x'.comap f)` and +`C.potential θ (x'.comap f)`. -/ +theorem frame_caller (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} {θ : C.Seed} {s : D.State} {x x' : Data r_caller} + {F : Finset ℕ} + (hr : C.rep θ s (x.comap f)) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hmem : Modifies F x.mem x'.mem) : + C.rep θ s (x'.comap f) ∧ C.potential θ (x'.comap f) = C.potential θ (x.comap f) := by + have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => + hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) + exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ + +/-- Coupled caller distribution after a caller transition kernel `μ_caller` that does not resample +the DS seed `θ`. -/ +noncomputable def frameDist (C : SubroutineDS P D w base params s₀) + (ν : PMF (C.Seed × Data r_caller)) + (μ_caller : C.Seed × Data r_caller → PMF (Data r_caller)) : + PMF (C.Seed × Data r_caller) := + ν.bind fun p => (μ_caller p).map fun x' => (p.1, x') + +@[simp] +theorem frameDist_map_fst (C : SubroutineDS P D w base params s₀) + (ν : PMF (C.Seed × Data r_caller)) + (μ_caller : C.Seed × Data r_caller → PMF (Data r_caller)) : + (C.frameDist ν μ_caller).map Prod.fst = ν.map Prod.fst := by + unfold frameDist + rw [PMF.map_bind, PMF.map] + refine congrArg (ν.bind ·) (funext fun p => ?_) + ext θ + rw [PMF.map_comp] + simp + +/-- Distributional caller framing: any deterministic or randomized caller transition kernel +`μ_caller` (including caller steps that execute `Instr.rand`) that preserves +`(x'.comap f).Bounded w`, leaves persistent DS registers (`P.input i = false`) unchanged, and +writes only to a set `F` disjoint from `[base, base + C.space)` preserves the coupled seed +marginal bound, `C.rep`, and expected potential under `C.frameDist ν μ_caller`. -/ +theorem frame_caller_dist (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} {s : D.State} + {ν : PMF (C.Seed × Data r_caller)} + {μ_caller : C.Seed × Data r_caller → PMF (Data r_caller)} + {F : Finset ℕ} + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hrep : ∀ p ∈ ν.support, C.rep p.1 s (p.2.comap f)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hframe : ∀ p ∈ ν.support, ∀ x' ∈ (μ_caller p).support, + (x'.comap f).Bounded w ∧ + (∀ i, P.input i = false → x'.regs (f i) = p.2.regs (f i)) ∧ + Modifies F p.2.mem x'.mem) : + (∀ θ, ((C.frameDist ν μ_caller).map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ q ∈ (C.frameDist ν μ_caller).support, C.rep q.1 s (q.2.comap f)) ∧ + (∑' q, C.frameDist ν μ_caller q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + refine ⟨fun θ => by rw [C.frameDist_map_fst]; exact hseed θ, ?_, ?_⟩ + · intro q hq + rw [frameDist, PMF.mem_support_bind_iff] at hq + obtain ⟨p, hp, hq_map⟩ := hq + rw [PMF.mem_support_map_iff] at hq_map + obtain ⟨x', hx', rfl⟩ := hq_map + obtain ⟨hb', hreg, hmem⟩ := hframe p hp x' hx' + exact (C.frame_caller (hrep p hp) hb' hreg hdisj hmem).1 + · rw [frameDist, tsum_pmf_bind] + refine tsum_congr fun p => ?_ + by_cases hp : p ∈ ν.support + · rw [tsum_pmf_map] + have hpot_eq : (∑' x', μ_caller p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = + ∑' x', μ_caller p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + refine tsum_congr fun x' => ?_ + by_cases hx' : x' ∈ (μ_caller p).support + · obtain ⟨hb', hreg, hmem⟩ := hframe p hp x' hx' + rw [(C.frame_caller (hrep p hp) hb' hreg hdisj hmem).2] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hx', zero_mul, zero_mul] + rw [hpot_eq, ENNReal.tsum_mul_right, (μ_caller p).tsum_coe, one_mul] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + +/-! ### Projection 5: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ + +/-- Execute `P.code.rename f` inline when arguments already match `(p.2.comap f).regs` on the +support of a coupled caller distribution `ν : PMF (C.Seed × Data r_caller)`. -/ +theorem code_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} (o : D.Op) (ν : PMF (C.Seed × Data r_caller)) + (hv : D.valid s o) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hsupp : ∀ p ∈ ν.support, C.rep p.1 s (p.2.comap f) ∧ C.argMatch o (p.2.comap f).regs) : + ∃ (μ_caller : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller)) + (T_caller : C.Seed × Data r_caller → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (P.code.rename f) p.2 ((μ_caller p).map Prod.snd) (T_caller p) ∧ + ∀ q ∈ (μ_caller p).support, + C.decode o (q.2.comap f).regs = some (D.step s o).2 ∧ + C.rep q.1 (D.step s o).1 (q.2.comap f) ∧ + Modifies (Block.mk base C.space).footprint p.2.mem q.2.mem ∧ + (∀ j ∉ Set.range f, q.2.regs j = p.2.regs j)) ∧ + (∀ θ', ((ν.bind μ_caller).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ + (∑' p, ν p * (T_caller p + ∑' q, μ_caller p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) ≤ + (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + let proj : C.Seed × Data r_caller → C.Seed × Data r_ds := fun p => (p.1, p.2.comap f) + let ν_ds : PMF (C.Seed × Data r_ds) := ν.map proj + have hseed_ds : ∀ θ, (ν_ds.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + intro θ + rw [PMF.map_comp] + exact hseed θ + have hsupp_ds : ∀ p_ds ∈ ν_ds.support, C.rep p_ds.1 s p_ds.2 ∧ C.argMatch o p_ds.2.regs := by + intro p_ds hp_ds + rw [PMF.mem_support_map_iff] at hp_ds + obtain ⟨p, hp, rfl⟩ := hp_ds + exact hsupp p hp + obtain ⟨μ_ds, T_ds, hstep_ds, hseed_ds', hamort_ds⟩ := + C.step_exec o ν_ds hv hseed_ds hsupp_ds + let μ_caller : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller) := + fun p => (μ_ds (proj p)).map fun q => (q.1, q.2.extend f p.2) + let T_caller : C.Seed × Data r_caller → ℝ≥0∞ := fun p => T_ds (proj p) + refine ⟨μ_caller, T_caller, ?_, ?_, ?_⟩ + · intro p hp + have hp_ds : proj p ∈ ν_ds.support := (PMF.mem_support_map_iff _ _ _).mpr ⟨p, hp, rfl⟩ + obtain ⟨he_ds, hq_ds⟩ := hstep_ds (proj p) hp_ds + refine ⟨?_, ?_⟩ + · have he_ren := he_ds.rename hf (x := p.2) + refine he_ren.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨q_ds, hq_mem, rfl⟩ := hq + obtain ⟨hdec, hrep', hmod⟩ := hq_ds q_ds hq_mem + refine ⟨?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem q_ds.2 p.2 hj⟩ + · rw [Data.comap_extend hf] + exact hdec + · rw [Data.comap_extend hf] + exact hrep' + · intro θ' + have hmap_eq : (ν.bind μ_caller).map Prod.fst = (ν_ds.bind μ_ds).map Prod.fst := by + rw [PMF.map_bind, PMF.map_bind, PMF.bind_map] + refine congrArg (ν.bind ·) (funext fun p => ?_) + rw [PMF.map_comp] + rfl + rw [hmap_eq] + exact hseed_ds' θ' + · have hPot_inner : ∀ p : C.Seed × Data r_caller, + (∑' q, μ_caller p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞) := by + intro p + rw [tsum_pmf_map] + refine tsum_congr fun q_ds => ?_ + rw [Data.comap_extend hf] + simp_rw [hPot_inner] + have hLHS : + (∑' p, ν p * (T_caller p + + ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞))) = + ∑' p_ds, ν_ds p_ds * (T_ds p_ds + + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)) := + (tsum_pmf_map ν proj fun p_ds => + T_ds p_ds + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)).symm + have hRHS : + (∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞)) = + ∑' p_ds, ν_ds p_ds * (C.potential p_ds.1 p_ds.2 : ℝ≥0∞) := + (tsum_pmf_map ν proj fun p_ds => (C.potential p_ds.1 p_ds.2 : ℝ≥0∞)).symm + rw [hLHS, hRHS] + exact hamort_ds + +/-- Execute `.seq prep (P.code.rename f)` inline after a (deterministic or randomized) caller +argument-preparation block `prep` that preserves persistent DS registers and memory in +`[base, base + C.space)`. -/ +theorem call_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} {prep : Code B r_caller} {B_prep : ℝ≥0∞} + (o : D.Op) (ν : PMF (C.Seed × Data r_caller)) + {μ_prep : C.Seed × Data r_caller → PMF (Data r_caller)} + {T_prep : C.Seed × Data r_caller → ℝ≥0∞} + (hv : D.valid s o) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hrep : ∀ p ∈ ν.support, C.rep p.1 s (p.2.comap f)) + (hprep : ∀ p ∈ ν.support, + ExecCodeDist w prep p.2 (μ_prep p) (T_prep p) ∧ + T_prep p ≤ B_prep ∧ + ∀ x' ∈ (μ_prep p).support, + (x'.comap f).Bounded w ∧ + (∀ i, P.input i = false → x'.regs (f i) = p.2.regs (f i)) ∧ + (∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = p.2.mem a) ∧ + C.argMatch o (x'.comap f).regs) : + ∃ (μ_call : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller)) + (T_call : C.Seed × Data r_caller → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (.seq prep (P.code.rename f)) p.2 ((μ_call p).map Prod.snd) (T_call p) ∧ + ∀ q ∈ (μ_call p).support, + C.decode o (q.2.comap f).regs = some (D.step s o).2 ∧ + C.rep q.1 (D.step s o).1 (q.2.comap f)) ∧ + (∀ θ', ((ν.bind μ_call).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ + (∑' p, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) ≤ + B_prep + (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + let ν_prep := C.frameDist ν μ_prep + have hseed_prep : ∀ θ, (ν_prep.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + intro θ + rw [C.frameDist_map_fst] + exact hseed θ + have hsupp_prep : ∀ p' ∈ ν_prep.support, + C.rep p'.1 s (p'.2.comap f) ∧ C.argMatch o (p'.2.comap f).regs := by + intro p' hp' + simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] at hp' + obtain ⟨p, hp, hp_map⟩ := hp' + rw [PMF.mem_support_map_iff] at hp_map + obtain ⟨x', hx', rfl⟩ := hp_map + obtain ⟨_, _, hfr_x'⟩ := hprep p hp + obtain ⟨hb', hreg, hmem, harg⟩ := hfr_x' x' hx' + exact ⟨C.rep_supported (x := p.2.comap f) (x' := x'.comap f) hreg hmem hb' (hrep p hp), harg⟩ + obtain ⟨μ_code, T_code, hcode, hseed_code, hamort_code⟩ := + C.code_rename hf o ν_prep hv hseed_prep hsupp_prep + let μ_call : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller) := + fun p => (μ_prep p).bind fun x' => μ_code (p.1, x') + let T_call : C.Seed × Data r_caller → ℝ≥0∞ := + fun p => T_prep p + ∑' x', μ_prep p x' * T_code (p.1, x') + refine ⟨μ_call, T_call, ?_, ?_, ?_⟩ + · intro p hp + obtain ⟨he_prep, _, hfr_x'⟩ := hprep p hp + have he_seq := he_prep.seq + (μ₂ := fun x' => (μ_code (p.1, x')).map Prod.snd) + (T₂ := fun x' => T_code (p.1, x')) + fun x' hx' => by + have hp' : (p.1, x') ∈ ν_prep.support := by + simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] + exact ⟨p, hp, (PMF.mem_support_map_iff _ _ _).mpr ⟨x', hx', rfl⟩⟩ + exact (hcode (p.1, x') hp').1 + refine ⟨he_seq.congr_dist ?_ le_rfl, ?_⟩ + · simp only [μ_call, PMF.map_bind] + · intro q hq + rw [PMF.mem_support_bind_iff] at hq + obtain ⟨x', hx', hq_code⟩ := hq + have hp' : (p.1, x') ∈ ν_prep.support := by + simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] + exact ⟨p, hp, (PMF.mem_support_map_iff _ _ _).mpr ⟨x', hx', rfl⟩⟩ + obtain ⟨hdec, hrep', _, _⟩ := (hcode (p.1, x') hp').2 q hq_code + exact ⟨hdec, hrep'⟩ + · intro θ' + have hbind_eq : ν.bind μ_call = ν_prep.bind μ_code := by + simp only [ν_prep, frameDist, μ_call, PMF.bind_bind, PMF.bind_map] + rfl + rw [hbind_eq] + exact hseed_code θ' + · have hpot_prep : + (∑' p', ν_prep p' * (C.potential p'.1 (p'.2.comap f) : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + simp only [ν_prep, frameDist, tsum_pmf_bind] + refine tsum_congr fun p => ?_ + by_cases hp : p ∈ ν.support + · rw [tsum_pmf_map] + have hpot_eq : (∑' x', μ_prep p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = + ∑' x', μ_prep p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + refine tsum_congr fun x' => ?_ + by_cases hx' : x' ∈ (μ_prep p).support + · obtain ⟨_, _, hfr_x'⟩ := hprep p hp + obtain ⟨_, hreg, hmem, _⟩ := hfr_x' x' hx' + rw [C.potential_supported (x := p.2.comap f) (x' := x'.comap f) hreg hmem (hrep p hp)] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hx', zero_mul, zero_mul] + rw [hpot_eq, ENNReal.tsum_mul_right, (μ_prep p).tsum_coe, one_mul] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + have hdecomp : + (∑' p, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) = + (∑' p, ν p * T_prep p) + + ∑' p', ν_prep p' * + (T_code p' + ∑' q, μ_code p' q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by + have hinner : ∀ p : C.Seed × Data r_caller, + T_call p + (∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + T_prep p + ∑' x', μ_prep p x' * + (T_code (p.1, x') + + ∑' q, μ_code (p.1, x') q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by + intro p + simp only [T_call, μ_call, tsum_pmf_bind, add_assoc] + congr 1 + rw [← ENNReal.tsum_add] + simp_rw [mul_add] + have hmul : ∀ p : C.Seed × Data r_caller, + ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + ν p * T_prep p + ν p * ∑' x', μ_prep p x' * + (T_code (p.1, x') + + ∑' q, μ_code (p.1, x') q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by + intro p + rw [hinner p, mul_add] + simp_rw [hmul] + rw [ENNReal.tsum_add] + congr 1 + symm + dsimp only [ν_prep, frameDist] + rw [tsum_pmf_bind] + refine tsum_congr fun p => ?_ + rw [tsum_pmf_map] + have hprep_le : (∑' p, ν p * T_prep p) ≤ B_prep := by + calc + (∑' p, ν p * T_prep p) + ≤ ∑' p, ν p * B_prep := by + refine ENNReal.tsum_le_tsum fun p => ?_ + by_cases hp : p ∈ ν.support + · exact mul_le_mul_right (hprep p hp).2.1 _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + _ = B_prep := by rw [ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + rw [hdecomp] + calc + (∑' p, ν p * T_prep p) + + ∑' p', ν_prep p' * + (T_code p' + ∑' q, μ_code p' q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) + ≤ B_prep + ((C.charge s o : ℝ≥0∞) + + ∑' p', ν_prep p' * (C.potential p'.1 (p'.2.comap f) : ℝ≥0∞)) := + add_le_add hprep_le hamort_code + _ = B_prep + (C.charge s o : ℝ≥0∞) + + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + rw [hpot_prep, add_assoc] + +/-! ### Projection 6: End-to-end expected-amortized sequence telescope (`steps_expected_amortized`, +`seq_expected_amortized`) -/ + +/-- Deliver standalone operation arguments for `o` without changing memory or persistent +registers (`P.input i = false`). -/ +def prepare (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : Data r_ds where + regs i := if P.input i then C.encode o i else x.regs i + mem := x.mem + +@[simp] +theorem prepare_mem (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : + (C.prepare o x).mem = x.mem := rfl + +theorem prepare_regs_of_not_input (C : SubroutineDS P D w base params s₀) + {i : Fin r_ds} (hi : P.input i = false) (o : D.Op) (x : Data r_ds) : + (C.prepare o x).regs i = x.regs i := by + simp [prepare, hi] + +theorem prepare_bounded (C : SubroutineDS P D w base params s₀) + {s : D.State} {o : D.Op} {x : Data r_ds} (hx : x.Bounded w) (hv : D.valid s o) : + (C.prepare o x).Bounded w := by + refine ⟨fun i => ?_, hx.2⟩ + change Fits w (if P.input i then C.encode o i else x.regs i) + split + · exact C.arguments_fit o hv i (by assumption) + · exact hx.1 i + +theorem prepare_rep (C : SubroutineDS P D w base params s₀) + {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} + (hr : C.rep θ s x) (hv : D.valid s o) : + C.rep θ s (C.prepare o x) := + C.rep_supported + (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) + (C.prepare_bounded (C.bounded hr) hv) + hr + +theorem prepare_potential (C : SubroutineDS P D w base params s₀) + {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} + (hr : C.rep θ s x) : + C.potential θ (C.prepare o x) = C.potential θ x := + C.potential_supported + (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) + hr + +theorem prepare_argMatch (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : + C.argMatch o (C.prepare o x).regs := + C.encode_matches o x.regs + +/-- **Multi-operation expected-amortized telescope (`steps_expected_amortized`).** +From any abstract state `s : D.State` and coupled state distribution `ν : PMF (C.Seed × Data r_ds)` +satisfying the seed-marginal bound and `C.rep p.1 s p.2` on `ν.support`, executing any admissible +operation sequence `ops : List D.Op` (`D.ValidHistory s ops`) produces a final coupled distribution +`ν_final` and total expected operation step count `T_ops : ℝ≥0∞` satisfying the addition-only +`ℝ≥0∞` telescope inequality +`T_ops + E_{ν_final}[Φ] ≤ D.totalCost C.charge s ops + E_ν[Φ]`. -/ +theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) + {s : D.State} (ops : List D.Op) (ν : PMF (C.Seed × Data r_ds)) + (hv : D.ValidHistory s ops) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hrep : ∀ p ∈ ν.support, C.rep p.1 s p.2) : + ∃ (ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), + (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ p ∈ ν_final.support, C.rep p.1 (D.run s ops).1 p.2) ∧ + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ + (D.totalCost C.charge s ops : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by + induction ops generalizing s ν with + | nil => + exact ⟨ν, 0, hseed, hrep, by simp⟩ + | cons o os ih => + obtain ⟨hvo, hvos⟩ := hv + let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.prepare o p.2) + have hseed_prep : ∀ θ, (ν_prep.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + intro θ + rw [PMF.map_comp] + exact hseed θ + have hsupp_prep : ∀ p' ∈ ν_prep.support, C.rep p'.1 s p'.2 ∧ C.argMatch o p'.2.regs := by + intro p' hp' + rw [PMF.mem_support_map_iff] at hp' + obtain ⟨p, hp, rfl⟩ := hp' + exact ⟨C.prepare_rep (hrep p hp) hvo, C.prepare_argMatch o p.2⟩ + have hpot_prep : + (∑' p', ν_prep p' * (C.potential p'.1 p'.2 : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by + rw [tsum_pmf_map] + refine tsum_congr fun p => ?_ + by_cases hp : p ∈ ν.support + · rw [C.prepare_potential (hrep p hp)] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + obtain ⟨μ_step, T_step, hstep, hseed₁, hamort₁⟩ := + C.step_exec o ν_prep hvo hseed_prep hsupp_prep + let ν₁ := ν_prep.bind μ_step + have hrep₁ : ∀ q ∈ ν₁.support, C.rep q.1 (D.step s o).1 q.2 := by + intro q hq + rw [PMF.mem_support_bind_iff] at hq + obtain ⟨p', hp', hq_step⟩ := hq + exact ((hstep p' hp').2 q hq_step).2.1 + obtain ⟨ν_final, T_rest, hseed_final, hrep_final, hamort_rest⟩ := + ih ν₁ hvos hseed₁ hrep₁ + let T₁ : ℝ≥0∞ := ∑' p', ν_prep p' * T_step p' + have hstep_split : + T₁ + (∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) = + ∑' p', ν_prep p' * (T_step p' + ∑' q, μ_step p' q * (C.potential q.1 q.2 : ℝ≥0∞)) := by + simp only [T₁, ν₁, tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] + refine ⟨ν_final, T₁ + T_rest, hseed_final, hrep_final, ?_⟩ + calc + (T₁ + T_rest) + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) + = T₁ + (T_rest + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl + _ ≤ T₁ + ((D.totalCost C.charge (D.step s o).1 os : ℝ≥0∞) + + ∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) := + add_le_add le_rfl hamort_rest + _ = (D.totalCost C.charge (D.step s o).1 os : ℝ≥0∞) + + (T₁ + ∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl + _ ≤ (D.totalCost C.charge (D.step s o).1 os : ℝ≥0∞) + + ((C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞)) := by + rw [hstep_split, ← hpot_prep] + exact add_le_add le_rfl hamort₁ + _ = (D.totalCost C.charge s (o :: os) : ℝ≥0∞) + + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by + simp only [Stateful.Spec.totalCost_cons, Nat.cast_add] + ac_rfl + +/-- **End-to-end expected-amortized theorem from fresh initialization (`seq_expected_amortized`).** +Starting from zeroed `[base, base + C.space)`, `P.initFresh` followed by any admissible operation +sequence `ops : List D.Op` (`D.ValidHistory s₀ ops`) has total expected instruction cost +`C.initTime + T_ops` bounded by +`C.initTime + C.initPotential + D.totalCost C.charge s₀ ops`. -/ +theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) + {x : Data r_ds} (ops : List D.Op) + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) + (hv : D.ValidHistory s₀ ops) : + ∃ (ν₀ ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), + ExecCodeDist w P.initFresh x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ p ∈ ν_final.support, C.rep p.1 (D.run s₀ ops).1 p.2) ∧ + C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ + C.initTime + (C.initPotential : ℝ≥0∞) + (D.totalCost C.charge s₀ ops : ℝ≥0∞) := by + obtain ⟨ν₀, he_init, hseed₀, hsupp₀⟩ := C.initFresh_exec hx hfr hpar hz + have hrep₀ : ∀ p ∈ ν₀.support, C.rep p.1 s₀ p.2 := fun p hp => (hsupp₀ p hp).2.1 + obtain ⟨ν_final, T_ops, hseed_final, hrep_final, hamort⟩ := + C.steps_expected_amortized ops ν₀ hv hseed₀ hrep₀ + have hpot₀ : (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) ≤ (C.initPotential : ℝ≥0∞) := by + calc + (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) + ≤ ∑' p, ν₀ p * (C.initPotential : ℝ≥0∞) := by + refine ENNReal.tsum_le_tsum fun p => ?_ + by_cases hp : p ∈ ν₀.support + · exact mul_le_mul_right (Nat.cast_le.mpr (hsupp₀ p hp).2.2.1) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + _ = (C.initPotential : ℝ≥0∞) := by + rw [ENNReal.tsum_mul_right, ν₀.tsum_coe, one_mul] + refine ⟨ν₀, ν_final, T_ops, he_init, hseed_final, hrep_final, ?_⟩ + calc + C.initTime + T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) + = C.initTime + (T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl + _ ≤ C.initTime + ((D.totalCost C.charge s₀ ops : ℝ≥0∞) + + ∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) := + add_le_add le_rfl hamort + _ ≤ C.initTime + ((D.totalCost C.charge s₀ ops : ℝ≥0∞) + (C.initPotential : ℝ≥0∞)) := + add_le_add le_rfl (add_le_add le_rfl hpot₀) + _ = C.initTime + (C.initPotential : ℝ≥0∞) + (D.totalCost C.charge s₀ ops : ℝ≥0∞) := by ac_rfl + +end Cslib.WordRAM.Randomized.SubroutineDS + +end diff --git a/ComplexityTests/WordRAMRandomizedSubroutine.lean b/ComplexityTests/WordRAMRandomizedSubroutine.lean new file mode 100644 index 000000000..d11f4056d --- /dev/null +++ b/ComplexityTests/WordRAMRandomizedSubroutine.lean @@ -0,0 +1,350 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Computability.WordRAM.Randomized.Expected +public import Complexity.Computability.WordRAM.Randomized.Subroutine + +/-! +# Randomized Word-RAM geometric loop and subroutine interface tests + +Verifies the distributional structured execution relation `ExecCodeDist`, the geometric Las Vegas +loop expected-time bound `ExecCodeDist.loop_geometric`, and the randomized Word-RAM subroutine +interface `Cslib.WordRAM.Randomized.SubroutineDS`: +* a concrete geometric coin-flipping Las Vegas loop (`while r0 == r1 do r0 ← rand r1` with + `r1 = 1` and exit probability `1 / 2` per iteration) almost surely halts with `r0 = 0` and + bounded expected step count; +* a concrete randomized data structure executing `Instr.rand` during its operations instantiates + `Cslib.WordRAM.Randomized.SubroutineDS` and exercises `toStandard`, `initFresh_exec`, + `initClear_exec`, `frame_caller_dist` (across a randomized caller step), `call_rename`, and + `seq_expected_amortized`. +-/ + +@[expose] public section + +namespace ComplexityTests.WordRAMRandomizedSubroutine + +open Cslib Cslib.WordRAM Cslib.WordRAM.Randomized +open Cslib.Probability.PMF (uniformOfFintype uniformOfFintype_apply) +open scoped ENNReal + +/-! ## 1. Geometric coin-flipping Las Vegas loop (`ExecCodeDist.loop_geometric`) -/ + +abbrev g0 : Fin 2 := 0 +abbrev g1 : Fin 2 := 1 + +/-- Geometric Las Vegas loop: while `r0 == r1` (with `r1 = 1`), sample `r0 ← rand r1` uniformly +from `{0, 1}`. Each iteration exits with probability `1 / 2` in `1` body instruction. -/ +def geomCoinLoop : Code .exact 2 := + .loop (.eq g0 g1) (.rand g0 g1) + +/-- Loop-head invariant: `x` is `w`-bounded and `r1 = 1`. -/ +def geomInv (w : ℕ) (x : Data 2) : Prop := + x.Bounded w ∧ x.regs g1 = 1 + +/-- Loop-exit postcondition: `x` is `w`-bounded, `r0 = 0`, and `r1 = 1`. -/ +def geomPost (w : ℕ) (x : Data 2) : Prop := + x.Bounded w ∧ x.regs g0 = 0 ∧ x.regs g1 = 1 + +/-- Single-iteration transition distribution of `r0 ← rand r1` when `r1 = 1`. -/ +noncomputable def geomBodyDist (x : Data 2) : PMF (Data 2) := + (uniformOfFintype (Fin (x.regs g1 + 1))).map fun k => x.set g0 k.val + +private theorem fits_one_of_pos {w : ℕ} (hw : 1 ≤ w) : Fits w 1 := + fits_one (by omega) + +theorem geomCoinLoop_execCodeDist {w : ℕ} (hw : 1 ≤ w) (x₀ : Data 2) + (hx₀ : geomInv w x₀ ∧ (Cond.eq g0 g1).Holds x₀) : + ∃ μ_exit : PMF (Data 2), + ExecCodeDist w geomCoinLoop x₀ μ_exit (((1 : ℝ≥0∞) + 2) / (2 : ℝ≥0∞)⁻¹ + 2) ∧ + ∀ y ∈ μ_exit.support, ¬(Cond.eq g0 g1).Holds y ∧ geomPost w y := by + have hp_pos : (0 : ℝ≥0∞) < (2 : ℝ≥0∞)⁻¹ := by simp + have hp_top : ((2 : ℝ≥0∞)⁻¹) ≠ ⊤ := by simp + have hbody : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → + ExecCodeDist (B := .exact) w (.rand g0 g1) z (geomBodyDist z) 1 := by + intro z ⟨hbz, _⟩ _ + simpa [geomBodyDist, ProbComp.eval_unifFin] using ExecCodeDist.rand (B := .exact) (hbz.1 g1) + have hB : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → (1 : ℝ≥0∞) ≤ 1 := + fun _ _ _ => le_rfl + have hinv : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → + ∀ y ∈ (geomBodyDist z).support, LoopStateInv (.eq g0 g1) (geomInv w) (geomPost w) y := by + intro x ⟨hbx, hr1⟩ _ y hy + rw [geomBodyDist, PMF.mem_support_map_iff] at hy + obtain ⟨k, _, rfl⟩ := hy + have hk_lt : k.val < 2 := by omega + have hby : (x.set g0 k.val).Bounded w := by + refine hbx.set ?_ + rcases Nat.lt_succ_iff_lt_or_eq.mp hk_lt with hk0 | hk1 + · have hk0' : k.val = 0 := Nat.lt_one_iff.mp hk0 + rw [hk0'] + exact fits_zero w + · rw [hk1] + exact fits_one_of_pos hw + have hy_r1 : (x.set g0 k.val).regs g1 = 1 := by + simp [Data.set, hr1] + rcases Nat.lt_succ_iff_lt_or_eq.mp hk_lt with hk0 | hk1 + · have hk0' : k.val = 0 := Nat.lt_one_iff.mp hk0 + right + refine ⟨?_, hby, ?_, hy_r1⟩ + · simp [Cond.Holds, Data.set, hr1, hk0'] + · simp [Data.set, hk0'] + · left + refine ⟨?_, hby, hy_r1⟩ + simp [Cond.Holds, Data.set, hr1, hk1] + have hp_exit : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → + (2 : ℝ≥0∞)⁻¹ ≤ (geomBodyDist z).toOuterMeasure {y | ¬(Cond.eq g0 g1).Holds y} := by + intro x ⟨_, hr1⟩ _ + classical + rw [PMF.toOuterMeasure_apply] + have h_zero_lt : 0 < x.regs g1 + 1 := by omega + let k₀ : Fin (x.regs g1 + 1) := ⟨0, h_zero_lt⟩ + let y₀ : Data 2 := x.set g0 0 + have hy₀_not : ¬(Cond.eq g0 g1).Holds y₀ := by + simp [y₀, Cond.Holds, Data.set, hr1] + have h1 : (2 : ℝ≥0∞)⁻¹ ≤ geomBodyDist x y₀ := by + rw [geomBodyDist, PMF.map_apply] + have hle := ENNReal.le_tsum + (f := fun k : Fin (x.regs g1 + 1) => + if y₀ = x.set g0 k.val then uniformOfFintype (Fin (x.regs g1 + 1)) k else 0) k₀ + simpa [y₀, k₀, uniformOfFintype_apply, hr1] using hle + have h2 : geomBodyDist x y₀ ≤ + ∑' y, Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y := by + calc + geomBodyDist x y₀ + = Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y₀ := by + rw [Set.indicator_of_mem hy₀_not] + _ ≤ ∑' y, Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y := + ENNReal.le_tsum y₀ + exact le_trans h1 h2 + have hx₀_inv : LoopStateInv (.eq g0 g1) (geomInv w) (geomPost w) x₀ := Or.inl ⟨hx₀.2, hx₀.1⟩ + refine ⟨loopExitPMF (.eq g0 g1) geomBodyDist x₀, + ExecCodeDist.loop_geometric (geomInv w) (geomPost w) hp_pos hp_top hbody hB hinv hp_exit + hx₀_inv, + fun y hy => + (mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx₀_inv hy).1⟩ + +/-- Complete flat program for the geometric coin-flipping loop exercises `program_halts`: +it halts almost surely from `r0 = 1, r1 = 1` with finite expected step count. -/ +theorem geomCoinLoop_halts {w : ℕ} (hw : 1 ≤ w) : + let x₀ : Data 2 := ((Data.init [] : Data 2).set g0 1).set g1 1 + AlmostSurelyHalts w (geomCoinLoop.program g0) ⟨0, x₀⟩ ∧ + expectedTime w (geomCoinLoop.program g0) ⟨0, x₀⟩ ≤ + ((1 : ℝ≥0∞) + 2) / (2 : ℝ≥0∞)⁻¹ + 2 + 1 := by + intro x₀ + have hx₀_inv : geomInv w x₀ := by + refine ⟨?_, by simp [x₀, Data.set]⟩ + exact ((Data.Bounded.init (by simp [InputFits])).set + (fits_one_of_pos hw)).set (fits_one_of_pos hw) + have hx₀_cond : (Cond.eq g0 g1).Holds x₀ := by + simp [x₀, Cond.Holds, Data.set] + obtain ⟨μ_exit, hex, _⟩ := geomCoinLoop_execCodeDist hw x₀ ⟨hx₀_inv, hx₀_cond⟩ + obtain ⟨hExp, _, hAS, _⟩ := hex.program_halts g0 + have h12_ne : (1 : ℝ≥0∞) + 2 ≠ ⊤ := + ENNReal.add_ne_top.mpr ⟨ENNReal.one_ne_top, ENNReal.ofNat_ne_top⟩ + have hT_ne : ((1 : ℝ≥0∞) + 2) / (2 : ℝ≥0∞)⁻¹ + 2 ≠ ⊤ := + ENNReal.add_ne_top.mpr ⟨ENNReal.div_ne_top h12_ne (by simp), ENNReal.ofNat_ne_top⟩ + exact ⟨hAS hT_ne, hExp⟩ + +/-! ## 2. Concrete randomized data structure (`Cslib.WordRAM.Randomized.SubroutineDS`) -/ + +/-- Trivial stateful specification for a randomized sampler subroutine whose implementation invokes +`Instr.rand` during each operation. -/ +def coinSpec : Stateful.Spec.{0, 0, 0} where + State := PUnit + Op := PUnit + Result _ := PUnit + step _ _ := (PUnit.unit, PUnit.unit) + +/-- Subroutine code with 5 registers (`0` = `baseReg`, `1` = `frontierReg`, `2` = `zeroReg`, +`3` = `curReg` / scratch output, `4` = `oneReg`). Its operation body `code` executes +`r3 ← rand r2` (`Instr.rand`). -/ +def coinSubroutineCode : SubroutineCode .exact 5 0 where + allocSpan := .op (.copy 0 1) + initSetup := .op (.const 2 0) + code := .rand 3 2 + baseReg := 0 + frontierReg := 1 + zeroReg := 2 + oneReg := 4 + curReg := 3 + incCur := Op.exactBin .add 3 3 4 + initParams := Fin.elim0 + input i := i == 3 + output := 3 + +/-- Representation invariant for `coinSubroutineCode`: machine words are `w`-bounded, `baseReg` +holds `base`, and `zeroReg` holds `0`. -/ +def coinRep (w base : ℕ) (_θ : Fin 2) (_s : PUnit) (x : Data 5) : Prop := + x.Bounded w ∧ x.regs 0 = base ∧ x.regs 2 = 0 + +private theorem pmf_map_fst_const {α β γ : Type*} (q : PMF α) (b : β) (g : α → γ) : + (q.map fun a => (b, g a)).map Prod.fst = PMF.pure b := by + ext θ + classical + rw [PMF.map_comp] + by_cases h : θ = b <;> simp [PMF.map_apply, h, q.tsum_coe] + +private theorem pmf_map_snd_const {α β γ : Type*} (q : PMF α) (f : α → β) (c : γ) : + (q.map fun a => (f a, c)).map Prod.snd = PMF.pure c := by + ext y + classical + rw [PMF.map_comp] + by_cases h : y = c <;> simp [PMF.map_apply, h, q.tsum_coe] + +/-- Concrete randomized `SubroutineDS` certificate for `coinSubroutineCode` with latent seed type +`Fin 2` and uniform reference distribution `uniformOfFintype (Fin 2)`. -/ +noncomputable def coinSubroutineDS (w base : ℕ) : + Cslib.WordRAM.Randomized.SubroutineDS + coinSubroutineCode coinSpec w base Fin.elim0 PUnit.unit where + Seed := Fin 2 + seedLaw := uniformOfFintype (Fin 2) + seedSlack := 1 + seedSlack_ne_top := ENNReal.one_ne_top + space := 0 + spanTime := 1 + setupTime := 1 + initPotential := 0 + charge _ _ := 1 + rep := coinRep w base + potential _ _ := 0 + encode _ _ := 0 + argMatch _ _ := True + decode _ _ := some PUnit.unit + bounded h := h.1 + rep_unique _ _ := rfl + arguments_fit _ _ _ _ := fits_zero w + encode_matches _ _ := trivial + rep_supported hreg _ hb' hrep := + ⟨hb', + by rw [hreg 0 rfl, hrep.2.1], + by rw [hreg 2 rfl, hrep.2.2]⟩ + potential_supported _ _ _ := rfl + clear_nodup := by decide + params_not_clear j := Fin.elim0 j + incCur_eval {x} h1 hfit := by + change Data.put w x 3 (x.regs 3 + x.regs 4) = + some (x.set coinSubroutineCode.curReg (x.regs coinSubroutineCode.curReg + 1)) + change x.regs 4 = 1 at h1 + rw [h1] + exact Data.put_of_fits hfit + span_exec hx hfr _ := by + refine ⟨_, ExecWithin.copy, rfl, ?_, ?_, fun j => Fin.elim0 j⟩ + · simpa [coinSubroutineCode, Data.set] using hfr + · simpa [coinSubroutineCode, Data.set] using hfr + setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by + let x₂ : Data 5 := x₁.set 2 0 + let ν₀ : PMF (Fin 2 × Data 5) := (uniformOfFintype (Fin 2)).map fun θ => (θ, x₂) + refine ⟨ν₀, ?_, ?_, ?_⟩ + · have hex : ExecCodeDist (B := .exact) w (.op (.const 2 0)) x₁ (PMF.pure x₂) (1 : ℕ) := + ExecCodeDist.of_exec (Exec.const (fits_zero w)) + exact hex.congr_dist (pmf_map_snd_const (uniformOfFintype (Fin 2)) id x₂).symm (by simp) + · intro θ + rw [PMF.map_comp] + simp + · intro p hp + rw [PMF.mem_support_map_iff] at hp + obtain ⟨θ, _, rfl⟩ := hp + refine ⟨?_, ?_, Nat.le_refl 0, Modifies.refl _ _⟩ + · simpa [coinSubroutineCode, x₂, Data.set] using hfr₁ + · refine ⟨hx₁.set (fits_zero w), ?_, ?_⟩ + · simp [x₂, Data.set, ← hb₁, coinSubroutineCode] + · simp [x₂, Data.set] + step_exec _ ν _ hseed hsupp := by + let μ_step : Fin 2 × Data 5 → PMF (Fin 2 × Data 5) := fun p => + (uniformOfFintype (Fin (p.2.regs 2 + 1))).map fun k => (p.1, p.2.set 3 k.val) + refine ⟨μ_step, fun _ => 1, ?_, ?_, ?_⟩ + · intro p hp + obtain ⟨⟨hbp, hb0, hz2⟩, _⟩ := hsupp p hp + refine ⟨?_, ?_⟩ + · have hr := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 3) (src := 2) + (hbp.1 2) + refine hr.congr_dist ?_ le_rfl + rw [ProbComp.eval_unifFin, PMF.map_comp] + rfl + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨k, _, rfl⟩ := hq + have hk0 : k.val = 0 := by omega + refine ⟨rfl, ⟨?_, ?_, ?_⟩, Modifies.refl _ _⟩ + · refine hbp.set ?_ + rw [hk0] + exact fits_zero w + · simp [Data.set, hb0] + · simp [Data.set, hz2] + · intro θ' + have hmap : (ν.bind μ_step).map Prod.fst = ν.map Prod.fst := by + rw [PMF.map_bind, PMF.map] + refine congrArg (ν.bind ·) (funext fun p => ?_) + exact pmf_map_fst_const (uniformOfFintype (Fin (p.2.regs 2 + 1))) p.1 + (fun k => p.2.set 3 k.val) + rw [hmap] + exact hseed θ' + · simp [ν.tsum_coe] + +/-- Standard-basis lifting of `coinSubroutineDS`. -/ +noncomputable def coinSubroutineDS_standard (w base : ℕ) : + Cslib.WordRAM.Randomized.SubroutineDS + coinSubroutineCode.toStandard coinSpec w base Fin.elim0 PUnit.unit := + (coinSubroutineDS w base).toStandard + +/-- Fresh initialization + multi-operation expected-amortized sequence bound for +`coinSubroutineDS`. -/ +theorem coinSubroutineDS_seq_amortized (w base : ℕ) + (ops : List PUnit) {x₀ : Data 5} + (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : + let C := coinSubroutineDS w base + ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), + ExecCodeDist w coinSubroutineCode.initFresh x₀ (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ p ∈ ν_final.support, C.rep p.1 (coinSpec.run PUnit.unit ops).1 p.2) ∧ + C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ + C.initTime + (C.initPotential : ℝ≥0∞) + + (coinSpec.totalCost C.charge PUnit.unit ops : ℝ≥0∞) := by + intro C + have hvalid : coinSpec.ValidHistory PUnit.unit ops := by + induction ops with + | nil => trivial + | cons _ _ ih => exact ⟨trivial, ih⟩ + have hz : ∀ a ∈ (Block.mk base C.space).footprint, x₀.mem a = 0 := by + change ∀ a ∈ (Block.mk base 0).footprint, x₀.mem a = 0 + simp [Block.footprint] + exact C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid + +/-- Caller framing across a randomized caller step: a caller with 6 registers embeds `Fin 5` via +`Fin.castSucc` and executes `r5 ← rand r2` on its own register `r5` (`5 : Fin 6`), preserving the +coupled seed marginal bound, `rep`, and expected potential via `frame_caller_dist`. -/ +theorem coinSubroutineDS_caller_rand_frame (w base : ℕ) + (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ + (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (hrep : ∀ p ∈ ν.support, + (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base + let μ_caller : C.Seed × Data 6 → PMF (Data 6) := fun p => + (uniformOfFintype (Fin 1)).map fun k => p.2.set 5 k.val + (∀ θ, ((C.frameDist ν μ_caller).map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ q ∈ (C.frameDist ν μ_caller).support, C.rep q.1 PUnit.unit (q.2.comap Fin.castSucc)) ∧ + (∑' q, C.frameDist ν μ_caller q * (C.potential q.1 (q.2.comap Fin.castSucc) : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 (p.2.comap Fin.castSucc) : ℝ≥0∞) := by + intro C μ_caller + refine C.frame_caller_dist (F := ∅) hseed hrep (Finset.disjoint_empty_right _) ?_ + intro p hp x' hx' + rw [PMF.mem_support_map_iff] at hx' + obtain ⟨k, _, rfl⟩ := hx' + have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) + refine ⟨?_, ?_, Modifies.refl _ _⟩ + · refine ⟨fun i => ?_, hbp.2⟩ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] + exact hbp.1 i + · intro i _ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp [Data.set, hne] + +end ComplexityTests.WordRAMRandomizedSubroutine From 8079947e5ee790950c5aab90f2073930f2a0fff9 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 17:00:40 +0000 Subject: [PATCH 03/18] Add ExecSteps trace to SubroutineDS and expand randomized subroutine tests --- .../WordRAM/Randomized/Expected.lean | 47 +++--- .../WordRAM/Randomized/Subroutine.lean | 159 +++++++++++------- .../WordRAMRandomizedSubroutine.lean | 67 ++++++++ 3 files changed, 188 insertions(+), 85 deletions(-) diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index acfe26e2d..60ca18eef 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -1431,6 +1431,23 @@ theorem mem_support_loopExitPMF {cond : Cond r} {I Q : Data r → Prop} · exact absurd hcy hnc · exact ⟨⟨hnc, hQy⟩, K, hK⟩ +/-- Expectation of `k : β → ℝ≥0∞` under `p.map g` equals expectation of `k ∘ g` under `p`. -/ +theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [ENNReal.tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +/-- Expectation of `k : β → ℝ≥0∞` under `p.bind f` decomposes as an iterated expectation. -/ +theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : + (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by + simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] + exact ENNReal.tsum_comm + /-! ### Compositional distributional execution (`ExecCodeDist`) -/ /-- `ExecCodeDist w c x μ T`: at word size `w`, running structured program `c` from data `x` @@ -1686,16 +1703,6 @@ private theorem pmf_map_apply_not_mem_range {α β : Type*} (p : PMF α) (g : α have hne : b ≠ g a := fun h => hb ⟨a, h.symm⟩ simp [hne] -private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : - (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by - simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] - refine tsum_congr fun a => ?_ - rw [tsum_eq_single (g a)] - · simp - · intro b hb - simp [hb] - private theorem loopIterPMF_rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r') (K : ℕ) (u : Data r) : loopIterPMF (cond.rename f) (fun x' => (μ_body (x'.comap f)).map (fun y => y.extend f x)) @@ -1884,9 +1891,8 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program ∑' z, (μ₁.bind μ₂) z * expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by congr 1 - simp only [PMF.bind_apply, Code.size, Nat.add_assoc] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + simp only [Code.size, Nat.add_assoc] + exact (tsum_pmf_bind μ₁ μ₂ _).symm calc T₁ + ∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩) ≤ T₁ + ((∑' y, μ₁ y * T₂ y) + @@ -1985,9 +1991,7 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program (∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y) = ∑' y, loopExitPMF cond μ_body z y * K_cont y := by rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz] - simp only [PMF.bind_apply] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + exact (tsum_pmf_bind (μ_body z) (loopExitPMF cond μ_body) K_cont).symm have hih_step : (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + @@ -2196,9 +2200,8 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + (c₁.seq c₂).size, y⟩ (.halted v d)) = ∑' y₁, μ₁ y₁ * ∑' y, (μ₂ y₁) y * ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c₁.size + c₂.size, y⟩ (.halted v d) := by - simp only [PMF.bind_apply, Code.size, Nat.add_assoc] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + simp only [Code.size, Nat.add_assoc] + exact tsum_pmf_bind μ₁ μ₂ _ rw [hbind] refine (ENNReal.tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) by_cases hy₁ : y₁ ∈ μ₁.support @@ -2270,10 +2273,8 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ · rw [loopIterPMF_succ, loopStepPMF_of_holds hcz] have hbind : (∑' y, ((μ_body z).bind (loopIterPMF cond μ_body K)) y * H_loop y) = - ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := by - simp only [PMF.bind_apply] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := + tsum_pmf_bind (μ_body z) (loopIterPMF cond μ_body K) H_loop rw [hbind] refine (ENNReal.tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) by_cases hu : u ∈ (μ_body z).support diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean index f89efc199..38527898f 100644 --- a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -53,6 +53,8 @@ density factor `seedSlack : ℝ≥0∞`: * `Cslib.WordRAM.Randomized.SubroutineDS`: reusable randomized Word-RAM subroutine certificate for a deterministic specification `D : Stateful.Spec`. +* `Cslib.WordRAM.Randomized.SubroutineDS.ExecSteps`: multi-operation distributional execution trace + threading coupled state distributions and expected step costs across `ops : List D.Op`. * `Cslib.WordRAM.SubroutineDS.toRandomized`: canonical embedding of any deterministic `Cslib.WordRAM.SubroutineDS` into `Cslib.WordRAM.Randomized.SubroutineDS`. * `Cslib.WordRAM.Randomized.SubroutineDS.mapSimulates`, `toStandard`: basis transport from @@ -386,14 +388,19 @@ theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} rw [hm₁] at hsupp exact ⟨ν₀, (ExecCodeDist.of_execWithin he₁).seq_pure he₂, hseed, hsupp⟩ -/-- Relocated fresh initialization along any injective register embedding `f`. -/ -theorem initFresh_rename (C : SubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} - (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) - (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) - (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : +private theorem init_rename_of_exec (C : SubroutineDS P D w base params s₀) + {c : Code B r_ds} {T : ℝ≥0∞} {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {x : Data r_caller} + (hex : ∃ ν₀ : PMF (C.Seed × Data r_ds), + ExecCodeDist w c (x.comap f) (ν₀.map Prod.snd) T ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + C.space ∧ + C.rep p.1 s₀ p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint (x.comap f).mem p.2.mem) : ∃ ν₀ : PMF (C.Seed × Data r_caller), - ExecCodeDist w (P.initFresh.rename f) x (ν₀.map Prod.snd) C.initTime ∧ + ExecCodeDist w (c.rename f) x (ν₀.map Prod.snd) T ∧ (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ ∀ p ∈ ν₀.support, p.2.regs (f P.frontierReg) = base + C.space ∧ @@ -401,7 +408,7 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by - obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initFresh_exec hx hfr hpar hz + obtain ⟨ν₀, he, hseed, hsupp⟩ := hex refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ · have he' := he.rename hf refine he'.congr_dist ?_ le_rfl @@ -423,6 +430,23 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) · rw [Data.comap_extend hf] exact hpot +/-- Relocated fresh initialization along any injective register embedding `f`. -/ +theorem initFresh_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ ν₀ : PMF (C.Seed × Data r_caller), + ExecCodeDist w (P.initFresh.rename f) x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs (f P.frontierReg) = base + C.space ∧ + C.rep p.1 s₀ (p.2.comap f) ∧ + C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := + C.init_rename_of_exec hf (C.initFresh_exec hx hfr hpar hz) + /-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`zeroFillCode_exec`, `initClear_exec`, `initClear_rename`) -/ @@ -521,45 +545,8 @@ theorem initClear_rename (C : SubroutineDS P D w base params s₀) C.rep p.1 s₀ (p.2.comap f) ∧ C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ - (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by - obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initClear_exec hx hfr hpar - refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ - · have he' := he.rename hf - refine he'.congr_dist ?_ le_rfl - rw [PMF.map_comp, PMF.map_comp] - rfl - · intro θ - rw [PMF.map_comp] - exact hseed θ - · intro q hq - rw [PMF.mem_support_map_iff] at hq - obtain ⟨p, hp, rfl⟩ := hq - obtain ⟨hfr', hrep, hpot, hmod⟩ := hsupp p hp - refine ⟨?_, ?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem p.2 x hj⟩ - · change ((p.2.extend f x).comap f).regs P.frontierReg = base + C.space - rw [Data.comap_extend hf] - exact hfr' - · rw [Data.comap_extend hf] - exact hrep - · rw [Data.comap_extend hf] - exact hpot - -/-! ### Helper lemmas for coupled PMF expectations -/ - -private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : - (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by - simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] - refine tsum_congr fun a => ?_ - rw [tsum_eq_single (g a)] - · simp - · intro b hb - simp [hb] - -private theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : - (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by - simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] - exact ENNReal.tsum_comm + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := + C.init_rename_of_exec hf (C.initClear_exec hx hfr hpar) /-! ### Projection 4: Caller-to-DS bidirectional framing (`frame_caller`, `frame_caller_dist`) -/ @@ -756,7 +743,10 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) ExecCodeDist w (.seq prep (P.code.rename f)) p.2 ((μ_call p).map Prod.snd) (T_call p) ∧ ∀ q ∈ (μ_call p).support, C.decode o (q.2.comap f).regs = some (D.step s o).2 ∧ - C.rep q.1 (D.step s o).1 (q.2.comap f)) ∧ + C.rep q.1 (D.step s o).1 (q.2.comap f) ∧ + ∃ x' ∈ (μ_prep p).support, + Modifies (Block.mk base C.space).footprint x'.mem q.2.mem ∧ + ∀ j ∉ Set.range f, q.2.regs j = x'.regs j) ∧ (∀ θ', ((ν.bind μ_call).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ (∑' p, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) ≤ B_prep + (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by @@ -800,8 +790,8 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) have hp' : (p.1, x') ∈ ν_prep.support := by simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] exact ⟨p, hp, (PMF.mem_support_map_iff _ _ _).mpr ⟨x', hx', rfl⟩⟩ - obtain ⟨hdec, hrep', _, _⟩ := (hcode (p.1, x') hp').2 q hq_code - exact ⟨hdec, hrep'⟩ + obtain ⟨hdec, hrep', hmod, hregs⟩ := (hcode (p.1, x') hp').2 q hq_code + exact ⟨hdec, hrep', x', hx', hmod, hregs⟩ · intro θ' have hbind_eq : ν.bind μ_call = ν_prep.bind μ_code := by simp only [ν_prep, frameDist, μ_call, PMF.bind_bind, PMF.bind_map] @@ -926,12 +916,36 @@ theorem prepare_argMatch (C : SubroutineDS P D w base params s₀) (o : D.Op) (x C.argMatch o (C.prepare o x).regs := C.encode_matches o x.regs +/-- Inductive multi-operation distributional execution relation for a randomized subroutine `C`. +`C.ExecSteps s ops ν ν_final T_ops` witnesses that starting from abstract state `s` and coupled +state distribution `ν : PMF (C.Seed × Data r_ds)`, executing each operation `o` in `ops` by +delivering arguments via `C.prepare o` and running `P.code` under `ExecCodeDist` produces the +final coupled distribution `ν_final`, decodes the exact abstract return value `(D.step s o).2` at +every step, modifies memory only within `[base, base + C.space)`, and incurs total expected +operation step cost `T_ops`. -/ +inductive ExecSteps (C : SubroutineDS P D w base params s₀) : + D.State → List D.Op → PMF (C.Seed × Data r_ds) → PMF (C.Seed × Data r_ds) → ℝ≥0∞ → Prop where + | nil (s : D.State) (ν : PMF (C.Seed × Data r_ds)) : + C.ExecSteps s [] ν ν 0 + | cons {s : D.State} {o : D.Op} {ops : List D.Op} + {ν ν_final : PMF (C.Seed × Data r_ds)} + {μ_step : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds)} + {T_step : C.Seed × Data r_ds → ℝ≥0∞} {T_rest : ℝ≥0∞} : + (∀ p ∈ ν.support, + ExecCodeDist w P.code (C.prepare o p.2) ((μ_step p).map Prod.snd) (T_step p) ∧ + ∀ q ∈ (μ_step p).support, + C.decode o q.2.regs = some (D.step s o).2 ∧ + C.rep q.1 (D.step s o).1 q.2 ∧ + Modifies (Block.mk base C.space).footprint p.2.mem q.2.mem) → + C.ExecSteps (D.step s o).1 ops (ν.bind μ_step) ν_final T_rest → + C.ExecSteps s (o :: ops) ν ν_final ((∑' p, ν p * T_step p) + T_rest) + /-- **Multi-operation expected-amortized telescope (`steps_expected_amortized`).** From any abstract state `s : D.State` and coupled state distribution `ν : PMF (C.Seed × Data r_ds)` satisfying the seed-marginal bound and `C.rep p.1 s p.2` on `ν.support`, executing any admissible operation sequence `ops : List D.Op` (`D.ValidHistory s ops`) produces a final coupled distribution -`ν_final` and total expected operation step count `T_ops : ℝ≥0∞` satisfying the addition-only -`ℝ≥0∞` telescope inequality +`ν_final` and total expected operation step count `T_ops : ℝ≥0∞` witnessing +`C.ExecSteps s ops ν ν_final T_ops` and satisfying the addition-only `ℝ≥0∞` telescope inequality `T_ops + E_{ν_final}[Φ] ≤ D.totalCost C.charge s ops + E_ν[Φ]`. -/ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) {s : D.State} (ops : List D.Op) (ν : PMF (C.Seed × Data r_ds)) @@ -939,13 +953,14 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) (hrep : ∀ p ∈ ν.support, C.rep p.1 s p.2) : ∃ (ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), + C.ExecSteps s ops ν ν_final T_ops ∧ (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ p ∈ ν_final.support, C.rep p.1 (D.run s ops).1 p.2) ∧ T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ (D.totalCost C.charge s ops : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by induction ops generalizing s ν with | nil => - exact ⟨ν, 0, hseed, hrep, by simp⟩ + exact ⟨ν, 0, ExecSteps.nil s ν, hseed, hrep, by simp⟩ | cons o os ih => obtain ⟨hvo, hvos⟩ := hv let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.prepare o p.2) @@ -968,20 +983,39 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] obtain ⟨μ_step, T_step, hstep, hseed₁, hamort₁⟩ := C.step_exec o ν_prep hvo hseed_prep hsupp_prep - let ν₁ := ν_prep.bind μ_step + let μ_op : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds) := + fun p => μ_step (p.1, C.prepare o p.2) + let T_op : C.Seed × Data r_ds → ℝ≥0∞ := + fun p => T_step (p.1, C.prepare o p.2) + let ν₁ := ν.bind μ_op + have hν₁_eq : ν₁ = ν_prep.bind μ_step := by + simp only [ν₁, ν_prep, μ_op, PMF.bind_map] + rfl + have hseed₁' : ∀ θ, (ν₁.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + rw [hν₁_eq] + exact hseed₁ have hrep₁ : ∀ q ∈ ν₁.support, C.rep q.1 (D.step s o).1 q.2 := by intro q hq - rw [PMF.mem_support_bind_iff] at hq + rw [hν₁_eq, PMF.mem_support_bind_iff] at hq obtain ⟨p', hp', hq_step⟩ := hq exact ((hstep p' hp').2 q hq_step).2.1 - obtain ⟨ν_final, T_rest, hseed_final, hrep_final, hamort_rest⟩ := - ih ν₁ hvos hseed₁ hrep₁ - let T₁ : ℝ≥0∞ := ∑' p', ν_prep p' * T_step p' + obtain ⟨ν_final, T_rest, hex_rest, hseed_final, hrep_final, hamort_rest⟩ := + ih ν₁ hvos hseed₁' hrep₁ + let T₁ : ℝ≥0∞ := ∑' p, ν p * T_op p + have hT₁_eq : T₁ = ∑' p', ν_prep p' * T_step p' := by + symm + exact tsum_pmf_map ν (fun p => (p.1, C.prepare o p.2)) T_step + have hex_cons : C.ExecSteps s (o :: os) ν ν_final (T₁ + T_rest) := by + refine ExecSteps.cons (μ_step := μ_op) (T_step := T_op) (fun p hp => ?_) hex_rest + have hp' : (p.1, C.prepare o p.2) ∈ ν_prep.support := + (PMF.mem_support_map_iff _ _ _).mpr ⟨p, hp, rfl⟩ + exact hstep (p.1, C.prepare o p.2) hp' have hstep_split : T₁ + (∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) = ∑' p', ν_prep p' * (T_step p' + ∑' q, μ_step p' q * (C.potential q.1 q.2 : ℝ≥0∞)) := by - simp only [T₁, ν₁, tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] - refine ⟨ν_final, T₁ + T_rest, hseed_final, hrep_final, ?_⟩ + rw [hT₁_eq, hν₁_eq] + simp only [tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] + refine ⟨ν_final, T₁ + T_rest, hex_cons, hseed_final, hrep_final, ?_⟩ calc (T₁ + T_rest) + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) = T₁ + (T_rest + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl @@ -1012,13 +1046,14 @@ theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) (hv : D.ValidHistory s₀ ops) : ∃ (ν₀ ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), ExecCodeDist w P.initFresh x (ν₀.map Prod.snd) C.initTime ∧ + C.ExecSteps s₀ ops ν₀ ν_final T_ops ∧ (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ p ∈ ν_final.support, C.rep p.1 (D.run s₀ ops).1 p.2) ∧ C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ C.initTime + (C.initPotential : ℝ≥0∞) + (D.totalCost C.charge s₀ ops : ℝ≥0∞) := by obtain ⟨ν₀, he_init, hseed₀, hsupp₀⟩ := C.initFresh_exec hx hfr hpar hz have hrep₀ : ∀ p ∈ ν₀.support, C.rep p.1 s₀ p.2 := fun p hp => (hsupp₀ p hp).2.1 - obtain ⟨ν_final, T_ops, hseed_final, hrep_final, hamort⟩ := + obtain ⟨ν_final, T_ops, hex_steps, hseed_final, hrep_final, hamort⟩ := C.steps_expected_amortized ops ν₀ hv hseed₀ hrep₀ have hpot₀ : (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) ≤ (C.initPotential : ℝ≥0∞) := by calc @@ -1030,7 +1065,7 @@ theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] _ = (C.initPotential : ℝ≥0∞) := by rw [ENNReal.tsum_mul_right, ν₀.tsum_coe, one_mul] - refine ⟨ν₀, ν_final, T_ops, he_init, hseed_final, hrep_final, ?_⟩ + refine ⟨ν₀, ν_final, T_ops, he_init, hex_steps, hseed_final, hrep_final, ?_⟩ calc C.initTime + T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) = C.initTime + (T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl diff --git a/ComplexityTests/WordRAMRandomizedSubroutine.lean b/ComplexityTests/WordRAMRandomizedSubroutine.lean index d11f4056d..1a335f4b5 100644 --- a/ComplexityTests/WordRAMRandomizedSubroutine.lean +++ b/ComplexityTests/WordRAMRandomizedSubroutine.lean @@ -301,6 +301,7 @@ theorem coinSubroutineDS_seq_amortized (w base : ℕ) let C := coinSubroutineDS w base ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), ExecCodeDist w coinSubroutineCode.initFresh x₀ (ν₀.map Prod.snd) C.initTime ∧ + C.ExecSteps PUnit.unit ops ν₀ ν_final T_ops ∧ (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ p ∈ ν_final.support, C.rep p.1 (coinSpec.run PUnit.unit ops).1 p.2) ∧ C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ @@ -316,6 +317,72 @@ theorem coinSubroutineDS_seq_amortized (w base : ℕ) simp [Block.footprint] exact C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid +/-- Dirty-memory initialization via `initClear_exec` for `coinSubroutineDS`. -/ +theorem coinSubroutineDS_initClear (w base : ℕ) {x₀ : Data 5} + (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : + let C := coinSubroutineDS w base + ∃ ν₀ : PMF (C.Seed × Data 5), + ExecCodeDist w coinSubroutineCode.initClear x₀ (ν₀.map Prod.snd) + (C.initTime + (4 * C.space + 6 : ℕ)) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs coinSubroutineCode.frontierReg = base + C.space ∧ + C.rep p.1 PUnit.unit p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x₀.mem p.2.mem := by + intro C + exact C.initClear_exec hx₀ hfr (fun j => Fin.elim0 j) + +/-- Inline caller subroutine invocation via `call_rename` for `coinSubroutineDS`: a caller with +6 registers embeds `Fin 5` via `Fin.castSucc`, executes `prep := .op (.const 5 0)` on its own +register `r5`, and invokes `coinSubroutineCode.code.rename Fin.castSucc`. -/ +theorem coinSubroutineDS_call_rename (w base : ℕ) + (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ + (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (hrep : ∀ p ∈ ν.support, + (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base + let prep : Code .exact 6 := .op (.const 5 0) + ∃ (μ_call : C.Seed × Data 6 → PMF (C.Seed × Data 6)) + (T_call : C.Seed × Data 6 → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (.seq prep (coinSubroutineCode.code.rename Fin.castSucc)) + p.2 ((μ_call p).map Prod.snd) (T_call p) ∧ + ∀ q ∈ (μ_call p).support, + C.decode PUnit.unit (q.2.comap Fin.castSucc).regs = + some (coinSpec.step PUnit.unit PUnit.unit).2 ∧ + C.rep q.1 (coinSpec.step PUnit.unit PUnit.unit).1 (q.2.comap Fin.castSucc) ∧ + ∃ x' ∈ (PMF.pure (p.2.set 5 0) : PMF (Data 6)).support, + Modifies (Block.mk base C.space).footprint x'.mem q.2.mem ∧ + ∀ j ∉ Set.range (Fin.castSucc : Fin 5 → Fin 6), q.2.regs j = x'.regs j) ∧ + (∀ θ', ((ν.bind μ_call).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ + (∑' p, ν p * (T_call p + + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap Fin.castSucc) : ℝ≥0∞))) ≤ + 1 + (C.charge PUnit.unit PUnit.unit : ℝ≥0∞) + + ∑' p, ν p * (C.potential p.1 (p.2.comap Fin.castSucc) : ℝ≥0∞) := by + intro C prep + refine C.call_rename (Fin.castSucc_injective 5) PUnit.unit ν + (prep := prep) (B_prep := 1) + (μ_prep := fun p => PMF.pure (p.2.set 5 0)) (T_prep := fun _ => 1) + trivial hseed hrep ?_ + intro p hp + refine ⟨?_, le_rfl, ?_⟩ + · simpa [prep] using ExecCodeDist.of_exec (B := .exact) (w := w) (x := p.2) + (Exec.const (d := (5 : Fin 6)) (fits_zero w)) + · intro x' hx' + rw [PMF.mem_support_pure_iff] at hx' + subst hx' + have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) + refine ⟨?_, ?_, fun _ _ => rfl, trivial⟩ + · refine ⟨fun i => ?_, hbp.2⟩ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] + exact hbp.1 i + · intro i _ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp [Data.set, hne] + /-- Caller framing across a randomized caller step: a caller with 6 registers embeds `Fin 5` via `Fin.castSucc` and executes `r5 ← rand r2` on its own register `r5` (`5 : Fin 6`), preserving the coupled seed marginal bound, `rep`, and expected potential via `frame_caller_dist`. -/ From cefa21faff0d923eb0fac5dd8e2588777173b9f1 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 17:25:29 +0000 Subject: [PATCH 04/18] Update author header and MitzenmacherUpfal2017 section locators --- .../Computability/WordRAM/Randomized/Expected.lean | 8 ++++---- .../WordRAM/Randomized/Subroutine.lean | 14 ++++++++------ 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index 60ca18eef..ebfa3b1b7 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -72,14 +72,14 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, geometric Las Vegas `while` loops and structured programs. Discrete and conditional expectation principles are classical background; see -[MitzenmacherUpfal2017], Sections 2.1 and 2.3. The particular native tail-sum/Bellman definitions, -finite structured-execution-tree bridge and ranked-loop expected-cost formalization are original -library constructions in this module. +[MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. The particular native tail-sum/Bellman +definitions, finite structured-execution-tree bridge and ranked-loop expected-cost formalization are +original library constructions in this module. ## References * [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017], - Sections 2.1 and 2.3. + Sections 1.5, 2.1, and 2.3. -/ @[expose] public section diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean index 38527898f..acfd953f7 100644 --- a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -1,7 +1,7 @@ /- -Copyright (c) 2026 Chaeyoung Lim. All rights reserved. +Copyright (c) 2026 Renfei Zhou. All rights reserved. Released under Apache 2.0 license as described in the file LICENSE. -Authors: Chaeyoung Lim +Authors: Renfei Zhou -/ module @@ -17,7 +17,8 @@ This file extends the deterministic `Cslib.WordRAM.SubroutineDS` interface to ** structures and Las Vegas subroutines** on the Word-RAM — covering both structures initialized from a random seed (such as universal hash tables, 2-choice hash tables, and cuckoo hash tables) and structures whose operations make random choices via `Instr.rand` (such as treaps, skip lists, -reservoir samplers, or randomized rehashing) `[MitzenmacherUpfal2017]`. +reservoir samplers, or randomized rehashing); see [MitzenmacherUpfal2017], Sections 1.5, 2.1, and +2.3. ## Core design @@ -43,7 +44,7 @@ density factor `seedSlack : ℝ≥0∞`: - Every admissible operation `o : D.Op` from abstract state `s : D.State` and coupled state distribution `ν : PMF (Seed × Data r_ds)` produces a transition kernel `μ_step : Seed × Data r_ds → PMF (Seed × Data r_ds)` and expected step count - `T_step : Seed × Data r_ds → ℝ≥0∞` satisfying + `T_step : Seed × Data r_ds → ℝ≥0∞` satisfying (for `Φ(p) := potential p.1 p.2`) `∑' p, ν p * (T_step p + ∑' q, μ_step p q * Φ(q)) ≤ charge s o + ∑' p, ν p * Φ(p)`. - Across any admissible finite operation sequence `ops : List D.Op`, `seq_expected_amortized` telescopes these one-step bounds without `ℝ≥0∞` subtraction: @@ -81,7 +82,7 @@ density factor `seedSlack : ℝ≥0∞`: * [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, Chapter 17][CLRS2009] * [M. Mitzenmacher and E. Upfal, *Probability and Computing: Randomization and Probabilistic - Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017] + Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. -/ @[expose] public section @@ -946,7 +947,8 @@ satisfying the seed-marginal bound and `C.rep p.1 s p.2` on `ν.support`, execut operation sequence `ops : List D.Op` (`D.ValidHistory s ops`) produces a final coupled distribution `ν_final` and total expected operation step count `T_ops : ℝ≥0∞` witnessing `C.ExecSteps s ops ν ν_final T_ops` and satisfying the addition-only `ℝ≥0∞` telescope inequality -`T_ops + E_{ν_final}[Φ] ≤ D.totalCost C.charge s ops + E_ν[Φ]`. -/ +(with `Φ(p) := C.potential p.1 p.2`) +`T_ops + ∑' q, ν_final q * Φ(q) ≤ D.totalCost C.charge s ops + ∑' p, ν p * Φ(p)`. -/ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) {s : D.State} (ops : List D.Op) (ν : PMF (C.Seed × Data r_ds)) (hv : D.ValidHistory s ops) From a7461e87d2534a4b73d2f0b05e9e06c0f41c541c Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 17:53:41 +0000 Subject: [PATCH 05/18] refactor(stateful,word-ram): share Execution/DynamicProgram/SubroutineBase and drop aliases --- .../Computability/WordRAM/RelSubroutine.lean | 521 +++--------------- .../Computability/WordRAM/Subroutine.lean | 349 ++++++------ .../Semantics/Stateful/Realization.lean | 16 +- .../Semantics/Stateful/Relational.lean | 138 ++--- ComplexityTests/StatefulRelational.lean | 8 +- ComplexityTests/WordRAMRelSubroutine.lean | 10 +- 6 files changed, 311 insertions(+), 731 deletions(-) diff --git a/Complexity/Computability/WordRAM/RelSubroutine.lean b/Complexity/Computability/WordRAM/RelSubroutine.lean index 9b92e4dee..7e3dece99 100644 --- a/Complexity/Computability/WordRAM/RelSubroutine.lean +++ b/Complexity/Computability/WordRAM/RelSubroutine.lean @@ -12,39 +12,40 @@ public import Complexity.Computability.WordRAM.Subroutine public import Complexity.Foundations.Semantics.Stateful.Relational /-! -# Relational Word-RAM dynamic programs and composable data-structure subroutines +# Relational Word-RAM dynamic certificates and composable data-structure subroutines This file extends `Complexity.Computability.WordRAM.Dynamic` and `Complexity.Computability.WordRAM.Subroutine` from deterministic specifications (`Stateful.Spec`) to relational specifications (`Stateful.RelSpec`): -1. **Standalone relational fixed-program implementations (`RelDynamicProgram`, - `RelDynamicCertificate`):** Generalizes `DynamicProgram` and `DynamicCertificate` to - `D : Stateful.RelSpec`, where `execute` witnesses an angelic outcome `(s', a)` satisfying - `D.step s o s' a`. Because the Word-RAM structured semantics `Exec` is deterministic, - `RelDynamicCertificate.realization` establishes `Stateful.RelRealization` directly via - `Exec.unique`. +1. **Standalone relational fixed-program certificates (`RelDynamicCertificate`):** Generalizes + `DynamicCertificate` over `DynamicProgram D r` for `D : Stateful.RelSpec`, where `execute` + witnesses an angelic outcome `(s', a)` satisfying `D.step s o s' a`. Because the Word-RAM + structured semantics `Exec` is deterministic, `RelDynamicCertificate.realization` establishes + `Stateful.RelRealization` directly via `Exec.unique`. 2. **Unified standalone and inline relational subroutine certificates (`RelSubroutineDS`):** - Mirrors `SubroutineDS` over `D : Stateful.RelSpec`, omitting `rep_unique` (since concrete - potential `potential : Data r_ds → ℕ` depends only on machine data) and existentially - quantifying over `(s', a)` with `D.step s o s' a` in `step_exec`. + Extends `SubroutineBase P D.Result D.valid w base params s₀` over `D : Stateful.RelSpec`, + inheriting `toDynamicProgram`, `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, + `initClear_exec`, `initClear_rename`, `frame_caller`, and `prepareRegs` without duplication, + omitting `rep_unique` (since concrete potential `potential : Data r_ds → ℕ` depends only on + machine data), and existentially quantifying over `(s', a)` with `D.step s o s' a` in + `step_exec`. 3. **Zero-cost embedding of deterministic subroutines (`SubroutineDS.toRelSubroutineDS`):** Every `SubroutineDS P D w base params s₀` embeds into `RelSubroutineDS P D.toRelSpec w base params s₀`. 4. **Basis lifting, standalone `Stateful` projections, inline subroutine calls, and flat-program - operation sequences:** Provides `mapSimulates`, `toStandard`, `toRelDynamicProgram`, - `toRelCertificate`, `toRelAmortized`, `toRelBound`, `run_client_amortized`, `run_client_bound`, - `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, - `initClear_rename`, `code_rename`, `call_rename`, `frame_caller`, and flat-program sequence - execution (`RunsRelOps`, `seq_runsRelOps`, `run_ops_exact_fresh`, `run_ops_exact_reusable`). + operation sequences:** Provides `mapSimulates`, `toStandard`, `toRelCertificate`, + `toRelAmortized`, `toRelBound`, `run_client_amortized`, `run_client_bound`, `code_rename`, + `call_rename`, and flat-program sequence execution (`RunsRelOps`, `seq_runsRelOps`, + `run_ops_exact_fresh`, `run_ops_exact_reusable`). ## Main definitions -* `Cslib.WordRAM.RelDynamicProgram`, `Cslib.WordRAM.RelDynamicCertificate`: standalone fixed-program - realization and certificate for a relational specification `D : Stateful.RelSpec`. +* `Cslib.WordRAM.RelDynamicCertificate`: standalone fixed-program certificate for + `DynamicProgram D r` realizing a relational specification `D : Stateful.RelSpec`. * `Cslib.WordRAM.RelSubroutineDS`: unified standalone and inline subroutine specification and amortized resource certificate for `P : SubroutineCode B r_ds k_init` realizing - `D : Stateful.RelSpec` (without `rep_unique`). + `D : Stateful.RelSpec` (extending `SubroutineBase` without `rep_unique`). * `Cslib.WordRAM.SubroutineDS.toRelSubroutineDS`: embedding of a deterministic `SubroutineDS` into `RelSubroutineDS`. * `Cslib.WordRAM.RelSubroutineDS.RunsRelOps`: sequential flat-program execution of an operation list @@ -58,11 +59,9 @@ to relational specifications (`Stateful.RelSpec`): * `Cslib.WordRAM.RelSubroutineDS.toRelAmortized`, `Cslib.WordRAM.RelSubroutineDS.run_client_amortized`: standalone amortized potential certificate and adaptive-client bound over `Basis.standard`. -* `Cslib.WordRAM.RelSubroutineDS.initFresh_rename`, - `Cslib.WordRAM.RelSubroutineDS.initClear_rename`, - `Cslib.WordRAM.RelSubroutineDS.call_rename`, - `Cslib.WordRAM.RelSubroutineDS.frame_caller`: relocated inline initialization, inline operation - calls, and bidirectional framing. +* `Cslib.WordRAM.RelSubroutineDS.code_rename`, + `Cslib.WordRAM.RelSubroutineDS.call_rename`: relocated inline operation calls with angelic + relational step outcomes. * `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_fresh`, `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_reusable`: end-to-end flat-program execution of any demonically admissible operation sequence `D.ValidOpSeq s₀ ops`. @@ -80,136 +79,13 @@ namespace Cslib.WordRAM universe u v z -/-! ## Standalone relational dynamic programs (`RelDynamicProgram`, `RelDynamicCertificate`) -/ +/-! ## Standalone relational dynamic certificates (`RelDynamicCertificate`) -/ variable {r : ℕ} {D : Stateful.RelSpec.{u, v, z}} -/-- A single `.standard`-basis Word-RAM program serving all operations of a relational -specification `D : Stateful.RelSpec` through a fixed register calling convention. -/ -structure RelDynamicProgram (D : Stateful.RelSpec.{u, v, z}) (r : ℕ) where - /-- The same structured program is executed for every operation. -/ - code : Code .standard r - /-- Register returned by the final `halt` instruction. -/ - output : Fin r - /-- Registers overwritten by externally supplied operation arguments. -/ - input : Fin r → Bool - /-- Encoded arguments, independent of the maintained machine state. -/ - encode : D.Op → Fin r → ℕ - /-- Interpretation of the final register tuple as the operation's typed observation. -/ - decode : (o : D.Op) → (Fin r → ℕ) → Option (D.Result o) - -/-- Embed a deterministic `DynamicProgram D r` as a `RelDynamicProgram D.toRelSpec r`. -/ -abbrev DynamicProgram.toRelDynamicProgram {D : Stateful.Spec.{u, v, z}} {r : ℕ} - (P : DynamicProgram D r) : RelDynamicProgram D.toRelSpec r where - code := P.code - output := P.output - input := P.input - encode := P.encode - decode := P.decode - -namespace RelDynamicProgram - -variable (P : RelDynamicProgram D r) - -/-- Deliver operation arguments without changing maintained memory or non-input registers. -/ -def prepare (o : D.Op) (x : Data r) : Data r where - regs i := if P.input i then P.encode o i else x.regs i - mem := x.mem - -/-- Delivering operation arguments preserves the machine memory unchanged. -/ -@[simp] -theorem prepare_mem (o : D.Op) (x : Data r) : (P.prepare o x).mem = x.mem := rfl - -/-- Registers outside the calling convention retain their contents. -/ -theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : D.Op) (x : Data r) : - (P.prepare o x).regs i = x.regs i := by - simp [prepare, hi] - -/-- Argument delivery preserves bounded words when the arguments fit. -/ -theorem prepare_bounded {w : ℕ} {o : D.Op} {x : Data r} (hx : x.Bounded w) - (ho : ∀ i, P.input i = true → Fits w (P.encode o i)) : (P.prepare o x).Bounded w := by - refine ⟨fun i => ?_, hx.2⟩ - change Fits w (if P.input i then P.encode o i else x.regs i) - split - · exact ho i (by assumption) - · exact hx.1 i - -/-- Evaluate one typed call, delivering its arguments and decoding its observation. The returned -cost includes the final `halt`. -/ -def eval (w fuel : ℕ) (o : D.Op) (x : Data r) : Option (Data r × D.Result o × ℕ) := - match P.code.eval w fuel (P.prepare o x) with - | none => none - | some (y, n) => (P.decode o y.regs).map fun a => (y, a, n + 1) - -/-- A bounded machine call with native execution evidence (`DynamicProgram.OperationTrace`). -/ -def ExecOp (w : ℕ) : Stateful.RelExecution D (Data r) DynamicProgram.OperationTrace := - fun x o a y trace => - (P.prepare o x).Bounded w ∧ ∃ n, TraceExec w P.code (P.prepare o x) y n trace.probes ∧ - P.decode o y.regs = some a ∧ trace.instructions = n + 1 - -/-- Successful typed evaluation supplies an exact operation trace when the delivered state is -bounded. -/ -theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result o} - (hb : (P.prepare o x).Bounded w) (h : P.eval w fuel o x = some (y, a, k)) : - ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := by - cases he : P.code.eval w fuel (P.prepare o x) with - | none => simp [eval, he] at h - | some yn => - obtain ⟨z, n⟩ := yn - cases hd : P.decode o z.regs with - | none => simp [eval, he, hd] at h - | some b => - simp only [eval, he, hd, Option.map_some, Option.some.injEq, Prod.mk.injEq] at h - obtain ⟨rfl, rfl, rfl⟩ := h - obtain ⟨probes, hp⟩ := (Exec.of_eval he).exists_trace - exact ⟨probes, hb, n, hp, hd, rfl⟩ - -/-- Every call is an execution of one fixed flat machine program. -/ -theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} - {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : - Executes w (P.code.program P.output) ⟨0, P.prepare o x⟩ (y.regs P.output) - trace.instructions := by - obtain ⟨_, n, he, _, ht⟩ := h - rw [ht] - exact he.exec.executes P.output - -/-- A call preserves bounded words. -/ -theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} - {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : y.Bounded w := by - obtain ⟨hb, _, he, _, _⟩ := h - exact he.exec.bounded hb - -/-- Calls are deterministic in their final representation, observation, and instruction count. -/ -theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} - {trace trace' : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) - (h' : P.ExecOp w x o a' y' trace') : - y = y' ∧ a = a' ∧ trace.instructions = trace'.instructions := by - obtain ⟨_, _, he, ha, ht⟩ := h - obtain ⟨_, _, he', ha', ht'⟩ := h' - obtain ⟨rfl, rfl⟩ := he.exec.unique he'.exec - exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ - -/-- A call's final state and instruction count agree with any structured execution of the code. -/ -theorem ExecOp.eq_of_exec {P : RelDynamicProgram D r} {w : ℕ} {x y z : Data r} {o : D.Op} - {a : D.Result o} {trace : DynamicProgram.OperationTrace} {m : ℕ} - (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : - y = z ∧ trace.instructions = m + 1 := by - obtain ⟨_, _, ht, _, hn⟩ := he - obtain ⟨rfl, rfl⟩ := ht.exec.unique h - exact ⟨rfl, hn⟩ - -/-- The write transcript supplies a frame rule for a call, including register-only delivery. -/ -theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} - {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) {Q : HeapAssertion} - (hQ : Q.holds x.mem) (hd : Disjoint Q.footprint (Probe.writes trace.probes)) : - Q.holds y.mem := by - obtain ⟨_, _, he, _, _⟩ := h - exact he.frame hQ hd - -end RelDynamicProgram - -/-- Per-operation angelic machine proof obligations for a `RelDynamicProgram`. -/ -structure RelDynamicCertificate (P : RelDynamicProgram D r) (w : ℕ) where +/-- Per-operation angelic machine proof obligations for a `DynamicProgram D r` over a relational +specification `D : Stateful.RelSpec`. -/ +structure RelDynamicCertificate (P : DynamicProgram D r) (w : ℕ) where /-- Abstract state represented by the persistent machine data. -/ rep : D.State → Data r → Prop /-- Represented configurations contain only machine words. -/ @@ -225,7 +101,8 @@ structure RelDynamicCertificate (P : RelDynamicProgram D r) (w : ℕ) where /-- Embed a deterministic `DynamicCertificate` as a `RelDynamicCertificate`. -/ def DynamicCertificate.toRelDynamicCertificate {D : Stateful.Spec.{u, v, z}} {r w : ℕ} {P : DynamicProgram D r} (C : DynamicCertificate P w) : - RelDynamicCertificate P.toRelDynamicProgram w where + RelDynamicCertificate (D := D.toRelSpec) + ⟨P.code, P.output, P.input, P.encode, P.decode⟩ w where rep := C.rep bounded := C.bounded arguments_fit := C.arguments_fit @@ -235,14 +112,14 @@ def DynamicCertificate.toRelDynamicCertificate {D : Stateful.Spec.{u, v, z}} {r namespace RelDynamicCertificate -variable {P : RelDynamicProgram D r} {w : ℕ} (C : RelDynamicCertificate P w) +variable {P : DynamicProgram D r} {w : ℕ} (C : RelDynamicCertificate P w) /-- Turn a successful typed evaluation on a represented state and admissible operation into certified `ExecOp` evidence. -/ theorem execOp_of_eval {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} {fuel k : ℕ} (hr : C.rep s x) (hv : D.valid s o) (h : P.eval w fuel o x = some (y, a, k)) : ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := - RelDynamicProgram.ExecOp.of_eval P + DynamicProgram.ExecOp.of_eval P (P.prepare_bounded (C.bounded hr) (C.arguments_fit o hv)) h /-- Fixed-program relational certificates realize `D` via `Stateful.RelRealization`. -/ @@ -291,7 +168,7 @@ theorem run_client_const {K : ℕ} {α : Type*} (p : D.Client α) {s : D.State} {x : Data r} (hr : C.rep s x) (hv : D.DemonicValid s p) : ∃ a y traces s' tr, - Stateful.RelExecutes (P.ExecOp w) p x a y traces ∧ + Stateful.Executes (P.ExecOp w) p x a y traces ∧ D.ClientRun s p a s' tr ∧ C.rep s' y ∧ traces.length = tr.length ∧ @@ -331,7 +208,7 @@ theorem executes_client_frame {j : ℕ} {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} {traces : List DynamicProgram.OperationTrace} (hr : C.rep s x) (hv : D.DemonicValid s p) - (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + (he : Stateful.Executes (P.ExecOp w) p x val y traces) : y.mem j = x.mem j := by induction he generalizing s with | pure _ _ => rfl @@ -348,7 +225,7 @@ theorem executes_client_modifies {F : Finset ℕ} {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} {traces : List DynamicProgram.OperationTrace} (hr : C.rep s x) (hv : D.DemonicValid s p) - (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + (he : Stateful.Executes (P.ExecOp w) p x val y traces) : Modifies F x.mem y.mem := fun j hj => C.executes_client_frame (fun hr' hv' he' => hmod hr' hv' he' j hj) hr hv he @@ -388,70 +265,14 @@ end SubroutineCode `P : SubroutineCode B r_ds k_init` realizing a relational specification `D : Stateful.RelSpec` at word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. -In contrast to `SubroutineDS`, `rep_unique` is omitted (because concrete potential -`potential : Data r_ds → ℕ` depends only on machine data) and `step_exec` existentially -quantifies over a permitted outcome `(s', a)` satisfying `D.step s o s' a`. -/ +Extends `SubroutineBase P D.Result D.valid w base params s₀` (inheriting `toDynamicProgram`, +`initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, `initClear_rename`, +`frame_caller`, and `prepareRegs`), omits `rep_unique` (because concrete potential +`potential : Data r_ds → ℕ` depends only on machine data), and existentially quantifies over a +permitted outcome `(s', a)` satisfying `D.step s o s' a` in `step_exec`. -/ structure RelSubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) - (D : Stateful.RelSpec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) where - /-- Contiguous memory footprint length `[base, base + space)`. -/ - space : ℕ - /-- Instruction bound for `P.allocSpan`. -/ - spanTime : ℕ - /-- Instruction bound for `P.initSetup`. -/ - setupTime : ℕ - /-- Upper bound on initial concrete potential after initialization. -/ - initPotential : ℕ - /-- Amortized instruction charge per operation (excluding trailing standalone `halt`). -/ - charge : D.State → D.Op → ℕ - /-- Representation invariant linking mathematical state `s` to concrete `Data r_ds`. -/ - rep : D.State → Data r_ds → Prop - /-- Concrete integer potential depending only on `Data r_ds`. -/ - potential : Data r_ds → ℕ - /-- Standalone register encoding for `RelDynamicProgram.prepare`. -/ - encode : D.Op → Fin r_ds → ℕ - /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ - argMatch : D.Op → (Fin r_ds → ℕ) → Prop - /-- Output decoder reading typed operation observations from final registers. -/ - decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) - /-- Represented configurations are word-bounded. -/ - bounded : ∀ {s x}, rep s x → x.Bounded w - /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ - arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) - /-- Full standalone `prepare` delivery satisfies `argMatch`. -/ - encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), - argMatch o (fun i => if P.input i then encode o i else regs i) - /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. - -/ - rep_supported : ∀ {s x x'}, - (∀ i, P.input i = false → x'.regs i = x.regs i) → - (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → - x'.Bounded w → rep s x → rep s x' - /-- `potential` depends only on persistent registers and `[base, base + space)` under `rep`. -/ - potential_supported : ∀ {s x x'}, - (∀ i, P.input i = false → x'.regs i = x.regs i) → - (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → - rep s x → potential x' = potential x - /-- Distinctness of registers used by `zeroFillCode` relative to `baseReg` and `frontierReg`. -/ - clear_nodup : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup - /-- Initialization parameter registers are not overwritten by `zeroFillCode`. -/ - params_not_clear : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg] - /-- `P.incCur` increments `P.curReg` by `1` when `P.oneReg = 1` and the sum fits in `w` bits. -/ - incCur_eval : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → - P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1)) - /-- Execution of `P.allocSpan`: records `baseReg = base` and advances `frontierReg`. -/ - span_exec : ∀ {x : Data r_ds}, - x.Bounded w → x.regs P.frontierReg = base → (∀ j, x.regs (P.initParams j) = params j) → - ∃ x₁, ExecWithin w P.allocSpan x x₁ spanTime ∧ x₁.mem = x.mem ∧ - x₁.regs P.baseReg = base ∧ x₁.regs P.frontierReg = base + space ∧ - (∀ j, x₁.regs (P.initParams j) = params j) - /-- Execution of `P.initSetup` once `[base, base + space)` holds zeros. -/ - setup_exec : ∀ {x₁ : Data r_ds}, - x₁.Bounded w → x₁.regs P.baseReg = base → x₁.regs P.frontierReg = base + space → - (∀ j, x₁.regs (P.initParams j) = params j) → - (∀ a ∈ (Block.mk base space).footprint, x₁.mem a = 0) → - ∃ y, ExecWithin w P.initSetup x₁ y setupTime ∧ y.regs P.frontierReg = base + space ∧ - rep s₀ y ∧ potential y ≤ initPotential ∧ - Modifies (Block.mk base space).footprint x₁.mem y.mem + (D : Stateful.RelSpec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) + extends SubroutineBase P D.Result D.valid w base params s₀ where /-- Angelic execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`, witnessing some permitted outcome `(s', a)` with `D.step s o s' a`. -/ step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), @@ -470,26 +291,7 @@ def SubroutineDS.toRelSubroutineDS {B : Basis} {r_ds k_init w base : ℕ} {params : Fin k_init → ℕ} {s₀ : D.State} (C : SubroutineDS P D w base params s₀) : RelSubroutineDS P D.toRelSpec w base params s₀ where - space := C.space - spanTime := C.spanTime - setupTime := C.setupTime - initPotential := C.initPotential - charge := C.charge - rep := C.rep - potential := C.potential - encode := C.encode - argMatch := C.argMatch - decode := C.decode - bounded := C.bounded - arguments_fit := C.arguments_fit - encode_matches := C.encode_matches - rep_supported := C.rep_supported - potential_supported := C.potential_supported - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval := C.incCur_eval - span_exec := C.span_exec - setup_exec := C.setup_exec + toSubroutineBase := C.toSubroutineBase step_exec o hr hv harg := by obtain ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg exact ⟨y, (D.step _ o).1, (D.step _ o).2, n, he, rfl, hdec, hrep, hmod, hpot⟩ @@ -500,39 +302,12 @@ variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} {P : SubroutineCode B r_ds k_init} {D : Stateful.RelSpec.{u, v, z}} {params : Fin k_init → ℕ} {s₀ : D.State} -/-- Total instruction bound for `P.initFresh`. -/ -def initTime (C : RelSubroutineDS P D w base params s₀) : ℕ := - C.spanTime + C.setupTime - -/-! ### Projection 1: Basis lifting and standalone `RelDynamicProgram` / `Stateful` theorems -/ +/-! ### Projection 1: Basis lifting and standalone `DynamicProgram` / `Stateful` theorems -/ /-- Lift a `RelSubroutineDS` certificate along any defined-evaluation basis simulation. -/ def mapSimulates (φ : B.Simulates B') (C : RelSubroutineDS P D w base params s₀) : RelSubroutineDS (P.mapSimulates φ) D w base params s₀ where - space := C.space - spanTime := C.spanTime - setupTime := C.setupTime - initPotential := C.initPotential - charge := C.charge - rep := C.rep - potential := C.potential - encode := C.encode - argMatch := C.argMatch - decode := C.decode - bounded := C.bounded - arguments_fit := C.arguments_fit - encode_matches := C.encode_matches - rep_supported := C.rep_supported - potential_supported := C.potential_supported - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval h1 hfit := Op.eval_mapSimulates_of_some φ (C.incCur_eval h1 hfit) - span_exec hx hf hp := - let ⟨x₁, he, hm, hb, hfr, hpar⟩ := C.span_exec hx hf hp - ⟨x₁, he.mapSimulates φ, hm, hb, hfr, hpar⟩ - setup_exec hx₁ hb hfr hpar hz := - let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz - ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ + toSubroutineBase := C.toSubroutineBase.mapSimulates φ step_exec o hr hv harg := let ⟨y, s', a, n, he, hstep, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg ⟨y, s', a, n, he.mapSimulates φ, hstep, hdec, hrep, hmod, hpot⟩ @@ -548,50 +323,42 @@ section Standard variable {P_std : SubroutineCode .standard r_ds k_init} (C : RelSubroutineDS P_std D w base params s₀) -/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicProgram`. -/ -def toRelDynamicProgram : RelDynamicProgram D r_ds where - code := P_std.code - output := P_std.output - input := P_std.input - encode := C.encode - decode := C.decode - /-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicCertificate`. -/ -def toRelCertificate : RelDynamicCertificate C.toRelDynamicProgram w where +def toRelCertificate : RelDynamicCertificate (D := D) C.toDynamicProgram w where rep := C.rep bounded := C.bounded arguments_fit := C.arguments_fit execute {s x} o hr hv := by - have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := - C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := C.rep_supported - (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hb' hr - have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := C.encode_matches o x.regs obtain ⟨y, s', a, n, he, hstep, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg exact ⟨y, s', a, n, he, hstep, hdec, hrep⟩ /-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `Stateful.RelAmortized` -certificate, charging `+ 1` per operation for `RelDynamicProgram`'s trailing `halt` instruction. -/ +certificate, charging `+ 1` per operation for `DynamicProgram`'s trailing `halt` instruction. -/ theorem toRelAmortized : Stateful.RelAmortized C.toRelCertificate.realization DynamicProgram.OperationTrace.instructions (fun _ x => C.potential x) (fun s o => C.charge s o + 1) where step {s x o a y trace} hr hv he := by - have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := - C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := C.rep_supported - (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hb' hr - have hp_eq : C.potential (C.toRelDynamicProgram.prepare o x) = C.potential x := + have hp_eq : C.potential (C.toDynamicProgram.prepare o x) = C.potential x := C.potential_supported - (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hr - have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := C.encode_matches o x.regs obtain ⟨z, s', a', m, hz, hstep, hdec, hrep, _, hpot⟩ := C.step_exec o hr' hv harg obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz @@ -603,9 +370,6 @@ theorem toRelAmortized : rw [hp_eq] at hpot omega -/-- Alias for `toRelAmortized` matching `SubroutineDS.toAmortized`. -/ -abbrev toAmortized := @toRelAmortized - /-- When `C.potential` is identically zero, `RelSubroutineDS` projects directly to a worst-case `Stateful.RelBound` certificate. -/ theorem toRelBound (hpot : ∀ x, C.potential x = 0) : @@ -617,14 +381,11 @@ theorem toRelBound (hpot : ∀ x, C.potential x = 0) : simp only [hpot] at h omega -/-- Alias for `toRelBound` matching `SubroutineDS.toBound`. -/ -abbrev toBound := @toRelBound - /-- Standalone adaptive-client amortized bound for any demonically admissible client `p`. -/ theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} (hr : C.rep s x) (hv : D.DemonicValid s p) : ∃ a y traces s' tr, - Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x a y traces ∧ D.ClientRun s p a s' tr ∧ C.rep s' y ∧ traces.length = tr.length ∧ @@ -636,7 +397,7 @@ theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : D theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} (hr : C.rep s x) (hv : D.DemonicValid s p) : ∃ a y traces s' tr, - Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x a y traces ∧ D.ClientRun s p a s' tr ∧ C.rep s' y ∧ traces.length = tr.length ∧ @@ -646,140 +407,7 @@ theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.C end Standard -/-! ### Projection 2: Fresh `O(1)` initialization and register relocation -/ - -/-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ -theorem initFresh_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} - (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) - (hpar : ∀ j, x.regs (P.initParams j) = params j) - (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : - ∃ y, ExecWithin w P.initFresh x y C.initTime ∧ - y.regs P.frontierReg = base + C.space ∧ - C.rep s₀ y ∧ - C.potential y ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem y.mem := by - obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar - have hx₁ : x₁.Bounded w := he₁.bounded hx - have hz₁ : ∀ a ∈ (Block.mk base C.space).footprint, x₁.mem a = 0 := by - intro a ha - rw [hm₁] - exact hz a ha - obtain ⟨y, he₂, hfr₂, hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb₁ hfr₁ hpar₁ hz₁ - rw [hm₁] at hmod - exact ⟨y, he₁.seq he₂, hfr₂, hrep, hpot, hmod⟩ - -/-- Relocated `O(1)` fresh initialization along any injective register embedding `f`. -/ -theorem initFresh_rename (C : RelSubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} - (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) - (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) - (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : - ∃ z, ExecWithin w (P.initFresh.rename f) x z C.initTime ∧ - z.regs (f P.frontierReg) = base + C.space ∧ - C.rep s₀ (z.comap f) ∧ - C.potential (z.comap f) ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ - (∀ j ∉ Set.range f, z.regs j = x.regs j) := by - obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initFresh_exec hx hfr hpar hz - refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, - fun j hj => Data.extend_regs_of_notMem y x hj⟩ - · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space - rw [Data.comap_extend hf] - exact hfr' - · rw [Data.comap_extend hf] - exact hrep - · rw [Data.comap_extend hf] - exact hpot - -/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`initClear`) -/ - -/-- Auxiliary deterministic `SubroutineDS` sharing `C`'s initialization fields so -`zeroFillCode_exec` is reused without duplicating its loop proof. -/ -private def toInitSubroutineDS (C : RelSubroutineDS P D w base params s₀) : - SubroutineDS P - { State := PUnit.{1}, Op := PUnit.{1}, Result := fun _ => PUnit.{1}, - step := fun _ _ => (⟨⟩, ⟨⟩), valid := fun _ _ => False } - w base params ⟨⟩ where - space := C.space - spanTime := C.spanTime - setupTime := C.setupTime - initPotential := C.initPotential - charge _ _ := 0 - rep _ x := C.rep s₀ x - potential := C.potential - encode _ _ := 0 - argMatch _ _ := True - decode _ _ := some ⟨⟩ - bounded := C.bounded - rep_unique _ _ := rfl - arguments_fit _ hv := False.elim hv - encode_matches _ _ := trivial - rep_supported := C.rep_supported - potential_supported := C.potential_supported - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval := C.incCur_eval - span_exec := C.span_exec - setup_exec := C.setup_exec - step_exec _ _ hv := False.elim hv - -/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` -instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : RelSubroutineDS P D w base params s₀) {x₁ : Data r_ds} - (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) - (hfr₁ : x₁.regs P.frontierReg = base + C.space) - (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : - ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ - x₂.Bounded w ∧ - x₂.regs P.baseReg = base ∧ - x₂.regs P.frontierReg = base + C.space ∧ - (∀ j, x₂.regs (P.initParams j) = params j) ∧ - (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ - Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := - C.toInitSubroutineDS.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ - -/-- Execute `P.initClear` on arbitrary dirty memory in `C.initTime + 4 * C.space + 6` -instructions. -/ -theorem initClear_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} - (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) - (hpar : ∀ j, x.regs (P.initParams j) = params j) : - ∃ y, ExecWithin w P.initClear x y (C.initTime + 4 * C.space + 6) ∧ - y.regs P.frontierReg = base + C.space ∧ - C.rep s₀ y ∧ - C.potential y ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem y.mem := by - obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar - obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := - C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ - obtain ⟨y, he₃, hfr₃, hrep, hpot, hmod₃⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ - rw [hm₁] at hmod₂ - have hmod : Modifies (Block.mk base C.space).footprint x.mem y.mem := by - simpa only [Finset.union_idempotent] using hmod₂.trans hmod₃ - exact ⟨y, (he₁.seq (he₂.seq he₃)).mono (by unfold initTime; omega), hfr₃, hrep, hpot, hmod⟩ - -/-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ -theorem initClear_rename (C : RelSubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} - (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) - (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : - ∃ z, ExecWithin w (P.initClear.rename f) x z (C.initTime + 4 * C.space + 6) ∧ - z.regs (f P.frontierReg) = base + C.space ∧ - C.rep s₀ (z.comap f) ∧ - C.potential (z.comap f) ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ - (∀ j ∉ Set.range f, z.regs j = x.regs j) := by - obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initClear_exec hx hfr hpar - refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, - fun j hj => Data.extend_regs_of_notMem y x hj⟩ - · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space - rw [Data.comap_extend hf] - exact hfr' - · rw [Data.comap_extend hf] - exact hrep - · rw [Data.comap_extend hf] - exact hpot - -/-! ### Projection 4: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ +/-! ### Projection 2: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ /-- Execute `P.code.rename f` inline when arguments already match `(x.comap f).regs`, witnessing a permitted outcome `(s', a)` with `D.step s o s' a`. -/ @@ -831,30 +459,7 @@ theorem call_rename (C : RelSubroutineDS P D w base params s₀) exact ⟨z, s', a, t_prep + t_op, hprep.seq he_op, hstep, hdec, hrep_z, hmod_z, hregs_z, by rw [hp_eq] at hpot_z; omega⟩ -/-! ### Projection 5: Caller-to-DS bidirectional framing (`frame_caller`) -/ - -/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS -registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from -`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ -theorem frame_caller (C : RelSubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} {s : D.State} {x x' : Data r_caller} {F : Finset ℕ} - (hr : C.rep s (x.comap f)) - (hb' : (x'.comap f).Bounded w) - (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) - (hdisj : Disjoint (Block.mk base C.space).footprint F) - (hmem : Modifies F x.mem x'.mem) : - C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by - have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => - hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) - exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, - C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ - -/-! ### Projection 6: Flat-program operation-sequence execution (`RunsRelOps`, `run_ops_exact_*`) -/ - -/-- Default register-preparation function overwriting `P.input` registers with `C.encode o`. -/ -def prepareRegs (C : RelSubroutineDS P D w base params s₀) (o : D.Op) - (regs : Fin r_ds → ℕ) : Fin r_ds → ℕ := - fun i => if P.input i then C.encode o i else regs i +/-! ### Projection 3: Flat-program operation-sequence execution (`RunsRelOps`, `run_ops_exact_*`) -/ /-- Sequential flat-program execution of `ops : List D.Op` by repeated calls to `P.opProg`: each call prepares input registers via `prepRegs`, runs `P.opProg` from `pc = 0` to `halt`, @@ -937,12 +542,12 @@ theorem seq_runsRelOps (C : RelSubroutineDS P D w base params s₀) D.totalCost (fun st o => C.charge st o + 1) s tr + C.potential x := by refine C.seq_runsRelOps_of_prep C.prepareRegs ?_ ?_ C.encode_matches ops hr hv · intro s' o regs hv_o hregs i - simp only [prepareRegs] + simp only [SubroutineBase.prepareRegs] split_ifs with hi · exact C.arguments_fit o hv_o i hi · exact hregs i · intro o regs i hi - simp [prepareRegs, hi] + simp [SubroutineBase.prepareRegs, hi] /-- Standalone flat-program operation-sequence theorem with `O(1)` fresh initialization on zeroed `[base, base + C.space)`. -/ diff --git a/Complexity/Computability/WordRAM/Subroutine.lean b/Complexity/Computability/WordRAM/Subroutine.lean index 2dae6c8e7..bb0969dcc 100644 --- a/Complexity/Computability/WordRAM/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Subroutine.lean @@ -40,11 +40,12 @@ A data structure is packaged once as a `SubroutineCode B r_ds k_init` and certif ## Main definitions -* `Cslib.WordRAM.SubroutineCode`, `Cslib.WordRAM.SubroutineDS`: uniform relocatable machine code and - unified standalone / inline specification and resource certificate for a Word-RAM data structure. +* `Cslib.WordRAM.SubroutineCode`, `Cslib.WordRAM.SubroutineBase`, `Cslib.WordRAM.SubroutineDS`: + uniform relocatable machine code, step-independent initialization/framing bundle, and unified + standalone / inline specification and resource certificate for a Word-RAM data structure. * `Cslib.WordRAM.SubroutineCode.initFresh`, `Cslib.WordRAM.SubroutineCode.initClear`: `O(1)` fresh initialization on zeroed memory and generic `O(space)` dirty-memory re-initialization. -* `Cslib.WordRAM.SubroutineDS.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: +* `Cslib.WordRAM.SubroutineBase.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: standalone `.standard`-basis `DynamicProgram` and `DynamicCertificate` projections. ## Main results @@ -53,8 +54,8 @@ A data structure is packaged once as a `SubroutineCode B r_ds k_init` and certif * `Cslib.WordRAM.SubroutineDS.toAmortized`, `Cslib.WordRAM.SubroutineDS.toBound`, `Cslib.WordRAM.SubroutineDS.run_client_amortized`, `Cslib.WordRAM.SubroutineDS.run_client_bound`: standalone amortized and worst-case instruction bounds over `Basis.standard`. -* `Cslib.WordRAM.SubroutineDS.initFresh_rename`, `Cslib.WordRAM.SubroutineDS.initClear_rename`, - `Cslib.WordRAM.SubroutineDS.call_rename`, `Cslib.WordRAM.SubroutineDS.frame_caller`: relocated +* `Cslib.WordRAM.SubroutineBase.initFresh_rename`, `Cslib.WordRAM.SubroutineBase.initClear_rename`, + `Cslib.WordRAM.SubroutineDS.call_rename`, `Cslib.WordRAM.SubroutineBase.frame_caller`: relocated inline initialization, inline operation calls, and bidirectional register/memory framing. ## References @@ -209,10 +210,18 @@ end SubroutineCode universe u v z -/-- Unified standalone and subroutine specification/resource certificate for `P` realizing `D` at -word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. -/ -structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) - (D : Stateful.Spec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) where +/-- Step-independent specification, representation, calling-convention, and initialization bundle +for `P : SubroutineCode B r_ds k_init` over a state type `State`, operation type `Op`, observation +family `Result`, and admissibility predicate `valid` at word width `w`, base address `base`, +runtime parameters `params`, and initial state `s₀`. + +Extended by `SubroutineDS` (for deterministic `Stateful.Spec`) and `RelSubroutineDS` (for +relational `Stateful.RelSpec`) so that initialization (`initFresh`, `zeroFillCode`, `initClear`), +register relocation, standalone `toDynamicProgram`, and bidirectional framing (`frame_caller`) are +stated and proved once. -/ +structure SubroutineBase {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) + {State : Type u} {O : Type v} (Result : O → Type z) (valid : State → O → Prop) + (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : State) where /-- Contiguous memory footprint length `[base, base + space)`. -/ space : ℕ /-- Instruction bound for `P.allocSpan`. -/ @@ -222,25 +231,23 @@ structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_d /-- Upper bound on initial concrete potential after initialization. -/ initPotential : ℕ /-- Amortized instruction charge per operation (excluding trailing standalone `halt`). -/ - charge : D.State → D.Op → ℕ + charge : State → O → ℕ /-- Representation invariant linking mathematical state `s` to concrete `Data r_ds`. -/ - rep : D.State → Data r_ds → Prop + rep : State → Data r_ds → Prop /-- Concrete integer potential depending only on `Data r_ds`. -/ potential : Data r_ds → ℕ /-- Standalone register encoding for `DynamicProgram.prepare`. -/ - encode : D.Op → Fin r_ds → ℕ + encode : O → Fin r_ds → ℕ /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ - argMatch : D.Op → (Fin r_ds → ℕ) → Prop + argMatch : O → (Fin r_ds → ℕ) → Prop /-- Output decoder reading typed operation observations from final registers. -/ - decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) + decode : (o : O) → (Fin r_ds → ℕ) → Option (Result o) /-- Represented configurations are word-bounded. -/ bounded : ∀ {s x}, rep s x → x.Bounded w - /-- Concrete machine data determines at most one represented abstract state. -/ - rep_unique : ∀ {s₁ s₂ x}, rep s₁ x → rep s₂ x → s₁ = s₂ /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ - arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) + arguments_fit : ∀ {s} (o), valid s o → ∀ i, P.input i = true → Fits w (encode o i) /-- Full standalone `prepare` delivery satisfies `argMatch`. -/ - encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), + encode_matches : ∀ (o : O) (regs : Fin r_ds → ℕ), argMatch o (fun i => if P.input i then encode o i else regs i) /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. -/ @@ -274,30 +281,25 @@ structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_d ∃ y, ExecWithin w P.initSetup x₁ y setupTime ∧ y.regs P.frontierReg = base + space ∧ rep s₀ y ∧ potential y ≤ initPotential ∧ Modifies (Block.mk base space).footprint x₁.mem y.mem - /-- Execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`. -/ - step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), - rep s x → D.valid s o → argMatch o x.regs → - ∃ y n, Exec w P.code x y n ∧ - decode o y.regs = some (D.step s o).2 ∧ - rep (D.step s o).1 y ∧ - Modifies (Block.mk base space).footprint x.mem y.mem ∧ - n + potential y ≤ charge s o + potential x -namespace SubroutineDS +namespace SubroutineBase variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} - {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} - {params : Fin k_init → ℕ} {s₀ : D.State} + {P : SubroutineCode B r_ds k_init} {State : Type u} {O : Type v} + {Result : O → Type z} {valid : State → O → Prop} {params : Fin k_init → ℕ} {s₀ : State} /-- Total instruction bound for `P.initFresh`. -/ -def initTime (C : SubroutineDS P D w base params s₀) : ℕ := +def initTime (C : SubroutineBase P Result valid w base params s₀) : ℕ := C.spanTime + C.setupTime -/-! ### Projection 1: Basis lifting and standalone `DynamicProgram` / `Stateful` theorems -/ +/-- Default register-preparation function overwriting `P.input` registers with `C.encode o`. -/ +def prepareRegs (C : SubroutineBase P Result valid w base params s₀) (o : O) + (regs : Fin r_ds → ℕ) : Fin r_ds → ℕ := + fun i => if P.input i then C.encode o i else regs i -/-- Lift a `SubroutineDS` certificate along any defined-evaluation basis simulation. -/ -def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) : - SubroutineDS (P.mapSimulates φ) D w base params s₀ where +/-- Lift a `SubroutineBase` bundle along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : SubroutineBase P Result valid w base params s₀) : + SubroutineBase (P.mapSimulates φ) Result valid w base params s₀ where space := C.space spanTime := C.spanTime setupTime := C.setupTime @@ -309,7 +311,6 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) argMatch := C.argMatch decode := C.decode bounded := C.bounded - rep_unique := C.rep_unique arguments_fit := C.arguments_fit encode_matches := C.encode_matches rep_supported := C.rep_supported @@ -323,110 +324,19 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) setup_exec hx₁ hb hfr hpar hz := let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ - step_exec o hr hv harg := - let ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg - ⟨y, n, he.mapSimulates φ, hdec, hrep, hmod, hpot⟩ - -/-- Lift an exact-basis `SubroutineDS` certificate to `Basis.standard`. -/ -abbrev toStandard {P : SubroutineCode .exact r_ds k_init} - (C : SubroutineDS P D w base params s₀) : - SubroutineDS P.toStandard D w base params s₀ := - C.mapSimulates .exactStandard - -section Standard - -variable {P_std : SubroutineCode .standard r_ds k_init} - (C : SubroutineDS P_std D w base params s₀) -/-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicProgram`. -/ -def toDynamicProgram : DynamicProgram D r_ds where +/-- Project a `.standard`-basis `SubroutineBase` to a standalone `DynamicProgram`. -/ +def toDynamicProgram {P_std : SubroutineCode .standard r_ds k_init} + (C : SubroutineBase P_std Result valid w base params s₀) : + DynamicProgram { State := Empty, Op := O, Result := Result, step := Empty.elim } r_ds where code := P_std.code output := P_std.output input := P_std.input encode := C.encode decode := C.decode -/-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicCertificate`. -/ -def toCertificate : DynamicCertificate C.toDynamicProgram w where - rep := C.rep - bounded := C.bounded - arguments_fit := C.arguments_fit - execute {s x} o hr hv := by - have hb' : (C.toDynamicProgram.prepare o x).Bounded w := - C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toDynamicProgram.prepare o x) := - C.rep_supported - (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) - (fun _ _ => rfl) hb' hr - have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := - C.encode_matches o x.regs - obtain ⟨y, n, he, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg - exact ⟨y, n, he, hdec, hrep⟩ - -/-- Project a `.standard`-basis `SubroutineDS` to a standalone `Stateful.Amortized` certificate, -charging `+ 1` per operation for `DynamicProgram`'s trailing `halt` instruction. -/ -theorem toAmortized : - Stateful.Amortized C.toCertificate.realization - DynamicProgram.OperationTrace.instructions - (fun _ x => C.potential x) - (fun s o => C.charge s o + 1) where - step {s x o a y trace} hr hv he := by - have hb' : (C.toDynamicProgram.prepare o x).Bounded w := - C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toDynamicProgram.prepare o x) := - C.rep_supported - (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) - (fun _ _ => rfl) hb' hr - have hp_eq : C.potential (C.toDynamicProgram.prepare o x) = C.potential x := - C.potential_supported - (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) - (fun _ _ => rfl) hr - have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := - C.encode_matches o x.regs - obtain ⟨z, m, hz, _, _, _, hpot⟩ := C.step_exec o hr' hv harg - obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz - rw [hinst] - rw [hp_eq] at hpot - omega - -/-- When `C.potential` is identically zero, `SubroutineDS` projects directly to a worst-case -`Stateful.Bound` certificate. -/ -theorem toBound (hpot : ∀ x, C.potential x = 0) : - Stateful.Bound C.toCertificate.realization - DynamicProgram.OperationTrace.instructions - (fun s o => C.charge s o + 1) := by - intro s x o a y trace hr hv he - have h := C.toAmortized.step hr hv he - simp only [hpot] at h - omega - -/-- Standalone adaptive-client amortized bound for any admissible client program `p`. -/ -theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} - (hr : C.rep s x) (hv : D.ValidHistory s (D.runClient s p).2.2) : - ∃ y traces, - Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ - C.rep (D.runClient s p).2.1 y ∧ - Stateful.total DynamicProgram.OperationTrace.instructions traces + C.potential y ≤ - D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 + C.potential x := - C.toAmortized.exists_executes p hr hv - -/-- Standalone adaptive-client worst-case bound when `C.potential` is identically zero. -/ -theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) - {s : D.State} {x : Data r_ds} (hr : C.rep s x) - (hv : D.ValidHistory s (D.runClient s p).2.2) : - ∃ y traces, - Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ - C.rep (D.runClient s p).2.1 y ∧ - Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ - D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 := - Stateful.Bound.exists_executes (C.toBound hpot) p hr hv - -end Standard - -/-! ### Projection 2: Fresh `O(1)` initialization and register relocation -/ - /-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ -theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} +theorem initFresh_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) (hpar : ∀ j, x.regs (P.initParams j) = params j) (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : @@ -446,7 +356,7 @@ theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} exact ⟨y, he₁.seq he₂, hfr₂, hrep, hpot, hmod⟩ /-- Relocated `O(1)` fresh initialization along any injective register embedding `f`. -/ -theorem initFresh_rename (C : SubroutineDS P D w base params s₀) +theorem initFresh_rename (C : SubroutineBase P Result valid w base params s₀) {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) @@ -468,11 +378,9 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) · rw [Data.comap_extend hf] exact hpot -/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`initClear`) -/ - /-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data r_ds} +theorem zeroFillCode_exec (C : SubroutineBase P Result valid w base params s₀) {x₁ : Data r_ds} (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) (hfr₁ : x₁.regs P.frontierReg = base + C.space) (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : @@ -618,7 +526,7 @@ theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data /-- Execute `P.initClear` on **arbitrary dirty memory** in `C.initTime + 4 * C.space + 6` instructions. -/ -theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} +theorem initClear_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) (hpar : ∀ j, x.regs (P.initParams j) = params j) : ∃ y, ExecWithin w P.initClear x y (C.initTime + 4 * C.space + 6) ∧ @@ -636,7 +544,7 @@ theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} exact ⟨y, (he₁.seq (he₂.seq he₃)).mono (by unfold initTime; omega), hfr₃, hrep, hpot, hmod⟩ /-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ -theorem initClear_rename (C : SubroutineDS P D w base params s₀) +theorem initClear_rename (C : SubroutineBase P Result valid w base params s₀) {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : @@ -657,7 +565,154 @@ theorem initClear_rename (C : SubroutineDS P D w base params s₀) · rw [Data.comap_extend hf] exact hpot -/-! ### Projection 4: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ +/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS +registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from +`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ +theorem frame_caller (C : SubroutineBase P Result valid w base params s₀) + {f : Fin r_ds → Fin r_caller} {s : State} {x x' : Data r_caller} {F : Finset ℕ} + (hr : C.rep s (x.comap f)) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hmem : Modifies F x.mem x'.mem) : + C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by + have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => + hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) + exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ + +end SubroutineBase + +/-- Unified standalone and subroutine specification/resource certificate for `P` realizing `D` at +word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. -/ +structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) + (D : Stateful.Spec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) + extends SubroutineBase P D.Result D.valid w base params s₀ where + /-- Concrete machine data determines at most one represented abstract state. -/ + rep_unique : ∀ {s₁ s₂ x}, rep s₁ x → rep s₂ x → s₁ = s₂ + /-- Execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`. -/ + step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), + rep s x → D.valid s o → argMatch o x.regs → + ∃ y n, Exec w P.code x y n ∧ + decode o y.regs = some (D.step s o).2 ∧ + rep (D.step s o).1 y ∧ + Modifies (Block.mk base space).footprint x.mem y.mem ∧ + n + potential y ≤ charge s o + potential x + +namespace SubroutineDS + +variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-! ### Projection 1: Basis lifting and standalone `DynamicProgram` / `Stateful` theorems -/ + +/-- Lift a `SubroutineDS` certificate along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) : + SubroutineDS (P.mapSimulates φ) D w base params s₀ where + toSubroutineBase := C.toSubroutineBase.mapSimulates φ + rep_unique := C.rep_unique + step_exec o hr hv harg := + let ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + ⟨y, n, he.mapSimulates φ, hdec, hrep, hmod, hpot⟩ + +/-- Lift an exact-basis `SubroutineDS` certificate to `Basis.standard`. -/ +abbrev toStandard {P : SubroutineCode .exact r_ds k_init} + (C : SubroutineDS P D w base params s₀) : + SubroutineDS P.toStandard D w base params s₀ := + C.mapSimulates .exactStandard + +section Standard + +variable {P_std : SubroutineCode .standard r_ds k_init} + (C : SubroutineDS P_std D w base params s₀) + +/-- Project a `.standard`-basis `SubroutineDS` to its underlying standalone `DynamicProgram`. -/ +def toDynamicProgram : DynamicProgram D r_ds where + code := P_std.code + output := P_std.output + input := P_std.input + encode := C.encode + decode := C.decode + +/-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicCertificate`. -/ +def toCertificate : DynamicCertificate C.toDynamicProgram w where + rep := C.rep + bounded := C.bounded + arguments_fit := C.arguments_fit + execute {s x} o hr hv := by + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨y, n, he, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg + exact ⟨y, n, he, hdec, hrep⟩ + +/-- Project a `.standard`-basis `SubroutineDS` to a standalone `Stateful.Amortized` certificate, +charging `+ 1` per operation for `DynamicProgram`'s trailing `halt` instruction. -/ +theorem toAmortized : + Stateful.Amortized C.toCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun _ x => C.potential x) + (fun s o => C.charge s o + 1) where + step {s x o a y trace} hr hv he := by + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have hp_eq : C.potential (C.toDynamicProgram.prepare o x) = C.potential x := + C.potential_supported + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hr + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨z, m, hz, _, _, _, hpot⟩ := C.step_exec o hr' hv harg + obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz + rw [hinst] + rw [hp_eq] at hpot + omega + +/-- When `C.potential` is identically zero, `SubroutineDS` projects directly to a worst-case +`Stateful.Bound` certificate. -/ +theorem toBound (hpot : ∀ x, C.potential x = 0) : + Stateful.Bound C.toCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun s o => C.charge s o + 1) := by + intro s x o a y trace hr hv he + have h := C.toAmortized.step hr hv he + simp only [hpot] at h + omega + +/-- Standalone adaptive-client amortized bound for any admissible client program `p`. -/ +theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} + (hr : C.rep s x) (hv : D.ValidHistory s (D.runClient s p).2.2) : + ∃ y traces, + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ + C.rep (D.runClient s p).2.1 y ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces + C.potential y ≤ + D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 + C.potential x := + C.toAmortized.exists_executes p hr hv + +/-- Standalone adaptive-client worst-case bound when `C.potential` is identically zero. -/ +theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) + {s : D.State} {x : Data r_ds} (hr : C.rep s x) + (hv : D.ValidHistory s (D.runClient s p).2.2) : + ∃ y traces, + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ + C.rep (D.runClient s p).2.1 y ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ + D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 := + Stateful.Bound.exists_executes (C.toBound hpot) p hr hv + +end Standard + +/-! ### Projection 2: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ /-- Execute `P.code.rename f` inline when arguments already match `(x.comap f).regs`. -/ theorem code_rename (C : SubroutineDS P D w base params s₀) @@ -706,24 +761,6 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) exact ⟨z, t_prep + t_op, hprep.seq he_op, hdec, hrep_z, hmod_z, hregs_z, by rw [hp_eq] at hpot_z; omega⟩ -/-! ### Projection 5: Caller-to-DS bidirectional framing (`frame_caller`) -/ - -/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS -registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from -`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ -theorem frame_caller (C : SubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} {s : D.State} {x x' : Data r_caller} {F : Finset ℕ} - (hr : C.rep s (x.comap f)) - (hb' : (x'.comap f).Bounded w) - (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) - (hdisj : Disjoint (Block.mk base C.space).footprint F) - (hmem : Modifies F x.mem x'.mem) : - C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by - have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => - hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) - exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, - C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ - end SubroutineDS end Cslib.WordRAM diff --git a/Complexity/Foundations/Semantics/Stateful/Realization.lean b/Complexity/Foundations/Semantics/Stateful/Realization.lean index 43afd738d..5761f7fb1 100644 --- a/Complexity/Foundations/Semantics/Stateful/Realization.lean +++ b/Complexity/Foundations/Semantics/Stateful/Realization.lean @@ -39,14 +39,16 @@ namespace Cslib.Stateful universe u v w x y -variable {D E : Spec.{u, v, w}} {C τ : Type x} {α β : Type y} +variable {S : PFunctor.{v, w}} {D E : Spec.{u, v, w}} {C τ : Type x} {α β : Type y} + +instance : Coe Spec.{u, v, w} PFunctor.{v, w} := ⟨Spec.signature⟩ /-- A backend chooses its own execution evidence, independently of a resource measure. -/ -abbrev Execution (D : Spec.{u, v, w}) (C Trace : Type x) := - C → (o : D.Op) → D.Result o → C → Trace → Prop +abbrev Execution (S : PFunctor.{v, w}) (C Trace : Type x) := + C → (o : S.A) → S.B o → C → Trace → Prop /-- Clients collect native execution evidence in call order; no costs occur in this semantics. -/ -inductive Executes (exec : Execution D C τ) : D.Client α → C → α → C → List τ → Prop +inductive Executes (exec : Execution S C τ) : S.FreeM α → C → α → C → List τ → Prop | pure (a c) : Executes exec (.pure a) c a c [] | call {o k c a c' trace b c'' traces} : exec c o a c' trace → Executes exec (k a) c' b c'' traces → @@ -97,12 +99,12 @@ end Realization /-- Execution of a sequential composition splits its native trace at the call boundary. -/ -theorem executes_bind_iff (exec : Execution D C τ) (p : D.Client α) - (k : α → D.Client β) {c c' : C} {b : β} {ts : List τ} : +theorem executes_bind_iff (exec : Execution S C τ) (p : S.FreeM α) + (k : α → S.FreeM β) {c c' : C} {b : β} {ts : List τ} : Executes exec (p.bind k) c b c' ts ↔ ∃ a mid front back, Executes exec p c a mid front ∧ Executes exec (k a) mid b c' back ∧ ts = front ++ back := by - induction p generalizing c ts with + induction p using PFunctor.FreeM.recOn generalizing c ts with | pure a => constructor · intro h; exact ⟨a, c, [], ts, .pure a c, h, rfl⟩ diff --git a/Complexity/Foundations/Semantics/Stateful/Relational.lean b/Complexity/Foundations/Semantics/Stateful/Relational.lean index 40d7761e7..722f77f02 100644 --- a/Complexity/Foundations/Semantics/Stateful/Relational.lean +++ b/Complexity/Foundations/Semantics/Stateful/Relational.lean @@ -52,9 +52,8 @@ deterministic counterparts on `D`. * `Cslib.Stateful.RelSpec.ClientRun`, `Cslib.Stateful.RelSpec.DemonicValid`, `Cslib.Stateful.RelSpec.Hoare`, `Cslib.Stateful.RelSpec.CostHoare`: adaptive client runs, demonic admissibility, and compositional Hoare / CostHoare triples. -* `Cslib.Stateful.RelExecution`, `Cslib.Stateful.RelExecutes`, - `Cslib.Stateful.RelRealization`, `Cslib.Stateful.RelBound`, `Cslib.Stateful.RelAmortized`: - native execution traces, angelic representation simulation, and potential-method certificates. +* `Cslib.Stateful.RelRealization`, `Cslib.Stateful.RelBound`, `Cslib.Stateful.RelAmortized`: + angelic representation simulation and potential-method certificates over `Execution` / `Executes`. ## Main results @@ -127,6 +126,19 @@ theorem exists_step [D.Total] {s : D.State} {o : D.Op} (hv : D.valid s o) : /-- The polynomial operation signature `⟨D.Op, D.Result⟩`. -/ abbrev signature : PFunctor.{v, w} := ⟨D.Op, D.Result⟩ +instance : Coe RelSpec.{u, v, w} PFunctor.{v, w} := ⟨RelSpec.signature⟩ + +/-- Project the operation signature `(D.Op, D.Result)` of a relational specification to a +canonical `Stateful.Spec` with `State := Empty`, allowing signature-only constructions over +`Stateful.Spec` (such as `WordRAM.DynamicProgram`) to apply directly to `D : RelSpec`. -/ +abbrev toSpec : Spec.{0, v, w} where + State := Empty + Op := D.Op + Result := D.Result + step s := s.elim + +instance : Coe RelSpec.{u, v, w} Spec.{0, v, w} := ⟨RelSpec.toSpec⟩ + /-- An adaptive client over `D`: each continuation may branch on the observed operation result. -/ abbrev Client (α : Type y) : Type (max v w y) := D.signature.FreeM α @@ -159,8 +171,6 @@ abbrev Spec.toRelSpec (D : Spec.{u, v, w}) : RelSpec.{u, v, w} where step s o s' a := (s', a) = D.step s o valid := D.valid -instance : Coe Spec.{u, v, w} RelSpec.{u, v, w} := ⟨Spec.toRelSpec⟩ - namespace Spec variable (D : Spec.{u, v, w}) @@ -858,74 +868,14 @@ theorem toRelSpec_costHoare_iff (charge : D.State → D.Op → ℕ) (pre : D.Sta end Spec -/-! ## Native backend executions (`RelExecution`, `RelExecutes`), `RelRealization`, `RelAmortized` --/ - -/-- A concrete backend execution relation for a relational specification `D : RelSpec`. -/ -abbrev RelExecution (D : RelSpec.{u, v, w}) (C Trace : Type x) := - C → (o : D.Op) → D.Result o → C → Trace → Prop - -/-- Adaptive client execution collecting native execution evidence in call order over a -relational specification `D : RelSpec`. -/ -inductive RelExecutes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - (exec : RelExecution D C τ) : D.Client α → C → α → C → List τ → Prop - | pure (a c) : RelExecutes exec (.pure a) c a c [] - | call {o k c a c' trace b c'' traces} : - exec c o a c' trace → RelExecutes exec (k a) c' b c'' traces → - RelExecutes exec (.liftBind o k) c b c'' (trace :: traces) - -/-- Sequential client execution splits its native trace at the call boundary. -/ -theorem relExecutes_bind_iff {D : RelSpec.{u, v, w}} {C τ : Type x} {α β : Type y} - (exec : RelExecution D C τ) (p : D.Client α) (k : α → D.Client β) - {c c' : C} {b : β} {ts : List τ} : - RelExecutes exec (p.bind k) c b c' ts ↔ - ∃ a mid front back, RelExecutes exec p c a mid front ∧ - RelExecutes exec (k a) mid b c' back ∧ ts = front ++ back := by - induction p generalizing c ts with - | pure a => - constructor - · intro h; exact ⟨a, c, [], ts, .pure a c, h, rfl⟩ - · rintro ⟨a', mid, front, back, hp, hk, ht⟩ - cases hp - simpa only [ht, List.nil_append, PFunctor.FreeM.bind] using hk - | liftBind o cont ih => - change RelExecutes exec (.liftBind o fun a => (cont a).bind k) c b c' ts ↔ _ - constructor - · intro h - cases h with - | call ho rest => - obtain ⟨a, mid, front, back, hp, hk, ht⟩ := (ih _).mp rest - exact ⟨a, mid, _, back, .call ho hp, hk, by simp [ht]⟩ - · rintro ⟨a, mid, front, back, hp, hk, ht⟩ - cases hp with - | call ho rest => - have hrest := (ih _).mpr ⟨a, mid, _, back, rest, hk, rfl⟩ - have h := RelExecutes.call (k := fun a => (cont a).bind k) ho hrest - simpa only [ht, List.cons_append] using h - -private theorem Spec.toRelSpec_relExecutes_imp {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : Execution D C τ} {p : D.toRelSpec.Client α} {c : C} {a : α} {c' : C} - {traces : List τ} (h : RelExecutes (D := D.toRelSpec) exec p c a c' traces) : - Executes exec p c a c' traces := by - induction h with - | pure a c => exact .pure a c - | call ho _ ih => exact .call ho ih - -/-- On `D.toRelSpec`, `RelExecutes` coincides with `Executes`. -/ -theorem Spec.toRelSpec_relExecutes_iff {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} - (exec : Execution D C τ) (p : D.Client α) (c : C) (a : α) (c' : C) (traces : List τ) : - RelExecutes (D := D.toRelSpec) exec p c a c' traces ↔ Executes exec p c a c' traces := by - refine ⟨Spec.toRelSpec_relExecutes_imp, fun h => ?_⟩ - induction h with - | pure a c => exact RelExecutes.pure (D := D.toRelSpec) a c - | call ho _ ih => exact RelExecutes.call (D := D.toRelSpec) ho ih +/-! ## Angelic backend realizations (`RelRealization`, `RelBound`, `RelAmortized`) -/ /-- Angelic semantic realization of a relational specification `D : RelSpec`: * `total`: every represented state and admissible operation admits a native execution; * `correct`: every native execution from a represented state on an admissible operation realizes *some* permitted abstract post-state `s'` with `D.step s o s' a ∧ rep s' c'`. -/ structure RelRealization (D : RelSpec.{u, v, w}) {C τ : Type x} - (exec : RelExecution D C τ) where + (exec : Execution D C τ) where /-- Representation invariant linking abstract states to concrete backend states. -/ rep : D.State → C → Prop /-- Existence of a native execution for every represented state and admissible operation. -/ @@ -937,13 +887,13 @@ structure RelRealization (D : RelSpec.{u, v, w}) {C τ : Type x} namespace RelRealization variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} (R : RelRealization D exec) + {exec : Execution D C τ} (R : RelRealization D exec) /-- From related states, every native execution of a demonically admissible client realizes a valid `ClientRun` ending in a related state. -/ theorem correct_client {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List τ} (hr : R.rep s c) (hv : D.DemonicValid s p) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by induction he generalizing s with | pure a c => @@ -959,7 +909,7 @@ theorem correct_client {p : D.Client α} {s : D.State} {c c' : C} {a : α} valid `ClientRun` ending in a related state. -/ theorem exists_executes (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces s' tr, Executes exec p c a c' traces ∧ D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by induction hv generalizing c with | pure s a => @@ -986,17 +936,17 @@ def Realization.toRelRealization {D : Spec.{u, v, w}} {C τ : Type x} /-- Every native execution of a verified demonic `Hoare` client satisfies its abstract postcondition in a related final state. -/ theorem RelSpec.Hoare.realize {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} - {exec : RelExecution D C Trace} (R : RelRealization D exec) + {exec : Execution D C Trace} (R : RelRealization D exec) {pre : D.State → Prop} {post : α → D.State → Prop} {p : D.Client α} (h : D.Hoare pre p post) {s : D.State} {c c' : C} {a : α} {traces : List Trace} - (hp : pre s) (hr : R.rep s c) (he : RelExecutes exec p c a c' traces) : + (hp : pre s) (hr : R.rep s c) (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' := by obtain ⟨hv, hpost⟩ := h s hp obtain ⟨s', tr, hrun, hr', _⟩ := R.correct_client hr hv he exact ⟨s', tr, hrun, hr', hpost a s' tr hrun⟩ /-- Per-operation worst-case resource certificate over a relational realization `R`. -/ -def RelBound {D : RelSpec.{u, v, w}} {C τ : Type x} {exec : RelExecution D C τ} +def RelBound {D : RelSpec.{u, v, w}} {C τ : Type x} {exec : Execution D C τ} (R : RelRealization D exec) (weight : τ → ℕ) (charge : D.State → D.Op → ℕ) : Prop := ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → weight trace ≤ charge s o @@ -1013,7 +963,7 @@ for every native operation execution from `R.rep s c` on an admissible `o`, ther permitted abstract post-state `s'` satisfying `D.step s o s' a`, `R.rep s' c'`, and the single-step potential inequality `weight trace + potential s' c' ≤ charge s o + potential s c`. -/ structure RelAmortized {D : RelSpec.{u, v, w}} {C Trace : Type x} - {exec : RelExecution D C Trace} (R : RelRealization D exec) (weight : Trace → ℕ) + {exec : Execution D C Trace} (R : RelRealization D exec) (weight : Trace → ℕ) (potential : D.State → C → ℕ) (charge : D.State → D.Op → ℕ) : Prop where /-- Angelic single-step potential inequality witnessing a permitted post-state `s'`. -/ step : ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → @@ -1023,19 +973,19 @@ structure RelAmortized {D : RelSpec.{u, v, w}} {C Trace : Type x} namespace RelAmortized variable {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} - {exec : RelExecution D C Trace} {R : RelRealization D exec} + {exec : Execution D C Trace} {R : RelRealization D exec} {weight : Trace → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} (A : RelAmortized R weight potential charge) include A in /-- **Amortized telescoping along adaptive clients (`executes_le`).** -For any demonically admissible client `p` and native execution `RelExecutes exec p c a c' traces`, +For any demonically admissible client `p` and native execution `Executes exec p c a c' traces`, there exist a final abstract state `s'` and transcript `tr` such that `D.ClientRun s p a s' tr`, `R.rep s' c'`, and `total weight traces + potential s' c' ≤ D.totalCost charge s tr + potential s c`. -/ theorem executes_le {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List Trace} (hr : R.rep s c) (hv : D.DemonicValid s p) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces + potential s' c' ≤ @@ -1052,16 +1002,13 @@ theorem executes_le {p : D.Client α} {s : D.State} {c c' : C} {a : α} simp only [total_cons, RelSpec.totalCost_cons] omega -/-- Alias for `RelAmortized.executes_le` matching `Amortized.client`. -/ -abbrev client := @executes_le - include A in /-- **Existence, correctness, and amortized bound (`run_client_amortized`).** From related initial states, every demonically admissible client `p` admits a native execution satisfying `D.ClientRun`, `R.rep`, and the telescoping potential bound. -/ theorem run_client_amortized (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces s' tr, Executes exec p c a c' traces ∧ D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces + potential s' c' ≤ @@ -1070,17 +1017,6 @@ theorem run_client_amortized (p : D.Client α) {s : D.State} {c : C} obtain ⟨s', tr, hrun, hr', hlen, hpot⟩ := A.executes_le hr hv he exact ⟨a, c', traces, s', tr, he, hrun, hr', hlen, hpot⟩ -include A in -/-- Alias for `RelAmortized.run_client_amortized` matching `Amortized.exists_executes`. -/ -theorem exists_executes (p : D.Client α) {s : D.State} {c : C} - (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ - D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ - traces.length = tr.length ∧ - total weight traces + potential s' c' ≤ - D.totalCost charge s tr + potential s c := - A.run_client_amortized p hr hv - /-- A worst-case `RelBound` is a zero-potential `RelAmortized` certificate. -/ theorem of_relBound (B : RelBound R weight charge) : RelAmortized R weight (fun _ _ => 0) charge where @@ -1104,7 +1040,7 @@ theorem Amortized.toRelAmortized {D : Spec.{u, v, w}} {C Trace : Type x} namespace RelBound variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} (B : RelBound R weight charge) @@ -1112,7 +1048,7 @@ include B in /-- Under a worst-case `RelBound`, every native execution of a demonically admissible client has total weight at most `D.totalCost charge s tr` for some valid `ClientRun` transcript `tr`. -/ theorem client {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List τ} - (hr : R.rep s c) (hv : D.DemonicValid s p) (he : RelExecutes exec p c a c' traces) : + (hr : R.rep s c) (hv : D.DemonicValid s p) (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces ≤ D.totalCost charge s tr := by @@ -1124,7 +1060,7 @@ include B in meeting the total charge bound. -/ theorem exists_executes (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces s' tr, Executes exec p c a c' traces ∧ D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces ≤ D.totalCost charge s tr := by @@ -1137,13 +1073,13 @@ end RelBound /-- Transport a demonic `CostHoare` triple through a `RelAmortized` certificate to bound native execution weight plus final potential by `budget s + potential s c`. -/ theorem RelSpec.CostHoare.realize_amortized {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} (A : RelAmortized R weight potential charge) (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' ∧ total weight traces + potential s' c' ≤ budget s + potential s c := by obtain ⟨hv, hpost⟩ := h.1 s hs @@ -1153,13 +1089,13 @@ theorem RelSpec.CostHoare.realize_amortized {D : RelSpec.{u, v, w}} {C τ : Type /-- Transport a demonic `CostHoare` triple through a worst-case `RelBound` certificate. -/ theorem RelSpec.CostHoare.realize {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} (B : RelBound R weight charge) (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by obtain ⟨s', _, _, hr', hpost, hpot⟩ := h.realize_amortized (RelAmortized.of_relBound B) hs hr he @@ -1167,13 +1103,13 @@ theorem RelSpec.CostHoare.realize {D : RelSpec.{u, v, w}} {C τ : Type x} {α : /-- Existence of a native execution satisfying a demonic `CostHoare` contract under `RelBound`. -/ theorem RelSpec.CostHoare.exists_executes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} (B : RelBound R weight charge) (h : D.CostHoare charge pre p post budget) {s : D.State} {c : C} (hs : pre s) (hr : R.rep s c) : - ∃ a c' traces, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces, Executes exec p c a c' traces ∧ (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by obtain ⟨a, c', traces, _, _, he, _, _, _⟩ := R.exists_executes p hr (h.1 s hs).1 exact ⟨a, c', traces, he, h.realize B hs hr he⟩ diff --git a/ComplexityTests/StatefulRelational.lean b/ComplexityTests/StatefulRelational.lean index 8d3373331..1b6980b9c 100644 --- a/ComplexityTests/StatefulRelational.lean +++ b/ComplexityTests/StatefulRelational.lean @@ -21,7 +21,7 @@ backend realization and amortized analysis (`RelRealization`, `RelBound`, `RelAm @[expose] public section -namespace Cslib.Stateful.Tests +namespace ComplexityTests.StatefulRelational open Cslib.Stateful @@ -144,7 +144,7 @@ theorem allocTwoAndFreeFirst_hoare : /-- Concrete first-fit implementation of `TwoSlotAlloc`: `alloc` chooses slot `0` when free and slot `1` otherwise, recording `1` unit of execution trace cost per operation. -/ -def firstFitExec : RelExecution TwoSlotAlloc (Bool × Bool) ℕ +def firstFitExec : Execution TwoSlotAlloc (Bool × Bool) ℕ | c, .alloc, i, c', n => n = 1 ∧ if c.1 = false then i = 0 ∧ c' = (true, c.2) @@ -199,7 +199,7 @@ theorem firstFitAmortized : `RelAmortized.run_client_amortized` and `RelSpec.CostHoare.realize_amortized`. -/ theorem firstFit_allocTwoAndFreeFirst_executes : ∃ v c' traces s' tr, - RelExecutes firstFitExec allocTwoAndFreeFirst (false, false) v c' traces ∧ + Executes firstFitExec allocTwoAndFreeFirst (false, false) v c' traces ∧ TwoSlotAlloc.ClientRun (false, false) allocTwoAndFreeFirst v s' tr ∧ firstFitRealization.rep s' c' ∧ (v.1 ≠ v.2 ∧ (if v.1 = 0 then s'.1 = false else s'.2 = false)) ∧ @@ -238,4 +238,4 @@ theorem counterTwice_relHoare : subst hs exact ⟨⟨trivial, trivial, trivial⟩, rfl, rfl⟩ -end Cslib.Stateful.Tests +end ComplexityTests.StatefulRelational diff --git a/ComplexityTests/WordRAMRelSubroutine.lean b/ComplexityTests/WordRAMRelSubroutine.lean index 1eecc6baa..b8aeee2fd 100644 --- a/ComplexityTests/WordRAMRelSubroutine.lean +++ b/ComplexityTests/WordRAMRelSubroutine.lean @@ -13,7 +13,7 @@ public import Complexity.Computability.WordRAM.RelSubroutine /-! # Tests for Relational Word-RAM Subroutine Realizations -Concrete non-vacuous tests for `Cslib.WordRAM.RelSubroutineDS`, `RelDynamicProgram`, +Concrete non-vacuous tests for `Cslib.WordRAM.RelSubroutineDS`, `DynamicProgram`, `RelDynamicCertificate`, `SubroutineDS.toRelSubroutineDS`, `RelSubroutineDS.toStandard`, `RelSubroutineDS.call_rename`, `RelSubroutineDS.run_client_amortized`, `RelSubroutineDS.run_ops_exact_fresh`, and `RelSubroutineDS.run_ops_exact_reusable` on a @@ -22,7 +22,7 @@ genuinely nondeterministic two-slot allocator specification. @[expose] public section -namespace Cslib.WordRAM.Tests +namespace ComplexityTests.WordRAMRelSubroutine open Cslib Cslib.Stateful Cslib.WordRAM @@ -176,8 +176,8 @@ theorem allocPair_demonicValid : theorem twoSlot_run_client_amortized (w base : ℕ) (h2 : Fits w 2) {x : Data 6} (hr : (twoSlotSubroutine w base h2).rep (false, false) x) : ∃ a y traces s' tr, - Stateful.RelExecutes - ((twoSlotSubroutine w base h2).toStandard.toRelDynamicProgram.ExecOp w) + Stateful.Executes + ((twoSlotSubroutine w base h2).toStandard.toDynamicProgram.ExecOp w) allocPair x a y traces ∧ TwoSlotAlloc.ClientRun (false, false) allocPair a s' tr ∧ (twoSlotSubroutine w base h2).rep s' y ∧ @@ -348,4 +348,4 @@ def detCounter2RelSubroutine (w base : ℕ) (h2 : Fits w 2) : RelSubroutineDS twoSlotCode DetCounter2.toRelSpec w base Fin.elim0 0 := (detCounter2Subroutine w base h2).toRelSubroutineDS -end Cslib.WordRAM.Tests +end ComplexityTests.WordRAMRelSubroutine From 888918d31bf556a04215a38559542b30a557bc44 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:10:19 -0400 Subject: [PATCH 06/18] chore(deps): track CSLib main and Lean rc2 Advance CSLib to 8a8ad748 and synchronize Mathlib, the documentation generator, Verso, and all three lockfiles with Lean v4.36.0-rc2. Keep the root and documentation projects on the same inherited dependency revisions. --- docbuild/lake-manifest.json | 34 +++++++++++++++++----------------- docbuild/lakefile.toml | 2 +- guide/lake-manifest.json | 30 +++++++++++++++--------------- guide/lakefile.toml | 2 +- lake-manifest.json | 26 +++++++++++++------------- lakefile.toml | 2 +- lean-toolchain | 2 +- 7 files changed, 49 insertions(+), 49 deletions(-) diff --git a/docbuild/lake-manifest.json b/docbuild/lake-manifest.json index 3609d26b5..a1f5e4692 100644 --- a/docbuild/lake-manifest.json +++ b/docbuild/lake-manifest.json @@ -5,10 +5,10 @@ "type": "git", "subDir": null, "scope": "leanprover", - "rev": "0656b7ebd305f089c8d6bd8684a0797f19414cc2", + "rev": "20afd4ddabd2ef38ffb1a8b930ee0f375e3f75e8", "name": "«doc-gen4»", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": false, "configFile": "lakefile.lean"}, {"type": "path", @@ -23,7 +23,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "e1faac1842cb10b3a68c644643b6e3568766343c", + "rev": "f582e4bc64d6f43e6e711f89b09567d2088fab8d", "name": "leansqlite", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -33,7 +33,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "34d2efffcb6dc8d5f3537cf6a7f88bfaa999ec49", + "rev": "c09cb1e4f1e6e1c231a1175a12b681b075054a1b", "name": "Cli", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -43,7 +43,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "a24bcde05a7971144ba0c0b5b1d60c3a0f0bc4af", + "rev": "91ece4f8d8f93aac566e247874f6b27b57249f1b", "name": "UnicodeBasic", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -53,7 +53,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "78f28e27259a4b2f5a8199800334cebd92730f3d", + "rev": "cdf8963f8b4c9a5593c5aaa9fa4f2759f8dec1c1", "name": "BibtexQuery", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -73,27 +73,27 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "rev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "name": "cslib", "manifestFile": "lake-manifest.json", - "inputRev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "inputRev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "inherited": true, "configFile": "lakefile.toml"}, {"url": "https://github.com/leanprover-community/mathlib4", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "625d8914199bc567cca9b91f90e8e5afafa5ae51", + "rev": "9d581dfff2e41de989072ea63abf3daf69311520", "name": "mathlib", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.lean"}, {"url": "https://github.com/leanprover-community/plausible", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "c2d7840c65673533e15b499f25beecdc6227f7d8", + "rev": "129d1ba9a16bc0660f46c79e9fb041ef5d5d80de", "name": "plausible", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -103,7 +103,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "9d39f9bb0c30aa2f6200808805ff16e6ce950890", + "rev": "c61286b7b63cbfd563b28e11e81d0a18233a044a", "name": "LeanSearchClient", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -113,7 +113,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "b3cc36c295aa5e3e3c0197b09a6556298e710447", + "rev": "b8cc5dd0a36233b88a6ed5a7dbf1207d09338c60", "name": "importGraph", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -123,7 +123,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "17137fad38a2cc782ff00b323a13c3a991d085ab", + "rev": "16259c0d07d61ee37d999d05b5cf96338b8daa11", "name": "proofwidgets", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -133,7 +133,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "cbdde3aa2665915fb286c48bcd4cdf885c629511", + "rev": "7d11bfa88558f8d3d90afdf6bd877ed70001498d", "name": "aesop", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -143,7 +143,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "6d26f8bb4b2bb78f929ad160feac508866821316", + "rev": "b0256621cb88178e9ad932a53633077b3029e621", "name": "Qq", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -153,7 +153,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "4453b52d2468f4c346a8711d728c0742e7483c9b", + "rev": "d4c39b228d4a06f3b00748c2c445a7b9f056a6e3", "name": "batteries", "manifestFile": "lake-manifest.json", "inputRev": "main", diff --git a/docbuild/lakefile.toml b/docbuild/lakefile.toml index ed4689727..b9c4092df 100644 --- a/docbuild/lakefile.toml +++ b/docbuild/lakefile.toml @@ -12,4 +12,4 @@ path = ".." [[require]] scope = "leanprover" name = "doc-gen4" -rev = "v4.36.0-rc1" +rev = "v4.36.0-rc2" diff --git a/guide/lake-manifest.json b/guide/lake-manifest.json index b8da10f21..3f19786a0 100644 --- a/guide/lake-manifest.json +++ b/guide/lake-manifest.json @@ -5,10 +5,10 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f7a4dc6867f8df27778b2b320763ec9e3b7be070", + "rev": "71a2035b44e7f54a797ce799fd1b03cdd3a44684", "name": "verso", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": false, "configFile": "lakefile.lean"}, {"type": "path", @@ -23,7 +23,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "34d2efffcb6dc8d5f3537cf6a7f88bfaa999ec49", + "rev": "c09cb1e4f1e6e1c231a1175a12b681b075054a1b", "name": "Cli", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -33,7 +33,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "d22e76ef3d2a925132315af902ea432233b4a44a", + "rev": "758bff3509b8e90ca82e37aa5de81de3c9e70fc0", "name": "illuminate", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -43,7 +43,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "c2d7840c65673533e15b499f25beecdc6227f7d8", + "rev": "129d1ba9a16bc0660f46c79e9fb041ef5d5d80de", "name": "plausible", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -73,27 +73,27 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "rev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "name": "cslib", "manifestFile": "lake-manifest.json", - "inputRev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "inputRev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "inherited": true, "configFile": "lakefile.toml"}, {"url": "https://github.com/leanprover-community/mathlib4", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "625d8914199bc567cca9b91f90e8e5afafa5ae51", + "rev": "9d581dfff2e41de989072ea63abf3daf69311520", "name": "mathlib", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.lean"}, {"url": "https://github.com/leanprover-community/LeanSearchClient", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "9d39f9bb0c30aa2f6200808805ff16e6ce950890", + "rev": "c61286b7b63cbfd563b28e11e81d0a18233a044a", "name": "LeanSearchClient", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -103,7 +103,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "b3cc36c295aa5e3e3c0197b09a6556298e710447", + "rev": "b8cc5dd0a36233b88a6ed5a7dbf1207d09338c60", "name": "importGraph", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -113,7 +113,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "17137fad38a2cc782ff00b323a13c3a991d085ab", + "rev": "16259c0d07d61ee37d999d05b5cf96338b8daa11", "name": "proofwidgets", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -123,7 +123,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "cbdde3aa2665915fb286c48bcd4cdf885c629511", + "rev": "7d11bfa88558f8d3d90afdf6bd877ed70001498d", "name": "aesop", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -133,7 +133,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "6d26f8bb4b2bb78f929ad160feac508866821316", + "rev": "b0256621cb88178e9ad932a53633077b3029e621", "name": "Qq", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -143,7 +143,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "4453b52d2468f4c346a8711d728c0742e7483c9b", + "rev": "d4c39b228d4a06f3b00748c2c445a7b9f056a6e3", "name": "batteries", "manifestFile": "lake-manifest.json", "inputRev": "main", diff --git a/guide/lakefile.toml b/guide/lakefile.toml index fb7601692..f62d2a91a 100644 --- a/guide/lakefile.toml +++ b/guide/lakefile.toml @@ -21,7 +21,7 @@ path = ".." [[require]] name = "verso" git = "https://github.com/leanprover/verso" -rev = "v4.36.0-rc1" +rev = "v4.36.0-rc2" [[lean_lib]] name = "ComplexityGuide" diff --git a/lake-manifest.json b/lake-manifest.json index a4892b4fc..a2c1eaffd 100644 --- a/lake-manifest.json +++ b/lake-manifest.json @@ -5,27 +5,27 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "rev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "name": "cslib", "manifestFile": "lake-manifest.json", - "inputRev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "inputRev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "inherited": false, "configFile": "lakefile.toml"}, {"url": "https://github.com/leanprover-community/mathlib4", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "625d8914199bc567cca9b91f90e8e5afafa5ae51", + "rev": "9d581dfff2e41de989072ea63abf3daf69311520", "name": "mathlib", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.lean"}, {"url": "https://github.com/leanprover-community/plausible", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "c2d7840c65673533e15b499f25beecdc6227f7d8", + "rev": "129d1ba9a16bc0660f46c79e9fb041ef5d5d80de", "name": "plausible", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -35,7 +35,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "9d39f9bb0c30aa2f6200808805ff16e6ce950890", + "rev": "c61286b7b63cbfd563b28e11e81d0a18233a044a", "name": "LeanSearchClient", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -45,7 +45,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "b3cc36c295aa5e3e3c0197b09a6556298e710447", + "rev": "b8cc5dd0a36233b88a6ed5a7dbf1207d09338c60", "name": "importGraph", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -55,7 +55,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "17137fad38a2cc782ff00b323a13c3a991d085ab", + "rev": "16259c0d07d61ee37d999d05b5cf96338b8daa11", "name": "proofwidgets", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -65,7 +65,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "cbdde3aa2665915fb286c48bcd4cdf885c629511", + "rev": "7d11bfa88558f8d3d90afdf6bd877ed70001498d", "name": "aesop", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -75,7 +75,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "6d26f8bb4b2bb78f929ad160feac508866821316", + "rev": "b0256621cb88178e9ad932a53633077b3029e621", "name": "Qq", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -85,7 +85,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "4453b52d2468f4c346a8711d728c0742e7483c9b", + "rev": "d4c39b228d4a06f3b00748c2c445a7b9f056a6e3", "name": "batteries", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -95,10 +95,10 @@ "type": "git", "subDir": null, "scope": "leanprover", - "rev": "34d2efffcb6dc8d5f3537cf6a7f88bfaa999ec49", + "rev": "c09cb1e4f1e6e1c231a1175a12b681b075054a1b", "name": "Cli", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.toml"}], "name": "complexitylib", diff --git a/lakefile.toml b/lakefile.toml index ce7b8cd77..00f09a874 100644 --- a/lakefile.toml +++ b/lakefile.toml @@ -33,7 +33,7 @@ autoImplicit = false [[require]] name = "cslib" git = "https://github.com/leanprover/cslib" -rev = "f4a472560f78c59bbedd795fe041ec507f7d0582" +rev = "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad" [[lean_lib]] name = "Complexity" diff --git a/lean-toolchain b/lean-toolchain index 2afd38716..0fc433692 100644 --- a/lean-toolchain +++ b/lean-toolchain @@ -1 +1 @@ -leanprover/lean4:v4.36.0-rc1 +leanprover/lean4:v4.36.0-rc2 From 9a1b75416b7723a073411f0ee9a7d2ccfdd49ec9 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:10:24 -0400 Subject: [PATCH 07/18] refactor(probability): use generic infinite-sum lemmas Replace 187 deprecated ENNReal wrapper uses across 55 proof files with their generic counterparts. The exact mapping is: - ENNReal.tsum_le_tsum -> tsum_le_tsum (94) - ENNReal.tsum_add -> tsum_add (54) - ENNReal.tsum_eq_zero -> tsum_eq_zero_iff (14) - ENNReal.summable -> CompleteLattice.summable (6) - ENNReal.tsum_comm -> tsum_comm (6) - ENNReal.tsum_prod' -> tsum_prod' (5) - ENNReal.le_tsum -> le_tsum (5) - ENNReal.tsum_eq_add_tsum_ite -> tsum_eq_add_tsum_ite (2) - ENNReal.tsum_sigma -> tsum_sigma (1) The old wrappers invoke these same theorems specialized to ENNReal. All five le_tsum calls already pass the function by name. No statements or definitions change. --- .../Algorithms/WordRAM/Karger/FastCut.lean | 2 +- .../Algorithms/WordRAM/Karger/Graph.lean | 6 +-- .../Algorithms/WordRAM/KargerMinCut.lean | 2 +- .../Algorithms/WordRAM/KargerStein.lean | 6 +-- .../Computability/Complexity/Classes/IP.lean | 2 +- .../Classes/IP/GraphNonIsomorphism.lean | 2 +- .../Complexity/Classes/IP/Guessing.lean | 6 +-- .../Complexity/Classes/IP/OptimalProver.lean | 8 ++-- .../Complexity/Classes/IP/Repetition.lean | 6 +-- .../LogSpace/Randomized/Amplification.lean | 2 +- .../Complexity/Classes/MIP/OptimalProver.lean | 4 +- .../Complexity/FineGrained/PPSZ/Basic.lean | 2 +- .../Complexity/FineGrained/PPZ.lean | 2 +- .../Computability/Interactive/Plain.lean | 4 +- .../Computability/Kolmogorov/AverageCase.lean | 4 +- .../Computability/PCP/TwoProverGame.lean | 2 +- .../WordRAM/Randomized/Expected.lean | 12 ++--- .../CramerShoup/ProjectiveHash.lean | 4 +- .../Computational/CramerShoup/Scheme.lean | 2 +- .../Crypto/Computational/DiffieHellman.lean | 16 +++---- .../Computational/DiffieHellman/Order.lean | 4 +- .../Encryption/Authenticated.lean | 14 +++--- .../Encryption/EncryptThenMAC.lean | 8 ++-- .../Crypto/Computational/GMW/Soundness.lean | 6 +-- Complexity/Crypto/Computational/MAC.lean | 8 ++-- .../Computational/NaorReingold/Security.lean | 10 ++-- .../Computational/OneWay/LiuPassConverse.lean | 2 +- .../Computational/OneWayToSignature.lean | 6 +-- .../Crypto/Computational/PRGToOneWay.lean | 2 +- .../Computational/PedersenCommitment.lean | 8 ++-- .../Pseudoentropy/MaskedExtraction.lean | 2 +- Complexity/Crypto/Game.lean | 20 ++++---- Complexity/Crypto/Game/UntilBad.lean | 16 +++---- Complexity/Crypto/Primitives/PRG/NextBit.lean | 4 +- .../Crypto/Protocols/Cipher/EvenMansour.lean | 6 +-- .../Crypto/Protocols/Cipher/LubyRackoff.lean | 48 +++++++++---------- .../Crypto/Protocols/MAC/CarterWegman.lean | 16 +++---- .../Protocols/SigmaProtocol/FiatShamir.lean | 6 +-- .../Probabilistic/BoundedUniform.lean | 6 +-- .../Languages/Probabilistic/Expect.lean | 8 ++-- .../Probabilistic/Oracle/Interaction.lean | 2 +- .../Probabilistic/RandomPermutation.lean | 8 ++-- .../Languages/Probabilistic/UntilBad.lean | 10 ++-- .../Languages/Probabilistic/Vector.lean | 2 +- Complexity/Probability/Conditioning.lean | 6 +-- .../Probability/CorrelatedSampling.lean | 2 +- Complexity/Probability/Forking.lean | 2 +- Complexity/Probability/IID.lean | 2 +- Complexity/Probability/MinEntropy.lean | 2 +- Complexity/Probability/Minimax.lean | 4 +- Complexity/Probability/PMF/Bind.lean | 14 +++--- Complexity/Probability/PMF/Event.lean | 16 +++---- Complexity/Probability/PMF/Map.lean | 2 +- .../ProductConditioning/SideInformation.lean | 2 +- Complexity/Probability/RejectionSampling.lean | 4 +- 55 files changed, 186 insertions(+), 186 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/Karger/FastCut.lean b/Complexity/Algorithms/WordRAM/Karger/FastCut.lean index a216c2ae8..30a2afa92 100644 --- a/Complexity/Algorithms/WordRAM/Karger/FastCut.lean +++ b/Complexity/Algorithms/WordRAM/Karger/FastCut.lean @@ -658,7 +658,7 @@ theorem two_branch_hit_prob_ge {α β γ : Type} (m₁ : ProbComp α) (m₂ : α have ha : a ∈ m₁.support := (ProbComp.support_eval m₁) ▸ ha_eval have ha_hit : a ∈ hit₁ := not_not.mp ha_not_miss rw [PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_eq_zero.mpr fun b => ?_ + refine tsum_eq_zero_iff.mpr fun b => ?_ by_cases hb_eval : b ∈ (m₂ a).eval.support · have hb : b ∈ (m₂ a).support := (ProbComp.support_eval (m₂ a)) ▸ hb_eval have hmem : f a b ∈ hit := hprop₁ a ha ha_hit b hb diff --git a/Complexity/Algorithms/WordRAM/Karger/Graph.lean b/Complexity/Algorithms/WordRAM/Karger/Graph.lean index 9afa1d805..c03da4013 100644 --- a/Complexity/Algorithms/WordRAM/Karger/Graph.lean +++ b/Complexity/Algorithms/WordRAM/Karger/Graph.lean @@ -1107,7 +1107,7 @@ theorem toOuterMeasure_bind_ge_mul_ennreal {α β : Type*} (p : PMF α) (kernel s₁ * s₂ ≤ p.toOuterMeasure pre * s₂ := mul_le_mul_of_nonneg_right hp zero_le _ ≤ (p.bind kernel).toOuterMeasure event := by rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ pre · simp only [Set.indicator_of_mem ha] by_cases hs : a ∈ p.support @@ -1131,7 +1131,7 @@ theorem toOuterMeasure_bind_le_mul_ennreal {α β : Type*} (p : PMF α) (kernel (p.bind kernel).toOuterMeasure failure ≤ p.toOuterMeasure stillFailing * q₂ := by rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases hs : a ∈ p.support · by_cases ha : a ∈ stillFailing · simp only [Set.indicator_of_mem ha] @@ -1146,7 +1146,7 @@ valid on arbitrary types without measurability hypotheses. -/ theorem toOuterMeasure_add_compl_ennreal {α : Type*} (p : PMF α) (s : Set α) : p.toOuterMeasure s + p.toOuterMeasure sᶜ = 1 := by classical - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] have hpt : (fun x => s.indicator p x + sᶜ.indicator p x) = p := by ext x by_cases hx : x ∈ s <;> simp [hx] diff --git a/Complexity/Algorithms/WordRAM/KargerMinCut.lean b/Complexity/Algorithms/WordRAM/KargerMinCut.lean index b3ef003a5..c07c84a3c 100644 --- a/Complexity/Algorithms/WordRAM/KargerMinCut.lean +++ b/Complexity/Algorithms/WordRAM/KargerMinCut.lean @@ -1181,7 +1181,7 @@ theorem ampLoop_spec {w sb n R : ℕ} {W : Fin n → Fin n → ℕ} {mem₀ : obtain ⟨m_r, hm_r, _, hsup_r, _⟩ := hnext z₁ hz₁ rw [hml_def, hm_r.codeComp_eq, PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - convert ENNReal.tsum_eq_zero.mpr _ + convert tsum_eq_zero_iff.mpr _ intro z₂ by_cases hz₂_eval : z₂ ∈ m_r.eval.support · have hz₂ : z₂ ∈ m_r.support := (ProbComp.support_eval m_r) ▸ hz₂_eval diff --git a/Complexity/Algorithms/WordRAM/KargerStein.lean b/Complexity/Algorithms/WordRAM/KargerStein.lean index 92368fc36..5a2a42a1a 100644 --- a/Complexity/Algorithms/WordRAM/KargerStein.lean +++ b/Complexity/Algorithms/WordRAM/KargerStein.lean @@ -168,7 +168,7 @@ private theorem toOuterMeasure_mono_support {α : Type} (m : ProbComp α) {S T : m.eval.toOuterMeasure S ≤ m.eval.toOuterMeasure T := by classical rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ S · simp only [Set.indicator_of_mem ha] by_cases hs : a ∈ m.support @@ -2684,7 +2684,7 @@ theorem fastCutNode_spec {n w : ℕ} {W₀ : Fin n → Fin n → ℕ} have hcompl0 : (ms2_fn z₂.1).eval.toOuterMeasure {zs | zs.1.regs retVal = minCut W₀}ᶜ = 0 := by rw [PMF.toOuterMeasure_apply] - refine ENNReal.tsum_eq_zero.mpr fun zs => ?_ + refine tsum_eq_zero_iff.mpr fun zs => ?_ by_cases hzs_ev : zs ∈ (ms2_fn z₂.1).eval.support · have hzs : zs ∈ (ms2_fn z₂.1).support := (ProbComp.support_eval (ms2_fn z₂.1)) ▸ hzs_ev @@ -3679,7 +3679,7 @@ theorem fastCutAmpLoop_spec {w sb n R : ℕ} {W : Fin n → Fin n → ℕ} {mem have hbv₁ : z₁.1.regs bestVal = minCut W := not_not.mp hhit obtain ⟨m_r, hm_r, _, hsup_r, _⟩ := hnext z₁ hz₁ rw [hml_def, hm_r.codeComp_eq, PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_eq_zero.mpr fun z₂ => ?_ + refine tsum_eq_zero_iff.mpr fun z₂ => ?_ by_cases hz₂_eval : z₂ ∈ m_r.eval.support · have hz₂ : z₂ ∈ m_r.support := (ProbComp.support_eval m_r) ▸ hz₂_eval have hbv₂ : z₂.1.regs bestVal = minCut W := diff --git a/Complexity/Computability/Complexity/Classes/IP.lean b/Complexity/Computability/Complexity/Classes/IP.lean index d2c73a5bc..d583e0f90 100644 --- a/Complexity/Computability/Complexity/Classes/IP.lean +++ b/Complexity/Computability/Complexity/Classes/IP.lean @@ -148,7 +148,7 @@ theorem acceptProb_eq_acceptProbWith (verifier : WordOracleComp Operation Bool) /-- The acceptance probability sums the masses of the accepting outcomes over final states. -/ theorem acceptProb_eq_tsum (verifier : WordOracleComp Operation Bool) (P : Prover Operation) : acceptProb verifier P = ∑' s, P.run verifier P.init (true, s) := by - rw [acceptProb, PMF.map_apply, ENNReal.tsum_prod', tsum_bool] + rw [acceptProb, PMF.map_apply, tsum_prod', tsum_bool] simp /-- Private coins drawn before any message average the acceptance probabilities of the diff --git a/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean b/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean index dfce7de33..659f372bf 100644 --- a/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean +++ b/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean @@ -413,7 +413,7 @@ private theorem card_filter_not_injective_div_le (n : ℕ) : private theorem tsum_map_mul {β γ : Type*} (μ : PMF β) (m : β → γ) (f : γ → ENNReal) : ∑' c, μ.map m c * f c = ∑' b, μ b * f (m b) := by simp only [PMF.map_apply, ENNReal.tsum_mul_right.symm] - rw [ENNReal.tsum_comm] + rw [tsum_comm] refine tsum_congr fun b => ?_ rw [tsum_eq_single (m b)] · simp diff --git a/Complexity/Computability/Complexity/Classes/IP/Guessing.lean b/Complexity/Computability/Complexity/Classes/IP/Guessing.lean index 6eaedc8da..02e8929d3 100644 --- a/Complexity/Computability/Complexity/Classes/IP/Guessing.lean +++ b/Complexity/Computability/Complexity/Classes/IP/Guessing.lean @@ -77,8 +77,8 @@ noncomputable def Prover.guessProb (P : Prover Unit) (s : P.State) (m : List Boo theorem Prover.guessProb_add_le_one (P : Prover Unit) (s : P.State) (m : List Bool) : P.guessProb s m false + P.guessProb s m true ≤ 1 := by simp only [Prover.guessProb, PMF.map_apply] - rw [← ENNReal.tsum_add, ← PMF.tsum_coe ((P.respond ((), m)).run s)] - refine ENNReal.tsum_le_tsum fun y => ?_ + rw [← tsum_add, ← PMF.tsum_coe ((P.respond ((), m)).run s)] + refine tsum_le_tsum fun y => ?_ by_cases h : List.take 1 y.1 = [false] · simp [h] · by_cases h' : List.take 1 y.1 = [true] <;> simp [h, h'] @@ -120,7 +120,7 @@ theorem acceptProb_guessingRound_le {message : Bool → ProbComp (List Bool)} { exact hclose _ fun m => PMF.coe_le_one _ _ _ = ∑' m, ProbComp.eval (message false) m * (P.guessProb P.init m false + P.guessProb P.init m true) + δ := by - rw [← add_assoc, ← ENNReal.tsum_add] + rw [← add_assoc, ← tsum_add] simp only [mul_add] _ ≤ 1 + δ := by gcongr diff --git a/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean b/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean index 3ad366c14..7fd4eec66 100644 --- a/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean +++ b/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean @@ -304,7 +304,7 @@ private theorem sum_acceptFrom_after_le (hF : IsBounded F D L) (P : Prover Unit) lia rw [Finset.sum_eq_zero fun m _ => ?_, add_zero] · split_ifs <;> simp - · refine ENNReal.tsum_eq_zero.mpr fun ys => ?_ + · refine tsum_eq_zero_iff.mpr fun ys => ?_ rw [after_append] unfold outcome at hnil rcases hah : after (F r) h with _ | ⟨b⟩ | ⟨q, next⟩ @@ -320,12 +320,12 @@ private theorem sum_acceptFrom_after_le (hF : IsBounded F D L) (P : Prover Unit) exact (Finset.sum_congr rfl fun r _ => by split_ifs <;> simp).le rw [Finset.sum_comm] refine Finset.sum_le_sum fun m _ => ?_ - rw [← Summable.tsum_finsetSum fun r _ => ENNReal.summable] + rw [← Summable.tsum_finsetSum fun r _ => CompleteLattice.summable] calc ∑' ys : List Bool × P.State, ∑ r, (P.respond ((), m)).run s ys * acceptFrom P ys.2 (after (F r) (h ++ [(m, ys.1)])) ≤ ∑' ys : List Bool × P.State, (P.respond ((), m)).run s ys * ((BitString.shortWords L).sup fun y => value F L n (h ++ [(m, y)]) : ℕ) := by - refine ENNReal.tsum_le_tsum fun ys => ?_ + refine tsum_le_tsum fun ys => ?_ rw [← Finset.mul_sum] refine mul_le_mul_right ?_ _ refine (ih (h ++ [(m, ys.1)]) (by @@ -395,7 +395,7 @@ private theorem sum_acceptFrom_optimalProver_after (hF : IsBounded F D L) (n : lia rw [Finset.sum_eq_zero fun m _ => ?_, add_zero] · split_ifs <;> simp - · refine ENNReal.tsum_eq_zero.mpr fun ys => ?_ + · refine tsum_eq_zero_iff.mpr fun ys => ?_ rw [after_append] unfold outcome at hnil rcases hah : after (F r) h with _ | ⟨b⟩ | ⟨q, next⟩ diff --git a/Complexity/Computability/Complexity/Classes/IP/Repetition.lean b/Complexity/Computability/Complexity/Classes/IP/Repetition.lean index 66023d528..77f8fb503 100644 --- a/Complexity/Computability/Complexity/Classes/IP/Repetition.lean +++ b/Complexity/Computability/Complexity/Classes/IP/Repetition.lean @@ -64,7 +64,7 @@ private theorem acceptProb_bind_ite (P : Prover Operation) acceptProb (do let ok ← round; if ok then rest else pure false) P = ∑' s, P.run round P.init (true, s) * acceptProb rest { P with init := s } := by simp only [acceptProb, Prover.run_bind, PMF.map_bind] - rw [PMF.bind_apply, ENNReal.tsum_prod', tsum_bool] + rw [PMF.bind_apply, tsum_prod', tsum_bool] simp only [Bool.false_eq_true, Bool.true_eq_false, ↓reduceIte, Prover.run_pure, PMF.pure_map, PMF.pure_apply, mul_zero, tsum_zero, zero_add] rfl @@ -81,7 +81,7 @@ theorem acceptProb_sequentialRepeat_le {round : WordOracleComp Operation Bool} { calc ∑' s, P.run round P.init (true, s) * acceptProb (sequentialRepeat k round) { P with init := s } ≤ ∑' s, P.run round P.init (true, s) * ε ^ k := - ENNReal.tsum_le_tsum fun s => mul_le_mul' le_rfl (ih _) + tsum_le_tsum fun s => mul_le_mul' le_rfl (ih _) _ = (∑' s, P.run round P.init (true, s)) * ε ^ k := ENNReal.tsum_mul_right _ ≤ ε * ε ^ k := by gcongr @@ -103,7 +103,7 @@ theorem le_acceptProb_sequentialRepeat {round : WordOracleComp Operation Bool} { gcongr exact (h s).trans_eq (acceptProb_eq_tsum round _) _ = ∑' s', P.run round s (true, s') * c ^ k := ENNReal.tsum_mul_right.symm - _ ≤ _ := ENNReal.tsum_le_tsum fun s' => mul_le_mul' le_rfl (ih s') + _ ≤ _ := tsum_le_tsum fun s' => mul_le_mul' le_rfl (ih s') /-- A constant number of sequential runs of a strict oracle PPT verifier is strict oracle PPT. -/ theorem isOraclePPTOn_sequentialRepeat [DecidableEq Operation] [Finite Operation] diff --git a/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean b/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean index 1c3606355..56fd01273 100644 --- a/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean +++ b/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean @@ -126,7 +126,7 @@ theorem eval_runClosed_orElse_apply (h₁ : first.toNTM.RunsInTime input t₁) exact congrArg _ (PMF.pure_apply_self _) · rw [add_zero] _ = q + ProbComp.eval (first.runClosed t₁ input) [true] * (1 - q) := by - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, tsum_ite_eq] + rw [tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, tsum_ite_eq] /-- **Rejections multiply under `orElse`.** Counted on coin strings, the probability that `first.orElse second` does not accept is the product of the probabilities that `first` and diff --git a/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean b/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean index ebf6834a9..0f1a57869 100644 --- a/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean +++ b/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean @@ -581,7 +581,7 @@ private theorem tsum_mul_le_bestOfList (μ : PMF α) (g : α → ℝ≥0∞) (de ∑' x, μ x * g x ≤ g (bestOfList g default l) := by calc ∑' x, μ x * g x ≤ ∑' x, μ x * g (bestOfList g default l) := by - refine ENNReal.tsum_le_tsum fun x => ?_ + refine tsum_le_tsum fun x => ?_ by_cases hx : x ∈ μ.support · exact mul_le_mul_right (le_bestOfList g default l (hsupp x hx)) (μ x) · rw [PMF.mem_support_iff, not_not] at hx @@ -618,7 +618,7 @@ theorem exists_deterministic_prover_family [Finite ι] [DecidableEq ι] {γ : Ty rw [run_eq_bind_piPresample (hF c) P (fun _ => d) (fun _ => le_rfl) (Prover.independent P).init, PMF.bind_apply] simp only [hc, ← ENNReal.tsum_mul_left] - rw [← Summable.tsum_finsetSum fun _ _ => ENNReal.summable] + rw [← Summable.tsum_finsetSum fun _ _ => CompleteLattice.summable] refine tsum_congr fun f => ?_ rw [Finset.mul_sum] exact Finset.sum_congr rfl fun c _ => by rw [mul_left_comm] diff --git a/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean b/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean index e47a1f654..a73035873 100644 --- a/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean +++ b/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean @@ -276,7 +276,7 @@ theorem ppz_round_apply_le {φ : CNF (Fin n)} {w : ℕ} (hw : 1 ≤ w) {σ : Fin (hσ : φ.eval σ = true) : PPZ.round φ (fun x => some (σ x)) ≤ round φ w (fun x => some (σ x)) := by rw [PPZ.round, round, PPZ.roundWith, PPZ.roundWith, PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun π => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun π => mul_le_mul_right ?_ _ rw [← PPZ.restrict_prefixSet_self σ π] rw [PPZ.runWith_apply_restrict (O := PPZ.forced φ) (fun _ hτ _ _ h => PPZ.forced_eq hσ hτ h) π le_rfl, PPZ.runWith_apply_restrict (O := implied φ w) (fun _ hτ _ _ h => implied_eq hσ hτ h) π diff --git a/Complexity/Computability/Complexity/FineGrained/PPZ.lean b/Complexity/Computability/Complexity/FineGrained/PPZ.lean index 29084952a..a7c4fa2c3 100644 --- a/Complexity/Computability/Complexity/FineGrained/PPZ.lean +++ b/Complexity/Computability/Complexity/FineGrained/PPZ.lean @@ -458,7 +458,7 @@ theorem runWith_apply_restrict {O : (Fin n → Option Bool) → Fin n → Option by_cases hsupp : τ ∈ (runWith O π i).support · rw [PMF.map_apply] convert mul_zero _ - refine ENNReal.tsum_eq_zero.mpr fun b => ite_eq_right fun e => hτ ?_ + refine tsum_eq_zero_iff.mpr fun b => ite_eq_right fun e => hτ ?_ funext y by_cases hy : y = x · subst hy diff --git a/Complexity/Computability/Interactive/Plain.lean b/Complexity/Computability/Interactive/Plain.lean index 23a157b84..822083780 100644 --- a/Complexity/Computability/Interactive/Plain.lean +++ b/Complexity/Computability/Interactive/Plain.lean @@ -201,9 +201,9 @@ private theorem rbrFrom_le : ∀ (s : Spec) (smp : PerChallenge ProbComp s) st' calc ∑' c, ProbComp.eval smp.1 c * _ ≤ ∑' c, ProbComp.eval smp.1 c * (A.indicator 1 c + PerChallenge.sum s ε.2) := - ENNReal.tsum_le_tsum fun c => by gcongr; exact hinner c + tsum_le_tsum fun c => by gcongr; exact hinner c _ = (ProbComp.eval smp.1).toOuterMeasure A + PerChallenge.sum s ε.2 := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun c => ?_ diff --git a/Complexity/Computability/Kolmogorov/AverageCase.lean b/Complexity/Computability/Kolmogorov/AverageCase.lean index 0ead74cf0..86083ba86 100644 --- a/Complexity/Computability/Kolmogorov/AverageCase.lean +++ b/Complexity/Computability/Kolmogorov/AverageCase.lean @@ -277,7 +277,7 @@ private theorem pmf_map_true_le_one_sub_of_imp_false {α : Type*} (q : PMF α) ( ((q.map P) true).toReal ≤ 1 - ((q.map Q) true).toReal := by have hle : (q.map P) true ≤ (q.map Q) false := by rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun v => ?_ + refine tsum_le_tsum fun v => ?_ by_cases hp : P v = true · have hq : Q v = false := hdisj v hp simp [hp, hq] @@ -292,7 +292,7 @@ private theorem pmf_map_true_le_of_imp_true {α : Type*} (q : PMF α) (P Q : α ((q.map P) true).toReal ≤ ((q.map Q) true).toReal := by apply ENNReal.toReal_mono (PMF.apply_ne_top _ _) rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun v => ?_ + refine tsum_le_tsum fun v => ?_ by_cases hp : P v = true · have hq : Q v = true := himp v hp simp [hp, hq] diff --git a/Complexity/Computability/PCP/TwoProverGame.lean b/Complexity/Computability/PCP/TwoProverGame.lean index d918d1eba..e742d8fc8 100644 --- a/Complexity/Computability/PCP/TwoProverGame.lean +++ b/Complexity/Computability/PCP/TwoProverGame.lean @@ -117,7 +117,7 @@ theorem winProb_shared_le_value [Nonempty A] [Nonempty B] {R : Type*} (ρ : PMF exact ENNReal.ofReal_le_ofReal (G.winProb_le_value (fun x => ha x r) (fun y => hb y r)) calc ∑' r, ρ r * (G.questions.map fun q => (q, ha q.1 r, hb q.2 r)).toOuterMeasure {z | G.accepts z.1.1 z.1.2 z.2.1 z.2.2} - ≤ ∑' r, ρ r * ENNReal.ofReal G.value := ENNReal.tsum_le_tsum fun r => by gcongr; exact hr r + ≤ ∑' r, ρ r * ENNReal.ofReal G.value := tsum_le_tsum fun r => by gcongr; exact hr r _ = ENNReal.ofReal G.value := by rw [ENNReal.tsum_mul_right, ρ.tsum_coe, one_mul] end Finite diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index 7d9e1844a..4f1881163 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -117,7 +117,7 @@ theorem evalRand_haltedSet_mono (w : ℕ) (p : Program B r) (s : State r) : rw [evalRand, runRand_add w p T 1, ProbComp.eval_bind] simp only [evalRand, runRand_succ, runRand_zero, bind_pure] rw [hstep, PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_le_tsum fun cfg => ?_ + refine tsum_le_tsum fun cfg => ?_ by_cases hcfg : cfg ∈ RandCfg.haltedSet r · have hmem := hcfg obtain ⟨v, d, rfl⟩ := hcfg @@ -139,7 +139,7 @@ theorem RandHaltsWithin.almostSurelyHalts {w T : ℕ} {p : Program B r} {s : Sta exact hnot (h cfg hmem) have hsum : (evalRand w p T s).toOuterMeasure (RandCfg.haltedSet r) + (evalRand w p T s).toOuterMeasure (RandCfg.haltedSet r)ᶜ = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert (evalRand w p T s).tsum_coe using 1 congr 1 funext cfg @@ -172,7 +172,7 @@ theorem expectedStepsFrom_faulted (w : ℕ) (p : Program B r) : private theorem tsum_nat_eq_zero_add (g : ℕ → ℝ≥0∞) : ∑' t : ℕ, g t = g 0 + ∑' t : ℕ, g (t + 1) := by - rw [ENNReal.tsum_eq_add_tsum_ite (f := g) 0] + rw [tsum_eq_add_tsum_ite (f := g) 0] congr 1 have hsh := Function.Injective.tsum_eq Nat.succ_injective (f := fun t => if t = 0 then 0 else g t) fun t ht => by @@ -199,7 +199,7 @@ theorem expectedStepsFrom_running (w : ℕ) (p : Program B r) (s : State r) : intro t rw [runRand_succ, ProbComp.eval_bind, PMF.toOuterMeasure_bind_apply] simp only [hsucc] - rw [ENNReal.tsum_comm] + rw [tsum_comm] simp only [ProbComp.expect, expectedStepsFrom, ENNReal.tsum_mul_left] /-- Under `RandFaultFree`, every horizon partitions probability `1` between `.running` and @@ -208,7 +208,7 @@ theorem RandFaultFree.running_add_halted_eq_one {w : ℕ} {p : Program B r} {s : (hf : RandFaultFree w p s) (t : ℕ) : (evalRand w p t s).toOuterMeasure {cfg | ∃ u, cfg = .running u} + (evalRand w p t s).toOuterMeasure (RandCfg.haltedSet r) = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert (evalRand w p t s).tsum_coe using 1 congr 1 funext cfg @@ -242,7 +242,7 @@ theorem AlmostSurelyHalts.of_expectedTime_ne_top {w : ℕ} {p : Program B r} {s have hinf : (⊤ : ℝ≥0∞) ≤ expectedTime w p s := by calc (⊤ : ℝ≥0∞) = ∑' _ : ℕ, (1 - supH) := (ENNReal.tsum_const_eq_top_of_ne_zero hpos.ne').symm - _ ≤ expectedTime w p s := ENNReal.tsum_le_tsum hle_run + _ ≤ expectedTime w p s := tsum_le_tsum hle_run exact htop (top_unique hinf) /-- Under `RandHaltsWithin w p s T`, the active-configuration tail sum vanishes from `T` onward and diff --git a/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean b/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean index 09e298a66..c52fe95e6 100644 --- a/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean +++ b/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean @@ -555,8 +555,8 @@ theorem uniform_projHash2_fiber_any_le (d : F.Desc) [DecidableEq (F.Carrier d)] have hor : (μ.map (fun τ₂ => (q :: qs).any (fun q' => hit q' τ₂))) true ≤ (μ.map (hit q)) true + (μ.map (fun τ₂ => qs.any (fun q' => hit q' τ₂))) true := by - simp only [List.any_cons, PMF.map_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun τ₂ => ?_ + simp only [List.any_cons, PMF.map_apply, ← tsum_add] + refine tsum_le_tsum fun τ₂ => ?_ cases hit q τ₂ <;> cases qs.any (fun q' => hit q' τ₂) <;> simp have hsingle : (μ.map (hit q)) true = (F.order d : ℝ≥0∞)⁻¹ := by have heq : diff --git a/Complexity/Crypto/Computational/CramerShoup/Scheme.lean b/Complexity/Crypto/Computational/CramerShoup/Scheme.lean index 58a4a0937..0e6744c8c 100644 --- a/Complexity/Crypto/Computational/CramerShoup/Scheme.lean +++ b/Complexity/Crypto/Computational/CramerShoup/Scheme.lean @@ -1008,7 +1008,7 @@ private theorem tsum_and_isCramerShoupBadDH_le rw [hpre, Bool.true_and, ← hvElem, ← hwElem, encodeProjHash2_encode, he_equiv τ] rw [hbad_eq] cases cond <;> cases F.encode d (e_equiv τ • F.generator d) == parts.2.2.2 <;> simp - exact (ENNReal.tsum_le_tsum fun τ => + exact (tsum_le_tsum fun τ => mul_le_mul_of_nonneg_left (hle τ) zero_le).trans (F.tsum_uniform_smul_generator_eq e_equiv parts.2.2.2) · simp only [Bool.not_eq_true] at hpre diff --git a/Complexity/Crypto/Computational/DiffieHellman.lean b/Complexity/Crypto/Computational/DiffieHellman.lean index 00a002fc8..2f8a35c45 100644 --- a/Complexity/Crypto/Computational/DiffieHellman.lean +++ b/Complexity/Crypto/Computational/DiffieHellman.lean @@ -173,14 +173,14 @@ theorem succProbability_discreteLogGame_le_cdhGame (setup : ℕ → PMF F.Desc) simp only [discreteLogGame, cdhGame, dlToCDHAdversary, ProbComp.eval_bind, ProbComp.eval_pure, PMF.monad_bind_eq_bind, PMF.monad_pure_eq_pure, PMF.bind_bind, PMF.pure_bind] rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ zero_le rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le rw [← PMF.bind_const (PMF.uniformOfFintype (ZMod (F.order d))) ((ProbComp.eval (adversary n (d, F.encode d (α • F.generator d)))).bind _)] rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le - refine ENNReal.tsum_le_tsum fun guess => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun guess => mul_le_mul_of_nonneg_left ?_ zero_le by_cases hα : (ofBits guess : ZMod (F.order d)) = α · simp only [hα, F.smul_encode, smul_smul, decide_true, le_rfl] · simp only [hα, decide_false, PMF.pure_apply, Bool.true_eq_false, ↓reduceIte, zero_le] @@ -301,7 +301,7 @@ theorem ddhIdealGame_cdhToDDHDistinguisher_true_le (setup : ℕ → PMF F.Desc) fun guess => PMF.pure (decide (guess = F.encode d (γ • F.generator d)))) true ≤ ∑' d, setup n d * (m : ℝ≥0∞)⁻¹ := by - refine ENNReal.tsum_le_tsum fun d => ?_ + refine tsum_le_tsum fun d => ?_ by_cases hd : d ∈ (setup n).support · refine mul_le_mul_of_nonneg_left ?_ zero_le have hord : (F.order d : ℝ≥0∞)⁻¹ ≤ (m : ℝ≥0∞)⁻¹ := @@ -315,8 +315,8 @@ theorem ddhIdealGame_cdhToDDHDistinguisher_true_le (setup : ℕ → PMF F.Desc) PMF.pure (decide (guess = F.encode d (γ • F.generator d)))) true ≤ ∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * ∑' β, PMF.uniformOfFintype (ZMod (F.order d)) β * (m : ℝ≥0∞)⁻¹ := by - refine ENNReal.tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le - refine ENNReal.tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le rw [PMF.bind_comm, PMF.bind_apply] calc ∑' guess, ProbComp.eval (adversary n (d, F.encode d (α • F.generator d), F.encode d (β • F.generator d))) guess * @@ -325,7 +325,7 @@ theorem ddhIdealGame_cdhToDDHDistinguisher_true_le (setup : ℕ → PMF F.Desc) ≤ ∑' guess, ProbComp.eval (adversary n (d, F.encode d (α • F.generator d), F.encode d (β • F.generator d))) guess * (m : ℝ≥0∞)⁻¹ := by - refine ENNReal.tsum_le_tsum fun guess => + refine tsum_le_tsum fun guess => mul_le_mul_of_nonneg_left ((F.uniform_decide_eq_encode_le d guess).trans hord) zero_le _ = (m : ℝ≥0∞)⁻¹ := by rw [ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] diff --git a/Complexity/Crypto/Computational/DiffieHellman/Order.lean b/Complexity/Crypto/Computational/DiffieHellman/Order.lean index b0a3dbafa..478cd484b 100644 --- a/Complexity/Crypto/Computational/DiffieHellman/Order.lean +++ b/Complexity/Crypto/Computational/DiffieHellman/Order.lean @@ -138,7 +138,7 @@ private theorem expected_inv_order_le_two_mul_advantage_ddh (F : PrimeOrderGroup (((F.order d - 1 : ℕ) : ℝ≥0∞) * ((F.order d : ℝ≥0∞)⁻¹ * (F.order d : ℝ≥0∞)⁻¹))) := by dsimp only [ddhRealGame, ddhZeroDistinguisher] simp only [ProbComp.eval_pure, PMF.monad_bind_eq_bind, PMF.bind_apply] - rw [← ENNReal.tsum_add] + rw [← tsum_add] refine tsum_congr fun d => ?_ have hstep : (∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * @@ -164,7 +164,7 @@ private theorem expected_inv_order_le_two_mul_advantage_ddh (F : PrimeOrderGroup 2 * ∑' d, setup n d * (((F.order d - 1 : ℕ) : ℝ≥0∞) * ((F.order d : ℝ≥0∞)⁻¹ * (F.order d : ℝ≥0∞)⁻¹)) := by rw [← ENNReal.tsum_mul_left] - refine ENNReal.tsum_le_tsum fun d => ?_ + refine tsum_le_tsum fun d => ?_ rw [← mul_assoc, mul_comm (2 : ℝ≥0∞) (setup n d), mul_assoc] refine mul_le_mul_of_nonneg_left ?_ zero_le have hq2 : 2 ≤ F.order d := (F.prime_order d).two_le diff --git a/Complexity/Crypto/Computational/Encryption/Authenticated.lean b/Complexity/Crypto/Computational/Encryption/Authenticated.lean index a177011ae..951e29dce 100644 --- a/Complexity/Crypto/Computational/Encryption/Authenticated.lean +++ b/Complexity/Crypto/Computational/Encryption/Authenticated.lean @@ -1662,7 +1662,7 @@ private theorem ennreal_tsum_finsetSum {α β : Type*} induction s using Finset.induction_on with | empty => simp | insert b s hb ih => - simp only [Finset.sum_insert hb, ENNReal.tsum_add, ih] + simp only [Finset.sum_insert hb, tsum_add, ih] private theorem eval_ccaRejectWithBadOracle_false (E : SymmetricCipher) (n : ℕ) (k : List Bool) (bit : Bool) (request : Bool × List Bool) (queried : List (List Bool)) : @@ -1761,7 +1761,7 @@ private theorem bad_ccaRejectWithBad_le_sum_ciCombined {α : Type} (E : Symmetri (fun x => PMF.pure (!x.2.2.contains x.2.1.2.2 && (E.decrypt n k x.2.1.2.2).isSome)) true := - ENNReal.tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (ih a queried) zero_le + tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (ih a queried) zero_le _ = _ := by simp only [Finset.mul_sum] rw [ennreal_tsum_finsetSum] @@ -1818,7 +1818,7 @@ private theorem bad_ccaRejectWithBad_le_sum_ciCombined {α : Type} (E : Symmetri PMF.pure (!x.2.2.contains x.2.1.2.2 && (E.decrypt n k x.2.1.2.2).isSome)) true := - ENNReal.tsum_le_tsum fun ct => + tsum_le_tsum fun ct => mul_le_mul_of_nonneg_left (ih ct (ct :: queried)) zero_le _ = _ := by simp only [Finset.mul_sum] @@ -1993,8 +1993,8 @@ private theorem advantage_ccaGame_ccaRejectGame_le_sum (E : SymmetricCipher) (ct ProbOracleComp.eval_run_interpret, hrejEq] exact PMF.bind_apply _ _ _ have hle1 : pCCA ≤ pRej + pSum := by - rw [hCCA_eq, hRej_eq, hsum_eq, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun k => ?_ + rw [hCCA_eq, hRej_eq, hsum_eq, ← tsum_add] + refine tsum_le_tsum fun k => ?_ rw [← mul_add] by_cases hk : k ∈ (ProbComp.eval (E.keyGen n)).support · refine mul_le_mul_of_nonneg_left ?_ zero_le @@ -2007,8 +2007,8 @@ private theorem advantage_ccaGame_ccaRejectGame_le_sum (E : SymmetricCipher) (ct rw [eval_ciGame_ccaToCIAtIndexAdversary_key E ctBound j bit adversary n k hk hctBound] · rw [(PMF.apply_eq_zero_iff _ _).mpr hk, zero_mul, zero_mul] have hle2 : pRej ≤ pCCA + pSum := by - rw [hCCA_eq, hRej_eq, hsum_eq, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun k => ?_ + rw [hCCA_eq, hRej_eq, hsum_eq, ← tsum_add] + refine tsum_le_tsum fun k => ?_ rw [← mul_add] by_cases hk : k ∈ (ProbComp.eval (E.keyGen n)).support · refine mul_le_mul_of_nonneg_left ?_ zero_le diff --git a/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean b/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean index d8d240fb8..e33925650 100644 --- a/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean +++ b/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean @@ -1629,7 +1629,7 @@ private theorem step_bad_prfCipherForgetfulWithBadOracle_le (n : ℕ) (bit : Boo {x | x.2.2 = true} ≤ ∑' r, (BitString.uniformList n) r * {r : List Bool | (cache.lookup r).isSome}.indicator 1 r := by - refine ENNReal.tsum_le_tsum fun r => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun r => mul_le_mul_of_nonneg_left ?_ zero_le by_cases hr : (cache.lookup r).isSome = true · rw [Set.indicator_of_mem (show r ∈ {r : List Bool | (cache.lookup r).isSome} from hr), Pi.one_apply] @@ -1697,7 +1697,7 @@ private theorem bad_prfCipherForgetfulWithBadOracle_le {α : Type} (n : ℕ) (bi (cont y.1)).run y.2).toOuterMeasure {x | x.2.2 = true} ≤ ∑' y, ProbOracleComp.handler (fun m => StateT.lift (ProbComp.unifOracle m)) (prfCipherForgetfulWithBadOracle n bit) (Sum.inl m) (cache, false) y * B := by - refine ENNReal.tsum_le_tsum fun y => ?_ + refine tsum_le_tsum fun y => ?_ by_cases hy : y ∈ (ProbOracleComp.handler (fun m => StateT.lift (ProbComp.unifOracle m)) (prfCipherForgetfulWithBadOracle n bit) (Sum.inl m) (cache, false)).support @@ -1749,10 +1749,10 @@ private theorem bad_prfCipherForgetfulWithBadOracle_le {α : Type} (n : ℕ) (bi ≤ ∑' y, prfCipherForgetfulWithBadOracle n bit request (cache, false) y * ({x : List Bool × (List (List Bool × List Bool) × Bool) | x.2.2 = true}.indicator 1 y + K) := - ENNReal.tsum_le_tsum hterm + tsum_le_tsum hterm _ = (prfCipherForgetfulWithBadOracle n bit request (cache, false)).toOuterMeasure {x | x.2.2 = true} + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun y => ?_ diff --git a/Complexity/Crypto/Computational/GMW/Soundness.lean b/Complexity/Crypto/Computational/GMW/Soundness.lean index 7841dd8ec..99dcd2c2f 100644 --- a/Complexity/Crypto/Computational/GMW/Soundness.lean +++ b/Complexity/Crypto/Computational/GMW/Soundness.lean @@ -562,7 +562,7 @@ theorem acceptProb_gmwVerifier_le {α : Type} {generator : List Bool → List Bo P.init x * ProbOracleComp.acceptProbWith P.respond (step2 r x.1) x.2 ≤ ∑' x, ProbOracleComp.runWith P.respond (List.take (K * m) <$> ProbOracleComp.query ((), r)) P.init x * (1 / 8) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_right (hstep2_le r x.1 x.2 hbind) _ + tsum_le_tsum fun x => mul_le_mul_right (hstep2_le r x.1 x.2 hbind) _ _ = (∑' x, ProbOracleComp.runWith P.respond (List.take (K * m) <$> ProbOracleComp.query ((), r)) P.init x) * (1 / 8) := ENNReal.tsum_mul_right @@ -591,9 +591,9 @@ theorem acceptProb_gmwVerifier_le {α : Type} {generator : List Bool → List Bo calc ∑' r, BitString.uniformList m r * acceptProb (step1 r) P ≤ ∑' r, (({r | ¬ NaorBinding generator s r}.indicator (BitString.uniformList m) r) + - BitString.uniformList m r * (1 / 8)) := ENNReal.tsum_le_tsum hterm_le + BitString.uniformList m r * (1 / 8)) := tsum_le_tsum hterm_le _ = (∑' r, {r | ¬ NaorBinding generator s r}.indicator (BitString.uniformList m) r) + - ∑' r, BitString.uniformList m r * (1 / 8) := ENNReal.tsum_add + ∑' r, BitString.uniformList m r * (1 / 8) := tsum_add _ = naorBindingProb generator length s + 1 / 8 := by rw [← PMF.toOuterMeasure_apply, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] rfl diff --git a/Complexity/Crypto/Computational/MAC.lean b/Complexity/Crypto/Computational/MAC.lean index 5b35fad27..5d8f4e36f 100644 --- a/Complexity/Crypto/Computational/MAC.lean +++ b/Complexity/Crypto/Computational/MAC.lean @@ -797,7 +797,7 @@ private theorem pmf_bind_pure_bool_le {α : Type} (p : PMF α) (f g : α → Boo (p.bind (fun a => PMF.pure (f a))) true ≤ (p.bind (fun a => PMF.pure (g a))) true := by rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ p.support · refine mul_le_mul_of_nonneg_left ?_ zero_le cases hf : f a @@ -816,7 +816,7 @@ theorem succProbability_eufCmaGame_le_sufCmaGame (M : MACScheme) simp only [Game.succProbability, sufCmaGame, ProbComp.eval_bind, ProbComp.eval_pure] refine ENNReal.toNNReal_mono (PMF.apply_ne_top _ true) ?_ rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun sk => ?_ + refine tsum_le_tsum fun sk => ?_ refine mul_le_mul_of_nonneg_left (pmf_bind_pure_bool_le _ _ _ fun out _ hout => ?_) zero_le simp only [Bool.and_eq_true, Bool.not_eq_true'] at hout ⊢ refine ⟨⟨hout.1.1, ?_⟩, hout.2⟩ @@ -1450,7 +1450,7 @@ theorem succProbability_prfIdealGame_macToPrfDistinguisher_le (adversary : Forge zero_le] have htotal : (phase1.bind phase2) true ≤ (1 : ℝ≥0∞) / 2 ^ n := by rw [PMF.bind_apply] - refine (ENNReal.tsum_le_tsum fun a => ?_).trans + refine (tsum_le_tsum fun a => ?_).trans (PMF.tsum_mul_le_of_forall_le phase1 fun _ => le_rfl) by_cases ha : a ∈ phase1.support · exact mul_le_mul_right (hstep_bound a ha) _ @@ -2012,7 +2012,7 @@ theorem succProbability_eufCmaGame_le_sufCmaGame (M : NonceMACScheme) simp only [Game.succProbability, eufCmaGame, sufCmaGame, ProbComp.eval_bind, ProbComp.eval_pure] refine ENNReal.toNNReal_mono (PMF.apply_ne_top _ true) ?_ rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun sk => ?_ + refine tsum_le_tsum fun sk => ?_ refine mul_le_mul_of_nonneg_left (MACScheme.pmf_bind_pure_bool_le _ _ _ fun out _ hout => ?_) zero_le simp only [Bool.and_eq_true, Bool.not_eq_true'] at hout ⊢ diff --git a/Complexity/Crypto/Computational/NaorReingold/Security.lean b/Complexity/Crypto/Computational/NaorReingold/Security.lean index ad05595ed..bcc1e4022 100644 --- a/Complexity/Crypto/Computational/NaorReingold/Security.lean +++ b/Complexity/Crypto/Computational/NaorReingold/Security.lean @@ -2995,7 +2995,7 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} refine ne_of_lt (lt_of_le_of_lt ?_ ENNReal.one_lt_top) calc (∑' d, setup n d * (F.order d : ℝ≥0∞)⁻¹) ≤ ∑' d, setup n d * 1 := by - refine ENNReal.tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ (by positivity) + refine tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ (by positivity) exact ENNReal.inv_le_one.mpr (by exact_mod_cast (F.prime_order d).pos) _ = 1 := by simp [(setup n).tsum_coe] have hε_pos : 0 < ε n := by @@ -3007,7 +3007,7 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} ENNReal.mul_pos hd₀_ne hord_ne exact ENNReal.toNNReal_pos (ne_of_gt (lt_of_lt_of_le hterm_pos - (ENNReal.le_tsum (f := fun d => setup n d * (F.order d : ℝ≥0∞)⁻¹) d₀))) + (le_tsum (f := fun d => setup n d * (F.order d : ℝ≥0∞)⁻¹) d₀))) hexp_top let τ : ℝ≥0 := NNReal.sqrt (ε n) have hτ_pos : 0 < τ := NNReal.sqrt_pos.mpr hε_pos @@ -3079,7 +3079,7 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} have hsum_err : (∑' d, setup n d * errTerm d) = ((lhlBound n : ℝ≥0) : ℝ≥0∞) := by dsimp only [errTerm, lhlBound] - simp_rw [mul_add, ENNReal.tsum_add, ← mul_assoc] + simp_rw [mul_add, tsum_add, ← mul_assoc] rw [ENNReal.tsum_mul_right, ENNReal.tsum_mul_right, (setup n).tsum_coe, one_mul, ENNReal.coe_add] congr 1 @@ -3095,8 +3095,8 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} (hfg : ∀ d ∈ (setup n).support, f d true ≤ g d true + ((Game.advantage (g1 d) (g2 d) : ℝ≥0) : ℝ≥0∞)) : ((setup n).bind f) true ≤ ((setup n).bind g) true + ((lhlBound n : ℝ≥0) : ℝ≥0∞) := by - rw [← hsum_err, PMF.bind_apply, PMF.bind_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun d => ?_ + rw [← hsum_err, PMF.bind_apply, PMF.bind_apply, ← tsum_add] + refine tsum_le_tsum fun d => ?_ by_cases hd : d ∈ (setup n).support · rw [← mul_add] exact mul_le_mul_of_nonneg_left ((hfg d hd).trans (add_le_add le_rfl (hpoint d hd))) diff --git a/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean b/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean index 3c997b6a5..2350d6382 100644 --- a/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean +++ b/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean @@ -610,7 +610,7 @@ private theorem heuristic_fail_mul_le_family_fail {t : ℕ → ℕ} (ht : ∀ n, rw [hfailA_eq] dsimp only [failH] rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun coins => ?_ + refine tsum_le_tsum fun coins => ?_ split_ifs with h1 h2 · rfl · exfalso diff --git a/Complexity/Crypto/Computational/OneWayToSignature.lean b/Complexity/Crypto/Computational/OneWayToSignature.lean index ce784ba7d..a3cc11acb 100644 --- a/Complexity/Crypto/Computational/OneWayToSignature.lean +++ b/Complexity/Crypto/Computational/OneWayToSignature.lean @@ -663,7 +663,7 @@ private theorem map_bool_apply_true {α : Type*} (p : PMF α) (P : α → Bool) private theorem ennreal_tsum_finsetSum {α ι : Type*} (s : Finset ι) (g : ι → α → ℝ≥0∞) : (∑' a, ∑ i ∈ s, g i a) = ∑ i ∈ s, ∑' a, g i a := - Summable.tsum_finsetSum fun _ _ => ENNReal.summable + Summable.tsum_finsetSum fun _ _ => CompleteLattice.summable /-- **Lamport's one-time signature theorem**: for any one-way function `f`, `lamportScheme f` is existentially unforgeable under one-time adaptive chosen-message attack (`OneTimeEUFCMA`). -/ @@ -817,7 +817,7 @@ theorem OneWay.oneTimeEUFCMA_lamport {f : List Bool → List Bool} (hf : OneWay ≤ ∑' out, ∑ j : Fin (dyadicSize (2 * n)), if j.val < 2 * n then runWith (pkOf w) (dummySkOf w j.val) out * (if invWin w j.val out then 1 else 0) - else 0 := ENNReal.tsum_le_tsum (hpointwise w hw) + else 0 := tsum_le_tsum (hpointwise w hw) _ = ∑ j : Fin (dyadicSize (2 * n)), ∑' out, if j.val < 2 * n then runWith (pkOf w) (dummySkOf w j.val) out * (if invWin w j.val out then 1 else 0) @@ -845,7 +845,7 @@ theorem OneWay.oneTimeEUFCMA_lamport {f : List Bool → List Bool} (hf : OneWay if j.val < 2 * n then ((runWith (pkOf w) (dummySkOf w j.val)).map (invWin w j.val)) true else 0 := by - refine ENNReal.tsum_le_tsum fun w => ?_ + refine tsum_le_tsum fun w => ?_ by_cases hw : w ∈ μ.support · exact mul_le_mul_right (hw_sum w hw) (μ w) · simp [(PMF.apply_eq_zero_iff μ w).mpr hw] diff --git a/Complexity/Crypto/Computational/PRGToOneWay.lean b/Complexity/Crypto/Computational/PRGToOneWay.lean index 1fb8c7c47..70e7f7aea 100644 --- a/Complexity/Crypto/Computational/PRGToOneWay.lean +++ b/Complexity/Crypto/Computational/PRGToOneWay.lean @@ -107,7 +107,7 @@ theorem succProbability_uniformList_test_le {generator : List Bool → List Bool split_ifs with hr · exact PMF.coe_le_one _ _ simp only [test, ProbComp.eval_bind, ProbComp.eval_pure, PMF.bind_apply, PMF.pure_apply] - refine le_of_eq (ENNReal.tsum_eq_zero.2 fun x => ?_) + refine le_of_eq (tsum_eq_zero_iff.2 fun x => ?_) have hne : generator x ≠ List.ofFn r := fun hx => hr <| mem_image.2 ⟨x, hP x (by rw [← hlen x, hx, List.length_ofFn]), hx⟩ simp [hne] diff --git a/Complexity/Crypto/Computational/PedersenCommitment.lean b/Complexity/Crypto/Computational/PedersenCommitment.lean index e77853f5a..09842b79b 100644 --- a/Complexity/Crypto/Computational/PedersenCommitment.lean +++ b/Complexity/Crypto/Computational/PedersenCommitment.lean @@ -405,11 +405,11 @@ theorem pedersenHidingGame_apply_true_le (setup : ℕ → PMF F.Desc) ≤ ∑' α, (PMF.uniformOfFintype (ZMod (F.order d)) α * F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n true + if α = 0 then (F.order d : ℝ≥0∞)⁻¹ else 0) := - ENNReal.tsum_le_tsum hα + tsum_le_tsum hα _ = (∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n true) + (F.order d : ℝ≥0∞)⁻¹ := by - rw [ENNReal.tsum_add, tsum_ite_eq] + rw [tsum_add, tsum_ite_eq] _ ≤ (∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n true) + (m : ℝ≥0∞)⁻¹ := add_le_add le_rfl hord @@ -423,7 +423,7 @@ theorem pedersenHidingGame_apply_true_le (setup : ℕ → PMF F.Desc) (((PMF.uniformOfFintype (ZMod (F.order d))).bind fun α => F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n) true + (m : ℝ≥0∞)⁻¹) := by - refine ENNReal.tsum_le_tsum fun d => ?_ + refine tsum_le_tsum fun d => ?_ by_cases hd : d ∈ (setup n).support · exact mul_le_mul_of_nonneg_left (hd_bound d hd) zero_le · have hd0 : setup n d = 0 := by rwa [PMF.mem_support_iff, not_not] at hd @@ -432,7 +432,7 @@ theorem pedersenHidingGame_apply_true_le (setup : ℕ → PMF F.Desc) ((PMF.uniformOfFintype (ZMod (F.order d))).bind fun α => F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n) true) + (m : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, (setup n).tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, (setup n).tsum_coe, one_mul] /-- **Unconditional hiding bound for Pedersen commitments**: when every supported group at parameter `n` has order at least `m > 0`, any (even computationally unbounded) diff --git a/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean b/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean index 9a46827f4..ffd5f9b0b 100644 --- a/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean +++ b/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean @@ -109,7 +109,7 @@ theorem maskedSample_mass_ge {α β : Type} (source : ProbComp α) rw [← ENNReal.toReal_mul] apply ENNReal.toReal_mono (PMF.apply_ne_top _ _) rw [PMF.bind_apply] - exact ENNReal.le_tsum (f := fun x => ProbComp.eval source x * + exact le_tsum (f := fun x => ProbComp.eval source x * ((ProbComp.eval (maskBit (mask x) (truth x))).map (observe x, ·)) (observe value, bit)) value exact (mul_le_mul (hsource value hvalue) (hbitMass.trans hmap) diff --git a/Complexity/Crypto/Game.lean b/Complexity/Crypto/Game.lean index e289aabc6..4f37d2714 100644 --- a/Complexity/Crypto/Game.lean +++ b/Complexity/Crypto/Game.lean @@ -295,16 +295,16 @@ theorem advantage_bind_le {α : Type*} (d : PMF α) (game1 game2 : α → Game) rw [PMF.bind_apply, PMF.bind_apply] calc ∑' a, d a * game1 a true ≤ ∑' a, d a * (game2 a true + ENNReal.ofReal ε) := - ENNReal.tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h1 a) zero_le + tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h1 a) zero_le _ = (∑' a, d a * game2 a true) + ENNReal.ofReal ε := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] have hsum2 : (d.bind game2) true ≤ (d.bind game1) true + ENNReal.ofReal ε := by rw [PMF.bind_apply, PMF.bind_apply] calc ∑' a, d a * game2 a true ≤ ∑' a, d a * (game1 a true + ENNReal.ofReal ε) := - ENNReal.tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h2 a) zero_le + tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h2 a) zero_le _ = (∑' a, d a * game1 a true) + ENNReal.ofReal ε := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] have hadv := advantage_le_of_apply_le ENNReal.ofReal_ne_top hsum1 hsum2 rwa [ENNReal.toReal_ofReal hε0] at hadv @@ -324,8 +324,8 @@ theorem advantage_bind_uniformOfFintype_of_ne_le {α : Type*} [Fintype α] [None have hle₁ : ((PMF.uniformOfFintype α).bind g) true ≤ p true + (Fintype.card α : ℝ≥0∞)⁻¹ := by rw [← PMF.bind_const (PMF.uniformOfFintype α) p, ← hsingle, PMF.bind_apply, PMF.bind_apply, - PMF.bind_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun a => ?_ + PMF.bind_apply, ← tsum_add] + refine tsum_le_tsum fun a => ?_ rw [← mul_add] refine mul_le_mul_of_nonneg_left ?_ zero_le by_cases ha : a = a₀ @@ -335,8 +335,8 @@ theorem advantage_bind_uniformOfFintype_of_ne_le {α : Type*} [Fintype α] [None have hle₂ : p true ≤ ((PMF.uniformOfFintype α).bind g) true + (Fintype.card α : ℝ≥0∞)⁻¹ := by nth_rw 1 [← PMF.bind_const (PMF.uniformOfFintype α) p] - rw [← hsingle, PMF.bind_apply, PMF.bind_apply, PMF.bind_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun a => ?_ + rw [← hsingle, PMF.bind_apply, PMF.bind_apply, PMF.bind_apply, ← tsum_add] + refine tsum_le_tsum fun a => ?_ rw [← mul_add] refine mul_le_mul_of_nonneg_left ?_ zero_le by_cases ha : a = a₀ @@ -363,8 +363,8 @@ theorem bind_apply_le_add_bind_apply {β : Type*} (μ : PMF β) (μ.bind game2) b ≤ (μ.bind game1) b + (μ.bind err) b := by have h1 (f g : β → Game) (hle : ∀ x, f x b ≤ g x b + err x b) : (μ.bind f) b ≤ (μ.bind g) b + (μ.bind err) b := by - simp only [PMF.bind_apply, ← ENNReal.tsum_add, ← mul_add] - exact ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le + simp only [PMF.bind_apply, ← tsum_add, ← mul_add] + exact tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le exact ⟨h1 game1 game2 fun x => (h x).1, h1 game2 game1 fun x => (h x).2⟩ /-- Averaging pointwise acceptance-probability bounds `game1 x true ≤ game2 x true + err x true` diff --git a/Complexity/Crypto/Game/UntilBad.lean b/Complexity/Crypto/Game/UntilBad.lean index 52f3c2892..f80d1e46d 100644 --- a/Complexity/Crypto/Game/UntilBad.lean +++ b/Complexity/Crypto/Game/UntilBad.lean @@ -164,9 +164,9 @@ theorem runStatelessOracle_bind_le {T α : Type} (μ : PMF T) (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * g x) + (Q : ℝ≥0∞) * δ := by calc ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * f x ≤ ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * (g x + (Q : ℝ≥0∞) * δ) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le _ = (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * g x) + (Q : ℝ≥0∞) * δ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] exact ⟨hsum _ _ (fun x => (ih x).1), hsum _ _ (fun x => (ih x).2)⟩ · simp only [Sum.elim_inr] have hinr₁ : @@ -184,13 +184,13 @@ theorem runStatelessOracle_bind_le {T α : Type} (μ : PMF T) simp only [PMF.bind_apply] calc ∑' τ, μ τ * f τ b ≤ ∑' τ, (μ τ * g τ b + μ τ * if isBad τ req.2 then 1 else 0) := by - refine ENNReal.tsum_le_tsum fun τ => ?_ + refine tsum_le_tsum fun τ => ?_ by_cases hb : isBad τ req.2 = true · simp only [hb, ↓reduceIte, mul_one] exact le_add_of_le_right (mul_le_of_le_one_right zero_le (PMF.coe_le_one _ _)) · simp only [Bool.not_eq_true] at hb rw [hfg τ hb]; simp [hb] - _ = (∑' τ, μ τ * g τ b) + ∑' τ, μ τ * if isBad τ req.2 then 1 else 0 := ENNReal.tsum_add + _ = (∑' τ, μ τ * g τ b) + ∑' τ, μ τ * if isBad τ req.2 then 1 else 0 := tsum_add _ ≤ (∑' τ, μ τ * g τ b) + δ := add_le_add le_rfl (hbad req.2) have hstep1 := hstep (fun τ => runStatelessOracle (o₁ τ) (k (o₁ τ req.2)) (next τ)) @@ -372,20 +372,20 @@ theorem runStatelessOracle_le_runFinderOracle {α : Type} ≤ ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * (runStatelessOracle o₂ (k x) next b + runFinderOracle o₂ isBad extract (k x) none isGood true) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).1 zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).1 zero_le _ = (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runStatelessOracle o₂ (k x) next b) + ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runFinderOracle o₂ isBad extract (k x) none isGood true := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] · calc ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runStatelessOracle o₂ (k x) next b ≤ ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * (runStatelessOracle o₁ (k x) next b + runFinderOracle o₂ isBad extract (k x) none isGood true) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).2 zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).2 zero_le _ = (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runStatelessOracle o₁ (k x) next b) + ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runFinderOracle o₂ isBad extract (k x) none isGood true := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] · rw [runStatelessOracle_query_inr, runStatelessOracle_query_inr, runFinderOracle_query_inr] by_cases hb : isBad req.2 = true · simp only [hb, ↓reduceIte] diff --git a/Complexity/Crypto/Primitives/PRG/NextBit.lean b/Complexity/Crypto/Primitives/PRG/NextBit.lean index 886b68555..76fe10070 100644 --- a/Complexity/Crypto/Primitives/PRG/NextBit.lean +++ b/Complexity/Crypto/Primitives/PRG/NextBit.lean @@ -121,9 +121,9 @@ private theorem bind_apply_add_of_forall {α β : Type*} (μ : PMF α) {f g h : {c : ℝ≥0∞} (hfg : ∀ x, f x b + g x b = h x b + c) : μ.bind f b + μ.bind g b = μ.bind h b + c := by simp only [PMF.bind_apply] - rw [← ENNReal.tsum_add] + rw [← tsum_add] simp_rw [← mul_add, hfg, mul_add] - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] + rw [tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] /-- **The distinguisher/predictor lemma** [Boneh and Shoup, Lemma 3.5][BonehShoup2023]. Draw `x` from `μ`. The guess `guessOfDistinguisher (d x)` equals `target x` with probability `1 / 2`, plus diff --git a/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean b/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean index b88c043a5..ca41828a6 100644 --- a/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean +++ b/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean @@ -1955,7 +1955,7 @@ private theorem idealGame_apply_le_realGame_add_of_keyDist (keyDist : PMF (G × calc (∑' res : Bool × (Cache G × Cache G), M_ideal res * keyDist.toOuterMeasure {k | isBadEvenMansourKey res.2.1 res.2.2 k = true}) ≤ ∑' res : Bool × (Cache G × Cache G), M_ideal res * ε := by - refine ENNReal.tsum_le_tsum fun res => ?_ + refine tsum_le_tsum fun res => ?_ by_cases hmem : res ∈ M_ideal.support · refine mul_le_mul_of_nonneg_left (hbad_le res.2.1 res.2.2 ?_ ?_) zero_le · simpa using length_fst_idealLazyHandler_le hcall.1 hmem @@ -1968,14 +1968,14 @@ private theorem idealGame_apply_le_realGame_add_of_keyDist (keyDist : PMF (G × ≤ ∑' k : G × G, keyDist k * ((((OracleComp.interpret (realLazyHandler k) program).run []).map Prod.fst) b + M_ideal.toOuterMeasure {res | isBadEvenMansourKey res.2.1 res.2.2 k = true}) := - ENNReal.tsum_le_tsum fun k => mul_le_mul_of_nonneg_left (hpoint k) zero_le + tsum_le_tsum fun k => mul_le_mul_of_nonneg_left (hpoint k) zero_le _ = (∑' k : G × G, keyDist k * (((OracleComp.interpret (realLazyHandler k) program).run []).map Prod.fst) b) + (∑' k : G × G, keyDist k * M_ideal.toOuterMeasure {res | isBadEvenMansourKey res.2.1 res.2.2 k = true}) := by - simp_rw [mul_add, ENNReal.tsum_add] + simp_rw [mul_add, tsum_add] _ ≤ _ := add_le_add le_rfl hsum_bad private theorem advantage_le_of_forall_bool_le {g₁ g₂ : Game} {ε : ℝ≥0∞} (hε : ε ≠ ⊤) diff --git a/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean b/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean index ac6f10219..a46a9a06e 100644 --- a/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean +++ b/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean @@ -1013,7 +1013,7 @@ private theorem tsum_seed_toOuterMeasure_hasCollision_le {seed₁ : PMF K₁} {h ∑' x, idealRun x * ∑' k₁, ({k₁ : K₁ | hasUCollision (h₁ k₁) x.2 = true}.indicator seed₁ k₁) := by simp only [← ENNReal.tsum_mul_left] - rw [ENNReal.tsum_comm] + rw [tsum_comm] refine tsum_congr fun x => tsum_congr fun k₁ => ?_ by_cases hx : hasUCollision (h₁ k₁) x.2 = true · rw [Set.indicator_of_mem hx, Set.indicator_of_mem hx, mul_comm] @@ -1022,7 +1022,7 @@ private theorem tsum_seed_toOuterMeasure_hasCollision_le {seed₁ : PMF K₁} {h calc ∑' x, idealRun x * ∑' k₁, ({k₁ : K₁ | hasUCollision (h₁ k₁) x.2 = true}.indicator seed₁ k₁) ≤ ∑' x, idealRun x * (Q.choose 2 * ε) := by - refine ENNReal.tsum_le_tsum fun x => ?_ + refine tsum_le_tsum fun x => ?_ by_cases hmem : x ∈ idealRun.support · refine mul_le_mul_of_nonneg_left ?_ zero_le rw [← PMF.toOuterMeasure_apply] @@ -1045,10 +1045,10 @@ private theorem tsum_seed_toOuterMeasure_hasCollision_le {seed₁ : PMF K₁} {h ≤ ∑' k₁, seed₁ k₁ * (idealRun.toOuterMeasure {x | hasUCollision (h₁ k₁) x.2 = true} + idealRun.toOuterMeasure {x | hasVCollision x.2 = true}) := - ENNReal.tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (hsplit k₁) zero_le + tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (hsplit k₁) zero_le _ = (∑' k₁, seed₁ k₁ * idealRun.toOuterMeasure {x | hasUCollision (h₁ k₁) x.2 = true}) + (∑' k₁, seed₁ k₁ * idealRun.toOuterMeasure {x | hasVCollision x.2 = true}) := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] _ ≤ Q.choose 2 * ε + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ := add_le_add hu hv /-- For every fixed `f₁`, the acceptance probability of `feistel3FixedGame f₁ program` and @@ -1092,13 +1092,13 @@ theorem feistel3HashIdealGame_apply_le {seed₁ : PMF K₁} {h₁ : K₁ → G ≤ ∑' k₁, seed₁ k₁ * (randomFunctionGame program b + ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true}) := - ENNReal.tsum_le_tsum fun k₁ => + tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (feistel3FixedGame_apply_le (h₁ k₁) program b) zero_le _ = randomFunctionGame program b + ∑' k₁, seed₁ k₁ * ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, seed₁.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, seed₁.tsum_coe, one_mul] _ ≤ randomFunctionGame program b + (Q.choose 2 * ε + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹) := add_le_add le_rfl (tsum_seed_toOuterMeasure_hasCollision_le hh₁ hQ) @@ -1120,14 +1120,14 @@ theorem randomFunctionGame_apply_le_feistel3HashIdealGame {seed₁ : PMF K₁} { ≤ ∑' k₁, seed₁ k₁ * (feistel3FixedGame (h₁ k₁) program b + ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true}) := - ENNReal.tsum_le_tsum fun k₁ => + tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (randomFunctionGame_apply_le_feistel3FixedGame (h₁ k₁) program b) zero_le _ = (∑' k₁, seed₁ k₁ * feistel3FixedGame (h₁ k₁) program b) + ∑' k₁, seed₁ k₁ * ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] _ ≤ (∑' k₁, seed₁ k₁ * feistel3FixedGame (h₁ k₁) program b) + (Q.choose 2 * ε + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹) := add_le_add le_rfl (tsum_seed_toOuterMeasure_hasCollision_le hh₁ hQ) @@ -1619,7 +1619,7 @@ theorem le_idealStrongGame_feistel3CCAAttack_false {l₀ l₁ l₂ r₀ : G} _ ≤ ∑' π : Equiv.Perm (G × G), PMF.uniformOfFintype (Equiv.Perm (G × G)) π * OracleComp.interpret (feistelStrongOracle π) (feistel3CCAAttack l₀ l₁ r₀) false := - ENNReal.le_tsum (f := fun π : Equiv.Perm (G × G) => + le_tsum (f := fun π : Equiv.Perm (G × G) => PMF.uniformOfFintype (Equiv.Perm (G × G)) π * OracleComp.interpret (feistelStrongOracle π) (feistel3CCAAttack l₀ l₁ r₀) false) π₀ @@ -1875,9 +1875,9 @@ theorem idealStrongGame_feistel3CCAAttack_true_le {l₀ l₁ : G} (r₀ : G) (h0 (fun x => decide (-vw₀.1 + x.1.2 = -vw₁.1 + r₀))) true ≤ ∑' vw₁, ((PMF.uniformOfFintype (G × G)) vw₁ * (Fintype.card G : ℝ≥0∞)⁻¹ + ({q₃} : Set (G × G)).indicator (PMF.uniformOfFintype (G × G)) vw₁) := - ENNReal.tsum_le_tsum hvw₁ + tsum_le_tsum hvw₁ _ = (Fintype.card G : ℝ≥0∞)⁻¹ + (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + rw [tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, ← PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply_singleton, PMF.uniformOfFintype_apply] calc ∑' vw₀, (PMF.uniformOfFintype (G × G)) vw₀ * @@ -1888,7 +1888,7 @@ theorem idealStrongGame_feistel3CCAAttack_true_le {l₀ l₁ : G} (r₀ : G) (h0 (fun x => decide (-vw₀.1 + x.1.2 = -vw₁.1 + r₀))) true ≤ ∑' vw₀, (PMF.uniformOfFintype (G × G)) vw₀ * ((Fintype.card G : ℝ≥0∞)⁻¹ + (Fintype.card (G × G) : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun vw₀ => mul_le_mul_of_nonneg_left (hvw₀ vw₀) zero_le + tsum_le_tsum fun vw₀ => mul_le_mul_of_nonneg_left (hvw₀ vw₀) zero_le _ = (Fintype.card G : ℝ≥0∞)⁻¹ + (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by rw [ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] have hchoose : ((Nat.choose 3 2 : ℕ) : ℝ≥0∞) = 3 := rfl @@ -3043,7 +3043,7 @@ private theorem tsum_toOuterMeasure_of_pointwise_le {α β : Type} (μ : PMF α) (∑' a, μ a * ∑' y, {y | P a y}.indicator d y) = ∑' y, d y * ∑' a, {a | P a y}.indicator μ a := by simp only [← ENNReal.tsum_mul_left] - rw [ENNReal.tsum_comm] + rw [tsum_comm] refine tsum_congr fun y => tsum_congr fun a => ?_ by_cases h : P a y · rw [Set.indicator_of_mem h, Set.indicator_of_mem h, mul_comm] @@ -3051,7 +3051,7 @@ private theorem tsum_toOuterMeasure_of_pointwise_le {α β : Type} (μ : PMF α) rw [hcomm] calc ∑' y, d y * ∑' a, {a | P a y}.indicator μ a ≤ ∑' y, d y * M := - ENNReal.tsum_le_tsum fun y => mul_le_mul_of_nonneg_left (by + tsum_le_tsum fun y => mul_le_mul_of_nonneg_left (by rw [← PMF.toOuterMeasure_apply] exact hM y) zero_le _ = M := by rw [ENNReal.tsum_mul_right, d.tsum_coe, one_mul] @@ -3155,14 +3155,14 @@ private theorem tsum_uniform_toOuterMeasure_hasCollision4_le d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasVCollision4 f₄ x.2 = true} + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun f₄ => + tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left (toOuterMeasure_hasCollision4_le f₁ f₄ hQ) zero_le _ = d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasUCollision f₁ x.2 = true} + (∑' f₄, unif f₄ * d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasVCollision4 f₄ x.2 = true}) + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] _ ≤ _ := add_le_add (add_le_add le_rfl (tsum_uniform_toOuterMeasure_hasVCollision4_le d Q)) le_rfl calc ∑' f₁, unif f₁ * ∑' f₄, unif f₄ * d.toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true} @@ -3171,12 +3171,12 @@ private theorem tsum_uniform_toOuterMeasure_hasCollision4_le {x | IsGoodCache4 Q x.2 ∧ hasUCollision f₁ x.2 = true} + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left (hinner f₁) zero_le + tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left (hinner f₁) zero_le _ = (∑' f₁, unif f₁ * d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasUCollision f₁ x.2 = true}) + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] _ ≤ Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := @@ -3253,13 +3253,13 @@ theorem feistel4StrongRandomRoundsGame_apply_le (forgetVal + ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true}) := - ENNReal.tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left - (ENNReal.tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left + tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left + (tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left (feistel4StrongFixedGame_apply_le f₁ f₄ program b) zero_le) zero_le _ = forgetVal + ∑' f₁, unif f₁ * ∑' f₄, unif f₄ * ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] _ ≤ _ := add_le_add le_rfl (tsum_uniform_toOuterMeasure_hasCollision4_le hQ') have hperm := RandomPermutation.forgetfulAnswer_bind_apply_le_uniformPerm hQ' PMF.pure b simp only [Probability.PMF.monad_bind_eq_bind, PMF.bind_pure] at hperm @@ -3303,14 +3303,14 @@ theorem idealStrongGame_apply_le_feistel4StrongRandomRoundsGame (feistel4StrongFixedGame f₁ f₄ program b + ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true}) := - ENNReal.tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left - (ENNReal.tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left + tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left + (tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left (forgetfulAnswer_apply_le_feistel4StrongFixedGame f₁ f₄ program b) zero_le) zero_le _ = (∑' f₁, unif f₁ * ∑' f₄, unif f₄ * feistel4StrongFixedGame f₁ f₄ program b) + ∑' f₁, unif f₁ * ∑' f₄, unif f₄ * ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] _ ≤ _ := add_le_add le_rfl (tsum_uniform_toOuterMeasure_hasCollision4_le hQ') have hperm := RandomPermutation.uniformPerm_bind_apply_le_forgetful hQ' PMF.pure b simp only [Probability.PMF.monad_bind_eq_bind, PMF.bind_pure] at hperm diff --git a/Complexity/Crypto/Protocols/MAC/CarterWegman.lean b/Complexity/Crypto/Protocols/MAC/CarterWegman.lean index dd32115b4..4b4f2bb30 100644 --- a/Complexity/Crypto/Protocols/MAC/CarterWegman.lean +++ b/Complexity/Crypto/Protocols/MAC/CarterWegman.lean @@ -3802,10 +3802,10 @@ theorem randomNonceSimHandler_not_nodup_le {α : Type} [Fintype N] [Nonempty N] PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true ≤ ∑' x, PMF.uniformOfFintype N x * ((PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le _ = ((PMF.uniformOfFintype N).bind fun x => PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, (PMF.uniformOfFintype N).tsum_coe, one_mul, PMF.bind_apply] _ ≤ (tr.length : ℝ≥0∞) / Fintype.card N + B := add_le_add hmem_le le_rfl _ = ((∑ i ∈ Finset.range (Q + 1), (tr.length + i) : ℕ) : ℝ≥0∞) / Fintype.card N := by @@ -4029,11 +4029,11 @@ theorem randomNonceStrongGame_le_sum_add_max {seed : PMF K} {hash : K → M → ≤ ∑' res, simPMF res * ((PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le + tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le _ = (simPMF.bind fun res => PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, PMF.bind_apply] _ ≤ ((∑ i ∈ Finset.range Q, i : ℕ) : ℝ≥0∞) / Fintype.card N + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by @@ -4485,10 +4485,10 @@ theorem probOracleComp_randomNonceSimHandler_not_nodup_le {α : Type} PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true ≤ ∑' x, PMF.uniformOfFintype N x * ((PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le _ = ((PMF.uniformOfFintype N).bind fun x => PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, (PMF.uniformOfFintype N).tsum_coe, one_mul, PMF.bind_apply] _ ≤ (tr.length : ℝ≥0∞) / Fintype.card N + B := add_le_add hmem_le le_rfl _ = ((∑ i ∈ Finset.range (Q + 1), (tr.length + i) : ℕ) : ℝ≥0∞) / Fintype.card N := by @@ -4610,11 +4610,11 @@ theorem seed_uniform_probOracleComp_checkRandomNonceSUF_le {α : Type} ≤ ∑' res, simPMF res * ((PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le + tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le _ = (simPMF.bind fun res => PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, PMF.bind_apply] _ ≤ ((∑ i ∈ Finset.range Q, i : ℕ) : ℝ≥0∞) / Fintype.card N + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by diff --git a/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean b/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean index 2075b932a..39a83a0cb 100644 --- a/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean +++ b/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean @@ -391,7 +391,7 @@ private theorem toOuterMeasure_flag_le (P : SigmaProtocol X Y T C Z) (key : X × {x | x.2.2 = true} ≤ ∑' ts, (P.commit key.1 key.2) ts * {ts : T × P.State | (st.1.1.lookup (m, ts.1)).isSome}.indicator 1 ts := by - refine ENNReal.tsum_le_tsum fun ts => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun ts => mul_le_mul_of_nonneg_left ?_ zero_le by_cases hts : (st.1.1.lookup (m, ts.1)).isSome · rw [Set.indicator_of_mem (show ts ∈ {ts : T × P.State | (st.1.1.lookup (m, ts.1)).isSome} from hts), Pi.one_apply] @@ -828,12 +828,12 @@ private theorem bind_apply_le_add {α β : Type} {μ : PMF α} {f g : α → PMF (μ.bind f) b ≤ (μ.bind g) b + K := by simp only [PMF.bind_apply] calc ∑' a, μ a * f a b ≤ ∑' a, μ a * (g a b + K) := by - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ μ.support · exact mul_le_mul_of_nonneg_left (h a ha) zero_le · rw [(PMF.apply_eq_zero_iff _ _).mpr ha, zero_mul, zero_mul] _ = ∑' a, μ a * g a b + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] omit [DecidableEq C] in /-- **The game hops** for one key and one choice of the forger's coins: the experiment wins at diff --git a/Complexity/Languages/Probabilistic/BoundedUniform.lean b/Complexity/Languages/Probabilistic/BoundedUniform.lean index 6fa41f00f..a4166310d 100644 --- a/Complexity/Languages/Probabilistic/BoundedUniform.lean +++ b/Complexity/Languages/Probabilistic/BoundedUniform.lean @@ -187,7 +187,7 @@ private theorem retry_some (bound value : ℕ) (next : PMF (Option ℕ)) : simp [hib, hiv] change (∑' i, μ i * (if i < bound then PMF.pure (some i) else next) (some value)) = _ simp_rw [hterm] - rw [ENNReal.tsum_add] + rw [tsum_add] simp only [tsum_ite_eq] rw [show μ bound = (intervalFailure bound : ℝ≥0∞) from eval_sampleBoundedIndex_sentinel bound] by_cases hv : value < bound @@ -342,7 +342,7 @@ private theorem uniformFin_nat_apply (bound value : ℕ) [NeZero bound] : · intro i hi have : value ≠ i.val := fun h => hi (Fin.ext h.symm) simp [this] - · rw [ENNReal.tsum_eq_zero.mpr] + · rw [tsum_eq_zero_iff.mpr] · simp [hv] · intro i have : value ≠ i.val := by omega @@ -364,7 +364,7 @@ theorem eval_sampleUniformInterval_some_eq (bound attempts value : ℕ) [NeZero private theorem tsum_option_eq {α : Type} (f : Option α → ℝ≥0∞) : ∑' x, f x = f none + ∑' x, f (some x) := by rw [← (Equiv.optionEquivSumPUnit.{0, 0} α).symm.tsum_eq f] - rw [ENNReal.summable.tsum_sum ENNReal.summable] + rw [CompleteLattice.summable.tsum_sum CompleteLattice.summable] simp [add_comm] private theorem optional_map_density {α β : Type} (μ : PMF (Option α)) (ν : PMF α) diff --git a/Complexity/Languages/Probabilistic/Expect.lean b/Complexity/Languages/Probabilistic/Expect.lean index 3785b103e..6b22d82f8 100644 --- a/Complexity/Languages/Probabilistic/Expect.lean +++ b/Complexity/Languages/Probabilistic/Expect.lean @@ -70,7 +70,7 @@ theorem expect_zero (m : ProbComp α) : expect m (fun _ => 0) = 0 := theorem expect_add (m : ProbComp α) (f g : α → ℝ≥0∞) : expect m (fun a => f a + g a) = expect m f + expect m g := by - simp only [expect, mul_add, ENNReal.tsum_add] + simp only [expect, mul_add, tsum_add] theorem expect_mul_left (m : ProbComp α) (c : ℝ≥0∞) (f : α → ℝ≥0∞) : expect m (fun a => c * f a) = c * expect m f := by @@ -86,7 +86,7 @@ theorem expect_mul_right (m : ProbComp α) (f : α → ℝ≥0∞) (c : ℝ≥0 theorem expect_bind (m : ProbComp α) (k : α → ProbComp β) (f : β → ℝ≥0∞) : expect (m >>= k) f = expect m (fun a => expect (k a) f) := by simp only [expect, eval_bind, PMF.bind_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] + rw [tsum_comm] congr 1 funext a simp only [mul_assoc, ENNReal.tsum_mul_left] @@ -107,7 +107,7 @@ theorem expect_congr_on_support {m : ProbComp α} {f g : α → ℝ≥0∞} theorem expect_mono_on_support {m : ProbComp α} {f g : α → ℝ≥0∞} (h : ∀ a ∈ m.support, f a ≤ g a) : expect m f ≤ expect m g := by - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ m.support · exact mul_le_mul_right (h a ha) _ · have hzero : (eval m) a = 0 := (PMF.apply_eq_zero_iff (eval m) a).mpr (by @@ -129,7 +129,7 @@ theorem expect_eq_top_of_mem_support {m : ProbComp α} {f : α → ℝ≥0∞} { refine top_unique ?_ calc (⊤ : ℝ≥0∞) = (eval m) a * f a := by rw [hf, ENNReal.mul_top hpos] - _ ≤ expect m f := ENNReal.le_tsum (f := fun b => (eval m) b * f b) a + _ ≤ expect m f := le_tsum (f := fun b => (eval m) b * f b) a theorem expect_unifFin (n : ℕ) (f : Fin (n + 1) → ℝ≥0∞) : expect (unifFin n) f = (∑ k : Fin (n + 1), f k) / (n + 1 : ℝ≥0∞) := by diff --git a/Complexity/Languages/Probabilistic/Oracle/Interaction.lean b/Complexity/Languages/Probabilistic/Oracle/Interaction.lean index 6aa1215f3..d822a739c 100644 --- a/Complexity/Languages/Probabilistic/Oracle/Interaction.lean +++ b/Complexity/Languages/Probabilistic/Oracle/Interaction.lean @@ -168,7 +168,7 @@ theorem acceptProbWith_bind_ite_le (program : ProbOracleComp spec Bool) acceptProbWith oracle program s := by conv_rhs => rw [← bind_pure program, acceptProbWith_bind] rw [acceptProbWith_bind] - refine ENNReal.tsum_le_tsum fun x => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun x => mul_le_mul_right ?_ _ cases x.1 · simp only [Bool.false_eq_true, ↓reduceIte, le_refl] · simp only [↓reduceIte, acceptProbWith_pure] diff --git a/Complexity/Languages/Probabilistic/RandomPermutation.lean b/Complexity/Languages/Probabilistic/RandomPermutation.lean index f8b175e6d..153ac0a84 100644 --- a/Complexity/Languages/Probabilistic/RandomPermutation.lean +++ b/Complexity/Languages/Probabilistic/RandomPermutation.lean @@ -1175,9 +1175,9 @@ theorem toOuterMeasure_bad_choose_two_le {ι σ β : Type} {spec : ι → Type} calc ∑' y, (handler q s) y * ((OracleComp.interpret handler (cont y.1)).run y.2).toOuterMeasure {x | bad x.2} ≤ ∑' y, (handler q s) y * ({x : spec q × σ | bad x.2}.indicator 1 y + K) := - ENNReal.tsum_le_tsum hterm + tsum_le_tsum hterm _ = (handler q s).toOuterMeasure {x | bad x.2} + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun y => ?_ @@ -1413,8 +1413,8 @@ theorem toOuterMeasure_forgetfulAnswer_le_pairwise_and_length_le_add {β : Type} {x | (x.2.Pairwise (fun a b => a.1 ≠ b.1) ∧ x.2.Pairwise (fun a b => a.2 ≠ b.2) ∧ x.2.length ≤ Q) ∧ P x.2}) + q.toOuterMeasure {x | x.2.2 = true} - simp only [PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun ⟨a, c, b⟩ => ?_ + simp only [PMF.toOuterMeasure_apply, ← tsum_add] + refine tsum_le_tsum fun ⟨a, c, b⟩ => ?_ by_cases hmem : (a, (c, b)) ∈ q.support · by_cases hP : P c · cases b with diff --git a/Complexity/Languages/Probabilistic/UntilBad.lean b/Complexity/Languages/Probabilistic/UntilBad.lean index 7bcd7e202..ac26422b6 100644 --- a/Complexity/Languages/Probabilistic/UntilBad.lean +++ b/Complexity/Languages/Probabilistic/UntilBad.lean @@ -129,8 +129,8 @@ theorem bind_apply_le (h : IdenticalUntilBad first second bad) (program : Oracle ((interpret first program).run s).toOuterMeasure {x | bad x.2} := by classical simp only [PMF.bind_apply, PMF.toOuterMeasure_apply] - rw [← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun x => ?_ + rw [← tsum_add] + refine tsum_le_tsum fun x => ?_ by_cases hx : bad x.2 · have hbad : x ∈ {x : α × σ | bad x.2} := hx rw [Set.indicator_of_mem hbad] @@ -153,7 +153,7 @@ private theorem toOuterMeasure_bad_add_good (p : PMF (α × σ)) : p.toOuterMeasure {x | bad x.2} + p.toOuterMeasure {x | ¬bad x.2} = 1 := by classical simp only [PMF.toOuterMeasure_apply] - rw [← ENNReal.tsum_add, ← p.tsum_coe] + rw [← tsum_add, ← p.tsum_coe] refine tsum_congr fun x => ?_ by_cases hx : bad x.2 · rw [Set.indicator_of_mem (show x ∈ {x : α × σ | bad x.2} from hx), @@ -244,9 +244,9 @@ theorem toOuterMeasure_bad_le {handler : (q : ι) → StateT σ PMF (spec q)} {b calc ∑' y, (handler q s) y * ((interpret handler (cont y.1)).run y.2).toOuterMeasure {x | bad x.2} ≤ ∑' y, (handler q s) y * ({x : spec q × σ | bad x.2}.indicator 1 y + K) := - ENNReal.tsum_le_tsum hterm + tsum_le_tsum hterm _ = (handler q s).toOuterMeasure {x | bad x.2} + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun y => ?_ diff --git a/Complexity/Languages/Probabilistic/Vector.lean b/Complexity/Languages/Probabilistic/Vector.lean index 5e43aaaa5..4f3e7ead7 100644 --- a/Complexity/Languages/Probabilistic/Vector.lean +++ b/Complexity/Languages/Probabilistic/Vector.lean @@ -274,7 +274,7 @@ theorem eval_pi {m : ℕ} {α : Fin m → Type} [∀ i, Finite (α i)] funext i simpa using (congrFun hEq (Fin.succ i)).symm · intro a ha - simp only [mul_eq_zero, ENNReal.tsum_eq_zero] + simp only [mul_eq_zero, tsum_eq_zero_iff] right intro b rw [ite_eq_right] diff --git a/Complexity/Probability/Conditioning.lean b/Complexity/Probability/Conditioning.lean index faca6d699..340e70ef4 100644 --- a/Complexity/Probability/Conditioning.lean +++ b/Complexity/Probability/Conditioning.lean @@ -48,7 +48,7 @@ variable {α β : Type*} theorem map_fst_apply (joint : PMF (α × β)) (a : α) : (joint.map Prod.fst) a = ∑' b, joint (a, b) := by classical - rw [PMF.map_apply, ENNReal.tsum_prod'] + rw [PMF.map_apply, tsum_prod'] rw [tsum_eq_single a] · simp · intro x hx @@ -72,9 +72,9 @@ theorem conditionalSnd_apply (joint : PMF (α × β)) (a : α) conditionalSnd joint a b = joint (a, b) / (joint.map Prod.fst) a := by classical have hpost : a ∈ (joint.bind (fun pair => PMF.pure pair.1)).support := ha - simp only [conditionalSnd, hpost, dite_true, PMF.map_apply, ENNReal.tsum_prod', + simp only [conditionalSnd, hpost, dite_true, PMF.map_apply, tsum_prod', posteriorDist_apply, bind_pair_apply, PMF.pure_apply] - simp [ENNReal.tsum_prod', div_eq_mul_inv] + simp [tsum_prod', div_eq_mul_inv] /-- Multiplying by the marginal recovers the joint mass, also at zero-probability observations. -/ theorem marginal_mul_conditionalSnd (joint : PMF (α × β)) (a : α) (b : β) : diff --git a/Complexity/Probability/CorrelatedSampling.lean b/Complexity/Probability/CorrelatedSampling.lean index 5e9f61775..ae9ebd647 100644 --- a/Complexity/Probability/CorrelatedSampling.lean +++ b/Complexity/Probability/CorrelatedSampling.lean @@ -171,7 +171,7 @@ theorem map_firstIn_weightedOrderAux (w : Ω → ℝ≥0) : rcases o with _ | c · rw [PMF.bind_apply, PMF.map_apply] simp only [reduceCtorEq, ite_false, tsum_zero] - refine ENNReal.tsum_eq_zero.mpr fun a => ?_ + refine tsum_eq_zero_iff.mpr fun a => ?_ split_ifs <;> simp [PMF.map_apply] rw [PMF.bind_apply, tsum_fintype, hrhs] set du := weightedDraw w U hU0 diff --git a/Complexity/Probability/Forking.lean b/Complexity/Probability/Forking.lean index 960ea5e2e..b13827992 100644 --- a/Complexity/Probability/Forking.lean +++ b/Complexity/Probability/Forking.lean @@ -215,7 +215,7 @@ private theorem bind_match_apply {X : Type} (μ : PMF X) (index : X → Option ( | some j => decide (event j x))) true = ∑ j, (μ.bind fun x => PMF.pure (decide (index x = some j ∧ event j x))) true := by simp only [PMF.bind_apply, PMF.pure_apply] - rw [← Summable.tsum_finsetSum fun _ _ => ENNReal.summable] + rw [← Summable.tsum_finsetSum fun _ _ => CompleteLattice.summable] refine tsum_congr fun x => ?_ rw [← Finset.mul_sum] congr 1 diff --git a/Complexity/Probability/IID.lean b/Complexity/Probability/IID.lean index 74388f5b5..0a600cf4a 100644 --- a/Complexity/Probability/IID.lean +++ b/Complexity/Probability/IID.lean @@ -73,7 +73,7 @@ noncomputable def iid (μ : PMF α) : (k : ℕ) → PMF (Fin k → α) intro h exact absurd (by rw [h, Fin.tail_cons]) hw · intro a ha - rw [PMF.map_apply, ENNReal.tsum_eq_zero.mpr, mul_zero] + rw [PMF.map_apply, tsum_eq_zero_iff.mpr, mul_zero] intro w simp only [ite_eq_right_iff] intro h diff --git a/Complexity/Probability/MinEntropy.lean b/Complexity/Probability/MinEntropy.lean index 15120ffd6..29fd28b12 100644 --- a/Complexity/Probability/MinEntropy.lean +++ b/Complexity/Probability/MinEntropy.lean @@ -183,7 +183,7 @@ theorem le_minEntropy_bind (μ : PMF β) (f : β → PMF α) {k : ℝ} apply ENNReal.toReal_le_of_le_ofReal hk calc μ.bind f a = ∑' b, μ b * f b a := PMF.bind_apply μ f a - _ ≤ ∑' b, μ b * ENNReal.ofReal (2 ^ (-k)) := ENNReal.tsum_le_tsum hcomponent + _ ≤ ∑' b, μ b * ENNReal.ofReal (2 ^ (-k)) := tsum_le_tsum hcomponent _ = ENNReal.ofReal (2 ^ (-k)) := by rw [ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] /-! ### Weak random sources -/ diff --git a/Complexity/Probability/Minimax.lean b/Complexity/Probability/Minimax.lean index e34129b86..ba395a192 100644 --- a/Complexity/Probability/Minimax.lean +++ b/Complexity/Probability/Minimax.lean @@ -213,10 +213,10 @@ theorem exists_mem_support_tsum_mul_le (μ : PMF α) (ν : PMF β) (loss : α calc ∑' a, μ a * ∑' b, ν b * loss a b = ∑' b, ν b * ∑' a, μ a * loss a b := by simp only [← ENNReal.tsum_mul_left] - rw [ENNReal.tsum_comm] + rw [tsum_comm] exact tsum_congr fun b => tsum_congr fun a => by ring _ ≤ ∑' b, ν b * ε := by - refine ENNReal.tsum_le_tsum fun b => ?_ + refine tsum_le_tsum fun b => ?_ by_cases hb : b ∈ ν.support · gcongr exact h b hb diff --git a/Complexity/Probability/PMF/Bind.lean b/Complexity/Probability/PMF/Bind.lean index fcdd9502b..c3b9858bc 100644 --- a/Complexity/Probability/PMF/Bind.lean +++ b/Complexity/Probability/PMF/Bind.lean @@ -53,7 +53,7 @@ theorem bind_sigma_apply {β : α → Type*} (p : PMF α) (f : (a : α) → PMF (p.bind (fun a => (f a).map (Sigma.mk a))) ⟨a, b⟩ = p a * f a b := by classical simp only [PMF.bind_apply, PMF.map_apply, ← ENNReal.tsum_mul_left, mul_ite, mul_zero] - rw [← ENNReal.tsum_sigma] + rw [← tsum_sigma] simp /-- A continuation only needs to agree on outcomes the preceding distribution can produce. -/ @@ -76,7 +76,7 @@ theorem le_bind_apply_on_support (μ : PMF α) {f : α → PMF β} {b : β} {c : (h : ∀ a ∈ μ.support, c ≤ f a b) : c ≤ μ.bind f b := by rw [PMF.bind_apply] calc c = ∑' a, μ a * c := by rw [ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] - _ ≤ ∑' a, μ a * f a b := ENNReal.tsum_le_tsum fun a => by + _ ≤ ∑' a, μ a * f a b := tsum_le_tsum fun a => by by_cases ha : a ∈ μ.support · exact mul_le_mul_right (h a ha) _ · simp [(PMF.apply_eq_zero_iff μ a).2 ha] @@ -87,7 +87,7 @@ theorem bind_apply_le_on_support (μ : PMF α) {f : α → PMF β} {b : β} {c : (h : ∀ a ∈ μ.support, f a b ≤ c) : μ.bind f b ≤ c := by rw [PMF.bind_apply] calc ∑' a, μ a * f a b - _ ≤ ∑' a, μ a * c := ENNReal.tsum_le_tsum fun a => by + _ ≤ ∑' a, μ a * c := tsum_le_tsum fun a => by by_cases ha : a ∈ μ.support · exact mul_le_mul_right (h a ha) _ · simp [(PMF.apply_eq_zero_iff μ a).2 ha] @@ -98,7 +98,7 @@ probability in the whole experiment. The sample spaces may be infinite. -/ theorem bind_apply_mono_on_support (μ : PMF α) {f g : α → PMF β} (b : β) (h : ∀ a ∈ μ.support, f a b ≤ g a b) : μ.bind f b ≤ μ.bind g b := by simp only [PMF.bind_apply] - apply ENNReal.tsum_le_tsum + apply tsum_le_tsum intro a by_cases ha : a ∈ μ.support · exact mul_le_mul_right (h a ha) _ @@ -117,7 +117,7 @@ theorem exists_bind_apply_le [Finite α] (μ : PMF α) (f : α → PMF β) (b : /-- An average of values at most `B` is at most `B`. -/ theorem tsum_mul_le_of_forall_le (μ : PMF α) {f : α → ℝ≥0∞} {B : ℝ≥0∞} (h : ∀ a, f a ≤ B) : ∑' a, μ a * f a ≤ B := - calc ∑' a, μ a * f a ≤ ∑' a, μ a * B := ENNReal.tsum_le_tsum fun a => mul_le_mul_right (h a) _ + calc ∑' a, μ a * f a ≤ ∑' a, μ a * B := tsum_le_tsum fun a => mul_le_mul_right (h a) _ _ = B := by rw [ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] /-- Values at most `1` on a finite set `T` and at most `B` elsewhere average to at most @@ -126,9 +126,9 @@ theorem tsum_mul_le_card_mul_add [DecidableEq α] (μ : PMF α) {ε : ℝ≥0∞ (T : Finset α) {f : α → ℝ≥0∞} {B : ℝ≥0∞} (h : ∀ a, f a ≤ (if a ∈ T then 1 else 0) + B) : ∑' a, μ a * f a ≤ T.card * ε + B := by calc ∑' a, μ a * f a ≤ ∑' a, (μ a * (if a ∈ T then 1 else 0) + μ a * B) := - ENNReal.tsum_le_tsum fun a => (mul_le_mul_right (h a) _).trans_eq (mul_add _ _ _) + tsum_le_tsum fun a => (mul_le_mul_right (h a) _).trans_eq (mul_add _ _ _) _ = ∑ a ∈ T, μ a + B := by - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul, + rw [tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul, tsum_eq_sum (s := T) fun a ha => by simp only [ha, ↓reduceIte, mul_zero]] congr 1 exact Finset.sum_congr rfl fun a ha => by simp only [ha, ↓reduceIte, mul_one] diff --git a/Complexity/Probability/PMF/Event.lean b/Complexity/Probability/PMF/Event.lean index ce673d7ce..16a45d6e8 100644 --- a/Complexity/Probability/PMF/Event.lean +++ b/Complexity/Probability/PMF/Event.lean @@ -73,7 +73,7 @@ theorem toOuterMeasure_ne_top (p : PMF α) (event : Set α) : p.toOuterMeasure e /-- An event has probability at most one. -/ theorem toOuterMeasure_le_one (p : PMF α) (event : Set α) : p.toOuterMeasure event ≤ 1 := by rw [PMF.toOuterMeasure_apply, ← p.tsum_coe] - exact ENNReal.tsum_le_tsum (fun a => Set.indicator_apply_le (fun _ => le_rfl)) + exact tsum_le_tsum (fun a => Set.indicator_apply_le (fun _ => le_rfl)) /-- The event that `f a` deviates from `m` by at least `ε` is covered by the upper tail `m + ε ≤ f a` and the lower tail `f a ≤ m - ε`. -/ @@ -111,7 +111,7 @@ theorem toOuterMeasure_apply_toReal [Fintype α] (p : PMF α) (event : Set α) : theorem toOuterMeasure_toReal_add_compl (p : PMF α) (event : Set α) : (p.toOuterMeasure event).toReal + (p.toOuterMeasure eventᶜ).toReal = 1 := by have h : p.toOuterMeasure event + p.toOuterMeasure eventᶜ = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert p.tsum_coe using 1 congr 1 funext a @@ -154,7 +154,7 @@ theorem toOuterMeasure_bind_failure_le (p : PMF α) (kernel : α → PMF β) rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] calc _ ≤ ∑' a, ((preᶜ).indicator p a + p a * error) := by - apply ENNReal.tsum_le_tsum + apply tsum_le_tsum intro a by_cases ha : a ∈ pre · simp only [Set.indicator_of_notMem (show a ∉ preᶜ by simpa), zero_add] @@ -165,7 +165,7 @@ theorem toOuterMeasure_bind_failure_le (p : PMF α) (kernel : α → PMF β) · simp only [Set.indicator_of_mem (show a ∈ preᶜ from ha)] exact (mul_le_mul_of_nonneg_left (toOuterMeasure_le_one (kernel a) failure) zero_le).trans (by simp) - _ = _ := by rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, p.tsum_coe, one_mul] + _ = _ := by rw [tsum_add, ENNReal.tsum_mul_right, p.tsum_coe, one_mul] /-- The real-valued error rule for a randomized continuation. -/ theorem toOuterMeasure_bind_failure_toReal_le (p : PMF α) (kernel : α → PMF β) @@ -193,7 +193,7 @@ theorem toOuterMeasure_bind_ge_mul (p : PMF α) (kernel : α → PMF β) classical have hle : p.toOuterMeasure pre * ENNReal.ofReal s₂ ≤ (p.bind kernel).toOuterMeasure event := by rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ pre · simp only [Set.indicator_of_mem ha] by_cases hs : a ∈ p.support @@ -242,7 +242,7 @@ theorem map_true_mono (p : PMF α) {E F : α → Bool} (h : ∀ a ∈ p.support, E a = true → F a = true) : (p.map E) true ≤ (p.map F) true := by rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ p.support · by_cases hE : true = E a · simp [← hE, h a ha hE.symm] @@ -253,8 +253,8 @@ theorem map_true_mono (p : PMF α) {E F : α → Bool} theorem map_true_le_add (p : PMF α) {E A B : α → Bool} (h : ∀ a ∈ p.support, E a = true → A a = true ∨ B a = true) : (p.map E) true ≤ (p.map A) true + (p.map B) true := by - rw [PMF.map_apply, PMF.map_apply, PMF.map_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun a => ?_ + rw [PMF.map_apply, PMF.map_apply, PMF.map_apply, ← tsum_add] + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ p.support · by_cases hE : true = E a · rcases h a ha hE.symm with hA | hB diff --git a/Complexity/Probability/PMF/Map.lean b/Complexity/Probability/PMF/Map.lean index eed796d9b..4aad0a45d 100644 --- a/Complexity/Probability/PMF/Map.lean +++ b/Complexity/Probability/PMF/Map.lean @@ -40,7 +40,7 @@ variable {α β : Type*} theorem le_map_apply (p : PMF α) (f : α → β) (a : α) : p a ≤ (p.map f) (f a) := by classical rw [PMF.map_apply] - simpa using ENNReal.le_tsum (f := fun x => if f a = f x then p x else 0) a + simpa using le_tsum (f := fun x => if f a = f x then p x else 0) a /-- Relabeling a distribution by an equivalence preserves each corresponding point mass. -/ theorem map_equiv_apply (p : PMF α) (e : α ≃ β) (b : β) : diff --git a/Complexity/Probability/ProductConditioning/SideInformation.lean b/Complexity/Probability/ProductConditioning/SideInformation.lean index 316b7c63c..11c12f8fe 100644 --- a/Complexity/Probability/ProductConditioning/SideInformation.lean +++ b/Complexity/Probability/ProductConditioning/SideInformation.lean @@ -186,7 +186,7 @@ theorem sideJoint_apply (μ : PMF T) (J : T → PMF ((∀ i, β i) × V)) (t : T exact hx (Prod.ext (congrArg Prod.snd h).symm (congrArg (·.2) (congrArg Prod.fst h)).symm) · intro t' ht' rw [PMF.map_apply] - refine mul_eq_zero_of_right _ (ENNReal.tsum_eq_zero.mpr fun x => ite_eq_right fun h => ?_) + refine mul_eq_zero_of_right _ (tsum_eq_zero_iff.mpr fun x => ite_eq_right fun h => ?_) exact absurd (congrArg (·.1) (congrArg Prod.fst h)) (Ne.symm ht') /-- The pairs `(u, v)` that complete `(t, v)` to an outcome in `W`. -/ diff --git a/Complexity/Probability/RejectionSampling.lean b/Complexity/Probability/RejectionSampling.lean index d317c0d2f..83b85bf82 100644 --- a/Complexity/Probability/RejectionSampling.lean +++ b/Complexity/Probability/RejectionSampling.lean @@ -144,7 +144,7 @@ theorem shiftReject_apply_some (hv : ∀ z ∈ S, z - v ∈ B) (z : α) : · intro y hy have hyz : y + v ≠ z := fun h => hy (eq_sub_of_add_eq h) split_ifs <;> simp_all [eq_comm] - · refine ENNReal.tsum_eq_zero.mpr fun y => ?_ + · refine tsum_eq_zero_iff.mpr fun y => ?_ split_ifs with h₁ h₂ <;> simp_all /-- **Rejection sampling, aborts.** Under the same hypothesis, the sampler aborts with @@ -156,7 +156,7 @@ theorem shiftReject_apply_none (hv : ∀ z ∈ S, z - v ∈ B) : rw [tsum_eq_sum (s := S) fun z hz => by simp [hz], Finset.sum_ite_mem, Finset.inter_self, Finset.sum_const, nsmul_eq_mul, div_eq_mul_inv] have htotal := (shiftReject B S hB v).tsum_coe - rw [ENNReal.tsum_eq_add_tsum_ite none, ← (Option.some_injective α).tsum_eq] at htotal + rw [tsum_eq_add_tsum_ite none, ← (Option.some_injective α).tsum_eq] at htotal · simp only [reduceCtorEq, ite_false, hsome] at htotal exact ENNReal.eq_sub_of_add_eq (ENNReal.div_ne_top (ENNReal.natCast_ne_top _) (by simpa using hB.ne_empty)) htotal From d7dcd7985e43c0678127230df389fa8ea3e3ea61 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:21:52 -0400 Subject: [PATCH 08/18] fix(build): adapt proofs to the rc2 Mathlib API Qualify newly protected matrix and finite-sum lemmas, publicly import the relocated Matrix.Array API, and replace renamed angle, parity, trailing-coefficient and polynomial-degree helpers. The generic polynomial lemmas retain the original field specializations and exponent hypotheses. No theorem statement or definition body changes. Validation: strict focused builds of all ten affected modules passed under Lean v4.36.0-rc2. Source comparison against d6ccb24f shows all Lean edits are inside theorem proofs except the required Bird import. --- Complexity/Computability/Circuit/Bird.lean | 1 + Complexity/Computability/Circuit/Quantum/Basic.lean | 4 ++-- Complexity/Computability/Circuit/Rational/Wronskian.lean | 2 +- .../Foundations/BooleanAnalysis/LowDegree/Addition.lean | 2 +- .../Foundations/BooleanAnalysis/SparseEquations.lean | 2 +- .../FieldTheory/Finite/LinearizedPolynomial.lean | 6 +++--- Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean | 4 ++-- Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean | 2 +- .../Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean | 8 ++++---- Complexity/Foundations/Probability/Gaussian/Sheppard.lean | 2 +- 10 files changed, 17 insertions(+), 16 deletions(-) diff --git a/Complexity/Computability/Circuit/Bird.lean b/Complexity/Computability/Circuit/Bird.lean index 62b11531b..a39c61ae9 100644 --- a/Complexity/Computability/Circuit/Bird.lean +++ b/Complexity/Computability/Circuit/Bird.lean @@ -8,6 +8,7 @@ module -- upstream: candidate public import Complexity.Init +public import Mathlib.LinearAlgebra.Matrix.Array public import Mathlib.LinearAlgebra.Matrix.Determinant.Bird.Defs public import Mathlib.LinearAlgebra.Matrix.Determinant.Basic import Mathlib.Algebra.Order.BigOperators.Group.LocallyFinite diff --git a/Complexity/Computability/Circuit/Quantum/Basic.lean b/Complexity/Computability/Circuit/Quantum/Basic.lean index 7797874a4..c9265f0f4 100644 --- a/Complexity/Computability/Circuit/Quantum/Basic.lean +++ b/Complexity/Computability/Circuit/Quantum/Basic.lean @@ -172,7 +172,7 @@ theorem place_apply (A : Fin k ↪ Fin n) (U : unitaryGroup (Fin k → Bool) ℂ (place A U : Matrix _ _ ℂ) y x = if ∀ j, (¬∃ i, A i = j) → y j = x j then (U : Matrix _ _ ℂ) (y ∘ A) (x ∘ A) else 0 := by simp only [place, MonoidHom.coe_mk, OneHom.coe_mk, submatrix_apply, kroneckerMap_apply, - splitEquiv_apply_fst, splitEquiv_apply_snd, one_apply, funext_iff, Subtype.forall] + splitEquiv_apply_fst, splitEquiv_apply_snd, Matrix.one_apply, funext_iff, Subtype.forall] split_ifs <;> simp namespace PlacedGate @@ -281,7 +281,7 @@ theorem hadamardMatrix_mem_unitaryGroup : hadamardMatrix ∈ unitaryGroup Bool rw [mem_unitaryGroup_iff] ext b c cases b <;> cases c <;> - simp [hadamardMatrix, mul_apply, star_eq_conjTranspose, Complex.conj_ofReal, h2, h1] + simp [hadamardMatrix, Matrix.mul_apply, star_eq_conjTranspose, Complex.conj_ofReal, h2, h1] theorem star_hadamardMatrix : star hadamardMatrix = hadamardMatrix := by ext b c diff --git a/Complexity/Computability/Circuit/Rational/Wronskian.lean b/Complexity/Computability/Circuit/Rational/Wronskian.lean index f2ce76817..138a37718 100644 --- a/Complexity/Computability/Circuit/Rational/Wronskian.lean +++ b/Complexity/Computability/Circuit/Rational/Wronskian.lean @@ -148,7 +148,7 @@ lemma wronskian_order_of_distinct_orders {K : Type*} [Field K] [CharZero K] Matrix.det_mul_row (fun j => (f j).trailingCoeff) (Matrix.of fun i j : Fin h => ((f j).natTrailingDegree.descFactorial i.val : K))] exact mul_ne_zero (Finset.prod_ne_zero_iff.mpr fun j _ => - trailingCoeff_nonzero_iff_nonzero.mpr (hf j)) (det_descFactorial_ne_zero _ hd) + trailingCoeff_ne_zero.mpr (hf j)) (det_descFactorial_ne_zero _ hd) have hB0 : B.det ≠ 0 := fun hz => by simp [hz] at hBc have hA0 : A.det ≠ 0 := hdet ▸ mul_ne_zero (pow_ne_zero _ X_ne_zero) hB0 have hscale : A.det = X ^ (∑ i : Fin h, i.val) * wronskianFamily f := by diff --git a/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean b/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean index 905b6c11a..271991b3b 100644 --- a/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean +++ b/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean @@ -82,7 +82,7 @@ theorem hasSliceDecomposition_add (p q : ℕ) : have hexpand : (fun x y : ι → Bool => monomial K S.1 (x + y)) = ∑ T ∈ S.1.powerset, (fun x y => monomial K T x * monomial K (S.1 \ T) y) := by funext x y - simp only [sum_apply, monomial_add] + simp only [Finset.sum_apply, monomial_add] rw [hexpand] apply (sliceSpace (lowDegree K ι p) (lowDegree K ι q)).sum_mem intro T hT diff --git a/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean b/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean index d9daca94e..d1bc719e3 100644 --- a/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean +++ b/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean @@ -143,7 +143,7 @@ is even. -/ theorem monomialSum_zmod_two_eq_zero_iff_even (𝓜 : Finset (Finset ι)) (x : ι → Bool) : monomialSum (ZMod 2) 𝓜 x = 0 ↔ Even (monomialSum ℤ 𝓜 x) := by rw [monomialSum_eq_card_filter (ZMod 2), monomialSum_eq_card_filter ℤ, - ZMod.natCast_eq_zero_iff_even, Int.even_coe_nat] + ZMod.natCast_eq_zero_iff_even, Int.even_natCast] /-- An assignment `x : ι → Bool` is a common zero of the family `{P i}_{i ∈ I}` over `𝔽₂` if every equation `P i` (`i ∈ I`) evaluates to `0` in `ZMod 2` at `x`. -/ diff --git a/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean b/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean index 09309146e..a27032be5 100644 --- a/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean +++ b/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean @@ -252,7 +252,7 @@ private theorem comp_eq_X_pow_card_sub_X [Fintype F] {P Q : F[X]} ⟨by rw [natDegree_comp, hdeg], hQm.comp hPm hPdeg⟩ have hX : IsMonicOfDegree (X ^ Fintype.card F - X : F[X]) (Fintype.card F) := by rw [isMonicOfDegree_iff] - refine ⟨(FiniteField.X_pow_card_sub_X_natDegree_eq F hq1).le, ?_⟩ + refine ⟨(natDegree_X_pow_sub_X (R := F) hq1).le, ?_⟩ have h1 : (1 : ℕ) ≠ Fintype.card F := hq1.ne simp [coeff_X, h1] rw [← sub_eq_zero] @@ -410,10 +410,10 @@ theorem exists_subfield_card_le [Finite F] {n k : ℕ} (hn : finrank (ZMod 2) F have h1 : 1 < 2 ^ k := Nat.one_lt_two_pow hk0.ne' have hcard : Fintype.card F = 2 ^ n := by rw [Module.card_eq_pow_finrank (K := ZMod 2) (V := F), ZMod.card, hn] - refine FiniteField.X_pow_card_sub_X_ne_zero F h1 + refine X_pow_sub_X_ne_zero (R := F) h1 (eq_zero_of_natDegree_lt_card_of_eval_eq_zero _ (f := id) Function.injective_id (fun x => by simp [hall x]) ?_) - rw [FiniteField.X_pow_card_sub_X_natDegree_eq F h1, hcard] + rw [natDegree_X_pow_sub_X (R := F) h1, hcard] exact Nat.pow_lt_pow_right (by norm_num) hkn · by_cases hW : ∃ y₀ ∈ W, y₀ ≠ 0 · obtain ⟨y₀, hy₀W, hy₀⟩ := hW diff --git a/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean b/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean index 11579b762..50ea8c7ef 100644 --- a/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean +++ b/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean @@ -38,7 +38,7 @@ theorem list_prod_ofFn_apply : ∀ {d : ℕ} (M : Fin d → Matrix ι ι R) (a b if p 0 = a ∧ p (Fin.last d) = b then ∏ i : Fin d, M i (p i.castSucc) (p i.succ) else 0 | 0, M, a, b => by rw [← (Equiv.funUnique (Fin 1) ι).symm.sum_comp] - simp only [List.ofFn_zero, List.prod_nil, one_apply, Finset.univ_eq_empty, + simp only [List.ofFn_zero, List.prod_nil, Matrix.one_apply, Finset.univ_eq_empty, Finset.prod_empty, Equiv.funUnique_symm_apply, Fin.last_zero, uniqueElim_const] by_cases hab : a = b · subst hab @@ -51,7 +51,7 @@ theorem list_prod_ofFn_apply : ∀ {d : ℕ} (M : Fin d → Matrix ι ι R) (a b have hL : (List.ofFn M).prod a b = ∑ q : Fin (d + 1) → ι, if q (Fin.last d) = b then M 0 a (q 0) * ∏ i : Fin d, M i.succ (q i.castSucc) (q i.succ) else 0 := by - rw [List.ofFn_succ, List.prod_cons, mul_apply] + rw [List.ofFn_succ, List.prod_cons, Matrix.mul_apply] simp only [list_prod_ofFn_apply (fun i => M i.succ), Finset.mul_sum, mul_ite, mul_zero] rw [Finset.sum_comm] refine Finset.sum_congr rfl fun q _ => ?_ diff --git a/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean b/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean index 04e6d3604..de84069b6 100644 --- a/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean +++ b/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean @@ -87,7 +87,7 @@ theorem exists_eq_mul_rank (A : Matrix m n K) : (b.sum_repr (col j)) simp only [Submodule.coe_sum, Submodule.coe_smul, Finset.sum_apply, Pi.smul_apply, smul_eq_mul] at h - simpa [mul_apply, col, mul_comm] using h.symm + simpa [Matrix.mul_apply, col, mul_comm] using h.symm /-- The natural rank of a finite matrix agrees with the cardinal rank of its linear map. -/ theorem natCast_rank_eq_rank_toLin' [DecidableEq n] (M : Matrix m n K) : diff --git a/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean b/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean index 16c25ab37..7d6f58267 100644 --- a/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean +++ b/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean @@ -340,17 +340,17 @@ theorem add_blockRank_le_cutRank_add_transpose [Finite ι] (hfg.symm.mono inter_subset_right inter_subset_right) (fun i hi j hj hj' => by have hjY : j ∉ Y := fun h => hj' ⟨hj, h⟩ - simp only [add_apply, transpose_apply, hMY i j hjY, hMY j i (hnotY i hi.2), add_zero]) + simp only [Matrix.add_apply, transpose_apply, hMY i j hjY, hMY j i (hnotY i hi.2), add_zero]) (fun i hi j hj hj' => by have hjX : j ∉ X := fun h => hj' ⟨hj, h⟩ - simp only [add_apply, transpose_apply, hMX i j (hnotX i hi.2), hMX j i hjX, add_zero]) + simp only [Matrix.add_apply, transpose_apply, hMX i j (hnotX i hi.2), hMX j i hjX, add_zero]) have hb₁ : R.blockRank (f ⁻¹' U) (g ⁻¹' Uᶜ) ≤ (M + Mᵀ).blockRank (U ∩ X) Uᶜ := by calc R.blockRank (f ⁻¹' U) (g ⁻¹' Uᶜ) = M.blockRank (U ∩ X) (Uᶜ ∩ Y) := by rw [← hMR, M.blockRank_submatrix hf hg, image_preimage_eq_inter_range, image_preimage_eq_inter_range] _ = (M + Mᵀ).blockRank (U ∩ X) (Uᶜ ∩ Y) := blockRank_congr fun i hi j _ => by - simp only [add_apply, transpose_apply, hMY j i (hnotY i hi.2), add_zero] + simp only [Matrix.add_apply, transpose_apply, hMY j i (hnotY i hi.2), add_zero] _ ≤ (M + Mᵀ).blockRank (U ∩ X) Uᶜ := blockRank_mono_right _ _ inter_subset_left have hb₂ : R.blockRank (f ⁻¹' Uᶜ) (g ⁻¹' U) ≤ (M + Mᵀ).blockRank (U ∩ Y) Uᶜ := by calc R.blockRank (f ⁻¹' Uᶜ) (g ⁻¹' U) @@ -359,7 +359,7 @@ theorem add_blockRank_le_cutRank_add_transpose [Finite ι] image_preimage_eq_inter_range] _ = Mᵀ.blockRank (U ∩ Y) (Uᶜ ∩ X) := (M.blockRank_transpose _ _).symm _ = (M + Mᵀ).blockRank (U ∩ Y) (Uᶜ ∩ X) := blockRank_congr fun i hi j _ => by - simp only [add_apply, transpose_apply, hMX i j (hnotX i hi.2), zero_add] + simp only [Matrix.add_apply, transpose_apply, hMX i j (hnotX i hi.2), zero_add] _ ≤ (M + Mᵀ).blockRank (U ∩ Y) Uᶜ := blockRank_mono_right _ _ inter_subset_left unfold cutRank omega diff --git a/Complexity/Foundations/Probability/Gaussian/Sheppard.lean b/Complexity/Foundations/Probability/Gaussian/Sheppard.lean index e3c419ba0..e18ca8f6d 100644 --- a/Complexity/Foundations/Probability/Gaussian/Sheppard.lean +++ b/Complexity/Foundations/Probability/Gaussian/Sheppard.lean @@ -308,7 +308,7 @@ theorem measureReal_stdGaussian_mem_uIoc_le_angle {u v : E} (huv : ‖u‖ = ‖ simp only [hu, hv, inner_zero_left, uIoc_of_le le_rfl, Ioc_self, mem_empty_iff_false, ofPred_false, measureReal_empty] exact div_nonneg (InnerProductGeometry.angle_nonneg _ _) pi_pos.le - · rw [eq_neg_of_add_eq_zero_right hne, InnerProductGeometry.angle_self_neg_of_nonzero hu, + · rw [eq_neg_of_add_eq_zero_right hne, InnerProductGeometry.angle_self_neg_of_ne_zero hu, div_self pi_ne_zero] exact measureReal_le_one · rw [InnerProductGeometry.angle_eq_two_mul_arctan huv hne] From f5a70cb9826c88744c1dfbc560340d3efebbfb04 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:26:19 -0400 Subject: [PATCH 09/18] test(quantum): qualify the protected matrix lemma Preserve the existing non-unitary matrix counterexample under rc2 Mathlib by explicitly selecting Matrix.mul_apply, as in the library proof repairs. --- ComplexityTests/QuantumCircuit.lean | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ComplexityTests/QuantumCircuit.lean b/ComplexityTests/QuantumCircuit.lean index a440e1a4b..cca9dbe2c 100644 --- a/ComplexityTests/QuantumCircuit.lean +++ b/ComplexityTests/QuantumCircuit.lean @@ -158,7 +158,7 @@ example : ¬Circuit.IsOver {⟨3, Gate.toffoli⟩} [h₀] := fun h => example : !![(1 : ℂ), 1; 0, 1] ∉ unitaryGroup (Fin 2) ℂ := by intro h have := congrFun (congrFun (mem_unitaryGroup_iff.mp h) 0) 0 - simp [mul_apply, Fin.sum_univ_two, star_eq_conjTranspose] at this + simp [Matrix.mul_apply, Fin.sum_univ_two, star_eq_conjTranspose] at this -- A two-qubit gate cannot be placed on a one-qubit register: the wires must be distinct. example : IsEmpty (Fin 2 ↪ Fin 1) := From bb3e910dfc197e46f4b11e0c666b6546a075b7c8 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:46:20 -0400 Subject: [PATCH 10/18] fix(build): normalize goals before proof automation Use expected-type-directed refinement for the generic zero-sum lemma and expose the determinant variable-count function before fun_prop. Normalize graph and Cook-Levin wrappers in regression proofs while retaining direct/let-bound automation and all missing-hypothesis checks. Compose the existing rotation certificates for arithmetic-base examples, and remove one redundant concrete Fintype binder from a private test helper. Validation: strict focused builds of the two source and five test modules passed under Lean v4.36.0-rc2. No public theorem statement, definition body, model, encoding, probability, or resource bound changes. No heartbeat or fun_prop compatibility option was introduced. --- Complexity/Algorithms/WordRAM/KargerMinCut.lean | 2 +- .../Computability/Circuit/Arithmetic/Determinant.lean | 4 +++- ComplexityTests/ExpanderContraction.lean | 3 ++- ComplexityTests/ExpanderFamily.lean | 1 + ComplexityTests/ExplicitExpanders.lean | 8 ++++++-- ComplexityTests/Iteration/SparseDisplacement.lean | 2 +- ComplexityTests/MultiTapePolyTime.lean | 1 + 7 files changed, 15 insertions(+), 6 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/KargerMinCut.lean b/Complexity/Algorithms/WordRAM/KargerMinCut.lean index c07c84a3c..5cf511394 100644 --- a/Complexity/Algorithms/WordRAM/KargerMinCut.lean +++ b/Complexity/Algorithms/WordRAM/KargerMinCut.lean @@ -1181,7 +1181,7 @@ theorem ampLoop_spec {w sb n R : ℕ} {W : Fin n → Fin n → ℕ} {mem₀ : obtain ⟨m_r, hm_r, _, hsup_r, _⟩ := hnext z₁ hz₁ rw [hml_def, hm_r.codeComp_eq, PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - convert tsum_eq_zero_iff.mpr _ + refine tsum_eq_zero_iff.mpr ?_ intro z₂ by_cases hz₂_eval : z₂ ∈ m_r.eval.support · have hz₂ : z₂ ∈ m_r.support := (ProbComp.support_eval m_r) ▸ hz₂_eval diff --git a/Complexity/Computability/Circuit/Arithmetic/Determinant.lean b/Complexity/Computability/Circuit/Arithmetic/Determinant.lean index 1f231b4cf..01cf49231 100644 --- a/Complexity/Computability/Circuit/Arithmetic/Determinant.lean +++ b/Complexity/Computability/Circuit/Arithmetic/Determinant.lean @@ -272,7 +272,9 @@ generic determinants has `n * n` variables, degree at most `n` and arithmetic ci `3 * n ^ 4 + 4` gates. -/ theorem detFamily_mem_VP : detFamily K ∈ VP K := by choose c hc hsize using exists_circuit_det (K := K) - refine ⟨⟨by fun_prop, PolynomiallyBounded.id.mono_of_le fun n => + refine ⟨⟨?_, PolynomiallyBounded.id.mono_of_le fun n => totalDegree_det_genericMatrix_le n⟩, c, PolynomiallyBounded.mono_of_le (by fun_prop) hsize, hc⟩ + change PolynomiallyBounded (fun n : ℕ => n * n) + fun_prop end Cslib.Circuits.Arithmetic diff --git a/ComplexityTests/ExpanderContraction.lean b/ComplexityTests/ExpanderContraction.lean index ac59474d9..1bd9d7150 100644 --- a/ComplexityTests/ExpanderContraction.lean +++ b/ComplexityTests/ExpanderContraction.lean @@ -68,7 +68,7 @@ example : ComputableInPolyTime threeVertices.rot have hg : ComputableInPolyTime completeFive.rot (encPair (encFinEnum (Fin 5)) (encFinEnum (Fin 5))) (encPair (encFinEnum (Fin 5)) (encFinEnum (Fin 5))) := .of_finite _ - fun_prop [threeVertices, RotationMap.contractFin] + exact ComputableInPolyTime.rotationMap_contract he hd hg example {V W I D : Type} (G : RotationMap V D) (encode : V → W × I) (decode : W × I → Option V) (hinv : Function.IsPartialInv encode decode) @@ -80,6 +80,7 @@ example {V W I D : Type} (G : RotationMap V D) (encode : V → W × I) (fun p : (W × (I × D)) × Bool => ((RotationMap.contract encode decode hinv G).rot p.1, !p.2)) (encPair (encPair ew (encPair ei ed)) encBool) (encPair (encPair ew (encPair ei ed)) encBool) := by + dsimp only [RotationMap.contract, RotationMap.contractRot, RotationMap.contractDart] fail_if_success (clear he; fun_prop) fail_if_success (clear hd; fun_prop) fail_if_success (clear hg; fun_prop) diff --git a/ComplexityTests/ExpanderFamily.lean b/ComplexityTests/ExpanderFamily.lean index 0dd5ac6e9..51163712e 100644 --- a/ComplexityTests/ExpanderFamily.lean +++ b/ComplexityTests/ExpanderFamily.lean @@ -96,6 +96,7 @@ example {V W D : Type} (G : RotationMap V D) (e : V ≃ W) let next := (G.reindex e).rot p.1 (next, !p.2)) (encPair (encPair ew ed) encBool) (encPair (encPair ew ed) encBool) := by + dsimp only [RotationMap.reindex, RotationMap.reindexRot] fail_if_success (clear hf; fun_prop) fail_if_success (clear hb; fun_prop) fail_if_success (clear hg; fun_prop) diff --git a/ComplexityTests/ExplicitExpanders.lean b/ComplexityTests/ExplicitExpanders.lean index a912aabc8..c8d706921 100644 --- a/ComplexityTests/ExplicitExpanders.lean +++ b/ComplexityTests/ExplicitExpanders.lean @@ -64,7 +64,8 @@ example {K : Type} [Field K] (encode : K ↪ List Bool) ComputableInPolyTime (RotationMap.affinePlaneTowerBase K).rot (encPair (baseEncoding encode) (planeEncoding encode)) (encPair (baseEncoding encode) (planeEncoding encode)) := by - fun_prop [RotationMap.affinePlaneTowerBase] + have hplane := ComputableInPolyTime.rotationMap_affinePlane hmul hsub + exact ((hplane.rotationMap_tensor hplane).rotationMap_zigzag hplane).rotationMap_zigzag hplane -- Only the fixed field operations use the finite-domain theorem; graph products compose them. example (encode : ZMod 257 ↪ List Bool) : @@ -75,7 +76,8 @@ example (encode : ZMod 257 ↪ List Bool) : (planeEncoding encode) encode := .of_finite _ have hsub : ComputableInPolyTime (fun p : ZMod 257 × ZMod 257 => p.1 - p.2) (planeEncoding encode) encode := .of_finite _ - fun_prop [RotationMap.affinePlaneTowerBase] + have hplane := ComputableInPolyTime.rotationMap_affinePlane hmul hsub + exact ((hplane.rotationMap_tensor hplane).rotationMap_zigzag hplane).rotationMap_zigzag hplane -- Equivalent direct and let-bound spellings, with a supplied rotation on an arbitrary type. example {V D E : Type} (G : RotationMap V D) (H : RotationMap D E) @@ -86,6 +88,7 @@ example {V D E : Type} (G : RotationMap V D) (H : RotationMap D E) (fun p : ((V × D) × (E × E)) × Bool => ((G.zigzag H).rot p.1, !p.2)) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) := by + dsimp only [RotationMap.zigzag, RotationMap.zigzagRot] fail_if_success (clear hG; fun_prop) fail_if_success (clear hH; fun_prop) fun_prop @@ -100,6 +103,7 @@ example {V D E : Type} (G : RotationMap V D) (H : RotationMap D E) (next, !p.2)) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) := by + dsimp only [RotationMap.zigzag, RotationMap.zigzagRot] fun_prop end ComplexityTests.ExplicitExpanders diff --git a/ComplexityTests/Iteration/SparseDisplacement.lean b/ComplexityTests/Iteration/SparseDisplacement.lean index 634e74f09..fbe746f4f 100644 --- a/ComplexityTests/Iteration/SparseDisplacement.lean +++ b/ComplexityTests/Iteration/SparseDisplacement.lean @@ -34,7 +34,7 @@ example : private def editedCounter : Equiv.Perm (BitVec 2) := (Equiv.swap 0 1).trans (Equiv.addRight 1) -private theorem editedCounter_disagreement [Fintype (BitVec 2)] : +private theorem editedCounter_disagreement : disagreement editedCounter (fun x => x + 1) = {0, 1} := by ext x simp only [mem_disagreement, Finset.mem_insert, Finset.mem_singleton] diff --git a/ComplexityTests/MultiTapePolyTime.lean b/ComplexityTests/MultiTapePolyTime.lean index 2042aace5..8fde4c4a0 100644 --- a/ComplexityTests/MultiTapePolyTime.lean +++ b/ComplexityTests/MultiTapePolyTime.lean @@ -139,6 +139,7 @@ example {α : Type} (enc : α ↪ List Bool) (p : Polynomial ℕ) (c d : ℕ) : ComputableTidilyInPolyTime (fun a => List.range (Cslib.Complexity.CookLevin.parameters p c d enc a).2.2) enc (encList encUnary) := by + dsimp only [Cslib.Complexity.CookLevin.parameters] fun_prop private def finish (move : SignType) (symbol : Bool) : Turing.MultiTapeTM 0 Bool Unit := From 1cdc80ed67756efebde20dfabe1570b5ad5bc678 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:07:03 -0400 Subject: [PATCH 11/18] test(wordram): exercise two-outcome randomized subroutines Initialize and preserve the source register at one, requiring 1 <= w, so each native rand samples Fin 2. Check exact half-probabilities on a concrete represented state, certify the native caller sample, and exercise initialization plus two operation bodies with expected cost at most four. Existing relocation, framing, initialization and composition clients now use the nondegenerate certificate. Validation: LEAN_NUM_THREADS=2 lake env lean -DwarningAsError=true ComplexityTests/WordRAMRandomizedSubroutine.lean; git diff --check. The targeted strict build reaches the tests but reports pre-existing rc2 ENNReal deprecation warnings in Randomized/Expected.lean and Randomized/Subroutine.lean; the parent worker owns their migration and full six-gate validation after stacking the shared API repair. No curated statement, scope or rendering changes. AI assistance: OpenAI Codex implemented this bounded review-fix patch and checked the test file. --- .../WordRAMRandomizedSubroutine.lean | 164 ++++++++++++------ 1 file changed, 109 insertions(+), 55 deletions(-) diff --git a/ComplexityTests/WordRAMRandomizedSubroutine.lean b/ComplexityTests/WordRAMRandomizedSubroutine.lean index 1a335f4b5..23f2c2292 100644 --- a/ComplexityTests/WordRAMRandomizedSubroutine.lean +++ b/ComplexityTests/WordRAMRandomizedSubroutine.lean @@ -109,7 +109,7 @@ theorem geomCoinLoop_execCodeDist {w : ℕ} (hw : 1 ≤ w) (x₀ : Data 2) simp [y₀, Cond.Holds, Data.set, hr1] have h1 : (2 : ℝ≥0∞)⁻¹ ≤ geomBodyDist x y₀ := by rw [geomBodyDist, PMF.map_apply] - have hle := ENNReal.le_tsum + have hle := le_tsum (f := fun k : Fin (x.regs g1 + 1) => if y₀ = x.set g0 k.val then uniformOfFintype (Fin (x.regs g1 + 1)) k else 0) k₀ simpa [y₀, k₀, uniformOfFintype_apply, hr1] using hle @@ -120,7 +120,8 @@ theorem geomCoinLoop_execCodeDist {w : ℕ} (hw : 1 ≤ w) (x₀ : Data 2) = Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y₀ := by rw [Set.indicator_of_mem hy₀_not] _ ≤ ∑' y, Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y := - ENNReal.le_tsum y₀ + le_tsum (f := fun y => Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} + (geomBodyDist x) y) y₀ exact le_trans h1 h2 have hx₀_inv : LoopStateInv (.eq g0 g1) (geomInv w) (geomPost w) x₀ := Or.inl ⟨hx₀.2, hx₀.1⟩ refine ⟨loopExitPMF (.eq g0 g1) geomBodyDist x₀, @@ -163,11 +164,11 @@ def coinSpec : Stateful.Spec.{0, 0, 0} where /-- Subroutine code with 5 registers (`0` = `baseReg`, `1` = `frontierReg`, `2` = `zeroReg`, `3` = `curReg` / scratch output, `4` = `oneReg`). Its operation body `code` executes -`r3 ← rand r2` (`Instr.rand`). -/ +`r3 ← rand r4` (`Instr.rand`), with `r4 = 1`, so it samples both `0` and `1`. -/ def coinSubroutineCode : SubroutineCode .exact 5 0 where allocSpan := .op (.copy 0 1) - initSetup := .op (.const 2 0) - code := .rand 3 2 + initSetup := .op (.const 4 1) + code := .rand 3 4 baseReg := 0 frontierReg := 1 zeroReg := 2 @@ -179,9 +180,9 @@ def coinSubroutineCode : SubroutineCode .exact 5 0 where output := 3 /-- Representation invariant for `coinSubroutineCode`: machine words are `w`-bounded, `baseReg` -holds `base`, and `zeroReg` holds `0`. -/ +holds `base`, and `oneReg` holds `1`. -/ def coinRep (w base : ℕ) (_θ : Fin 2) (_s : PUnit) (x : Data 5) : Prop := - x.Bounded w ∧ x.regs 0 = base ∧ x.regs 2 = 0 + x.Bounded w ∧ x.regs 0 = base ∧ x.regs 4 = 1 private theorem pmf_map_fst_const {α β γ : Type*} (q : PMF α) (b : β) (g : α → γ) : (q.map fun a => (b, g a)).map Prod.fst = PMF.pure b := by @@ -198,8 +199,9 @@ private theorem pmf_map_snd_const {α β γ : Type*} (q : PMF α) (f : α → β by_cases h : y = c <;> simp [PMF.map_apply, h, q.tsum_coe] /-- Concrete randomized `SubroutineDS` certificate for `coinSubroutineCode` with latent seed type -`Fin 2` and uniform reference distribution `uniformOfFintype (Fin 2)`. -/ -noncomputable def coinSubroutineDS (w base : ℕ) : +`Fin 2` and uniform reference distribution `uniformOfFintype (Fin 2)`. The native operation +samples from `Fin 2`; its randomness is separate from this latent coupling seed. -/ +noncomputable def coinSubroutineDS (w base : ℕ) (hw : 1 ≤ w) : Cslib.WordRAM.Randomized.SubroutineDS coinSubroutineCode coinSpec w base Fin.elim0 PUnit.unit where Seed := Fin 2 @@ -223,7 +225,7 @@ noncomputable def coinSubroutineDS (w base : ℕ) : rep_supported hreg _ hb' hrep := ⟨hb', by rw [hreg 0 rfl, hrep.2.1], - by rw [hreg 2 rfl, hrep.2.2]⟩ + by rw [hreg 4 rfl, hrep.2.2]⟩ potential_supported _ _ _ := rfl clear_nodup := by decide params_not_clear j := Fin.elim0 j @@ -238,11 +240,11 @@ noncomputable def coinSubroutineDS (w base : ℕ) : · simpa [coinSubroutineCode, Data.set] using hfr · simpa [coinSubroutineCode, Data.set] using hfr setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by - let x₂ : Data 5 := x₁.set 2 0 + let x₂ : Data 5 := x₁.set 4 1 let ν₀ : PMF (Fin 2 × Data 5) := (uniformOfFintype (Fin 2)).map fun θ => (θ, x₂) refine ⟨ν₀, ?_, ?_, ?_⟩ - · have hex : ExecCodeDist (B := .exact) w (.op (.const 2 0)) x₁ (PMF.pure x₂) (1 : ℕ) := - ExecCodeDist.of_exec (Exec.const (fits_zero w)) + · have hex : ExecCodeDist (B := .exact) w (.op (.const 4 1)) x₁ (PMF.pure x₂) (1 : ℕ) := + ExecCodeDist.of_exec (Exec.const (fits_one_of_pos hw)) exact hex.congr_dist (pmf_map_snd_const (uniformOfFintype (Fin 2)) id x₂).symm (by simp) · intro θ rw [PMF.map_comp] @@ -252,53 +254,73 @@ noncomputable def coinSubroutineDS (w base : ℕ) : obtain ⟨θ, _, rfl⟩ := hp refine ⟨?_, ?_, Nat.le_refl 0, Modifies.refl _ _⟩ · simpa [coinSubroutineCode, x₂, Data.set] using hfr₁ - · refine ⟨hx₁.set (fits_zero w), ?_, ?_⟩ + · refine ⟨hx₁.set (fits_one_of_pos hw), ?_, ?_⟩ · simp [x₂, Data.set, ← hb₁, coinSubroutineCode] · simp [x₂, Data.set] step_exec _ ν _ hseed hsupp := by let μ_step : Fin 2 × Data 5 → PMF (Fin 2 × Data 5) := fun p => - (uniformOfFintype (Fin (p.2.regs 2 + 1))).map fun k => (p.1, p.2.set 3 k.val) + (uniformOfFintype (Fin (p.2.regs 4 + 1))).map fun k => (p.1, p.2.set 3 k.val) refine ⟨μ_step, fun _ => 1, ?_, ?_, ?_⟩ · intro p hp - obtain ⟨⟨hbp, hb0, hz2⟩, _⟩ := hsupp p hp + obtain ⟨⟨hbp, hb0, ho4⟩, _⟩ := hsupp p hp refine ⟨?_, ?_⟩ - · have hr := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 3) (src := 2) - (hbp.1 2) + · have hr := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 3) (src := 4) + (hbp.1 4) refine hr.congr_dist ?_ le_rfl rw [ProbComp.eval_unifFin, PMF.map_comp] rfl · intro q hq rw [PMF.mem_support_map_iff] at hq obtain ⟨k, _, rfl⟩ := hq - have hk0 : k.val = 0 := by omega + have hk1 : k.val ≤ 1 := by omega refine ⟨rfl, ⟨?_, ?_, ?_⟩, Modifies.refl _ _⟩ - · refine hbp.set ?_ - rw [hk0] - exact fits_zero w + · exact hbp.set ((fits_one_of_pos hw).of_le hk1) · simp [Data.set, hb0] - · simp [Data.set, hz2] + · simp [Data.set, ho4] · intro θ' have hmap : (ν.bind μ_step).map Prod.fst = ν.map Prod.fst := by rw [PMF.map_bind, PMF.map] refine congrArg (ν.bind ·) (funext fun p => ?_) - exact pmf_map_fst_const (uniformOfFintype (Fin (p.2.regs 2 + 1))) p.1 + exact pmf_map_fst_const (uniformOfFintype (Fin (p.2.regs 4 + 1))) p.1 (fun k => p.2.set 3 k.val) rw [hmap] exact hseed θ' · simp [ν.tsum_coe] +/-- The concrete represented state with `oneReg = 1` executes the native sampler in one body +instruction, and its output register is exactly fair on both `0` and `1`. -/ +theorem coinSubroutineCode_two_outcomes {w : ℕ} (hw : 1 ≤ w) : + let x : Data 5 := (Data.init []).set 4 1 + let μ : PMF (Data 5) := (uniformOfFintype (Fin 2)).map fun k => x.set 3 k.val + coinRep w 0 0 PUnit.unit x ∧ + ExecCodeDist w coinSubroutineCode.code x μ 1 ∧ + (μ.map fun y => y.regs 3) 0 = (2 : ℝ≥0∞)⁻¹ ∧ + (μ.map fun y => y.regs 3) 1 = (2 : ℝ≥0∞)⁻¹ := by + intro x μ + have hx : x.Bounded w := + (Data.Bounded.init (by simp [InputFits])).set (fits_one_of_pos hw) + refine ⟨⟨hx, by simp [x, Data.set], by simp [x, Data.set]⟩, ?_, ?_, ?_⟩ + · have hsource : x.regs 4 = 1 := by simp [x, Data.set] + have he := ExecCodeDist.rand (B := .exact) (w := w) (x := x) (dst := 3) (src := 4) (hx.1 4) + rw [ProbComp.eval_unifFin, hsource] at he + exact he + · simp [μ, PMF.map_comp, Function.comp_def, PMF.map_apply, tsum_fintype, + Fin.sum_univ_two, uniformOfFintype_apply, Data.set] + · simp [μ, PMF.map_comp, Function.comp_def, PMF.map_apply, tsum_fintype, + Fin.sum_univ_two, uniformOfFintype_apply, Data.set] + /-- Standard-basis lifting of `coinSubroutineDS`. -/ -noncomputable def coinSubroutineDS_standard (w base : ℕ) : +noncomputable def coinSubroutineDS_standard (w base : ℕ) (hw : 1 ≤ w) : Cslib.WordRAM.Randomized.SubroutineDS coinSubroutineCode.toStandard coinSpec w base Fin.elim0 PUnit.unit := - (coinSubroutineDS w base).toStandard + (coinSubroutineDS w base hw).toStandard /-- Fresh initialization + multi-operation expected-amortized sequence bound for `coinSubroutineDS`. -/ -theorem coinSubroutineDS_seq_amortized (w base : ℕ) +theorem coinSubroutineDS_seq_amortized (w base : ℕ) (hw : 1 ≤ w) (ops : List PUnit) {x₀ : Data 5} (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : - let C := coinSubroutineDS w base + let C := coinSubroutineDS w base hw ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), ExecCodeDist w coinSubroutineCode.initFresh x₀ (ν₀.map Prod.snd) C.initTime ∧ C.ExecSteps PUnit.unit ops ν₀ ν_final T_ops ∧ @@ -317,10 +339,34 @@ theorem coinSubroutineDS_seq_amortized (w base : ℕ) simp [Block.footprint] exact C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid +/-- From concrete empty memory, initialization and two genuine coin-sampling bodies have +expected cost at most four; external argument delivery is uncharged by `ExecSteps`. -/ +example : + let C := coinSubroutineDS 1 0 (by decide) + ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), + ExecCodeDist 1 coinSubroutineCode.initFresh (Data.init []) (ν₀.map Prod.snd) C.initTime ∧ + C.ExecSteps PUnit.unit [PUnit.unit, PUnit.unit] ν₀ ν_final T_ops ∧ + C.initTime + T_ops ≤ 4 := by + intro C + obtain ⟨ν₀, ν_final, T_ops, hinit, hsteps, _, _, hcost⟩ := + coinSubroutineDS_seq_amortized 1 0 (by decide) [PUnit.unit, PUnit.unit] + (x₀ := (Data.init [] : Data 5)) + (Data.Bounded.init (by simp [InputFits])) (by simp [Data.init]) + refine ⟨ν₀, ν_final, T_ops, hinit, hsteps, ?_⟩ + have hT : T_ops ≤ 2 := by + simpa [C, coinSubroutineDS, Cslib.WordRAM.Randomized.SubroutineDS.initTime, + Stateful.Spec.totalCost, coinSpec] using hcost + have hinitTime : C.initTime = 2 := by + norm_num [C, coinSubroutineDS, Cslib.WordRAM.Randomized.SubroutineDS.initTime] + rw [hinitTime] + calc + 2 + T_ops ≤ 2 + 2 := add_le_add le_rfl hT + _ = 4 := by norm_num + /-- Dirty-memory initialization via `initClear_exec` for `coinSubroutineDS`. -/ -theorem coinSubroutineDS_initClear (w base : ℕ) {x₀ : Data 5} +theorem coinSubroutineDS_initClear (w base : ℕ) (hw : 1 ≤ w) {x₀ : Data 5} (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : - let C := coinSubroutineDS w base + let C := coinSubroutineDS w base hw ∃ ν₀ : PMF (C.Seed × Data 5), ExecCodeDist w coinSubroutineCode.initClear x₀ (ν₀.map Prod.snd) (C.initTime + (4 * C.space + 6 : ℕ)) ∧ @@ -336,13 +382,13 @@ theorem coinSubroutineDS_initClear (w base : ℕ) {x₀ : Data 5} /-- Inline caller subroutine invocation via `call_rename` for `coinSubroutineDS`: a caller with 6 registers embeds `Fin 5` via `Fin.castSucc`, executes `prep := .op (.const 5 0)` on its own register `r5`, and invokes `coinSubroutineCode.code.rename Fin.castSucc`. -/ -theorem coinSubroutineDS_call_rename (w base : ℕ) - (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) +theorem coinSubroutineDS_call_rename (w base : ℕ) (hw : 1 ≤ w) + (ν : PMF ((coinSubroutineDS w base hw).Seed × Data 6)) (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ - (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (coinSubroutineDS w base hw).seedSlack * (coinSubroutineDS w base hw).seedLaw θ) (hrep : ∀ p ∈ ν.support, - (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : - let C := coinSubroutineDS w base + (coinSubroutineDS w base hw).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base hw let prep : Code .exact 6 := .op (.const 5 0) ∃ (μ_call : C.Seed × Data 6 → PMF (C.Seed × Data 6)) (T_call : C.Seed × Data 6 → ℝ≥0∞), @@ -384,34 +430,42 @@ theorem coinSubroutineDS_call_rename (w base : ℕ) simp [Data.set, hne] /-- Caller framing across a randomized caller step: a caller with 6 registers embeds `Fin 5` via -`Fin.castSucc` and executes `r5 ← rand r2` on its own register `r5` (`5 : Fin 6`), preserving the +`Fin.castSucc` and executes `r5 ← rand r4` on its own register `r5` (`5 : Fin 6`), preserving the coupled seed marginal bound, `rep`, and expected potential via `frame_caller_dist`. -/ -theorem coinSubroutineDS_caller_rand_frame (w base : ℕ) - (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) +theorem coinSubroutineDS_caller_rand_frame (w base : ℕ) (hw : 1 ≤ w) + (ν : PMF ((coinSubroutineDS w base hw).Seed × Data 6)) (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ - (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (coinSubroutineDS w base hw).seedSlack * (coinSubroutineDS w base hw).seedLaw θ) (hrep : ∀ p ∈ ν.support, - (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : - let C := coinSubroutineDS w base + (coinSubroutineDS w base hw).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base hw let μ_caller : C.Seed × Data 6 → PMF (Data 6) := fun p => - (uniformOfFintype (Fin 1)).map fun k => p.2.set 5 k.val + (uniformOfFintype (Fin 2)).map fun k => p.2.set 5 k.val + (∀ p ∈ ν.support, ExecCodeDist (B := .exact) w (.rand 5 4) p.2 (μ_caller p) 1) ∧ (∀ θ, ((C.frameDist ν μ_caller).map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ q ∈ (C.frameDist ν μ_caller).support, C.rep q.1 PUnit.unit (q.2.comap Fin.castSucc)) ∧ (∑' q, C.frameDist ν μ_caller q * (C.potential q.1 (q.2.comap Fin.castSucc) : ℝ≥0∞)) = ∑' p, ν p * (C.potential p.1 (p.2.comap Fin.castSucc) : ℝ≥0∞) := by intro C μ_caller - refine C.frame_caller_dist (F := ∅) hseed hrep (Finset.disjoint_empty_right _) ?_ - intro p hp x' hx' - rw [PMF.mem_support_map_iff] at hx' - obtain ⟨k, _, rfl⟩ := hx' - have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) - refine ⟨?_, ?_, Modifies.refl _ _⟩ - · refine ⟨fun i => ?_, hbp.2⟩ - have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i - simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] - exact hbp.1 i - · intro i _ - have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i - simp [Data.set, hne] + refine ⟨?_, ?_⟩ + · intro p hp + have hsource : p.2.regs 4 = 1 := (hrep p hp).2.2 + have hfit : Fits w (p.2.regs 4) := by rw [hsource]; exact fits_one_of_pos hw + have he := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 5) (src := 4) hfit + rw [ProbComp.eval_unifFin, hsource] at he + exact he + · refine C.frame_caller_dist (F := ∅) hseed hrep (Finset.disjoint_empty_right _) ?_ + intro p hp x' hx' + rw [PMF.mem_support_map_iff] at hx' + obtain ⟨k, _, rfl⟩ := hx' + have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) + refine ⟨?_, ?_, Modifies.refl _ _⟩ + · refine ⟨fun i => ?_, hbp.2⟩ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] + exact hbp.1 i + · intro i _ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp [Data.set, hne] end ComplexityTests.WordRAMRandomizedSubroutine From e62c359d802b888d1576882481152dfe650bce0d Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:09:05 -0400 Subject: [PATCH 12/18] refactor(wordram): index dynamic programs by operation signatures Remove the implicit relational-to-Empty-state deterministic specification. Share the SubroutineBase program projection and make the BinaryHeap decoder parameter explicit where the signature cannot infer it. Validation: scripts/check.sh passed all six gates under the published rc2 support candidate 0d38523e. Hygiene, changed-file lints and the tidy-surface audit passed. Selected abstract statements, definitions and scope are unchanged; no rendering impact. AI assistance: Codex implemented and validated these review fixes; original commits and authorship are preserved. --- .../WordRAM/PriorityQueue/BinaryHeap.lean | 2 +- Complexity/Computability/WordRAM/Dynamic.lean | 38 ++++++++++--------- .../Computability/WordRAM/Subroutine.lean | 10 +---- .../Semantics/Stateful/Relational.lean | 11 ------ 4 files changed, 22 insertions(+), 39 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean b/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean index f0605c71b..b9de086eb 100644 --- a/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean +++ b/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean @@ -691,7 +691,7 @@ def dynamicProgram (n K_max : ℕ) : DynamicProgram (IndexedMinPQ.spec n K_max) output := pqCode.output input := pqCode.input encode := pqCode.encode - decode := pqCode.decode + decode := pqCode.decode (K_max := K_max) /-- Standalone `.standard`-basis `DynamicCertificate` for the binary min-heap. -/ def certificate {n K_max w base : ℕ} (hS : Fits w (base + space n)) (hK : Fits w K_max) : diff --git a/Complexity/Computability/WordRAM/Dynamic.lean b/Complexity/Computability/WordRAM/Dynamic.lean index ab80464d6..3121bebc1 100644 --- a/Complexity/Computability/WordRAM/Dynamic.lean +++ b/Complexity/Computability/WordRAM/Dynamic.lean @@ -74,8 +74,10 @@ universe u v z variable {r : ℕ} {D : Stateful.Spec.{u, v, z}} -/-- A single program serving all operations through a fixed register calling convention. -/ -structure DynamicProgram (D : Stateful.Spec.{u, v, z}) (r : ℕ) where +/-- A single program serving an operation signature through a fixed register calling convention. +Deterministic and relational stateful specifications supply their signatures by coercion; +the program itself makes no assumption about abstract states or permitted transitions. -/ +structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) where /-- The same code is executed for every operation. -/ code : Code .standard r /-- Register returned by the final halt instruction. -/ @@ -83,30 +85,30 @@ structure DynamicProgram (D : Stateful.Spec.{u, v, z}) (r : ℕ) where /-- Registers overwritten by externally supplied operation arguments. -/ input : Fin r → Bool /-- Encoded arguments, independent of the maintained machine state. -/ - encode : D.Op → Fin r → ℕ + encode : S.A → Fin r → ℕ /-- Interpretation of the final register tuple as the operation's typed observation. -/ - decode : (o : D.Op) → (Fin r → ℕ) → Option (D.Result o) + decode : (o : S.A) → (Fin r → ℕ) → Option (S.B o) namespace DynamicProgram -variable (P : DynamicProgram D r) +variable {S : PFunctor.{v, z}} (P : DynamicProgram S r) /-- Deliver operation arguments without changing maintained memory or non-input registers. -/ -def prepare (o : D.Op) (x : Data r) : Data r where +def prepare (o : S.A) (x : Data r) : Data r where regs i := if P.input i then P.encode o i else x.regs i mem := x.mem /-- Delivering operation arguments preserves the machine memory unchanged. -/ @[simp] -theorem prepare_mem (o : D.Op) (x : Data r) : (P.prepare o x).mem = x.mem := rfl +theorem prepare_mem (o : S.A) (x : Data r) : (P.prepare o x).mem = x.mem := rfl /-- Registers outside the calling convention retain their contents. -/ -theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : D.Op) (x : Data r) : +theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : S.A) (x : Data r) : (P.prepare o x).regs i = x.regs i := by simp [prepare, hi] /-- Argument delivery preserves bounded words when the arguments fit. -/ -theorem prepare_bounded {w : ℕ} {o : D.Op} {x : Data r} (hx : x.Bounded w) +theorem prepare_bounded {w : ℕ} {o : S.A} {x : Data r} (hx : x.Bounded w) (ho : ∀ i, P.input i = true → Fits w (P.encode o i)) : (P.prepare o x).Bounded w := by refine ⟨fun i => ?_, hx.2⟩ change Fits w (if P.input i then P.encode o i else x.regs i) @@ -120,7 +122,7 @@ instructions; exhaustion, a machine fault, or decoder failure returns `none`. This interpreter does not check the initial state's global word bound or operation admissibility. Use `DynamicCertificate.execOp_of_eval` to connect a successful call to certified execution. -/ -def eval (w fuel : ℕ) (o : D.Op) (x : Data r) : Option (Data r × D.Result o × ℕ) := +def eval (w fuel : ℕ) (o : S.A) (x : Data r) : Option (Data r × S.B o × ℕ) := match P.code.eval w fuel (P.prepare o x) with | none => none | some (y, n) => (P.decode o y.regs).map fun a => (y, a, n + 1) @@ -133,13 +135,13 @@ structure OperationTrace where probes : List Probe /-- A bounded machine call with native execution evidence independent of any resource bound. -/ -def ExecOp (w : ℕ) : Stateful.Execution D (Data r) OperationTrace := fun x o a y trace => +def ExecOp (w : ℕ) : Stateful.Execution S (Data r) OperationTrace := fun x o a y trace => (P.prepare o x).Bounded w ∧ ∃ n, TraceExec w P.code (P.prepare o x) y n trace.probes ∧ P.decode o y.regs = some a ∧ trace.instructions = n + 1 /-- Successful typed evaluation supplies an exact operation trace when the delivered state is bounded. Evaluator success alone does not imply a bound on untouched memory. -/ -theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result o} +theorem ExecOp.of_eval {w fuel k : ℕ} {o : S.A} {x y : Data r} {a : S.B o} (hb : (P.prepare o x).Bounded w) (h : P.eval w fuel o x = some (y, a, k)) : ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := by cases he : P.code.eval w fuel (P.prepare o x) with @@ -156,7 +158,7 @@ theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result /-- Every call is an execution of one fixed flat machine program. The returned register tuple is observed at the final state; the designated output register is also the halt value. -/ -theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} +theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} (h : P.ExecOp w x o a y trace) : Executes w (P.code.program P.output) ⟨0, P.prepare o x⟩ (y.regs P.output) trace.instructions := by @@ -165,13 +167,13 @@ theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} exact he.exec.executes P.output /-- A call preserves bounded words, including every word in the observed register tuple. -/ -theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} +theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} (h : P.ExecOp w x o a y trace) : y.Bounded w := by obtain ⟨hb, n, he, _, _⟩ := h exact he.exec.bounded hb /-- Calls are deterministic in their final representation, observation, and instruction count. -/ -theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} +theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : S.A} {a a' : S.B o} {trace trace' : OperationTrace} (h : P.ExecOp w x o a y trace) (h' : P.ExecOp w x o a' y' trace') : y = y' ∧ a = a' ∧ trace.instructions = trace'.instructions := by @@ -181,8 +183,8 @@ theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ /-- A call's final state and instruction count agree with any structured execution of the code. -/ -theorem ExecOp.eq_of_exec {P : DynamicProgram D r} {w : ℕ} {x y z : Data r} {o : D.Op} - {a : D.Result o} {trace : OperationTrace} {m : ℕ} (he : P.ExecOp w x o a y trace) +theorem ExecOp.eq_of_exec {P : DynamicProgram S r} {w : ℕ} {x y z : Data r} {o : S.A} + {a : S.B o} {trace : OperationTrace} {m : ℕ} (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : y = z ∧ trace.instructions = m + 1 := by obtain ⟨_, n, ht, _, hn⟩ := he @@ -190,7 +192,7 @@ theorem ExecOp.eq_of_exec {P : DynamicProgram D r} {w : ℕ} {x y z : Data r} {o exact ⟨rfl, hn⟩ /-- The write transcript supplies a frame rule for a call, including register-only delivery. -/ -theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} +theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} (h : P.ExecOp w x o a y trace) {Q : HeapAssertion} (hQ : Q.holds x.mem) (hd : Disjoint Q.footprint (Probe.writes trace.probes)) : Q.holds y.mem := by diff --git a/Complexity/Computability/WordRAM/Subroutine.lean b/Complexity/Computability/WordRAM/Subroutine.lean index bb0969dcc..feffa199e 100644 --- a/Complexity/Computability/WordRAM/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Subroutine.lean @@ -328,7 +328,7 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineBase P Result valid w base /-- Project a `.standard`-basis `SubroutineBase` to a standalone `DynamicProgram`. -/ def toDynamicProgram {P_std : SubroutineCode .standard r_ds k_init} (C : SubroutineBase P_std Result valid w base params s₀) : - DynamicProgram { State := Empty, Op := O, Result := Result, step := Empty.elim } r_ds where + DynamicProgram ⟨O, Result⟩ r_ds where code := P_std.code output := P_std.output input := P_std.input @@ -627,14 +627,6 @@ section Standard variable {P_std : SubroutineCode .standard r_ds k_init} (C : SubroutineDS P_std D w base params s₀) -/-- Project a `.standard`-basis `SubroutineDS` to its underlying standalone `DynamicProgram`. -/ -def toDynamicProgram : DynamicProgram D r_ds where - code := P_std.code - output := P_std.output - input := P_std.input - encode := C.encode - decode := C.decode - /-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicCertificate`. -/ def toCertificate : DynamicCertificate C.toDynamicProgram w where rep := C.rep diff --git a/Complexity/Foundations/Semantics/Stateful/Relational.lean b/Complexity/Foundations/Semantics/Stateful/Relational.lean index 722f77f02..6a709a6d1 100644 --- a/Complexity/Foundations/Semantics/Stateful/Relational.lean +++ b/Complexity/Foundations/Semantics/Stateful/Relational.lean @@ -128,17 +128,6 @@ abbrev signature : PFunctor.{v, w} := ⟨D.Op, D.Result⟩ instance : Coe RelSpec.{u, v, w} PFunctor.{v, w} := ⟨RelSpec.signature⟩ -/-- Project the operation signature `(D.Op, D.Result)` of a relational specification to a -canonical `Stateful.Spec` with `State := Empty`, allowing signature-only constructions over -`Stateful.Spec` (such as `WordRAM.DynamicProgram`) to apply directly to `D : RelSpec`. -/ -abbrev toSpec : Spec.{0, v, w} where - State := Empty - Op := D.Op - Result := D.Result - step s := s.elim - -instance : Coe RelSpec.{u, v, w} Spec.{0, v, w} := ⟨RelSpec.toSpec⟩ - /-- An adaptive client over `D`: each continuation may branch on the observed operation result. -/ abbrev Client (α : Type y) : Type (max v w y) := D.signature.FreeM α From a115e42b53774978d96e2202cb74ea05bea5f7e7 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:14:24 -0400 Subject: [PATCH 13/18] refactor(wordram): share basis-preserving subroutine plumbing Keep the standard DynamicProgram specialization while allowing an explicit basis. Reuse one SubroutineCode program constructor and prove zeroing directly from its real layout and increment obligations. Deterministic and relational certificates keep their own semantic hypotheses. Validation: scripts/check.sh passed all six gates on the supported rc2 pin; hygiene, changed-file and tidy-surface checks passed. Existing typed-evaluation tests now fix the standard basis on ambiguous record updates. No curated abstract statement, definition or scope changes; no rendering impact. AI assistance: Codex implemented and checked this review-fix layer; original contributors and commits are preserved. --- Complexity/Computability/WordRAM/Dynamic.lean | 12 +- .../Computability/WordRAM/RelSubroutine.lean | 4 +- .../Computability/WordRAM/Subroutine.lean | 327 +++++++++--------- ComplexityTests/WordRAMDynamic.lean | 6 +- 4 files changed, 184 insertions(+), 165 deletions(-) diff --git a/Complexity/Computability/WordRAM/Dynamic.lean b/Complexity/Computability/WordRAM/Dynamic.lean index 3121bebc1..ace6fd08a 100644 --- a/Complexity/Computability/WordRAM/Dynamic.lean +++ b/Complexity/Computability/WordRAM/Dynamic.lean @@ -14,6 +14,10 @@ public import Complexity.Computability.WordRAM.Trace /-! # Fixed-program implementations of dynamic data structures +`DynamicProgram` depends only on the existing `PFunctor` operation signature and its instruction +basis, which defaults to `.standard`. Deterministic and relational specifications supply their +signatures by coercion; correctness certificates retain the actual specification separately. + An implementation has one structured program, one output register, and a register-only calling convention. Each call replaces designated input registers with encoded words, preserving all other registers and the entire memory. Argument encoding, external delivery, and result decoding @@ -77,9 +81,9 @@ variable {r : ℕ} {D : Stateful.Spec.{u, v, z}} /-- A single program serving an operation signature through a fixed register calling convention. Deterministic and relational stateful specifications supply their signatures by coercion; the program itself makes no assumption about abstract states or permitted transitions. -/ -structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) where +structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) (B : Basis := .standard) where /-- The same code is executed for every operation. -/ - code : Code .standard r + code : Code B r /-- Register returned by the final halt instruction. -/ output : Fin r /-- Registers overwritten by externally supplied operation arguments. -/ @@ -91,7 +95,7 @@ structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) where namespace DynamicProgram -variable {S : PFunctor.{v, z}} (P : DynamicProgram S r) +variable {S : PFunctor.{v, z}} {B : Basis} (P : DynamicProgram S r B) /-- Deliver operation arguments without changing maintained memory or non-input registers. -/ def prepare (o : S.A) (x : Data r) : Data r where @@ -183,7 +187,7 @@ theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : S.A} {a a' : S.B o} exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ /-- A call's final state and instruction count agree with any structured execution of the code. -/ -theorem ExecOp.eq_of_exec {P : DynamicProgram S r} {w : ℕ} {x y z : Data r} {o : S.A} +theorem ExecOp.eq_of_exec {P : DynamicProgram S r B} {w : ℕ} {x y z : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} {m : ℕ} (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : y = z ∧ trace.instructions = m + 1 := by diff --git a/Complexity/Computability/WordRAM/RelSubroutine.lean b/Complexity/Computability/WordRAM/RelSubroutine.lean index 7e3dece99..ba187f24c 100644 --- a/Complexity/Computability/WordRAM/RelSubroutine.lean +++ b/Complexity/Computability/WordRAM/RelSubroutine.lean @@ -25,7 +25,7 @@ to relational specifications (`Stateful.RelSpec`): `Stateful.RelRealization` directly via `Exec.unique`. 2. **Unified standalone and inline relational subroutine certificates (`RelSubroutineDS`):** Extends `SubroutineBase P D.Result D.valid w base params s₀` over `D : Stateful.RelSpec`, - inheriting `toDynamicProgram`, `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, + inheriting `toDynamicProgram`, `initFresh_exec`, `initFresh_rename`, `initClear_exec`, `initClear_rename`, `frame_caller`, and `prepareRegs` without duplication, omitting `rep_unique` (since concrete potential `potential : Data r_ds → ℕ` depends only on machine data), and existentially quantifying over `(s', a)` with `D.step s o s' a` in @@ -266,7 +266,7 @@ end SubroutineCode at word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. Extends `SubroutineBase P D.Result D.valid w base params s₀` (inheriting `toDynamicProgram`, -`initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, `initClear_rename`, +`initFresh_exec`, `initFresh_rename`, `initClear_exec`, `initClear_rename`, `frame_caller`, and `prepareRegs`), omits `rep_unique` (because concrete potential `potential : Data r_ds → ℕ` depends only on machine data), and existentially quantifies over a permitted outcome `(s', a)` satisfying `D.step s o s' a` in `step_exec`. -/ diff --git a/Complexity/Computability/WordRAM/Subroutine.lean b/Complexity/Computability/WordRAM/Subroutine.lean index feffa199e..b6cd0bd1b 100644 --- a/Complexity/Computability/WordRAM/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Subroutine.lean @@ -45,7 +45,7 @@ A data structure is packaged once as a `SubroutineCode B r_ds k_init` and certif standalone / inline specification and resource certificate for a Word-RAM data structure. * `Cslib.WordRAM.SubroutineCode.initFresh`, `Cslib.WordRAM.SubroutineCode.initClear`: `O(1)` fresh initialization on zeroed memory and generic `O(space)` dirty-memory re-initialization. -* `Cslib.WordRAM.SubroutineBase.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: +* `Cslib.WordRAM.SubroutineCode.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: standalone `.standard`-basis `DynamicProgram` and `DynamicCertificate` projections. ## Main results @@ -206,6 +206,169 @@ def mapSimulates (φ : B.Simulates B') (P : SubroutineCode B r_ds k_init) : abbrev toStandard (P : SubroutineCode .exact r_ds k_init) : SubroutineCode .standard r_ds k_init := P.mapSimulates .exactStandard +/-- Project subroutine code and its calling convention to the same operation signature. +The basis is retained, so this projection does not translate instructions or resource claims. -/ +def toDynamicProgram (P : SubroutineCode B r_ds k_init) {O : Type*} {Result : O → Type*} + (encode : O → Fin r_ds → ℕ) (decode : (o : O) → (Fin r_ds → ℕ) → Option (Result o)) : + DynamicProgram ⟨O, Result⟩ r_ds B where + code := P.code + output := P.output + input := P.input + encode := encode + decode := decode + +/-- Generic execution of `P.zeroFillCode` clearing `[base, base + space)` in `4 * space + 6` +instructions over any basis `B`. -/ +theorem zeroFillCode_exec (P : SubroutineCode B r_ds k_init) + {w base space : ℕ} {params : Fin k_init → ℕ} + (hclear : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup) + (hparams : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg]) + (hinc : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → + P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1))) + {x₁ : Data r_ds} + (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) + (hfr₁ : x₁.regs P.frontierReg = base + space) + (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : + ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * space + 6) ∧ + x₂.Bounded w ∧ + x₂.regs P.baseReg = base ∧ + x₂.regs P.frontierReg = base + space ∧ + (∀ j, x₂.regs (P.initParams j) = params j) ∧ + (∀ a ∈ (Block.mk base space).footprint, x₂.mem a = 0) ∧ + Modifies (Block.mk base space).footprint x₁.mem x₂.mem := by + have hfit0 : Fits w 0 := (hx₁.1 P.baseReg).of_le (Nat.zero_le _) + have hfit_stop : Fits w (base + space) := hfr₁ ▸ hx₁.1 P.frontierReg + obtain ⟨hcur_not, hzero_not, hone_not, hbase_ne_fr, _⟩ := + (List.nodup_cons.mp hclear).imp_right fun h => + (List.nodup_cons.mp h).imp_right fun h => + (List.nodup_cons.mp h).imp_right List.nodup_cons.mp + have hcur_ne_zero : P.curReg ≠ P.zeroReg := fun h => hcur_not (by simp [h]) + have hcur_ne_one : P.curReg ≠ P.oneReg := fun h => hcur_not (by simp [h]) + have hcur_ne_base : P.curReg ≠ P.baseReg := fun h => hcur_not (by simp [h]) + have hcur_ne_fr : P.curReg ≠ P.frontierReg := fun h => hcur_not (by simp [h]) + have hzero_ne_one : P.zeroReg ≠ P.oneReg := fun h => hzero_not (by simp [h]) + have hzero_ne_base : P.zeroReg ≠ P.baseReg := fun h => hzero_not (by simp [h]) + have hzero_ne_fr : P.zeroReg ≠ P.frontierReg := fun h => hzero_not (by simp [h]) + have hone_ne_base : P.oneReg ≠ P.baseReg := fun h => hone_not (by simp [h]) + have hone_ne_fr : P.oneReg ≠ P.frontierReg := fun h => hone_not (by simp [h]) + by_cases hsp : 0 < space + · have hc_pos : (Cond.lt P.baseReg P.frontierReg).Holds x₁ := by + simp only [Cond.Holds, hb₁, hfr₁] + omega + have hfit1 : Fits w 1 := hfit_stop.of_le (by omega) + let x_z := x₁.set P.zeroReg 0 + let x_o := x_z.set P.oneReg 1 + let x_c := x_o.set P.curReg (x_o.regs P.baseReg) + have hxc_base : x_c.regs P.baseReg = base := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_base.symm, + Function.update_of_ne hone_ne_base.symm, Function.update_of_ne hzero_ne_base.symm, hb₁] + have hxc_fr : x_c.regs P.frontierReg = base + space := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_fr.symm, + Function.update_of_ne hone_ne_fr.symm, Function.update_of_ne hzero_ne_fr.symm, hfr₁] + have hxc_zero : x_c.regs P.zeroReg = 0 := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_zero.symm, + Function.update_of_ne hzero_ne_one] + have hxc_one : x_c.regs P.oneReg = 1 := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_one.symm] + have hxc_cur : x_c.regs P.curReg = base := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hone_ne_base.symm, + Function.update_of_ne hzero_ne_base.symm, hb₁] + let xs : ℕ → Data r_ds := fun i => + ⟨Function.update x_c.regs P.curReg (base + i), clear x₁.mem base i⟩ + have hxs0 : xs 0 = x_c := by + ext r + · by_cases hr : r = P.curReg + · subst hr + simp [xs, hxc_cur] + · simp [xs, Function.update_of_ne hr, x_c, x_o, x_z, Data.set] + · simp [xs, x_c, x_o, x_z, Data.set] + have hloop_cond : ∀ i < space, (Cond.lt P.curReg P.frontierReg).Holds (xs i) := by + intro i hi + simp only [Cond.Holds, xs, Function.update_self, + Function.update_of_ne hcur_ne_fr.symm, hxc_fr] + omega + have hloop_exit : ¬(Cond.lt P.curReg P.frontierReg).Holds (xs space) := by + simp only [Cond.Holds, xs, Function.update_self, + Function.update_of_ne hcur_ne_fr.symm, hxc_fr] + omega + have hloop_body : ∀ i < space, + ExecWithin w (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur)) + (xs i) (xs (i + 1)) 2 := by + intro i hi + have hsi_zero : (xs i).regs P.zeroReg = 0 := by + simp [xs, Function.update_of_ne hcur_ne_zero.symm, hxc_zero] + have hsi_one : (xs i).regs P.oneReg = 1 := by + simp [xs, Function.update_of_ne hcur_ne_one.symm, hxc_one] + have hsi_cur : (xs i).regs P.curReg = base + i := by simp [xs] + let x_st := (xs i).write ((xs i).regs P.curReg) ((xs i).regs P.zeroReg) + have hst_one : x_st.regs P.oneReg = 1 := hsi_one + have hst_fit : Fits w (x_st.regs P.curReg + 1) := by + change Fits w ((xs i).regs P.curReg + 1) + rw [hsi_cur] + exact hfit_stop.of_le (by omega) + have hinc : ExecWithin w (.op P.incCur) x_st (x_st.set P.curReg (x_st.regs P.curReg + 1)) 1 := + (Exec.op (hinc hst_one hst_fit)).within + have htarget : x_st.set P.curReg (x_st.regs P.curReg + 1) = xs (i + 1) := by + ext r + · simp only [x_st, Data.set, Data.write, xs, hsi_cur] + by_cases hr : r = P.curReg + · subst hr + simp [Nat.add_assoc] + · simp [Function.update_of_ne hr] + · simp only [x_st, Data.set, Data.write, hsi_cur, hsi_zero] + simp [xs, clear_succ] + rw [htarget] at hinc + exact ExecWithin.store.seq hinc + have hloop : ExecWithin w + (.loop (.lt P.curReg P.frontierReg) (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) + x_c (xs space) (space * (2 + 2) + 2) := by + simpa [hxs0] using ExecWithin.loop xs space 2 hloop_cond hloop_exit hloop_body + have hbranch : ExecWithin w P.zeroFillCode x₁ (xs space) (4 * space + 6) := by + have hseq : ExecWithin w + (.seq (.op (.const P.zeroReg 0)) <| + .seq (.op (.const P.oneReg 1)) <| + .seq (.op (.copy P.curReg P.baseReg)) <| + .loop (.lt P.curReg P.frontierReg) + (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) + x₁ (xs space) (3 + (space * 4 + 2)) := by + obtain ⟨n₁, hn₁, he₁⟩ := (ExecWithin.const (x := x₁) (d := P.zeroReg) hfit0) + obtain ⟨n₂, hn₂, he₂⟩ := (ExecWithin.const (x := x_z) (d := P.oneReg) hfit1) + obtain ⟨n₃, hn₃, he₃⟩ := (ExecWithin.copy (B := B) (w := w) (x := x_o) + (d := P.curReg) (s := P.baseReg)) + obtain ⟨n₄, hn₄, he₄⟩ := hloop + exact ⟨n₁ + (n₂ + (n₃ + n₄)), by omega, he₁.seq (he₂.seq (he₃.seq he₄))⟩ + exact (ExecWithin.branch_pos hc_pos hseq).mono (by omega) + refine ⟨xs space, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ + · simp [xs, Function.update_of_ne hcur_ne_base.symm, hxc_base] + · simp [xs, Function.update_of_ne hcur_ne_fr.symm, hxc_fr] + · intro j + have hne := hparams j + have h1 : P.initParams j ≠ P.curReg := fun h => hne (by simp [h]) + have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) + have h3 : P.initParams j ≠ P.oneReg := fun h => hne (by simp [h]) + simp [xs, x_c, x_o, x_z, Data.set, Function.update_of_ne h1, Function.update_of_ne h2, + Function.update_of_ne h3, hpar₁ j] + · intro a ha + exact clear_of_owns (Block.mem_footprint.mp ha) + · intro a ha + exact clear_of_not_owns (mt Block.mem_footprint.mpr ha) + · have hsp0 : space = 0 := by omega + have hc_neg : ¬(Cond.lt P.baseReg P.frontierReg).Holds x₁ := by + simp only [Cond.Holds, hb₁, hfr₁, hsp0] + omega + have hbranch : ExecWithin w P.zeroFillCode x₁ (x₁.set P.zeroReg 0) (4 * space + 6) := + (ExecWithin.branch_neg hc_neg (ExecWithin.const hfit0)).mono (by omega) + refine ⟨x₁.set P.zeroReg 0, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ + · simp [Data.set, Function.update_of_ne hzero_ne_base.symm, hb₁] + · simp [Data.set, Function.update_of_ne hzero_ne_fr.symm, hfr₁] + · intro j + have hne := hparams j + have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) + simp [Data.set, Function.update_of_ne h2, hpar₁ j] + · intro a ha + simp [hsp0, Block.footprint] at ha + · exact Modifies.refl _ _ + end SubroutineCode universe u v z @@ -325,15 +488,10 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineBase P Result valid w base let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ -/-- Project a `.standard`-basis `SubroutineBase` to a standalone `DynamicProgram`. -/ -def toDynamicProgram {P_std : SubroutineCode .standard r_ds k_init} - (C : SubroutineBase P_std Result valid w base params s₀) : - DynamicProgram ⟨O, Result⟩ r_ds where - code := P_std.code - output := P_std.output - input := P_std.input - encode := C.encode - decode := C.decode +/-- Project a subroutine bundle to its signature-level program, retaining the instruction basis. -/ +def toDynamicProgram (C : SubroutineBase P Result valid w base params s₀) : + DynamicProgram ⟨O, Result⟩ r_ds B := + P.toDynamicProgram C.encode C.decode /-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ theorem initFresh_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} @@ -378,152 +536,6 @@ theorem initFresh_rename (C : SubroutineBase P Result valid w base params s₀) · rw [Data.comap_extend hf] exact hpot -/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` -instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : SubroutineBase P Result valid w base params s₀) {x₁ : Data r_ds} - (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) - (hfr₁ : x₁.regs P.frontierReg = base + C.space) - (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : - ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ - x₂.Bounded w ∧ - x₂.regs P.baseReg = base ∧ - x₂.regs P.frontierReg = base + C.space ∧ - (∀ j, x₂.regs (P.initParams j) = params j) ∧ - (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ - Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := by - have hfit0 : Fits w 0 := (hx₁.1 P.baseReg).of_le (Nat.zero_le _) - have hfit_stop : Fits w (base + C.space) := hfr₁ ▸ hx₁.1 P.frontierReg - obtain ⟨hcur_not, hzero_not, hone_not, hbase_ne_fr, _⟩ := - (List.nodup_cons.mp C.clear_nodup).imp_right fun h => - (List.nodup_cons.mp h).imp_right fun h => - (List.nodup_cons.mp h).imp_right List.nodup_cons.mp - have hcur_ne_zero : P.curReg ≠ P.zeroReg := fun h => hcur_not (by simp [h]) - have hcur_ne_one : P.curReg ≠ P.oneReg := fun h => hcur_not (by simp [h]) - have hcur_ne_base : P.curReg ≠ P.baseReg := fun h => hcur_not (by simp [h]) - have hcur_ne_fr : P.curReg ≠ P.frontierReg := fun h => hcur_not (by simp [h]) - have hzero_ne_one : P.zeroReg ≠ P.oneReg := fun h => hzero_not (by simp [h]) - have hzero_ne_base : P.zeroReg ≠ P.baseReg := fun h => hzero_not (by simp [h]) - have hzero_ne_fr : P.zeroReg ≠ P.frontierReg := fun h => hzero_not (by simp [h]) - have hone_ne_base : P.oneReg ≠ P.baseReg := fun h => hone_not (by simp [h]) - have hone_ne_fr : P.oneReg ≠ P.frontierReg := fun h => hone_not (by simp [h]) - by_cases hsp : 0 < C.space - · have hc_pos : (Cond.lt P.baseReg P.frontierReg).Holds x₁ := by - simp only [Cond.Holds, hb₁, hfr₁] - omega - have hfit1 : Fits w 1 := hfit_stop.of_le (by omega) - let x_z := x₁.set P.zeroReg 0 - let x_o := x_z.set P.oneReg 1 - let x_c := x_o.set P.curReg (x_o.regs P.baseReg) - have hxc_base : x_c.regs P.baseReg = base := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_base.symm, - Function.update_of_ne hone_ne_base.symm, Function.update_of_ne hzero_ne_base.symm, hb₁] - have hxc_fr : x_c.regs P.frontierReg = base + C.space := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_fr.symm, - Function.update_of_ne hone_ne_fr.symm, Function.update_of_ne hzero_ne_fr.symm, hfr₁] - have hxc_zero : x_c.regs P.zeroReg = 0 := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_zero.symm, - Function.update_of_ne hzero_ne_one] - have hxc_one : x_c.regs P.oneReg = 1 := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_one.symm] - have hxc_cur : x_c.regs P.curReg = base := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hone_ne_base.symm, - Function.update_of_ne hzero_ne_base.symm, hb₁] - let xs : ℕ → Data r_ds := fun i => - ⟨Function.update x_c.regs P.curReg (base + i), clear x₁.mem base i⟩ - have hxs0 : xs 0 = x_c := by - ext r - · by_cases hr : r = P.curReg - · subst hr - simp [xs, hxc_cur] - · simp [xs, Function.update_of_ne hr, x_c, x_o, x_z, Data.set] - · simp [xs, x_c, x_o, x_z, Data.set] - have hloop_cond : ∀ i < C.space, (Cond.lt P.curReg P.frontierReg).Holds (xs i) := by - intro i hi - simp only [Cond.Holds, xs, Function.update_self, - Function.update_of_ne hcur_ne_fr.symm, hxc_fr] - omega - have hloop_exit : ¬(Cond.lt P.curReg P.frontierReg).Holds (xs C.space) := by - simp only [Cond.Holds, xs, Function.update_self, - Function.update_of_ne hcur_ne_fr.symm, hxc_fr] - omega - have hloop_body : ∀ i < C.space, - ExecWithin w (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur)) - (xs i) (xs (i + 1)) 2 := by - intro i hi - have hsi_zero : (xs i).regs P.zeroReg = 0 := by - simp [xs, Function.update_of_ne hcur_ne_zero.symm, hxc_zero] - have hsi_one : (xs i).regs P.oneReg = 1 := by - simp [xs, Function.update_of_ne hcur_ne_one.symm, hxc_one] - have hsi_cur : (xs i).regs P.curReg = base + i := by simp [xs] - let x_st := (xs i).write ((xs i).regs P.curReg) ((xs i).regs P.zeroReg) - have hst_one : x_st.regs P.oneReg = 1 := hsi_one - have hst_fit : Fits w (x_st.regs P.curReg + 1) := by - change Fits w ((xs i).regs P.curReg + 1) - rw [hsi_cur] - exact hfit_stop.of_le (by omega) - have hinc : ExecWithin w (.op P.incCur) x_st (x_st.set P.curReg (x_st.regs P.curReg + 1)) 1 := - (Exec.op (C.incCur_eval hst_one hst_fit)).within - have htarget : x_st.set P.curReg (x_st.regs P.curReg + 1) = xs (i + 1) := by - ext r - · simp only [x_st, Data.set, Data.write, xs, hsi_cur] - by_cases hr : r = P.curReg - · subst hr - simp [Nat.add_assoc] - · simp [Function.update_of_ne hr] - · simp only [x_st, Data.set, Data.write, hsi_cur, hsi_zero] - simp [xs, clear_succ] - rw [htarget] at hinc - exact ExecWithin.store.seq hinc - have hloop : ExecWithin w - (.loop (.lt P.curReg P.frontierReg) (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) - x_c (xs C.space) (C.space * (2 + 2) + 2) := by - simpa [hxs0] using ExecWithin.loop xs C.space 2 hloop_cond hloop_exit hloop_body - have hbranch : ExecWithin w P.zeroFillCode x₁ (xs C.space) (4 * C.space + 6) := by - have hseq : ExecWithin w - (.seq (.op (.const P.zeroReg 0)) <| - .seq (.op (.const P.oneReg 1)) <| - .seq (.op (.copy P.curReg P.baseReg)) <| - .loop (.lt P.curReg P.frontierReg) - (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) - x₁ (xs C.space) (3 + (C.space * 4 + 2)) := by - obtain ⟨n₁, hn₁, he₁⟩ := (ExecWithin.const (x := x₁) (d := P.zeroReg) hfit0) - obtain ⟨n₂, hn₂, he₂⟩ := (ExecWithin.const (x := x_z) (d := P.oneReg) hfit1) - obtain ⟨n₃, hn₃, he₃⟩ := (ExecWithin.copy (B := B) (w := w) (x := x_o) - (d := P.curReg) (s := P.baseReg)) - obtain ⟨n₄, hn₄, he₄⟩ := hloop - exact ⟨n₁ + (n₂ + (n₃ + n₄)), by omega, he₁.seq (he₂.seq (he₃.seq he₄))⟩ - exact (ExecWithin.branch_pos hc_pos hseq).mono (by omega) - refine ⟨xs C.space, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ - · simp [xs, Function.update_of_ne hcur_ne_base.symm, hxc_base] - · simp [xs, Function.update_of_ne hcur_ne_fr.symm, hxc_fr] - · intro j - have hne := C.params_not_clear j - have h1 : P.initParams j ≠ P.curReg := fun h => hne (by simp [h]) - have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) - have h3 : P.initParams j ≠ P.oneReg := fun h => hne (by simp [h]) - simp [xs, x_c, x_o, x_z, Data.set, Function.update_of_ne h1, Function.update_of_ne h2, - Function.update_of_ne h3, hpar₁ j] - · intro a ha - exact clear_of_owns (Block.mem_footprint.mp ha) - · intro a ha - exact clear_of_not_owns (mt Block.mem_footprint.mpr ha) - · have hsp0 : C.space = 0 := by omega - have hc_neg : ¬(Cond.lt P.baseReg P.frontierReg).Holds x₁ := by - simp only [Cond.Holds, hb₁, hfr₁, hsp0] - omega - have hbranch : ExecWithin w P.zeroFillCode x₁ (x₁.set P.zeroReg 0) (4 * C.space + 6) := - (ExecWithin.branch_neg hc_neg (ExecWithin.const hfit0)).mono (by omega) - refine ⟨x₁.set P.zeroReg 0, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ - · simp [Data.set, Function.update_of_ne hzero_ne_base.symm, hb₁] - · simp [Data.set, Function.update_of_ne hzero_ne_fr.symm, hfr₁] - · intro j - have hne := C.params_not_clear j - have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) - simp [Data.set, Function.update_of_ne h2, hpar₁ j] - · intro a ha - simp [hsp0, Block.footprint] at ha - · exact Modifies.refl _ _ - /-- Execute `P.initClear` on **arbitrary dirty memory** in `C.initTime + 4 * C.space + 6` instructions. -/ theorem initClear_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} @@ -536,7 +548,8 @@ theorem initClear_exec (C : SubroutineBase P Result valid w base params s₀) {x Modifies (Block.mk base C.space).footprint x.mem y.mem := by obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := - C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + P.zeroFillCode_exec C.clear_nodup C.params_not_clear C.incCur_eval + (he₁.bounded hx) hb₁ hfr₁ hpar₁ obtain ⟨y, he₃, hfr₃, hrep, hpot, hmod₃⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ rw [hm₁] at hmod₂ have hmod : Modifies (Block.mk base C.space).footprint x.mem y.mem := by diff --git a/ComplexityTests/WordRAMDynamic.lean b/ComplexityTests/WordRAMDynamic.lean index 3e98d9e66..d27a344b5 100644 --- a/ComplexityTests/WordRAMDynamic.lean +++ b/ComplexityTests/WordRAMDynamic.lean @@ -147,9 +147,11 @@ example : (fun (_, answer, _) => answer) = some none ∧ ((Stack.implementation 4 2).eval 4 0 .size (Stack.initial 3)).map (fun (_, answer, _) => answer) = some 0 ∧ - ({ exchangeProgram 4 with decode := fun _ _ => none }).eval 4 0 7 + ({ exchangeProgram 4 with decode := fun _ _ => none } : + DynamicProgram (exchangeSpec 4) 3 .standard).eval 4 0 7 ⟨fun _ => 0, fun _ => 0⟩ = none ∧ - ({ exchangeProgram 4 with code := .loop (.eq 0 0) (.op (.copy 0 0)) }).eval 4 2 7 + ({ exchangeProgram 4 with code := .loop (.eq 0 0) (.op (.copy 0 0)) } : + DynamicProgram (exchangeSpec 4) 3 .standard).eval 4 2 7 ⟨fun _ => 0, fun _ => 0⟩ = none := by decide From fb42ef60e8b0a8a96fcf3cb1b6cccc7ea3ef5057 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:31:40 -0400 Subject: [PATCH 14/18] refactor(wordram): reuse randomized subroutine plumbing and probability APIs Remove surrogate deterministic zeroing and copied argument delivery in favor of the actual basis-preserving SubroutineCode/DynamicProgram APIs. Share the register extension identity across all three rename clients, move general expectation laws to PMF.Expect and reuse Mathlib measure and finite-sum facts. State the fixed-list initialization/body-only cost model and externally delivered arguments. Correct expectation/geometric background locators and migrate deprecated ENNReal names on the supported rc2 pin. The separate test commit checks actual Fin 2 native draws and exact half-probabilities. Validation: default scripts/check.sh passed all six gates; hygiene, changed-file lints against the shared a115e42b layer, and the tidy-surface audit passed. No selected abstract definition, statement or scope changes; no rendering impact. Original geometric/native mathematics is unchanged. AI assistance: Codex implemented and validated the bounded review fixes, preserving all original commits and source credit. --- .../WordRAM/Randomized/Expected.lean | 145 +++++---------- .../WordRAM/Randomized/Structured.lean | 14 +- .../WordRAM/Randomized/Subroutine.lean | 170 ++++++------------ .../Computability/WordRAM/Structured.lean | 3 +- Complexity/Probability/PMF/Expect.lean | 23 ++- 5 files changed, 126 insertions(+), 229 deletions(-) diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index b0c4ee3e8..d26d77662 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -12,6 +12,7 @@ public import Complexity.Computability.WordRAM.Randomized.Structured public import Complexity.Languages.Probabilistic.Expect public import Complexity.Probability.PMF.Bind public import Complexity.Probability.PMF.Event +import Complexity.Probability.PMF.Expect /-! # Expected running time for randomized word RAMs @@ -72,14 +73,14 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, geometric Las Vegas `while` loops and structured programs. Discrete and conditional expectation principles are classical background; see -[MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. The particular native tail-sum/Bellman +[MitzenmacherUpfal2017], Sections 2.1, 2.3, and 2.4. The particular native tail-sum/Bellman definitions, finite structured-execution-tree bridge and ranked-loop expected-cost formalization are original library constructions in this module. ## References * [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017], - Sections 1.5, 2.1, and 2.3. + Sections 2.1, 2.3, and 2.4. -/ @[expose] public section @@ -673,44 +674,17 @@ theorem evalRand_halted_mono (w : ℕ) (p : Program B r) (s : State r) (v : ℕ) = evalRand w p T s (.halted v d) * (stepRand w p (.halted v d)).eval (.halted v d) := by simp [stepRand_halted] _ ≤ ∑' cfg, evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d) := - ENNReal.le_tsum + le_tsum (f := fun cfg => evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d)) (RandCfg.halted v d) -private theorem finsetSum_iSup_of_monotone {α : Type*} (s : Finset α) {f : ℕ → α → ℝ≥0∞} - (hf : ∀ a, Monotone fun n => f n a) : - (∑ a ∈ s, ⨆ n, f n a) = ⨆ n, ∑ a ∈ s, f n a := by - classical - induction s using Finset.induction_on with - | empty => simp - | @insert a₀ s ha₀ ih => - rw [Finset.sum_insert ha₀, ih] - refine le_antisymm ?_ (iSup_le fun n => ?_) - · rw [ENNReal.add_iSup] - simp_rw [ENNReal.iSup_add] - refine iSup_le fun j => iSup_le fun i => le_iSup_of_le (max i j) ?_ - rw [Finset.sum_insert ha₀] - exact add_le_add (hf a₀ (le_max_left i j)) - (Finset.sum_le_sum fun a _ => hf a (le_max_right i j)) - · rw [Finset.sum_insert ha₀] - exact add_le_add (le_iSup (fun k => f k a₀) n) - (le_iSup (fun k => ∑ a ∈ s, f k a) n) - private theorem tsum_iSup_of_monotone {α : Type*} {f : ℕ → α → ℝ≥0∞} (hf : ∀ a, Monotone fun n => f n a) : (∑' a, ⨆ n, f n a) = ⨆ n, ∑' a, f n a := by - rw [ENNReal.tsum_eq_iSup_sum] - simp_rw [finsetSum_iSup_of_monotone _ hf] + rw [tsum_eq_iSup_sum] + simp_rw [ENNReal.finsetSum_iSup_of_monotone hf] rw [iSup_comm] - simp_rw [← ENNReal.tsum_eq_iSup_sum] - -private theorem tsum_finsetSum {α β : Type*} (s : Finset β) (f : β → α → ℝ≥0∞) : - (∑' a, ∑ b ∈ s, f b a) = ∑ b ∈ s, ∑' a, f b a := by - classical - induction s using Finset.induction_on with - | empty => simp - | @insert b₀ s hb₀ ih => - simp only [Finset.sum_insert hb₀, ENNReal.tsum_add, ih] + simp_rw [← tsum_eq_iSup_sum] /-- Finite-horizon approximation of `expectedStepsFrom`: expected number of active (`.running`) steps within the first `N` steps starting from configuration `cfg`. -/ @@ -766,7 +740,7 @@ theorem expectedStepsWithin_running_succ (w : ℕ) (p : Program B r) (N : ℕ) ( intro t rw [runRand_succ, ProbComp.eval_bind, PMF.toOuterMeasure_bind_apply] simp only [hsucc] - rw [← tsum_finsetSum] + rw [← Summable.tsum_finsetSum (fun _ _ => CompleteLattice.summable)] simp only [ProbComp.expect, expectedStepsWithin, Finset.mul_sum] theorem expectedStepsWithin_running_le (w : ℕ) (p : Program B r) (N : ℕ) (s : State r) : @@ -910,7 +884,7 @@ theorem ExecRand.evalRand_faulted_eq_zero {w : ℕ} {c : Code B r} {x : Data r} | zero => simp [evalRand] | succ k => rw [evalRand, runRand_succ, stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, bind_assoc, - ProbComp.eval_bind, PMF.bind_apply, ENNReal.tsum_eq_zero] + ProbComp.eval_bind, PMF.bind_apply, tsum_eq_zero_iff] intro sample have hmem : (x.set dst sample.val, 1) ∈ (ProbComp.unifFin (x.regs src) >>= fun s => pure (x.set dst s.val, 1)).support := @@ -1162,7 +1136,7 @@ theorem loopIterPMF_mono_of_not_holds (cond : Cond r) (μ_body : Data r → PMF = loopIterPMF cond μ_body K x y * loopStepPMF cond μ_body y y := by rw [loopStepPMF_of_not_holds hy, PMF.pure_apply_self, mul_one] _ ≤ ∑' z, loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y := - ENNReal.le_tsum + le_tsum (f := fun z => loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y) y theorem loopIterPMF_indicator_mono (cond : Cond r) (μ_body : Data r → PMF (Data r)) @@ -1203,7 +1177,7 @@ theorem loopIterPMF_support_inv {cond : Cond r} {I Q : Data r → Prop} private theorem pmf_toOuterMeasure_compl_add (μ : PMF (Data r)) (cond : Cond r) : μ.toOuterMeasure {y | ¬cond.Holds y} + μ.toOuterMeasure {y | cond.Holds y} = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert μ.tsum_coe using 1 congr 1 funext y @@ -1257,7 +1231,7 @@ private theorem loopActiveSteps_succ_of_holds {cond : Cond r} {μ_body : Data r intro K rw [loopIterPMF_succ, loopStepPMF_of_holds hc, PMF.toOuterMeasure_bind_apply] simp only [hsucc] - rw [← tsum_finsetSum] + rw [← Summable.tsum_finsetSum (fun _ _ => CompleteLattice.summable)] simp only [loopActiveSteps, Finset.mul_sum] private theorem loopActiveSteps_le {cond : Cond r} {I Q : Data r → Prop} @@ -1273,7 +1247,7 @@ private theorem loopActiveSteps_le {cond : Cond r} {I Q : Data r → Prop} rcases hx with ⟨hcx, hIx⟩ | ⟨hcx, _⟩ · simp only [hcx, ↓reduceIte] rw [loopActiveSteps_succ_of_holds hcx] - refine (add_le_add le_rfl (ENNReal.tsum_le_tsum fun z => ?_)).trans + refine (add_le_add le_rfl (tsum_le_tsum fun z => ?_)).trans (pmf_step_drift_bound (μ_body x) cond hp_pos hp_top (hp_exit x hIx hcx)) by_cases hz : z ∈ (μ_body x).support · exact mul_le_mul_right (ih (hinv x hIx hcx z hz)) _ @@ -1288,7 +1262,8 @@ theorem hasSum_loopExitDensity {cond : Cond r} {I Q : Data r → Prop} {x : Data r} (hx : LoopStateInv cond I Q x) : HasSum (fun y => ⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1 := by - rw [ENNReal.summable.hasSum_iff, tsum_iSup_of_monotone (loopIterPMF_indicator_mono cond μ_body x)] + rw [CompleteLattice.summable.hasSum_iff, + tsum_iSup_of_monotone (loopIterPMF_indicator_mono cond μ_body x)] simp_rw [← PMF.toOuterMeasure_apply] set b := ⨆ K : ℕ, (loopIterPMF cond μ_body K x).toOuterMeasure {z | ¬cond.Holds z} have hb_le : b ≤ 1 := iSup_le fun K => Cslib.Probability.PMF.toOuterMeasure_le_one _ _ @@ -1431,23 +1406,6 @@ theorem mem_support_loopExitPMF {cond : Cond r} {I Q : Data r → Prop} · exact absurd hcy hnc · exact ⟨⟨hnc, hQy⟩, K, hK⟩ -/-- Expectation of `k : β → ℝ≥0∞` under `p.map g` equals expectation of `k ∘ g` under `p`. -/ -theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : - (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by - simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] - refine tsum_congr fun a => ?_ - rw [tsum_eq_single (g a)] - · simp - · intro b hb - simp [hb] - -/-- Expectation of `k : β → ℝ≥0∞` under `p.bind f` decomposes as an iterated expectation. -/ -theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : - (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by - simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] - exact ENNReal.tsum_comm - /-! ### Compositional distributional execution (`ExecCodeDist`) -/ /-- `ExecCodeDist w c x μ T`: at word size `w`, running structured program `c` from data `x` @@ -1543,7 +1501,7 @@ theorem seq_const {μ₂ : Data r → PMF (Data r)} {T₂ : Data r → ℝ≥0 calc (∑' y, μ₁ y * T₂ y) ≤ ∑' y, μ₁ y * B₂ := by - refine ENNReal.tsum_le_tsum fun y => ?_ + refine tsum_le_tsum fun y => ?_ by_cases hy : y ∈ μ₁.support · exact mul_le_mul_right (hB₂ y hy) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] @@ -1663,29 +1621,12 @@ theorem mapSimulates {B' : Basis} (ψ : B.Simulates B') (h : ExecCodeDist w c x | loop_geometric I Q hp_pos hp_top _ hB hinv hp_exit hx ih_body => exact loop_geometric I Q hp_pos hp_top ih_body hB hinv hp_exit hx -private theorem extend_comap_eq {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (x : Data r') : - (x.comap f).extend f x = x := by - ext j - · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j - by_cases hj : ∃ i, f i = j - · obtain ⟨i, rfl⟩ := hj - exact hf.extend_apply _ _ i - · exact Function.extend_apply' _ _ _ hj - · rfl - private theorem extend_left_injective {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (x : Data r') : Function.Injective (fun y : Data r => y.extend f x) := by intro y₁ y₂ h have h' := congrArg (Data.comap f) h simpa [Data.comap_extend hf] using h' -private theorem pmf_map_toOuterMeasure {α β : Type*} (p : PMF α) (g : α → β) (S : Set β) : - (p.map g).toOuterMeasure S = p.toOuterMeasure (g ⁻¹' S) := by - rw [PMF.map, PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] - refine tsum_congr fun a => ?_ - rw [Function.comp_apply, PMF.toOuterMeasure_pure_apply] - by_cases ha : g a ∈ S <;> simp [ha] - private theorem pmf_map_apply_injective {α β : Type*} (p : PMF α) {g : α → β} (hg : g.Injective) (a : α) : p.map g (g a) = p a := by @@ -1698,7 +1639,7 @@ private theorem pmf_map_apply_injective {α β : Type*} (p : PMF α) {g : α → private theorem pmf_map_apply_not_mem_range {α β : Type*} (p : PMF α) (g : α → β) {b : β} (hb : b ∉ Set.range g) : p.map g b = 0 := by - rw [PMF.map_apply, ENNReal.tsum_eq_zero] + rw [PMF.map_apply, tsum_eq_zero_iff] intro a have hne : b ≠ g a := fun h => hb ⟨a, h.symm⟩ simp [hne] @@ -1757,7 +1698,7 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' refine congrArg (μ₁.bind ·) (funext fun y₁ => ?_) simp [Data.comap_extend hf] · refine add_le_add le_rfl ?_ - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] simp [Data.comap_extend hf] | branch_pos hc _ ih => subst hx₀ @@ -1791,7 +1732,7 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' rintro z' ⟨u, hIu, rfl⟩ hcz' have hcu : cond.Holds u := by simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' - simp only [μ_body', g, Data.comap_extend hf, pmf_map_toOuterMeasure] + simp only [μ_body', g, Data.comap_extend hf, PMF.toOuterMeasure_map_apply] have hset : ((fun y : Data r => y.extend f x) ⁻¹' {y' | ¬(cond.rename f).Holds y'}) = {y | ¬cond.Holds y} := by ext y @@ -1800,9 +1741,10 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' exact hp_exit u hIu hcu have hx_inv' : LoopStateInv (cond.rename f) I' Q' x := by rcases hx_inv with ⟨hcx, hIx⟩ | ⟨hcx, hQx⟩ - · exact Or.inl ⟨(Cond.holds_rename _ _).2 hcx, ⟨x.comap f, hIx, (extend_comap_eq hf x).symm⟩⟩ + · exact Or.inl ⟨(Cond.holds_rename _ _).2 hcx, + ⟨x.comap f, hIx, (Data.extend_comap hf x).symm⟩⟩ · exact Or.inr ⟨fun h => hcx ((Cond.holds_rename _ _).1 h), - ⟨x.comap f, hQx, (extend_comap_eq hf x).symm⟩⟩ + ⟨x.comap f, hQx, (Data.extend_comap hf x).symm⟩⟩ have hloop := loop_geometric (cond := cond.rename f) (body := body.rename f) I' Q' hp_pos hp_top (μ_body := μ_body') (T_body := T_body') (B_body := B_body) (x := x) @@ -1821,7 +1763,7 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' refine hloop.congr_dist ?_ le_rfl ext y' rw [loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv' hp_exit' hx_inv')] - have hx_eq : x = g (x.comap f) := (extend_comap_eq hf x).symm + have hx_eq : x = g (x.comap f) := (Data.extend_comap hf x).symm by_cases hy' : y' ∈ Set.range g · obtain ⟨y, rfl⟩ := hy' rw [pmf_map_apply_injective _ hg, @@ -1879,20 +1821,20 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program ≤ ∑' y, μ₁ y * (T₂ y + ∑' z, (μ₂ y) z * expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩)) := by - refine ENNReal.tsum_le_tsum fun y => ?_ + refine tsum_le_tsum fun y => ?_ by_cases hy : y ∈ μ₁.support · exact mul_le_mul_right (ih₂ y hy hc₂ N) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] _ = (∑' y, μ₁ y * T₂ y) + ∑' y, μ₁ y * ∑' z, (μ₂ y) z * expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩) := by - simp_rw [mul_add, ENNReal.tsum_add] + simp_rw [mul_add, tsum_add] _ = (∑' y, μ₁ y * T₂ y) + ∑' z, (μ₁.bind μ₂) z * expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by congr 1 simp only [Code.size, Nat.add_assoc] - exact (tsum_pmf_bind μ₁ μ₂ _).symm + exact (Cslib.Probability.PMF.tsum_bind_mul μ₁ μ₂ _).symm calc T₁ + ∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩) ≤ T₁ + ((∑' y, μ₁ y * T₂ y) + @@ -1931,13 +1873,13 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program (∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) ≤ ∑' y, μ y * (1 + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩)) := by - refine ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun y => mul_le_mul_right ?_ _ refine (expectedStepsWithin_running_le w p N ⟨pc + 1 + neg.size, y⟩).trans (le_of_eq ?_) rw [stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj, ProbComp.expect_pure, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] _ = 1 + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩) := by - simp_rw [mul_add, ENNReal.tsum_add, mul_one, μ.tsum_coe] + simp_rw [mul_add, tsum_add, mul_one, μ.tsum_coe] calc 1 + (T + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) ≤ 1 + (T + (1 + ∑' y, μ y * @@ -1981,17 +1923,18 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program calc (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc + 2 + body.size, u⟩)) ≤ ∑' u, μ_body z u * (1 + expectedStepsWithin w p n (.running ⟨pc, u⟩)) := by - refine ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun u => mul_le_mul_right ?_ _ refine (expectedStepsWithin_running_le w p n ⟨pc + 2 + body.size, u⟩).trans (le_of_eq ?_) rw [stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back, ProbComp.expect_pure] _ = 1 + ∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩) := by - simp_rw [mul_add, ENNReal.tsum_add, mul_one, (μ_body z).tsum_coe] + simp_rw [mul_add, tsum_add, mul_one, (μ_body z).tsum_coe] have hbind_k : (∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y) = ∑' y, loopExitPMF cond μ_body z y * K_cont y := by rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz] - exact (tsum_pmf_bind (μ_body z) (loopExitPMF cond μ_body) K_cont).symm + exact (Cslib.Probability.PMF.tsum_bind_mul (μ_body z) + (loopExitPMF cond μ_body) K_cont).symm have hih_step : (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + @@ -2000,14 +1943,14 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ ∑' u, μ_body z u * ((if cond.Holds u then M else 0) + 2 + ∑' y, loopExitPMF cond μ_body u y * K_cont y) := by - refine ENNReal.tsum_le_tsum fun u => ?_ + refine tsum_le_tsum fun u => ?_ by_cases hu : u ∈ (μ_body z).support · exact mul_le_mul_right (ihn (Nat.le_of_succ_le hn) u (hinv z hIz hcz u hu)) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + ∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y := by - simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, (μ_body z).tsum_coe, + simp_rw [mul_add, tsum_add, ENNReal.tsum_mul_right, (μ_body z).tsum_coe, one_mul] _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by @@ -2054,7 +1997,7 @@ theorem expectedStepsFrom_assemble (h : ExecCodeDist w c x μ T) {p : Program B T + ∑' y, μ y * expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) := by rw [← iSup_expectedStepsWithin w p (.running ⟨pc, x⟩)] refine iSup_le fun N => (h.expectedStepsWithin_assemble hp N).trans ?_ - refine add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _) + refine add_le_add le_rfl (tsum_le_tsum fun y => mul_le_mul_right ?_ _) exact expectedStepsWithin_le_expectedStepsFrom w p N (.running ⟨pc + c.size, y⟩) /-- Continuation upper-bound form of `expectedStepsFrom_assemble`. -/ @@ -2063,7 +2006,7 @@ theorem expectedStepsFrom_le (h : ExecCodeDist w c x μ T) {p : Program B r} {pc (hk : ∀ y ∈ μ.support, expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) ≤ k y) : expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ T + ∑' y, μ y * k y := by refine (h.expectedStepsFrom_assemble hp).trans - (add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => ?_)) + (add_le_add le_rfl (tsum_le_tsum fun y => ?_)) by_cases hy : y ∈ μ.support · exact mul_le_mul_right (hk y hy) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] @@ -2201,9 +2144,9 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} ∑' y₁, μ₁ y₁ * ∑' y, (μ₂ y₁) y * ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c₁.size + c₂.size, y⟩ (.halted v d) := by simp only [Code.size, Nat.add_assoc] - exact tsum_pmf_bind μ₁ μ₂ _ + exact Cslib.Probability.PMF.tsum_bind_mul μ₁ μ₂ _ rw [hbind] - refine (ENNReal.tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) + refine (tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) by_cases hy₁ : y₁ ∈ μ₁.support · exact mul_le_mul_right (ih₂ y₁ hy₁ hc₂) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy₁, zero_mul, zero_mul] @@ -2226,7 +2169,7 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by rw [stepRand_branch (s := ⟨pc, x⟩) hbr] simp [hc] - refine (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _).trans + refine (tsum_le_tsum fun y => mul_le_mul_right ?_ _).trans ((ih hp.cons_tail.append_left).trans (iSup_evalRand_halted_le_of_step hstep v d)) simpa [Code.size, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] using iSup_evalRand_halted_le_of_step @@ -2256,7 +2199,7 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} have hbr : stepRand w p (.running ⟨pc, z⟩) = pure (.running ⟨pc + 2, z⟩) := by rw [stepRand_branch (s := ⟨pc, z⟩) hp.head] simp [hcz] - refine (ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _).trans + refine (tsum_le_tsum fun u => mul_le_mul_right ?_ _).trans ((ih_body z hIz hcz hbody_contains).trans (iSup_evalRand_halted_le_of_step hbr v d)) exact iSup_evalRand_halted_le_of_step (stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back) v d @@ -2274,9 +2217,9 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} · rw [loopIterPMF_succ, loopStepPMF_of_holds hcz] have hbind : (∑' y, ((μ_body z).bind (loopIterPMF cond μ_body K)) y * H_loop y) = ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := - tsum_pmf_bind (μ_body z) (loopIterPMF cond μ_body K) H_loop + Cslib.Probability.PMF.tsum_bind_mul (μ_body z) (loopIterPMF cond μ_body K) H_loop rw [hbind] - refine (ENNReal.tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) + refine (tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) by_cases hu : u ∈ (μ_body z).support · exact mul_le_mul_right (ihK u (hinv z hIz hcz u hu)) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] @@ -2292,7 +2235,7 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} exact tsum_iSup_of_monotone fun y K₁ K₂ hK => mul_le_mul_left (loopIterPMF_indicator_mono cond μ_body x y hK) (H_exit y) rw [hrewrite_sum] - refine iSup_le fun K => (ENNReal.tsum_le_tsum fun y => ?_).trans (hiter_le K x hx_inv) + refine iSup_le fun K => (tsum_le_tsum fun y => ?_).trans (hiter_le K x hx_inv) by_cases hcy : ¬cond.Holds y · rw [Set.indicator_of_mem hcy] exact mul_le_mul_right (hexit_le y hcy) _ @@ -2334,7 +2277,7 @@ theorem program_halts (h : ExecCodeDist w c x μ T) (out : Fin r) : exact ne_top_of_le_ne_top (ENNReal.add_ne_top.mpr ⟨hT, ENNReal.one_ne_top⟩) hexp · have hle := h.le_iSup_evalRand_halted hp (y.regs out) y refine (le_trans ?_ - (ENNReal.le_tsum + (le_tsum (f := fun z => μ z * ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0 + c.size, z⟩ (.halted (y.regs out) y)) y)).trans hle diff --git a/Complexity/Computability/WordRAM/Randomized/Structured.lean b/Complexity/Computability/WordRAM/Randomized/Structured.lean index 1c63438b9..e037de1bd 100644 --- a/Complexity/Computability/WordRAM/Randomized/Structured.lean +++ b/Complexity/Computability/WordRAM/Randomized/Structured.lean @@ -305,16 +305,6 @@ section Rename variable {f : Fin r → Fin r'} -private theorem extend_comap_eq (hf : f.Injective) (x : Data r') : - (x.comap f).extend f x = x := by - ext j - · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j - by_cases hj : ∃ i, f i = j - · obtain ⟨i, rfl⟩ := hj - exact hf.extend_apply _ _ i - · exact Function.extend_apply' _ _ _ hj - · rfl - private theorem extend_set_eq (hf : f.Injective) (y : Data r) (x : Data r') (i : Fin r) (v : ℕ) : (y.set i v).extend f x = (y.extend f x).set (f i) v := Data.ext (hf.extend_update _ _ _ _) rfl @@ -342,7 +332,7 @@ theorem ExecRand.rename (hf : f.Injective) {w : ℕ} {c : Code B r} {x : Data r' pure (x.set (f dst) k.val, 1)) := by rw [← bind_pure_comp, bind_assoc] refine congrArg (ProbComp.unifFin (x.regs (f src)) >>= ·) (funext fun k => ?_) - simp only [pure_bind, extend_set_eq hf, extend_comap_eq hf] + simp only [pure_bind, extend_set_eq hf, Data.extend_comap hf] simp only [Code.rename, Data.comap_regs] rw [heq] exact ExecRand.rand hfit' @@ -386,7 +376,7 @@ theorem ExecRand.rename (hf : f.Injective) {w : ℕ} {c : Code B r} {x : Data r' simpa only [Code.rename, Functor.map_map] using h' | @loop_exit cond body _ hc => subst hx - simpa only [Code.rename, map_pure, extend_comap_eq hf] using + simpa only [Code.rename, map_pure, Data.extend_comap hf] using ExecRand.loop_exit (body := body.rename f) (fun h => hc ((Cond.holds_rename _ _).1 h)) | @loop_iter cond body _ mb ml hc _ hl ihb ihl => subst hx diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean index acfd953f7..6ec488459 100644 --- a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -9,6 +9,7 @@ module public import Complexity.Computability.WordRAM.Subroutine public import Complexity.Computability.WordRAM.Randomized.Expected +import Complexity.Probability.PMF.Expect /-! # Composable Randomized Word-RAM Data-Structure Subroutines @@ -17,8 +18,8 @@ This file extends the deterministic `Cslib.WordRAM.SubroutineDS` interface to ** structures and Las Vegas subroutines** on the Word-RAM — covering both structures initialized from a random seed (such as universal hash tables, 2-choice hash tables, and cuckoo hash tables) and structures whose operations make random choices via `Instr.rand` (such as treaps, skip lists, -reservoir samplers, or randomized rehashing); see [MitzenmacherUpfal2017], Sections 1.5, 2.1, and -2.3. +reservoir samplers, or randomized rehashing); see [MitzenmacherUpfal2017], Sections 2.1, 2.3, and +2.4. ## Core design @@ -50,6 +51,12 @@ density factor `seedSlack : ℝ≥0∞`: telescopes these one-step bounds without `ℝ≥0∞` subtraction: total expected step cost is at most `initTime + initPotential + D.totalCost charge s₀ ops`. +The sequence theorems charge initialization and subroutine bodies for a fixed operation list. +`prepare` delivers each operation's arguments by a semantic register replacement; its encoding +and delivery, caller dispatch and a whole-client driver are not charged. Callers requiring native +argument preparation can use `call_rename`, whose `B_prep` explicitly charges the preparation +code. Compiling a standalone body adds the final halt separately via `ExecCodeDist.program_halts`. + ## Main definitions * `Cslib.WordRAM.Randomized.SubroutineDS`: reusable randomized Word-RAM subroutine certificate for @@ -65,7 +72,8 @@ density factor `seedSlack : ℝ≥0∞`: * `Cslib.WordRAM.Randomized.SubroutineDS.initFresh_exec`, `initFresh_rename`: expected-time initialization on fresh zeroed memory. -* `Cslib.WordRAM.Randomized.SubroutineDS.zeroFillCode_exec`, `initClear_exec`, `initClear_rename`: +* `Cslib.WordRAM.SubroutineCode.zeroFillCode_exec`, + `Cslib.WordRAM.Randomized.SubroutineDS.initClear_exec`, `initClear_rename`: expected-time initialization on arbitrary dirty memory. * `Cslib.WordRAM.Randomized.SubroutineDS.code_rename`, `call_rename`: inline subroutine invocation on caller state space `Data r_caller` via register injection `f : Fin r_ds → Fin r_caller`. @@ -74,7 +82,7 @@ density factor `seedSlack : ℝ≥0∞`: or randomized caller steps that do not modify persistent DS registers (`P.input i = false`) or `[base, base + space)`. * `Cslib.WordRAM.Randomized.SubroutineDS.steps_expected_amortized`, `seq_expected_amortized`: - end-to-end telescope theorems bounding total expected time across any admissible operation + telescope theorems bounding initialization and expected body time across any admissible operation sequence `ops : List D.Op`. ## References @@ -82,7 +90,7 @@ density factor `seedSlack : ℝ≥0∞`: * [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, Chapter 17][CLRS2009] * [M. Mitzenmacher and E. Upfal, *Probability and Computing: Randomization and Probabilistic - Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. + Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017], Sections 2.1, 2.3, and 2.4. -/ @[expose] public section @@ -281,7 +289,7 @@ noncomputable def toRandomized (C : SubroutineDS P D w base params s₀) : · calc (∑' p, ν p * ((n p : ℝ≥0∞) + ∑' q, PMF.pure ((), y p) q * (C.potential q.2 : ℝ≥0∞))) ≤ ∑' p, ν p * ((C.charge s o : ℝ≥0∞) + (C.potential p.2 : ℝ≥0∞)) := by - refine ENNReal.tsum_le_tsum fun p => ?_ + refine tsum_le_tsum fun p => ?_ by_cases hp_mem : p ∈ ν.support · refine mul_le_mul_right ?_ _ have hp : C.rep s p.2 ∧ C.argMatch o p.2.regs := hsupp p hp_mem @@ -297,7 +305,7 @@ noncomputable def toRandomized (C : SubroutineDS P D w base params s₀) : exact_mod_cast hpot · rw [(PMF.apply_eq_zero_iff _ _).mpr hp_mem, zero_mul, zero_mul] _ = (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.2 : ℝ≥0∞) := by - simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + simp_rw [mul_add, tsum_add, ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] end Cslib.WordRAM.SubroutineDS @@ -451,57 +459,6 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) /-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`zeroFillCode_exec`, `initClear_exec`, `initClear_rename`) -/ -private def zeroFillSpec : Stateful.Spec.{0, 0, 0} where - State := PUnit - Op := PEmpty - Result o := PEmpty.elim o - step _ o := PEmpty.elim o - -private noncomputable def toZeroFillDet (C : SubroutineDS P D w base params s₀) : - Cslib.WordRAM.SubroutineDS - { P with initSetup := .op (.copy P.zeroReg P.zeroReg) } - zeroFillSpec w base params PUnit.unit where - space := C.space - spanTime := C.spanTime - setupTime := 1 - initPotential := 0 - charge _ o := PEmpty.elim o - rep _ x := x.Bounded w - potential _ := 0 - encode o := PEmpty.elim o - argMatch o := PEmpty.elim o - decode o := PEmpty.elim o - bounded h := h - rep_unique _ _ := rfl - arguments_fit o := PEmpty.elim o - encode_matches o := PEmpty.elim o - rep_supported _ _ hb _ := hb - potential_supported _ _ _ := rfl - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval := C.incCur_eval - span_exec := C.span_exec - setup_exec {x₁} hx₁ _ hfr _ _ := by - refine ⟨x₁.set P.zeroReg (x₁.regs P.zeroReg), ExecWithin.copy, ?_, - hx₁.set (hx₁.1 P.zeroReg), Nat.le_refl 0, Modifies.refl _ _⟩ - simp [Data.set, hfr] - step_exec o := PEmpty.elim o - -/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` -instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data r_ds} - (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) - (hfr₁ : x₁.regs P.frontierReg = base + C.space) - (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : - ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ - x₂.Bounded w ∧ - x₂.regs P.baseReg = base ∧ - x₂.regs P.frontierReg = base + C.space ∧ - (∀ j, x₂.regs (P.initParams j) = params j) ∧ - (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ - Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := - C.toZeroFillDet.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ - /-- Execute `P.initClear` on **arbitrary dirty memory** in `C.initTime + (4 * C.space + 6 : ℕ)` expected instructions. -/ theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} @@ -518,7 +475,8 @@ theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := - C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + P.zeroFillCode_exec C.clear_nodup C.params_not_clear C.incCur_eval + (he₁.bounded hx) hb₁ hfr₁ hpar₁ obtain ⟨ν₀, he₃, hseed, hsupp⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ rw [hm₁] at hmod₂ have he_clear : ExecCodeDist w P.initClear x (ν₀.map Prod.snd) @@ -618,10 +576,10 @@ theorem frame_caller_dist (C : SubroutineDS P D w base params s₀) obtain ⟨x', hx', rfl⟩ := hq_map obtain ⟨hb', hreg, hmem⟩ := hframe p hp x' hx' exact (C.frame_caller (hrep p hp) hb' hreg hdisj hmem).1 - · rw [frameDist, tsum_pmf_bind] + · rw [frameDist, Cslib.Probability.PMF.tsum_bind_mul] refine tsum_congr fun p => ?_ by_cases hp : p ∈ ν.support - · rw [tsum_pmf_map] + · rw [Cslib.Probability.PMF.tsum_map_mul] have hpot_eq : (∑' x', μ_caller p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = ∑' x', μ_caller p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by refine tsum_congr fun x' => ?_ @@ -700,7 +658,7 @@ theorem code_rename (C : SubroutineDS P D w base params s₀) (∑' q, μ_caller p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞) := by intro p - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] refine tsum_congr fun q_ds => ?_ rw [Data.comap_extend hf] simp_rw [hPot_inner] @@ -709,12 +667,13 @@ theorem code_rename (C : SubroutineDS P D w base params s₀) ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞))) = ∑' p_ds, ν_ds p_ds * (T_ds p_ds + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)) := - (tsum_pmf_map ν proj fun p_ds => + (Cslib.Probability.PMF.tsum_map_mul ν proj fun p_ds => T_ds p_ds + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)).symm have hRHS : (∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞)) = ∑' p_ds, ν_ds p_ds * (C.potential p_ds.1 p_ds.2 : ℝ≥0∞) := - (tsum_pmf_map ν proj fun p_ds => (C.potential p_ds.1 p_ds.2 : ℝ≥0∞)).symm + (Cslib.Probability.PMF.tsum_map_mul ν proj + fun p_ds => (C.potential p_ds.1 p_ds.2 : ℝ≥0∞)).symm rw [hLHS, hRHS] exact hamort_ds @@ -802,10 +761,10 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) · have hpot_prep : (∑' p', ν_prep p' * (C.potential p'.1 (p'.2.comap f) : ℝ≥0∞)) = ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by - simp only [ν_prep, frameDist, tsum_pmf_bind] + simp only [ν_prep, frameDist, Cslib.Probability.PMF.tsum_bind_mul] refine tsum_congr fun p => ?_ by_cases hp : p ∈ ν.support - · rw [tsum_pmf_map] + · rw [Cslib.Probability.PMF.tsum_map_mul] have hpot_eq : (∑' x', μ_prep p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = ∑' x', μ_prep p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by refine tsum_congr fun x' => ?_ @@ -827,9 +786,9 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) (T_code (p.1, x') + ∑' q, μ_code (p.1, x') q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by intro p - simp only [T_call, μ_call, tsum_pmf_bind, add_assoc] + simp only [T_call, μ_call, Cslib.Probability.PMF.tsum_bind_mul, add_assoc] congr 1 - rw [← ENNReal.tsum_add] + rw [← tsum_add] simp_rw [mul_add] have hmul : ∀ p : C.Seed × Data r_caller, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = @@ -839,18 +798,18 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) intro p rw [hinner p, mul_add] simp_rw [hmul] - rw [ENNReal.tsum_add] + rw [tsum_add] congr 1 symm dsimp only [ν_prep, frameDist] - rw [tsum_pmf_bind] + rw [Cslib.Probability.PMF.tsum_bind_mul] refine tsum_congr fun p => ?_ - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] have hprep_le : (∑' p, ν p * T_prep p) ≤ B_prep := by calc (∑' p, ν p * T_prep p) ≤ ∑' p, ν p * B_prep := by - refine ENNReal.tsum_le_tsum fun p => ?_ + refine tsum_le_tsum fun p => ?_ by_cases hp : p ∈ ν.support · exact mul_le_mul_right (hprep p hp).2.1 _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] @@ -870,60 +829,43 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) /-! ### Projection 6: End-to-end expected-amortized sequence telescope (`steps_expected_amortized`, `seq_expected_amortized`) -/ -/-- Deliver standalone operation arguments for `o` without changing memory or persistent -registers (`P.input i = false`). -/ -def prepare (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : Data r_ds where - regs i := if P.input i then C.encode o i else x.regs i - mem := x.mem - -@[simp] -theorem prepare_mem (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : - (C.prepare o x).mem = x.mem := rfl - -theorem prepare_regs_of_not_input (C : SubroutineDS P D w base params s₀) - {i : Fin r_ds} (hi : P.input i = false) (o : D.Op) (x : Data r_ds) : - (C.prepare o x).regs i = x.regs i := by - simp [prepare, hi] - -theorem prepare_bounded (C : SubroutineDS P D w base params s₀) - {s : D.State} {o : D.Op} {x : Data r_ds} (hx : x.Bounded w) (hv : D.valid s o) : - (C.prepare o x).Bounded w := by - refine ⟨fun i => ?_, hx.2⟩ - change Fits w (if P.input i then C.encode o i else x.regs i) - split - · exact C.arguments_fit o hv i (by assumption) - · exact hx.1 i +/-- Reuse the signature-level program and argument-delivery API with the actual instruction basis. +This projection carries code and the calling convention, without a deterministic certificate. -/ +def toDynamicProgram (C : SubroutineDS P D w base params s₀) : DynamicProgram D r_ds B := + P.toDynamicProgram C.encode C.decode theorem prepare_rep (C : SubroutineDS P D w base params s₀) {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} (hr : C.rep θ s x) (hv : D.valid s o) : - C.rep θ s (C.prepare o x) := + C.rep θ s (C.toDynamicProgram.prepare o x) := C.rep_supported - (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) - (C.prepare_bounded (C.bounded hr) hv) + (C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit _ hv)) hr theorem prepare_potential (C : SubroutineDS P D w base params s₀) {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} (hr : C.rep θ s x) : - C.potential θ (C.prepare o x) = C.potential θ x := + C.potential θ (C.toDynamicProgram.prepare o x) = C.potential θ x := C.potential_supported - (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hr theorem prepare_argMatch (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : - C.argMatch o (C.prepare o x).regs := + C.argMatch o (C.toDynamicProgram.prepare o x).regs := C.encode_matches o x.regs /-- Inductive multi-operation distributional execution relation for a randomized subroutine `C`. `C.ExecSteps s ops ν ν_final T_ops` witnesses that starting from abstract state `s` and coupled state distribution `ν : PMF (C.Seed × Data r_ds)`, executing each operation `o` in `ops` by -delivering arguments via `C.prepare o` and running `P.code` under `ExecCodeDist` produces the +delivering arguments via `C.toDynamicProgram.prepare o` and running `P.code` under `ExecCodeDist` +produces the final coupled distribution `ν_final`, decodes the exact abstract return value `(D.step s o).2` at every step, modifies memory only within `[base, base + C.space)`, and incurs total expected -operation step cost `T_ops`. -/ +operation body cost `T_ops`. Argument delivery is an external register replacement and +contributes no native instructions to this relation. -/ inductive ExecSteps (C : SubroutineDS P D w base params s₀) : D.State → List D.Op → PMF (C.Seed × Data r_ds) → PMF (C.Seed × Data r_ds) → ℝ≥0∞ → Prop where | nil (s : D.State) (ν : PMF (C.Seed × Data r_ds)) : @@ -933,7 +875,8 @@ inductive ExecSteps (C : SubroutineDS P D w base params s₀) : {μ_step : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds)} {T_step : C.Seed × Data r_ds → ℝ≥0∞} {T_rest : ℝ≥0∞} : (∀ p ∈ ν.support, - ExecCodeDist w P.code (C.prepare o p.2) ((μ_step p).map Prod.snd) (T_step p) ∧ + ExecCodeDist w P.code (C.toDynamicProgram.prepare o p.2) + ((μ_step p).map Prod.snd) (T_step p) ∧ ∀ q ∈ (μ_step p).support, C.decode o q.2.regs = some (D.step s o).2 ∧ C.rep q.1 (D.step s o).1 q.2 ∧ @@ -965,7 +908,7 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) exact ⟨ν, 0, ExecSteps.nil s ν, hseed, hrep, by simp⟩ | cons o os ih => obtain ⟨hvo, hvos⟩ := hv - let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.prepare o p.2) + let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.toDynamicProgram.prepare o p.2) have hseed_prep : ∀ θ, (ν_prep.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by intro θ rw [PMF.map_comp] @@ -978,7 +921,7 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) have hpot_prep : (∑' p', ν_prep p' * (C.potential p'.1 p'.2 : ℝ≥0∞)) = ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] refine tsum_congr fun p => ?_ by_cases hp : p ∈ ν.support · rw [C.prepare_potential (hrep p hp)] @@ -986,9 +929,9 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) obtain ⟨μ_step, T_step, hstep, hseed₁, hamort₁⟩ := C.step_exec o ν_prep hvo hseed_prep hsupp_prep let μ_op : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds) := - fun p => μ_step (p.1, C.prepare o p.2) + fun p => μ_step (p.1, C.toDynamicProgram.prepare o p.2) let T_op : C.Seed × Data r_ds → ℝ≥0∞ := - fun p => T_step (p.1, C.prepare o p.2) + fun p => T_step (p.1, C.toDynamicProgram.prepare o p.2) let ν₁ := ν.bind μ_op have hν₁_eq : ν₁ = ν_prep.bind μ_step := by simp only [ν₁, ν_prep, μ_op, PMF.bind_map] @@ -1006,17 +949,18 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) let T₁ : ℝ≥0∞ := ∑' p, ν p * T_op p have hT₁_eq : T₁ = ∑' p', ν_prep p' * T_step p' := by symm - exact tsum_pmf_map ν (fun p => (p.1, C.prepare o p.2)) T_step + exact Cslib.Probability.PMF.tsum_map_mul ν + (fun p => (p.1, C.toDynamicProgram.prepare o p.2)) T_step have hex_cons : C.ExecSteps s (o :: os) ν ν_final (T₁ + T_rest) := by refine ExecSteps.cons (μ_step := μ_op) (T_step := T_op) (fun p hp => ?_) hex_rest - have hp' : (p.1, C.prepare o p.2) ∈ ν_prep.support := + have hp' : (p.1, C.toDynamicProgram.prepare o p.2) ∈ ν_prep.support := (PMF.mem_support_map_iff _ _ _).mpr ⟨p, hp, rfl⟩ - exact hstep (p.1, C.prepare o p.2) hp' + exact hstep (p.1, C.toDynamicProgram.prepare o p.2) hp' have hstep_split : T₁ + (∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) = ∑' p', ν_prep p' * (T_step p' + ∑' q, μ_step p' q * (C.potential q.1 q.2 : ℝ≥0∞)) := by rw [hT₁_eq, hν₁_eq] - simp only [tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] + simp only [Cslib.Probability.PMF.tsum_bind_mul, ← tsum_add, mul_add] refine ⟨ν_final, T₁ + T_rest, hex_cons, hseed_final, hrep_final, ?_⟩ calc (T₁ + T_rest) + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) @@ -1061,7 +1005,7 @@ theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) calc (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) ≤ ∑' p, ν₀ p * (C.initPotential : ℝ≥0∞) := by - refine ENNReal.tsum_le_tsum fun p => ?_ + refine tsum_le_tsum fun p => ?_ by_cases hp : p ∈ ν₀.support · exact mul_le_mul_right (Nat.cast_le.mpr (hsupp₀ p hp).2.2.1) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] diff --git a/Complexity/Computability/WordRAM/Structured.lean b/Complexity/Computability/WordRAM/Structured.lean index a83d3abfb..82425958f 100644 --- a/Complexity/Computability/WordRAM/Structured.lean +++ b/Complexity/Computability/WordRAM/Structured.lean @@ -686,7 +686,8 @@ theorem Data.extend_extend (hf : f.Injective) (y z : Data r) (x : Data r') : · simp [Function.extend_apply' _ _ _ hj] · rfl -private theorem Data.extend_comap (hf : f.Injective) (x : Data r') : +/-- Writing a register view back into its source data changes nothing. -/ +theorem Data.extend_comap (hf : f.Injective) (x : Data r') : (x.comap f).extend f x = x := by ext j · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j diff --git a/Complexity/Probability/PMF/Expect.lean b/Complexity/Probability/PMF/Expect.lean index 4a00e266b..6606eb95d 100644 --- a/Complexity/Probability/PMF/Expect.lean +++ b/Complexity/Probability/PMF/Expect.lean @@ -1,7 +1,7 @@ /- -Copyright (c) 2026 Samuel Schlesinger. All rights reserved. +Copyright (c) 2026 Samuel Schlesinger, Renfei Zhou. All rights reserved. Released under Apache 2.0 license as described in the file LICENSE. -Authors: Samuel Schlesinger +Authors: Samuel Schlesinger, Renfei Zhou -/ module @@ -29,6 +29,8 @@ into the vocabulary of `Finset.expect` (`𝔼`) and `Finset.dens`. * `Cslib.Probability.PMF.sum_le_affine`: average pointwise affine bounds. * `Cslib.Probability.PMF.sum_map_mul`, `.sum_bind_mul`: average deterministic and random scores. +* `Cslib.Probability.PMF.tsum_map_mul`, `.tsum_bind_mul`: the same laws for extended + nonnegative scores over arbitrary types. * `Cslib.Probability.PMF.toReal_uniformOfFintype_bind_apply`: a uniform first stage averages the second stages. @@ -44,6 +46,23 @@ namespace Cslib.Probability.PMF open Finset open scoped BigOperators ENNReal +/-- Expectation of `k : β → ℝ≥0∞` under `p.map g` equals expectation of `k ∘ g` under `p`. -/ +theorem tsum_map_mul {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +/-- Expectation of `k : β → ℝ≥0∞` under `p.bind f` decomposes as an iterated expectation. -/ +theorem tsum_bind_mul {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : + (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by + simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] + exact tsum_comm + /-- Averaging a score after a deterministic map averages its composite with that map. Adapted from the cslib `crypto` branch at `5ff69ca`. -/ theorem sum_map_mul {α β : Type*} [Fintype α] [Fintype β] From 78981acccfe01c9eb996e96b66d00b24d1f40006 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 18:37:07 +0000 Subject: [PATCH 15/18] feat(WordRAM/Hashing): add self-contained k-wise independent and tabulation hashing programs --- Complexity.lean | 2 + .../Algorithms/WordRAM/Hashing/KWise.lean | 2730 +++++++++++++++++ .../WordRAM/Hashing/Tabulation.lean | 1565 ++++++++++ ComplexityTests/WordRAMKWiseHash.lean | 431 +++ 4 files changed, 4728 insertions(+) create mode 100644 Complexity/Algorithms/WordRAM/Hashing/KWise.lean create mode 100644 Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean create mode 100644 ComplexityTests/WordRAMKWiseHash.lean diff --git a/Complexity.lean b/Complexity.lean index f59b96f22..34622b270 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -148,7 +148,9 @@ public import Complexity.Algorithms.WordRAM.FusionTree.Query.Traversal public import Complexity.Algorithms.WordRAM.FusionTree.Rank public import Complexity.Algorithms.WordRAM.FusionTree.ReadHeader public import Complexity.Algorithms.WordRAM.FusionTree.Static +public import Complexity.Algorithms.WordRAM.Hashing.KWise public import Complexity.Algorithms.WordRAM.Hashing.MultiplyShift +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation public import Complexity.Algorithms.WordRAM.Histogram public import Complexity.Algorithms.WordRAM.Karger.Contract public import Complexity.Algorithms.WordRAM.Karger.FastCut diff --git a/Complexity/Algorithms/WordRAM/Hashing/KWise.lean b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean new file mode 100644 index 000000000..3697a633c --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean @@ -0,0 +1,2730 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation +public import Complexity.Foundations.FieldTheory.Finite.Trinomial +public import Complexity.Probability.KWiseIndependent.Trinomial + +/-! +# Self-contained Word-RAM `k`-wise independent hashing + +This file implements self-contained Word-RAM `k`-wise independent hashing over +`Cslib.WordRAM.Basis.exact` by evaluating degree-`(k - 1)` polynomials over the characteristic-`2` +finite field $\mathbb{F}_{2^{2 \cdot 3^t}} = \mathbb{F}_2[X] / (X^{2 \cdot 3^t} + X^{3^t} + 1)$ +(`Cslib.TrinomialField t`, where `t = trinomialStep U M` is chosen at runtime so that +$q = 2 \cdot 3^t \ge \max(\lceil \log_2 \max(U, 2) \rceil, \max(M, 1))$) and projecting to the low +`M` bits (`Cslib.Probability.trinomialPolyHash`) [Carter and Wegman][CarterWegman1979]; +[Wegman and Carter][WegmanCarter1981]; [Lidl and Niederreiter][LidlNiederreiter1997]. + +To achieve `O(1)` Word-RAM multiplication in $\mathbb{F}_{2^q}$ without dedicated carryless +multiplication hardware when $U \le n_{\max}^c$, we precompute the Four-Russians carryless +multiplication lookup table `clmulTab[0 .. 2 ^ (2 * b) - 1]` for $b$-bit chunks +(`b = chunkBits n_max = max 1 (⌊log₂ (max n_max 2)⌋ / 3)`, so $2^{2b} \le \max(4, n_{\max}) = +O(n_{\max}^{2/3}) \le O(n_{\max})$ words), decompose each $q$-bit field element into +$L = \lceil q / b \rceil$ limbs of $b$ bits, accumulate the $L \times L$ chunk carryless products, +and fold the resulting $< 2^{2q}$ polynomial mod $X^{2r} + X^r + 1$ in 8 straight-line bit +operations (`reduceTrinomialCode`). + +The initialization is factored into two reusable stages: +1. **`initTableCode k`**: deterministically computes $r = 3^t$, $q = 2r$, $b$, masks, and populates + `clmulTab[0 .. 2 ^ (2 * b) - 1]` in $O(n_{\max})$ deterministic instructions from + `(base, n_max, U, M)` alone (zero external primes or lookup hints). +2. **`resampleCode k`**: samples the $k$ polynomial coefficients `seed[0 .. k - 1]` uniformly and + independently in `Fin (2 ^ q)` using $k$ `Code.rand` calls in $O(k)$ instructions while + preserving `clmulTab`, so Cuckoo Hashing can rehash in $O(k) = O(1)$ instructions without + rebuilding `clmulTab`. +3. **`initCode k := .seq (initTableCode k) (resampleCode k)`**: full one-shot initialization. +4. **`evalCode k`**: evaluates Horner's rule for `trinomialPolyHash` in $O(k \cdot L^2 + 1)$ + deterministic instructions, preserving all memory and writing + `(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `regOut`. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.Hashing.KWise.chunkBits`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.limbCount`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.space`: chunk bit-width, limb count, and memory footprint. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.TableRep`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.SeedRep`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.Rep`: memory representation predicates separating the + deterministic `clmulTab` / header from the $k$ random seed words. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.initCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode`: structured Word-RAM programs over + `Basis.exact`. + +## Main results + +* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode_execWithin`: deterministic $O(n_{\max})$ + table construction establishing `TableRep`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRand`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.map_extractSeeds_eval_resampleComp`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRandWithin`: $O(k)$-step, $k$-query seed + resampling preserving `TableRep` and inducing `PMF.uniformOfFintype (Fin k → Fin (2 ^ q))`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode_execWithin`: deterministic $O(k \cdot L^2 + 1)$ + Horner evaluation matching `trinomialPolyHash`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.isKWiseIndependentHash_eval`: end-to-end Word-RAM + $k$-wise independence theorem combining `resampleComp`, `evalCode_execWithin`, and + `isKWiseIndependentHash_trinomialPolyHash`. + +## References + +* [J. L. Carter and M. N. Wegman, *Universal Classes of Hash Functions*][CarterWegman1979] +* [M. N. Wegman and J. L. Carter, *New Hash Functions and Their Use in Authentication and Set + Equality*][WegmanCarter1981] +* [R. Lidl and H. Niederreiter, *Finite Fields*][LidlNiederreiter1997] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.Hashing.KWise + +open Cslib.WordRAM Cslib.Probability +open Cslib.Algorithms.WordRAM.Hashing.Tabulation +open scoped ENNReal + +/-! ## Section 1: Parameters, bounds, and memory layout -/ + +/-- Chunk bit-width `b = max 1 (⌊log₂ (max n_max 2)⌋ / 3)` for Four-Russians carryless +multiplication, ensuring `1 ≤ b` and `2 ^ (2 * b) ≤ max 4 n_max`. -/ +@[expose] def chunkBits (n_max : ℕ) : ℕ := + max 1 (Nat.log 2 (max n_max 2) / 3) + +/-- `chunkBits n_max` is always at least `1`. -/ +theorem one_le_chunkBits (n_max : ℕ) : 1 ≤ chunkBits n_max := + Nat.le_max_left 1 _ + +/-- `chunkBits n_max` is always positive. -/ +theorem chunkBits_pos (n_max : ℕ) : 0 < chunkBits n_max := + one_le_chunkBits n_max + +/-- The Four-Russians carryless multiplication table size `2 ^ (2 * chunkBits n_max)` is at most +`max 4 n_max` (`= O(n_max^(2/3)) ≤ O(n_max)` words). -/ +theorem two_pow_two_mul_chunkBits_le (n_max : ℕ) : + 2 ^ (2 * chunkBits n_max) ≤ max 4 n_max := by + by_cases hdiv : Nat.log 2 (max n_max 2) / 3 ≤ 1 + · have hcb : chunkBits n_max = 1 := by simp [chunkBits]; omega + rw [hcb] + exact Nat.le_max_left 4 n_max + · have hcb : chunkBits n_max = Nat.log 2 (max n_max 2) / 3 := by simp [chunkBits]; omega + have h2cb : 2 * chunkBits n_max ≤ Nat.log 2 (max n_max 2) := by omega + calc + 2 ^ (2 * chunkBits n_max) ≤ 2 ^ Nat.log 2 (max n_max 2) := + Nat.pow_le_pow_right (by decide) h2cb + _ ≤ max n_max 2 := Nat.pow_log_le_self 2 (by omega) + _ ≤ max 4 n_max := by omega + +/-- Number of `b`-bit limbs `L = ⌈q / b⌉ = (q + b - 1) / b` needed to cover a `q`-bit field +element. -/ +@[expose] def limbCountFromB (b q : ℕ) : ℕ := + (q + b - 1) / b + +/-- Number of `chunkBits n_max`-bit limbs covering `trinomialDeg U M` bits. -/ +@[expose] def limbCount (n_max U M : ℕ) : ℕ := + limbCountFromB (chunkBits n_max) (trinomialDeg U M) + +/-- When `0 < b`, `limbCountFromB b q * b` is at least `q`. -/ +theorem le_limbCountFromB_mul {b : ℕ} (hb : 0 < b) (q : ℕ) : + q ≤ limbCountFromB b q * b := by + dsimp only [limbCountFromB] + have hmod := Nat.div_add_mod (q + b - 1) b + have hlt : (q + b - 1) % b < b := Nat.mod_lt _ hb + rw [Nat.mul_comm] + omega + +/-- `limbCount n_max U M * chunkBits n_max` covers all `trinomialDeg U M` bits. -/ +theorem trinomialDeg_le_limbCount_mul_chunkBits (n_max U M : ℕ) : + trinomialDeg U M ≤ limbCount n_max U M * chunkBits n_max := + le_limbCountFromB_mul (chunkBits_pos n_max) (trinomialDeg U M) + +/-- Memory footprint `8 + k + 2 ^ (2 * b)` words for `k`-wise independent hashing with chunk width +`b`: 8 header words, `k` seed words, and `2 ^ (2 * b)` carryless multiplication table entries. -/ +@[expose] def spaceFromB (b k : ℕ) : ℕ := + 8 + k + 2 ^ (2 * b) + +/-- Memory footprint `8 + k + 2 ^ (2 * chunkBits n_max)` words at capacity parameter `n_max`. -/ +@[expose] def space (n_max k : ℕ) : ℕ := + spaceFromB (chunkBits n_max) k + +/-- `space n_max k` is at most `8 + k + max 4 n_max = O(n_max + k)`. -/ +theorem space_le (n_max k : ℕ) : space n_max k ≤ 8 + k + max 4 n_max := by + dsimp only [space, spaceFromB] + have := two_pow_two_mul_chunkBits_le n_max + omega + +/-- Deterministic table representation predicate `TableRep base b U M k mem`: +stores the 8 header parameters `(r, q, 2 ^ r - 1, 2 ^ q - 1, 2 ^ M - 1, b, 2 ^ b - 1, L)` at +`base + 0 .. base + 7` and the Four-Russians carryless multiplication table +`clmul (idx >>> b) (idx &&& (2 ^ b - 1))` at `base + 8 + k + idx` for `idx < 2 ^ (2 * b)`. -/ +@[expose] def TableRep (base b U M k : ℕ) (mem : ℕ → ℕ) : Prop := + let r := 3 ^ trinomialStep U M + let q := trinomialDeg U M + mem base = r ∧ + mem (base + 1) = q ∧ + mem (base + 2) = 2 ^ r - 1 ∧ + mem (base + 3) = 2 ^ q - 1 ∧ + mem (base + 4) = 2 ^ M - 1 ∧ + mem (base + 5) = b ∧ + mem (base + 6) = 2 ^ b - 1 ∧ + mem (base + 7) = limbCountFromB b q ∧ + ∀ idx < 2 ^ (2 * b), + mem (base + 8 + k + idx) = clmul (idx >>> b) (idx &&& (2 ^ b - 1)) + +/-- Seed representation predicate `SeedRep base U M k c mem`: stores the `k` polynomial +coefficients `(c j).val < 2 ^ (trinomialDeg U M)` at `base + 8 + j.val`. -/ +@[expose] def SeedRep (base U M k : ℕ) + (c : Fin k → Fin (2 ^ trinomialDeg U M)) (mem : ℕ → ℕ) : Prop := + ∀ j : Fin k, mem (base + 8 + j.val) = (c j).val + +/-- Full representation predicate combining `TableRep` and `SeedRep`. -/ +@[expose] def Rep (base b U M k : ℕ) + (c : Fin k → Fin (2 ^ trinomialDeg U M)) (mem : ℕ → ℕ) : Prop := + TableRep base b U M k mem ∧ SeedRep base U M k c mem + +/-- Extract the `k`-tuple of seeds `Fin k → Fin (2 ^ trinomialDeg U M)` from +`[base + 8, base + 8 + k)`. -/ +@[expose] def extractSeeds (base U M k : ℕ) (mem : ℕ → ℕ) : + Fin k → Fin (2 ^ trinomialDeg U M) := + extractSlice (base + 8) (trinomialDeg U M) k mem + +/-! ## Section 2: Helper loops for `initTableCode` -/ + +/-- Body of `clog2Loop` on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, `reg 2 = 1`): updates +`rem ← (rem + 1) >>> 1` (`= (rem + 1) / 2`) and `acc ← acc + 1`. -/ +@[expose] def clog2Body : Code .exact 3 := + .seq (.op (.exactBin .add 0 0 2)) + (.seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2))) + +/-- Ceiling-`log₂` loop `while 1 < rem do [ rem ← (rem + 1) >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def clog2Loop : Code .exact 3 := + .loop (.lt 2 0) clog2Body + +/-- Closed-form final state of `clog2Loop`. -/ +@[expose] def clog2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.clog 2 rem) + +/-- `clog2Loop` executes deterministically in `5 * Nat.clog 2 rem + 2` instructions, accumulating +`acc + Nat.clog 2 rem` into register `1` and preserving all memory. -/ +theorem clog2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem1 : Fits w (rem + 1)) (hfit_acc : Fits w (acc + Nat.clog 2 rem)) : + Exec w clog2Loop x (clog2LoopData rem acc x) (5 * Nat.clog 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : (rem + 1) >>> 1 = (rem + 1) / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hclog : Nat.clog 2 rem = Nat.clog 2 ((rem + 1) / 2) + 1 := + Nat.clog_of_one_lt (by decide) hlt + have hdiv_lt : (rem + 1) / 2 < rem := by omega + have hfit_div1 : Fits w ((rem + 1) / 2 + 1) := hfit_rem1.of_le (by omega) + have hfit_div : Fits w ((rem + 1) / 2) := hfit_div1.of_le (by omega) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem + 1) + have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone]) hfit_rem1 + set d2 := d1.set 0 ((rem + 1) / 2) + have s2 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hone, hshr]) hfit_div + set d3 := d2.set 1 (acc + 1) + have s3 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d2 d3 1 := + exec_exactBin (by simp +decide [d2, d1, hacc, hone]) hfit_acc1 + have hrec := ih ((rem + 1) / 2) hdiv_lt (acc := acc + 1) (x := d3) + (by simp +decide [d3, d2]) + (by simp +decide [d3]) + (by simp +decide [d3, d2, d1, hone]) + hfit_div1 + (by rwa [show acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem by omega]) + have hd : clog2LoopData ((rem + 1) / 2) (acc + 1) d3 = clog2LoopData rem acc x := by + have hdiv_ne : (rem + 1) / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, d3, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem + omega + · simp +decide [clog2LoopData, d3, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec + simpa only [clog2Loop, clog2Body, hclog, + show 1 + (1 + 1) + (5 * Nat.clog 2 ((rem + 1) / 2) + 2) + 2 = + 5 * (Nat.clog 2 ((rem + 1) / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hclog0 : Nat.clog 2 rem = 0 := Nat.clog_of_right_le_one (by omega) 2 + have hd : clog2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, hrem_eq, hrem] + · simp +decide [clog2LoopData, hclog0, hacc] + · simp +decide [clog2LoopData] + · rfl + rw [hd, hclog0] + exact Exec.loop_exit hc + +/-- Final remainder reached by `trinomialStepAux` when repeatedly mapping `B ↦ (B + 2) / 3` until +`≤ 2`. -/ +@[expose] def trinomialStepRem (B : ℕ) : ℕ := + if B ≤ 2 then B else trinomialStepRem ((B + 2) / 3) +decreasing_by omega + +/-- `trinomialStepRem B` is always at most `2`. -/ +theorem trinomialStepRem_le_two (B : ℕ) : trinomialStepRem B ≤ 2 := by + induction B using Nat.strong_induction_on with + | h B ih => + rw [trinomialStepRem] + split_ifs with hB + · exact hB + · exact ih ((B + 2) / 3) (by omega) + +/-- `trinomialStepAux B` is at most `B`. -/ +theorem trinomialStepAux_le (B : ℕ) : trinomialStepAux B ≤ B := by + induction B using Nat.strong_induction_on with + | h B ih => + rw [trinomialStepAux] + split_ifs with hB + · exact Nat.zero_le B + · have := ih ((B + 2) / 3) (by omega) + omega + +/-- Body of `trinomialStepLoop` on `Data 4` (`reg 0 = rem`, `reg 1 = r`, `reg 2 = 2`, +`reg 3 = 3`): updates `rem ← (rem + 2) / 3` and `r ← r * 3`. -/ +@[expose] def trinomialStepBody : Code .exact 4 := + .seq (.op (.exactBin .add 0 0 2)) + (.seq (.op (.exactBin .div 0 0 3)) + (.op (.exactBin .mul 1 1 3))) + +/-- Loop `while 2 < rem do [ rem ← (rem + 2) / 3; r ← r * 3 ]` computing +`3 ^ trinomialStepAux B`. -/ +@[expose] def trinomialStepLoop : Code .exact 4 := + .loop (.lt 2 0) trinomialStepBody + +/-- Closed-form final state of `trinomialStepLoop`. -/ +@[expose] def trinomialStepLoopData (B acc : ℕ) (x : Data 4) : Data 4 := + (x.set 0 (trinomialStepRem B)).set 1 (acc * 3 ^ trinomialStepAux B) + +/-- `trinomialStepLoop` executes deterministically in `5 * trinomialStepAux B + 2` instructions, +accumulating `acc * 3 ^ trinomialStepAux B` into register `1` and preserving all memory. -/ +theorem trinomialStepLoop_exec {w : ℕ} (B : ℕ) {acc : ℕ} {x : Data 4} + (hrem : x.regs 0 = B) (hacc : x.regs 1 = acc) (htwo : x.regs 2 = 2) (hthree : x.regs 3 = 3) + (hfit_B2 : Fits w (B + 2)) (hfit_res : Fits w (acc * 3 ^ trinomialStepAux B)) : + Exec w trinomialStepLoop x (trinomialStepLoopData B acc x) + (5 * trinomialStepAux B + 2) := by + induction B using Nat.strong_induction_on generalizing acc x with + | h B ih => + by_cases hlt : 2 < B + · have hc : (Cond.lt (2 : Fin 4) 0).Holds x := by + simp [Cond.Holds, hrem, htwo, hlt] + have hstep : trinomialStepAux B = trinomialStepAux ((B + 2) / 3) + 1 := by + rw [trinomialStepAux, ite_eq_right (by omega)] + have hrem_step : trinomialStepRem B = trinomialStepRem ((B + 2) / 3) := by + rw [trinomialStepRem, ite_eq_right (by omega)] + have hmul_eq : (acc * 3) * 3 ^ trinomialStepAux ((B + 2) / 3) = + acc * 3 ^ trinomialStepAux B := by + rw [hstep, Nat.pow_succ'] + ring + have hfit_div2 : Fits w ((B + 2) / 3 + 2) := hfit_B2.of_le (by omega) + have hfit_div : Fits w ((B + 2) / 3) := hfit_div2.of_le (by omega) + have hfit_acc3 : Fits w (acc * 3) := by + have hpos : 1 ≤ 3 ^ trinomialStepAux ((B + 2) / 3) := Nat.one_le_pow _ 3 (by decide) + have hle : acc * 3 ≤ (acc * 3) * 3 ^ trinomialStepAux ((B + 2) / 3) := + Nat.le_mul_of_pos_right _ hpos + exact hfit_res.of_le (hmul_eq ▸ hle) + set d1 := x.set 0 (B + 2) + have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 4) x d1 1 := + exec_exactBin (by simp [hrem, htwo]) hfit_B2 + set d2 := d1.set 0 ((B + 2) / 3) + have s2 : Exec w (.op (.exactBin .div 0 0 3) : Code .exact 4) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hthree]) hfit_div + set d3 := d2.set 1 (acc * 3) + have s3 : Exec w (.op (.exactBin .mul 1 1 3) : Code .exact 4) d2 d3 1 := + exec_exactBin (by simp +decide [d2, d1, hacc, hthree]) hfit_acc3 + have hrec := ih ((B + 2) / 3) (by omega) (acc := acc * 3) (x := d3) + (by simp +decide [d3, d2]) + (by simp +decide [d3]) + (by simp +decide [d3, d2, d1, htwo]) + (by simp +decide [d3, d2, d1, hthree]) + hfit_div2 + (by rwa [hmul_eq]) + have hd : trinomialStepLoopData ((B + 2) / 3) (acc * 3) d3 = + trinomialStepLoopData B acc x := by + ext i : 2 + · fin_cases i + · simp +decide [trinomialStepLoopData, d3, d2, d1, hrem_step] + · simp +decide [trinomialStepLoopData, d3, d2, d1, hmul_eq] + · simp +decide [trinomialStepLoopData, d3, d2, d1] + · simp +decide [trinomialStepLoopData, d3, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec + simpa only [trinomialStepLoop, trinomialStepBody, hstep, + show 1 + (1 + 1) + (5 * trinomialStepAux ((B + 2) / 3) + 2) + 2 = + 5 * (trinomialStepAux ((B + 2) / 3) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 4) 0).Holds x := by + simp [Cond.Holds, hrem, htwo, hlt] + have hstep0 : trinomialStepAux B = 0 := by + rw [trinomialStepAux, ite_eq_left (by omega)] + have hrem0 : trinomialStepRem B = B := by + rw [trinomialStepRem, ite_eq_left (by omega)] + have hd : trinomialStepLoopData B acc x = x := by + ext i : 2 + · fin_cases i + · simp +decide [trinomialStepLoopData, hrem0, hrem] + · simp +decide [trinomialStepLoopData, hstep0, hacc] + · simp +decide [trinomialStepLoopData] + · simp +decide [trinomialStepLoopData] + · rfl + rw [hd, hstep0] + exact Exec.loop_exit hc + +/-- Bitwise decomposition of `idx < 2 ^ (2 * b)` into high `b`-bit chunk `u = idx >>> b` and low +`b`-bit chunk `v = idx &&& (2 ^ b - 1)`, together with properties of +`prevIdx = (u <<< b) + (v >>> 1)`. -/ +private theorem clmulTab_index_props {b idx : ℕ} (hidx : idx < 2 ^ (2 * b)) : + let u := idx >>> b + let v := idx &&& (2 ^ b - 1) + let prevIdx := (u <<< b) + (v >>> 1) + u < 2 ^ b ∧ + v < 2 ^ b ∧ + (u <<< b) + v = idx ∧ + prevIdx ≤ idx ∧ + (v ≠ 0 → prevIdx < idx) ∧ + (v = 0 → prevIdx = idx) ∧ + prevIdx >>> b = u ∧ + prevIdx &&& (2 ^ b - 1) = v >>> 1 := by + intro u v prevIdx + have hpow_pos : 0 < 2 ^ b := Nat.two_pow_pos b + have hu_eq : u = idx / 2 ^ b := Nat.shiftRight_eq_div_pow idx b + have hv_eq : v = idx % 2 ^ b := Nat.and_two_pow_sub_one_eq_mod idx b + have hu_lt : u < 2 ^ b := by + rw [hu_eq] + apply Nat.div_lt_of_lt_mul + rwa [← pow_add, ← two_mul] + have hv_lt : v < 2 ^ b := by + rw [hv_eq] + exact Nat.mod_lt idx hpow_pos + have huv_eq : (u <<< b) + v = idx := by + rw [Nat.shiftLeft_eq, hu_eq, hv_eq, Nat.div_add_mod' idx (2 ^ b)] + have hv_shr : v >>> 1 = v / 2 := Nat.shiftRight_eq_div_pow v 1 + have hv_shr_lt : v >>> 1 < 2 ^ b := by + rw [hv_shr] + exact Nat.lt_of_le_of_lt (Nat.div_le_self v 2) hv_lt + have hprev_eq : prevIdx = u * 2 ^ b + (v >>> 1) := by + dsimp only [prevIdx] + rw [Nat.shiftLeft_eq] + have hprev_le : prevIdx ≤ idx := by + have : v >>> 1 ≤ v := by rw [hv_shr]; exact Nat.div_le_self v 2 + dsimp only [prevIdx] + omega + have hprev_lt : v ≠ 0 → prevIdx < idx := by + intro hv0 + have : v >>> 1 < v := by rw [hv_shr]; omega + dsimp only [prevIdx] + omega + have hprev_zero : v = 0 → prevIdx = idx := by + intro hv0 + have : v >>> 1 = 0 := by rw [hv_shr, hv0] + dsimp only [prevIdx] + omega + have hprev_u : prevIdx >>> b = u := by + rw [Nat.shiftRight_eq_div_pow, hprev_eq, Nat.add_comm, + Nat.add_mul_div_right _ _ hpow_pos, Nat.div_eq_of_lt hv_shr_lt, zero_add] + have hprev_v : prevIdx &&& (2 ^ b - 1) = v >>> 1 := by + rw [Nat.and_two_pow_sub_one_eq_mod, hprev_eq, Nat.add_comm, + Nat.add_mul_mod_self_right, Nat.mod_eq_of_lt hv_shr_lt] + exact ⟨hu_lt, hv_lt, huv_eq, hprev_le, hprev_lt, hprev_zero, hprev_u, hprev_v⟩ + +/-- Step recurrence for `clmul u v` in `clmulTabBody`: +`((v &&& 1) * u) ^^^ (clmul u (v >>> 1) <<< 1)` equals `clmul u v`. -/ +private theorem clmul_eq_bit_mul_xor (u v : ℕ) : + ((v &&& 1) * u) ^^^ (clmul u (v >>> 1) <<< 1) = clmul u v := by + rw [clmul_step u v] + congr 1 + have hand : v &&& 1 = v % 2 := Nat.and_two_pow_sub_one_eq_mod v 1 + rw [hand] + rcases Nat.mod_two_eq_zero_or_one v with h0 | h1 + · simp [h0] + · simp [h1] + +/-- Body of `clmulTabLoop` on `Data 18` (`reg 0 = tabBase`, `reg 1 = idx`, `reg 2 = tabSize`, +`reg 3 = b`, `reg 4 = 2 ^ b - 1`, `reg 5 = 1`, `reg 6..9` scratch): populates +`mem[tabBase + idx] ← clmul (idx >>> b) (idx &&& (2 ^ b - 1))` using previously filled entry +`prevIdx = ((idx >>> b) <<< b) + ((idx &&& (2 ^ b - 1)) >>> 1)` and increments `idx`. -/ +@[expose] def clmulTabBody : Code .exact 18 := + .seq (.op (.exactBin .add 6 0 1)) + (.seq (.op (.const 9 0)) + (.seq (.op (.store 6 9)) + (.seq (.op (.exactBin .shiftRight 7 1 3)) + (.seq (.op (.exactBin .and 8 1 4)) + (.seq (.op (.exactBin .shiftRight 9 8 5)) + (.seq (.op (.exactBin .shiftLeft 6 7 3)) + (.seq (.op (.exactBin .add 9 6 9)) + (.seq (.op (.exactBin .add 9 0 9)) + (.seq (.op (.load 9 9)) + (.seq (.op (.exactBin .shiftLeft 9 9 5)) + (.seq (.op (.exactBin .and 8 8 5)) + (.seq (.op (.exactBin .mul 8 8 7)) + (.seq (.op (.exactBin .xor 9 8 9)) + (.seq (.op (.exactBin .add 6 0 1)) + (.seq (.op (.store 6 9)) + (.op (.exactBin .add 1 1 5))))))))))))))))) + +/-- Loop `while idx < tabSize do clmulTabBody` populating the Four-Russians carryless +multiplication table `clmulTab[0 .. 2 ^ (2 * b) - 1]`. -/ +@[expose] def clmulTabLoop : Code .exact 18 := + .loop (.lt 1 2) clmulTabBody + +/-- `clmulTabLoop` executes deterministically in `19 * (2 ^ (2 * b) - idx) + 2` instructions, +filling `mem[tabBase + i] = clmul (i >>> b) (i &&& (2 ^ b - 1))` for all `i < 2 ^ (2 * b)` and +preserving `reg 0 = tabBase` and memory outside `[tabBase + idx, tabBase + 2 ^ (2 * b))`. -/ +theorem clmulTabLoop_exec {w tabBase b : ℕ} + (hfit_end : Fits w (tabBase + 2 ^ (2 * b))) : + ∀ rem idx : ℕ, ∀ x : Data 18, + idx + rem = 2 ^ (2 * b) → + x.regs 0 = tabBase → + x.regs 1 = idx → + x.regs 2 = 2 ^ (2 * b) → + x.regs 3 = b → + x.regs 4 = 2 ^ b - 1 → + x.regs 5 = 1 → + (∀ i < idx, x.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) → + ∃ y : Data 18, + Exec w clmulTabLoop x y (19 * rem + 2) ∧ + y.regs 0 = tabBase ∧ + (∀ i < 2 ^ (2 * b), y.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) ∧ + (∀ a, (a < tabBase + idx ∨ tabBase + 2 ^ (2 * b) ≤ a) → y.mem a = x.mem a) := by + intro rem + induction rem with + | zero => + intro idx x hsum h0 h1 h2 _ _ _ hinv + have hidx_eq : idx = 2 ^ (2 * b) := by omega + have hc : ¬(Cond.lt (1 : Fin 18) 2).Holds x := by + simp [Cond.Holds, h1, h2, hidx_eq] + refine ⟨x, Exec.loop_exit hc, h0, ?_, fun _ _ => rfl⟩ + intro i hi + exact hinv i (hidx_eq ▸ hi) + | succ rem ih => + intro idx x hsum h0 h1 h2 h3 h4 h5 hinv + have hidx_lt : idx < 2 ^ (2 * b) := by omega + have hc : (Cond.lt (1 : Fin 18) 2).Holds x := by + simp [Cond.Holds, h1, h2, hidx_lt] + obtain ⟨hu_lt, hv_lt, huv_eq, hprev_le, hprev_lt, hprev_zero, hprev_u, hprev_v⟩ := + clmulTab_index_props hidx_lt + set u := idx >>> b + set v := idx &&& (2 ^ b - 1) + set prevIdx := (u <<< b) + (v >>> 1) + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hfit_idx : Fits w idx := hfit_end.of_le (by omega) + have hfit_addr : Fits w (tabBase + idx) := hfit_end.of_le (by omega) + have hfit_0 : Fits w 0 := hfit_end.of_le (Nat.zero_le _) + have hfit_u : Fits w u := hfit_end.of_le (by omega) + have hfit_v : Fits w v := hfit_end.of_le (by omega) + have hfit_vshr : Fits w (v >>> 1) := hfit_v.of_le (Nat.shiftRight_le v 1) + have hfit_ushl : Fits w (u <<< b) := hfit_idx.of_le (by omega) + have hfit_prev : Fits w prevIdx := hfit_idx.of_le hprev_le + have hfit_prevAddr : Fits w (tabBase + prevIdx) := hfit_addr.of_le (by omega) + have hprev_val_lt : clmul u (v >>> 1) < 2 ^ (2 * b - 1) := by + by_cases hb0 : b = 0 + · subst hb0 + have hu0 : u = 0 := by omega + simp [hu0] + · have hv_shr_lt : v >>> 1 < 2 ^ (b - 1) := by + rw [Nat.shiftRight_eq_div_pow, pow_one] + apply Nat.div_lt_of_lt_mul + rwa [← pow_succ', Nat.sub_add_cancel (by omega)] + have hlt := clmul_lt_two_pow hu_lt hv_shr_lt + rwa [show b + (b - 1) = 2 * b - 1 by omega] at hlt + have hsh_lt : clmul u (v >>> 1) <<< 1 < 2 ^ (2 * b) := by + by_cases hb0 : b = 0 + · subst hb0 + have hu0 : u = 0 := by omega + simp [hu0] + · rw [Nat.shiftLeft_eq, pow_one] + have hpow_split : 2 ^ (2 * b) = 2 ^ (2 * b - 1) * 2 := by + conv_lhs => rw [← Nat.sub_add_cancel (show 1 ≤ 2 * b by omega), pow_succ] + omega + have hfit_sh : Fits w (clmul u (v >>> 1) <<< 1) := hfit_end.of_le (by omega) + have hfit_vbit : Fits w (v &&& 1) := hfit_v.of_le Nat.and_le_left + have hbit_le : (v &&& 1) * u ≤ u := by + calc (v &&& 1) * u ≤ 1 * u := Nat.mul_le_mul_right u Nat.and_le_right + _ = u := Nat.one_mul u + have hfit_term : Fits w ((v &&& 1) * u) := hfit_u.of_le hbit_le + have hval_lt : clmul u v < 2 ^ (2 * b) := clmul_lt_two_pow_two_mul hu_lt hv_lt + have hfit_val : Fits w (clmul u v) := hfit_end.of_le (by omega) + have hfit_idx1 : Fits w (idx + 1) := hfit_end.of_le (by omega) + set s1 := x.set 6 (tabBase + idx) + have e1 : Exec w (.op (.exactBin .add 6 0 1) : Code .exact 18) x s1 1 := + exec_exactBin (by simp [h0, h1]) hfit_addr + set s2 := s1.set 9 0 + have e2 : Exec w (.op (.const 9 0) : Code .exact 18) s1 s2 1 := Exec.const hfit_0 + set s3 := s2.write (tabBase + idx) 0 + have e3 : Exec w (.op (.store 6 9) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1] using + (Exec.store (B := .exact) (w := w) (a := 6) (s := 9) (x := s2)) + have hprev_mem : s3.mem (tabBase + prevIdx) = clmul u (v >>> 1) := by + dsimp only [s3, s2, s1, Data.write, Data.set] + by_cases hv0 : v = 0 + · have hpeq : prevIdx = idx := hprev_zero hv0 + have hv_shr0 : v >>> 1 = 0 := by rw [hv0]; rfl + simp [hpeq, hv_shr0] + · have hplt : prevIdx < idx := hprev_lt hv0 + have hne : tabBase + prevIdx ≠ tabBase + idx := by omega + rw [Function.update_of_ne hne, hinv prevIdx hplt, hprev_u, hprev_v] + set s4 := s3.set 7 u + have e4 : Exec w (.op (.exactBin .shiftRight 7 1 3) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, h1, h3, u]) hfit_u + set s5 := s4.set 8 v + have e5 : Exec w (.op (.exactBin .and 8 1 4) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h1, h4, v]) hfit_v + set s6 := s5.set 9 (v >>> 1) + have e6 : Exec w (.op (.exactBin .shiftRight 9 8 5) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h5]) hfit_vshr + set s7 := s6.set 6 (u <<< b) + have e7 : Exec w (.op (.exactBin .shiftLeft 6 7 3) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, s4, s3, s2, s1, h3]) hfit_ushl + set s8 := s7.set 9 prevIdx + have e8 : Exec w (.op (.exactBin .add 9 6 9) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, prevIdx]) hfit_prev + set s9 := s8.set 9 (tabBase + prevIdx) + have e9 : Exec w (.op (.exactBin .add 9 0 9) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h0]) hfit_prevAddr + set s10 := s9.set 9 (clmul u (v >>> 1)) + have e10 : Exec w (.op (.load 9 9) : Code .exact 18) s9 s10 1 := by + simpa +decide [s10, s9, s8, s7, s6, s5, s4, hprev_mem] using + (Exec.load (B := .exact) (w := w) (d := 9) (a := 9) (x := s9)) + set s11 := s10.set 9 (clmul u (v >>> 1) <<< 1) + have e11 : Exec w (.op (.exactBin .shiftLeft 9 9 5) : Code .exact 18) s10 s11 1 := + exec_exactBin (by simp +decide [s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5]) hfit_sh + set s12 := s11.set 8 (v &&& 1) + have e12 : Exec w (.op (.exactBin .and 8 8 5) : Code .exact 18) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5]) hfit_vbit + set s13 := s12.set 8 ((v &&& 1) * u) + have e13 : Exec w (.op (.exactBin .mul 8 8 7) : Code .exact 18) s12 s13 1 := + exec_exactBin (by simp +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4]) hfit_term + have hxor_eq : s13.regs 8 ^^^ s13.regs 9 = clmul u v := by + simpa +decide [s13, s12, s11] using clmul_eq_bit_mul_xor u v + set s14 := s13.set 9 (clmul u v) + have e14 : Exec w (.op (.exactBin .xor 9 8 9) : Code .exact 18) s13 s14 1 := + exec_exactBin (by simp [hxor_eq]) hfit_val + set s15 := s14.set 6 (tabBase + idx) + have e15 : Exec w (.op (.exactBin .add 6 0 1) : Code .exact 18) s14 s15 1 := + exec_exactBin + (by simp +decide [s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0, h1]) + hfit_addr + set s16 := s15.write (tabBase + idx) (clmul u v) + have e16 : Exec w (.op (.store 6 9) : Code .exact 18) s15 s16 1 := by + simpa +decide [s16, s15, s14] using + (Exec.store (B := .exact) (w := w) (a := 6) (s := 9) (x := s15)) + set s17 := s16.set 1 (idx + 1) + have e17 : Exec w (.op (.exactBin .add 1 1 5) : Code .exact 18) s16 s17 1 := + exec_exactBin + (by simp +decide [s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + h1, h5]) + hfit_idx1 + have hbody := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq (e16.seq e17))))))))))))))) + have hs17_mem : s17.mem = Function.update x.mem (tabBase + idx) (clmul u v) := by + dsimp only [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + Data.write, Data.set] + rw [Function.update_idem] + have hinv' : ∀ i < idx + 1, s17.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1)) := by + intro i hi + rw [hs17_mem] + rcases eq_or_ne i idx with rfl | hne + · simp [u, v] + · have hilt : i < idx := by omega + rw [Function.update_of_ne (by omega)] + exact hinv i hilt + have hs17_0 : s17.regs 0 = tabBase := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0] + have hs17_1 : s17.regs 1 = idx + 1 := by simp +decide [s17] + have hs17_2 : s17.regs 2 = 2 ^ (2 * b) := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h2] + have hs17_3 : s17.regs 3 = b := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h3] + have hs17_4 : s17.regs 4 = 2 ^ b - 1 := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h4] + have hs17_5 : s17.regs 5 = 1 := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] + obtain ⟨y, hrec, hy0, hytab, hyout⟩ := + ih (idx + 1) s17 (by omega) hs17_0 hs17_1 hs17_2 hs17_3 hs17_4 hs17_5 hinv' + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, hy0, hytab, ?_⟩ + · simpa only [clmulTabLoop, clmulTabBody, + show 1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + + (1 + (1 + (1 + (1 + (1 + 1))))))))))))))) + (19 * rem + 2) + 2 = + 19 * (rem + 1) + 2 by omega] using hiter + · intro a ha + have ha' : a < tabBase + (idx + 1) ∨ tabBase + 2 ^ (2 * b) ≤ a := by omega + have hne : a ≠ tabBase + idx := by omega + rw [hyout a ha', hs17_mem, Function.update_of_ne hne] + +/-! ## Section 3: Table construction (`initTableCode`) and seed resampling (`resampleCode`) -/ + +/-- Register `0`: holds the base memory address `base`. -/ +abbrev regBase : Fin 18 := 0 + +/-- Register `1`: holds `n_max` on entry to `initTableCode` (and `key` on entry to `evalCode`). -/ +abbrev regNMax : Fin 18 := 1 + +/-- Register `2`: holds `U` on entry to `initTableCode` (and output in `evalCode`). -/ +abbrev regU : Fin 18 := 2 + +/-- Register `3`: holds `M` on entry to `initTableCode`. -/ +abbrev regM : Fin 18 := 3 + +/-- Subprogram on `Data 18` computing `chunkBits n_max` into register `5` and `1` into register `6` +from `n_max` in register `1`, while preserving registers `0, 1, 2, 3` and memory. -/ +@[expose] def chunkBitsCode : Code .exact 18 := + .seq (.op (.const 4 2)) + (.seq (maxCode 4 1 4) + (.seq (.op (.const 5 0)) + (.seq (.op (.const 6 1)) + (.seq (log2Loop.rename ![4, 5, 6]) + (.seq (.op (.const 4 3)) + (.seq (.op (.exactBin .div 5 5 4)) + (maxCode 5 6 5))))))) + +/-- `chunkBitsCode` executes deterministically in `4 * Nat.log 2 (max n_max 2) + 13` steps. -/ +theorem chunkBitsCode_exec {w base n_max U M : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = n_max) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (hfit_n : Fits w (max n_max 2)) (hfit_3 : Fits w 3) : + ∃ y : Data 18, + Exec w chunkBitsCode x y (4 * Nat.log 2 (max n_max 2) + 13) ∧ + y.regs 0 = base ∧ + y.regs 2 = U ∧ + y.regs 3 = M ∧ + y.regs 5 = chunkBits n_max ∧ + y.regs 6 = 1 ∧ + y.mem = x.mem := by + have hfit_2 : Fits w 2 := hfit_3.of_le (by decide) + have hfit_1 : Fits w 1 := hfit_3.of_le (by decide) + have hfit_0 : Fits w 0 := hfit_3.of_le (by decide) + have hlog_le : Nat.log 2 (max n_max 2) ≤ max n_max 2 := Nat.log_le_self 2 _ + have hfit_log : Fits w (0 + Nat.log 2 (max n_max 2)) := hfit_n.of_le (by omega) + have hfit_div : Fits w (Nat.log 2 (max n_max 2) / 3) := hfit_n.of_le (by omega) + set s1 := x.set 4 2 + have e1 : Exec w (.op (.const 4 2) : Code .exact 18) x s1 1 := Exec.const hfit_2 + set s2 := s1.set 4 (max n_max 2) + have e2 : Exec w (maxCode 4 1 4) s1 s2 3 := by + simpa +decide [s2, s1, h1] using maxCode_exec (w := w) 4 1 4 s1 + set s3 := s2.set 5 0 + have e3 : Exec w (.op (.const 5 0) : Code .exact 18) s2 s3 1 := Exec.const hfit_0 + set s4 := s3.set 6 1 + have e4 : Exec w (.op (.const 6 1) : Code .exact 18) s3 s4 1 := Exec.const hfit_1 + set f : Fin 3 → Fin 18 := ![4, 5, 6] + have hmap_inj : Function.Injective f := by decide + have hloop_inner := log2Loop_exec (w := w) (max n_max 2) (acc := 0) + (x := s4.comap f) + (by simp +decide [s4, s3, s2, f, Data.comap]) + (by simp +decide [s4, s3, f, Data.comap]) + (by simp +decide [s4, f, Data.comap]) + hfit_n hfit_log + have e5 := hloop_inner.rename hmap_inj + set s5 := (log2LoopData (max n_max 2) 0 (s4.comap f)).extend f s4 + have hs5_5 : s5.regs 5 = Nat.log 2 (max n_max 2) := by + change (s5.comap f).regs 1 = Nat.log 2 (max n_max 2) + rw [Data.comap_extend hmap_inj] + simp +decide [log2LoopData, s4, s3, s2, s1, f, Data.comap] + have hs5_6 : s5.regs 6 = 1 := by + change (s5.comap f).regs 2 = 1 + rw [Data.comap_extend hmap_inj] + simp +decide [log2LoopData, s4, s3, s2, s1, f, Data.comap] + have hs5_0 : s5.regs 0 = base := by + simp +decide [s5, Data.extend, f, s4, s3, s2, s1, h0] + have hs5_2 : s5.regs 2 = U := by + simp +decide [s5, Data.extend, f, s4, s3, s2, s1, h2] + have hs5_3 : s5.regs 3 = M := by + simp +decide [s5, Data.extend, f, s4, s3, s2, s1, h3] + set s6 := s5.set 4 3 + have e6 : Exec w (.op (.const 4 3) : Code .exact 18) s5 s6 1 := Exec.const hfit_3 + set s7 := s6.set 5 (Nat.log 2 (max n_max 2) / 3) + have e7 : Exec w (.op (.exactBin .div 5 5 4) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, hs5_5]) hfit_div + set s8 := s7.set 5 (chunkBits n_max) + have e8 : Exec w (maxCode 5 6 5) s7 s8 3 := by + simpa +decide [s8, s7, s6, hs5_6, chunkBits] using maxCode_exec (w := w) 5 6 5 s7 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq e8)))))) + refine ⟨s8, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simpa only [chunkBitsCode, f, + show 1 + (3 + (1 + (1 + (4 * Nat.log 2 (max n_max 2) + 2 + (1 + (1 + 3)))))) = + 4 * Nat.log 2 (max n_max 2) + 13 by omega] using hall + · simp +decide [s8, s7, s6, hs5_0] + · simp +decide [s8, s7, s6, hs5_2] + · simp +decide [s8, s7, s6, hs5_3] + · simp +decide [s8] + · simp +decide [s8, s7, s6, hs5_6] + · rfl + +private theorem add_two_le_two_pow {q : ℕ} (hq : 2 ≤ q) : q + 2 ≤ 2 ^ q := by + induction q with + | zero => omega + | succ n ih => + rcases (by omega : n < 2 ∨ 2 ≤ n) with hlt | hge + · have hn1 : n = 1 := by omega + subst hn1 + decide + · have hrec := ih hge + have hpos : 2 ≤ 2 ^ n := + calc 2 = 2 ^ 1 := by decide + _ ≤ 2 ^ n := Nat.pow_le_pow_right (by decide) (by omega) + rw [pow_succ] + omega + +/-- Subprogram on `Data 18` computing `r = 3 ^ trinomialStep U M` into register `1`, +`q = trinomialDeg U M` into register `2`, and `2` into register `4` from `U` in register `2`, +`M` in register `3`, and `1` in register `6`, while preserving `reg 0 = base`, `reg 3 = M`, +`reg 5 = b`, `reg 6 = 1`, and memory. -/ +@[expose] def trinomialParamsCode : Code .exact 18 := + .seq (.op (.const 4 2)) + (.seq (maxCode 7 2 4) + (.seq (.op (.const 8 0)) + (.seq (clog2Loop.rename ![7, 8, 6]) + (.seq (maxCode 7 3 6) + (.seq (maxCode 7 8 7) + (.seq (.op (.const 1 1)) + (.seq (.op (.const 8 3)) + (.seq (trinomialStepLoop.rename ![7, 1, 4, 8]) + (.op (.exactBin .mul 2 4 1)))))))))) + +/-- `trinomialParamsCode` executes deterministically in +`5 * Nat.clog 2 (max U 2) + 5 * trinomialStep U M + 18` steps. -/ +theorem trinomialParamsCode_exec {w base U M b : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (h5 : x.regs 5 = b) (h6 : x.regs 6 = 1) + (hfit_U : Fits w (max U 2 + 1)) (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ∃ y : Data 18, + Exec w trinomialParamsCode x y + (5 * Nat.clog 2 (max U 2) + 5 * trinomialStep U M + 18) ∧ + y.regs 0 = base ∧ + y.regs 1 = 3 ^ trinomialStep U M ∧ + y.regs 2 = trinomialDeg U M ∧ + y.regs 3 = M ∧ + y.regs 4 = 2 ∧ + y.regs 5 = b ∧ + y.regs 6 = 1 ∧ + y.mem = x.mem := by + set B := trinomialTargetBits U M + have hB_le_q : B ≤ trinomialDeg U M := trinomialTargetBits_le_trinomialDeg U M + have hq_ge_2 : 2 ≤ trinomialDeg U M := by + dsimp only [trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + omega + have hpow_q : trinomialDeg U M + 2 ≤ 2 ^ trinomialDeg U M := add_two_le_two_pow hq_ge_2 + have hfit_q2 : Fits w (trinomialDeg U M + 2) := hfit_q.of_le hpow_q + have hfit_B2 : Fits w (B + 2) := hfit_q2.of_le (by omega) + have hclog_le_B : Nat.clog 2 (max U 2) ≤ B := le_max_left _ _ + have hfit_clog : Fits w (0 + Nat.clog 2 (max U 2)) := hfit_B2.of_le (by omega) + have hr_le_q : 3 ^ trinomialStepAux B ≤ trinomialDeg U M := by + change 3 ^ trinomialStep U M ≤ 2 * 3 ^ trinomialStep U M + omega + have hfit_r : Fits w (1 * 3 ^ trinomialStepAux B) := by + rw [Nat.one_mul] + exact hfit_q2.of_le (by omega) + have hfit_q_val : Fits w (trinomialDeg U M) := hfit_q2.of_le (by omega) + have hfit_3 : Fits w 3 := hfit_q2.of_le (by omega) + have hfit_2 : Fits w 2 := hfit_q2.of_le (by omega) + have hfit_1 : Fits w 1 := hfit_q2.of_le (by omega) + have hfit_0 : Fits w 0 := hfit_q2.of_le (by omega) + set s1 := x.set 4 2 + have e1 : Exec w (.op (.const 4 2) : Code .exact 18) x s1 1 := Exec.const hfit_2 + set s2 := s1.set 7 (max U 2) + have e2 : Exec w (maxCode 7 2 4) s1 s2 3 := by + simpa +decide [s2, s1, h2] using maxCode_exec (w := w) 7 2 4 s1 + set s3 := s2.set 8 0 + have e3 : Exec w (.op (.const 8 0) : Code .exact 18) s2 s3 1 := Exec.const hfit_0 + set f1 : Fin 3 → Fin 18 := ![7, 8, 6] + have hmap1 : Function.Injective f1 := by decide + have hclog_inner := clog2Loop_exec (w := w) (max U 2) (acc := 0) + (x := s3.comap f1) + (by simp +decide [s3, s2, f1, Data.comap]) + (by simp +decide [s3, f1, Data.comap]) + (by simp +decide [s3, s2, s1, h6, f1, Data.comap]) + hfit_U hfit_clog + have e4 := hclog_inner.rename hmap1 + set s4 := (clog2LoopData (max U 2) 0 (s3.comap f1)).extend f1 s3 + have hs4_8 : s4.regs 8 = Nat.clog 2 (max U 2) := by + change (s4.comap f1).regs 1 = Nat.clog 2 (max U 2) + rw [Data.comap_extend hmap1] + simp +decide [clog2LoopData, s3, s2, s1, f1, Data.comap] + have hs4_6 : s4.regs 6 = 1 := by + change (s4.comap f1).regs 2 = 1 + rw [Data.comap_extend hmap1] + simp +decide [clog2LoopData, s3, s2, s1, h6, f1, Data.comap] + have hs4_0 : s4.regs 0 = base := by + simp +decide [s4, Data.extend, f1, s3, s2, s1, h0] + have hs4_3 : s4.regs 3 = M := by + simp +decide [s4, Data.extend, f1, s3, s2, s1, h3] + have hs4_4 : s4.regs 4 = 2 := by + simp +decide [s4, Data.extend, f1, s3, s2, s1] + have hs4_5 : s4.regs 5 = b := by + simp +decide [s4, Data.extend, f1, s3, s2, s1, h5] + set s5 := s4.set 7 (max M 1) + have e5 : Exec w (maxCode 7 3 6) s4 s5 3 := by + simpa +decide [s5, hs4_3, hs4_6] using maxCode_exec (w := w) 7 3 6 s4 + set s6 := s5.set 7 B + have e6 : Exec w (maxCode 7 8 7) s5 s6 3 := by + simpa +decide [s6, s5, hs4_8, B, trinomialTargetBits] using maxCode_exec (w := w) 7 8 7 s5 + set s7 := s6.set 1 1 + have e7 : Exec w (.op (.const 1 1) : Code .exact 18) s6 s7 1 := Exec.const hfit_1 + set s8 := s7.set 8 3 + have e8 : Exec w (.op (.const 8 3) : Code .exact 18) s7 s8 1 := Exec.const hfit_3 + set f2 : Fin 4 → Fin 18 := ![7, 1, 4, 8] + have hmap2 : Function.Injective f2 := by decide + have hstep_inner := trinomialStepLoop_exec (w := w) B (acc := 1) + (x := s8.comap f2) + (by simp +decide [s8, s7, s6, f2, Data.comap]) + (by simp +decide [s8, s7, f2, Data.comap]) + (by simp +decide [s8, s7, s6, s5, hs4_4, f2, Data.comap]) + (by simp +decide [s8, f2, Data.comap]) + hfit_B2 hfit_r + have e9 := hstep_inner.rename hmap2 + set s9 := (trinomialStepLoopData B 1 (s8.comap f2)).extend f2 s8 + have hs9_1 : s9.regs 1 = 3 ^ trinomialStep U M := by + change (s9.comap f2).regs 1 = 3 ^ trinomialStep U M + rw [Data.comap_extend hmap2] + simp +decide [trinomialStepLoopData, trinomialStep, B, s8, s7, f2, Data.comap] + have hs9_4 : s9.regs 4 = 2 := by + change (s9.comap f2).regs 2 = 2 + rw [Data.comap_extend hmap2] + simp +decide [trinomialStepLoopData, s8, s7, s6, s5, hs4_4, f2, Data.comap] + have hs9_0 : s9.regs 0 = base := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_0] + have hs9_3 : s9.regs 3 = M := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_3] + have hs9_5 : s9.regs 5 = b := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_5] + have hs9_6 : s9.regs 6 = 1 := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_6] + set s10 := s9.set 2 (trinomialDeg U M) + have e10 : Exec w (.op (.exactBin .mul 2 4 1) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [hs9_4, hs9_1, trinomialDeg]) hfit_q_val + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq e10)))))))) + refine ⟨s10, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simpa only [trinomialParamsCode, trinomialStep, B, f1, f2, + show 1 + (3 + (1 + (5 * Nat.clog 2 (max U 2) + 2 + (3 + (3 + (1 + (1 + + (5 * trinomialStepAux (trinomialTargetBits U M) + 2 + 1)))))))) = + 5 * Nat.clog 2 (max U 2) + + 5 * trinomialStepAux (trinomialTargetBits U M) + 18 by omega] using hall + · simp +decide [s10, hs9_0] + · simp +decide [s10, hs9_1] + · simp +decide [s10] + · simp +decide [s10, hs9_3] + · simp +decide [s10, hs9_4] + · simp +decide [s10, hs9_5] + · simp +decide [s10, hs9_6] + · rfl + +/-- First 17-instruction header-store block: writes `r, q, 2 ^ r - 1, 2 ^ q - 1, 2 ^ M - 1, b` to +`base + 0 .. base + 5`. -/ +@[expose] def storeHeaderPart1 : Code .exact 18 := + .seq (.op (.store 0 1)) + (.seq (.op (.exactBin .add 7 0 6)) + (.seq (.op (.store 7 2)) + (.seq (.op (.exactBin .shiftLeft 8 6 1)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .shiftLeft 8 6 2)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .shiftLeft 8 6 3)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .add 7 7 6)) + (.op (.store 7 5))))))))))))))))) + +/-- Second 17-instruction header-store and loop-prep block: writes `2 ^ b - 1` and +`limbCountFromB b q` to `base + 6, base + 7` and sets registers `0..5` for `clmulTabLoop`. -/ +@[expose] def storeHeaderPart2 (k : ℕ) : Code .exact 18 := + .seq (.op (.exactBin .shiftLeft 8 6 5)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .sub 9 2 6)) + (.seq (.op (.exactBin .div 9 9 5)) + (.seq (.op (.exactBin .add 9 9 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 9)) + (.seq (.op (.exactBin .mul 2 4 5)) + (.seq (.op (.exactBin .shiftLeft 2 6 2)) + (.seq (.op (.copy 3 5)) + (.seq (.op (.copy 4 8)) + (.seq (.op (.copy 5 6)) + (.seq (.op (.const 1 0)) + (.seq (.op (.const 9 (8 + k))) + (.op (.exactBin .add 0 0 9))))))))))))))))) + +/-- When `0 < b` and `0 < q`, `(q - 1) / b + 1 = limbCountFromB b q`. -/ +private theorem div_pred_add_one_eq_limbCountFromB {b q : ℕ} (hb : 0 < b) (hq : 0 < q) : + (q - 1) / b + 1 = limbCountFromB b q := by + dsimp only [limbCountFromB] + have h : q + b - 1 = (q - 1) + 1 * b := by omega + rw [h, Nat.add_mul_div_right _ _ hb] + +/-- `storeHeaderPart1` executes deterministically in `17` steps. -/ +theorem storeHeaderPart1_exec {w base r q M b : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = r) (h2 : x.regs 2 = q) + (h3 : x.regs 3 = M) (h4 : x.regs 4 = 2) (h5 : x.regs 5 = b) (h6 : x.regs 6 = 1) + (hr_le : r ≤ q) (hM_le : M ≤ q) + (hfit_end : Fits w (base + 8)) (hfit_q : Fits w (2 ^ q)) : + ∃ y : Data 18, + Exec w storeHeaderPart1 x y 17 ∧ + y.regs 0 = base ∧ + y.regs 2 = q ∧ + y.regs 4 = 2 ∧ + y.regs 5 = b ∧ + y.regs 6 = 1 ∧ + y.regs 7 = base + 5 ∧ + y.mem base = r ∧ + y.mem (base + 1) = q ∧ + y.mem (base + 2) = 2 ^ r - 1 ∧ + y.mem (base + 3) = 2 ^ q - 1 ∧ + y.mem (base + 4) = 2 ^ M - 1 ∧ + y.mem (base + 5) = b ∧ + (∀ a, (a < base ∨ base + 6 ≤ a) → y.mem a = x.mem a) := by + have hpow_r : 2 ^ r ≤ 2 ^ q := Nat.pow_le_pow_right (by decide) hr_le + have hpow_M : 2 ^ M ≤ 2 ^ q := Nat.pow_le_pow_right (by decide) hM_le + have hpos_r : 1 ≤ 2 ^ r := Nat.one_le_two_pow + have hpos_q : 1 ≤ 2 ^ q := Nat.one_le_two_pow + have hpos_M : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hfit_pow_r : Fits w (2 ^ r) := hfit_q.of_le hpow_r + have hfit_mask_r : Fits w (2 ^ r - 1) := hfit_pow_r.of_le (by omega) + have hfit_mask_q : Fits w (2 ^ q - 1) := hfit_q.of_le (by omega) + have hfit_pow_M : Fits w (2 ^ M) := hfit_q.of_le hpow_M + have hfit_mask_M : Fits w (2 ^ M - 1) := hfit_pow_M.of_le (by omega) + have hfit_b1 : Fits w (base + 1) := hfit_end.of_le (by omega) + have hfit_b2 : Fits w (base + 2) := hfit_end.of_le (by omega) + have hfit_b3 : Fits w (base + 3) := hfit_end.of_le (by omega) + have hfit_b4 : Fits w (base + 4) := hfit_end.of_le (by omega) + have hfit_b5 : Fits w (base + 5) := hfit_end.of_le (by omega) + set s1 := x.write base r + have e1 : Exec w (.op (.store 0 1) : Code .exact 18) x s1 1 := by + simpa +decide [s1, h0, h1] using (Exec.store (B := .exact) (w := w) (a := 0) (s := 1) (x := x)) + set s2 := s1.set 7 (base + 1) + have e2 : Exec w (.op (.exactBin .add 7 0 6) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h0, h6]) hfit_b1 + set s3 := s2.write (base + 1) q + have e3 : Exec w (.op (.store 7 2) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, h2] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 2) (x := s2)) + set s4 := s3.set 8 (2 ^ r) + have e4 : Exec w (.op (.exactBin .shiftLeft 8 6 1) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, h6, h1, Nat.shiftLeft_eq]) hfit_pow_r + set s5 := s4.set 8 (2 ^ r - 1) + have e5 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h6, hpos_r]) hfit_mask_r + set s6 := s5.set 7 (base + 2) + have e6 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h6, Nat.add_assoc]) hfit_b2 + set s7 := s6.write (base + 2) (2 ^ r - 1) + have e7 : Exec w (.op (.store 7 8) : Code .exact 18) s6 s7 1 := by + simpa +decide [s7, s6, s5] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s6)) + set s8 := s7.set 8 (2 ^ q) + have e8 : Exec w (.op (.exactBin .shiftLeft 8 6 2) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, h6, h2, Nat.shiftLeft_eq]) hfit_q + set s9 := s8.set 8 (2 ^ q - 1) + have e9 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h6, hpos_q]) hfit_mask_q + set s10 := s9.set 7 (base + 3) + have e10 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, Nat.add_assoc]) hfit_b3 + set s11 := s10.write (base + 3) (2 ^ q - 1) + have e11 : Exec w (.op (.store 7 8) : Code .exact 18) s10 s11 1 := by + simpa +decide [s11, s10, s9] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s10)) + set s12 := s11.set 8 (2 ^ M) + have e12 : Exec w (.op (.exactBin .shiftLeft 8 6 3) : Code .exact 18) s11 s12 1 := + exec_exactBin + (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, h3, Nat.shiftLeft_eq]) + hfit_pow_M + set s13 := s12.set 8 (2 ^ M - 1) + have e13 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s12 s13 1 := + exec_exactBin (by simp +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, hpos_M]) + hfit_mask_M + set s14 := s13.set 7 (base + 4) + have e14 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s13 s14 1 := + exec_exactBin + (by simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, Nat.add_assoc]) + hfit_b4 + set s15 := s14.write (base + 4) (2 ^ M - 1) + have e15 : Exec w (.op (.store 7 8) : Code .exact 18) s14 s15 1 := by + simpa +decide [s15, s14, s13] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s14)) + set s16 := s15.set 7 (base + 5) + have e16 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s15 s16 1 := + exec_exactBin + (by simp +decide [s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, + Nat.add_assoc]) + hfit_b5 + set s17 := s16.write (base + 5) b + have e17 : Exec w (.op (.store 7 5) : Code .exact 18) s16 s17 1 := by + simpa +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] + using (Exec.store (B := .exact) (w := w) (a := 7) (s := 5) (x := s16)) + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq (e16.seq e17))))))))))))))) + refine ⟨s17, hall, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h2] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h4] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6] + · simp +decide [s17, s16] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · intro a ha + dsimp only [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + Data.write, Data.set] + rw [Function.update_of_ne (by omega), Function.update_of_ne (by omega), + Function.update_of_ne (by omega), Function.update_of_ne (by omega), + Function.update_of_ne (by omega), Function.update_of_ne (by omega)] + +/-- `storeHeaderPart2 k` executes deterministically in `17` steps. -/ +theorem storeHeaderPart2_exec {w base q b k : ℕ} {x : Data 18} + (hb : 0 < b) (hq : 0 < q) + (h0 : x.regs 0 = base) (h2 : x.regs 2 = q) (h4 : x.regs 4 = 2) + (h5 : x.regs 5 = b) (h6 : x.regs 6 = 1) (h7 : x.regs 7 = base + 5) + (hfit_space : Fits w (base + spaceFromB b k)) (hfit_q : Fits w q) : + ∃ y : Data 18, + Exec w (storeHeaderPart2 k) x y 17 ∧ + y.regs 0 = base + 8 + k ∧ + y.regs 1 = 0 ∧ + y.regs 2 = 2 ^ (2 * b) ∧ + y.regs 3 = b ∧ + y.regs 4 = 2 ^ b - 1 ∧ + y.regs 5 = 1 ∧ + y.mem (base + 6) = 2 ^ b - 1 ∧ + y.mem (base + 7) = limbCountFromB b q ∧ + (∀ a, (a < base + 6 ∨ base + 8 ≤ a) → y.mem a = x.mem a) := by + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hpos_b : 1 ≤ 2 ^ b := Nat.one_le_two_pow + have h2b_lt : 2 * b < 2 ^ (2 * b) := Nat.lt_two_pow_self + have hfit_pow2b : Fits w (2 ^ (2 * b)) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_powb : Fits w (2 ^ b) := hfit_pow2b.of_le hpow_le + have hfit_maskb : Fits w (2 ^ b - 1) := hfit_powb.of_le (by omega) + have hfit_b6 : Fits w (base + 6) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b7 : Fits w (base + 7) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_qm1 : Fits w (q - 1) := hfit_q.of_le (by omega) + have hfit_div : Fits w ((q - 1) / b) := hfit_qm1.of_le (Nat.div_le_self _ _) + have hdiv1_le : (q - 1) / b + 1 ≤ q := by + have := Nat.div_le_self (q - 1) b + omega + have hfit_L : Fits w (limbCountFromB b q) := by + rw [← div_pred_add_one_eq_limbCountFromB hb hq] + exact hfit_q.of_le hdiv1_le + have hfit_2b : Fits w (2 * b) := hfit_pow2b.of_le (le_of_lt h2b_lt) + have hfit_0 : Fits w 0 := hfit_space.of_le (Nat.zero_le _) + have hfit_8k : Fits w (8 + k) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_tabBase : Fits w (base + 8 + k) := hfit_space.of_le (by simp [spaceFromB]; omega) + set s1 := x.set 8 (2 ^ b) + have e1 : Exec w (.op (.exactBin .shiftLeft 8 6 5) : Code .exact 18) x s1 1 := + exec_exactBin (by simp +decide [h6, h5, Nat.shiftLeft_eq]) hfit_powb + set s2 := s1.set 8 (2 ^ b - 1) + have e2 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h6, hpos_b]) hfit_maskb + set s3 := s2.set 7 (base + 6) + have e3 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h7, h6, Nat.add_assoc]) hfit_b6 + set s4 := s3.write (base + 6) (2 ^ b - 1) + have e4 : Exec w (.op (.store 7 8) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, s2] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s3)) + set s5 := s4.set 9 (q - 1) + have e5 : Exec w (.op (.exactBin .sub 9 2 6) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h2, h6]; omega) hfit_qm1 + set s6 := s5.set 9 ((q - 1) / b) + have e6 : Exec w (.op (.exactBin .div 9 9 5) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h5, hb.ne']) hfit_div + set s7 := s6.set 9 (limbCountFromB b q) + have e7 : Exec w (.op (.exactBin .add 9 9 6) : Code .exact 18) s6 s7 1 := + exec_exactBin + (by simp +decide [s6, s5, s4, s3, s2, s1, h6, div_pred_add_one_eq_limbCountFromB hb hq]) + hfit_L + set s8 := s7.set 7 (base + 7) + have e8 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, h6, Nat.add_assoc]) hfit_b7 + set s9 := s8.write (base + 7) (limbCountFromB b q) + have e9 : Exec w (.op (.store 7 9) : Code .exact 18) s8 s9 1 := by + simpa +decide [s9, s8, s7] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 9) (x := s8)) + set s10 := s9.set 2 (2 * b) + have e10 : Exec w (.op (.exactBin .mul 2 4 5) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h4, h5]) hfit_2b + set s11 := s10.set 2 (2 ^ (2 * b)) + have e11 : Exec w (.op (.exactBin .shiftLeft 2 6 2) : Code .exact 18) s10 s11 1 := + exec_exactBin + (by simp +decide [s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, Nat.shiftLeft_eq]) + hfit_pow2b + set s12 := s11.set 3 b + have e12 : Exec w (.op (.copy 3 5) : Code .exact 18) s11 s12 1 := by + simpa +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] using + (Exec.copy (B := .exact) (w := w) (d := 3) (s := 5) (x := s11)) + set s13 := s12.set 4 (2 ^ b - 1) + have e13 : Exec w (.op (.copy 4 8) : Code .exact 18) s12 s13 1 := by + simpa +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2] using + (Exec.copy (B := .exact) (w := w) (d := 4) (s := 8) (x := s12)) + set s14 := s13.set 5 1 + have e14 : Exec w (.op (.copy 5 6) : Code .exact 18) s13 s14 1 := by + simpa +decide [s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6] using + (Exec.copy (B := .exact) (w := w) (d := 5) (s := 6) (x := s13)) + set s15 := s14.set 1 0 + have e15 : Exec w (.op (.const 1 0) : Code .exact 18) s14 s15 1 := Exec.const hfit_0 + set s16 := s15.set 9 (8 + k) + have e16 : Exec w (.op (.const 9 (8 + k)) : Code .exact 18) s15 s16 1 := Exec.const hfit_8k + set s17 := s16.set 0 (base + 8 + k) + have e17 : Exec w (.op (.exactBin .add 0 0 9) : Code .exact 18) s16 s17 1 := + exec_exactBin + (by simp +decide [s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0, + Nat.add_assoc]) + hfit_tabBase + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq (e16.seq e17))))))))))))))) + refine ⟨s17, hall, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simp +decide [s17] + · simp +decide [s17, s16, s15] + · simp +decide [s17, s16, s15, s14, s13, s12, s11] + · simp +decide [s17, s16, s15, s14, s13, s12] + · simp +decide [s17, s16, s15, s14, s13] + · simp +decide [s17, s16, s15, s14] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9] + · intro a ha + dsimp only [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + Data.write, Data.set] + rw [Function.update_of_ne (by omega), Function.update_of_ne (by omega)] + +/-- 2-instruction suffix restoring `regBase = base` from `base + 8 + k`. -/ +@[expose] def restoreBaseCode (k : ℕ) : Code .exact 18 := + .seq (.op (.const 9 (8 + k))) + (.op (.exactBin .sub 0 0 9)) + +/-- Complete deterministic table initialization program `initTableCode k`: computes +`(r, q, b, L)` and masks from `(base, n_max, U, M)` in registers `0..3` and populates +`clmulTab[0 .. 2 ^ (2 * b) - 1]`. -/ +@[expose] def initTableCode (k : ℕ) : Code .exact 18 := + .seq chunkBitsCode + (.seq trinomialParamsCode + (.seq storeHeaderPart1 + (.seq (storeHeaderPart2 k) + (.seq clmulTabLoop + (restoreBaseCode k))))) + +/-- Exact step count of `initTableCode k`. -/ +@[expose] def initTableSteps (n_max U M : ℕ) : ℕ := + 19 * 2 ^ (2 * chunkBits n_max) + + 4 * Nat.log 2 (max n_max 2) + + 5 * Nat.clog 2 (max U 2) + + 5 * trinomialStep U M + 69 + +/-- **`KWise.initTableCode k` correctness and complexity**: from `reg 0 = base`, `reg 1 = n_max`, +`reg 2 = U`, `reg 3 = M`, `initTableCode k` halts deterministically in +`initTableSteps n_max U M = O(n_max + log U + log M)` instructions, establishing +`TableRep base (chunkBits n_max) U M k y.mem`, preserving `reg 0 = base`, and modifying memory +only within `[base, base + space n_max k)`. -/ +theorem initTableCode_execWithin {w base n_max U M k : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = n_max) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (hfit_space : Fits w (base + space n_max k)) + (hfit_n : Fits w (max n_max 2)) (hfit_U : Fits w (max U 2 + 1)) + (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ∃ y : Data 18, + ExecWithin w (initTableCode k) x y (initTableSteps n_max U M) ∧ + TableRep base (chunkBits n_max) U M k y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base ∨ base + space n_max k ≤ a) → y.mem a = x.mem a) := by + set b := chunkBits n_max + set r := 3 ^ trinomialStep U M + set q := trinomialDeg U M + have hb_pos : 0 < b := chunkBits_pos n_max + have hq_pos : 0 < q := by + dsimp only [q, trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + omega + have hr_le : r ≤ q := by dsimp only [q, trinomialDeg, r]; omega + have hM_le : M ≤ q := le_trinomialDeg U M + have hsp : space n_max k = 8 + k + 2 ^ (2 * b) := rfl + have hpow_pos : 0 < 2 ^ (2 * b) := by positivity + have hfit_3 : Fits w 3 := hfit_space.of_le (by omega) + have hfit_end8 : Fits w (base + 8) := hfit_space.of_le (by omega) + have hfit_qval : Fits w q := hfit_q.of_le (le_of_lt Nat.lt_two_pow_self) + have hfit_tabEnd : Fits w (base + 8 + k + 2 ^ (2 * b)) := hfit_space.of_le (by omega) + have hfit_8k : Fits w (8 + k) := hfit_space.of_le (by omega) + have hfit_base : Fits w base := hfit_space.of_le (by omega) + obtain ⟨d1, e1, hd1_0, hd1_2, hd1_3, hd1_5, hd1_6, hd1_mem⟩ := + chunkBitsCode_exec h0 h1 h2 h3 hfit_n hfit_3 + obtain ⟨d2, e2, hd2_0, hd2_1, hd2_2, hd2_3, hd2_4, hd2_5, hd2_6, hd2_mem⟩ := + trinomialParamsCode_exec hd1_0 hd1_2 hd1_3 hd1_5 hd1_6 hfit_U hfit_q + obtain ⟨d3, e3, hd3_0, hd3_2, hd3_4, hd3_5, hd3_6, hd3_7, + hm0, hm1, hm2, hm3, hm4, hm5, hd3_out⟩ := + storeHeaderPart1_exec hd2_0 hd2_1 hd2_2 hd2_3 hd2_4 hd2_5 hd2_6 hr_le hM_le + hfit_end8 hfit_q + obtain ⟨d4, e4, hd4_0, hd4_1, hd4_2, hd4_3, hd4_4, hd4_5, hm6, hm7, hd4_out⟩ := + storeHeaderPart2_exec hb_pos hq_pos hd3_0 hd3_2 hd3_4 hd3_5 hd3_6 hd3_7 hfit_space hfit_qval + obtain ⟨d5, e5, hd5_0, htab, hd5_out⟩ := + clmulTabLoop_exec hfit_tabEnd (2 ^ (2 * b)) 0 d4 (by omega) + hd4_0 hd4_1 hd4_2 hd4_3 hd4_4 hd4_5 (by intro i hi; omega) + set d6 := d5.set 9 (8 + k) + have er1 : Exec w (.op (.const 9 (8 + k)) : Code .exact 18) d5 d6 1 := Exec.const hfit_8k + set d7 := d6.set 0 base + have er2 : Exec w (.op (.exactBin .sub 0 0 9) : Code .exact 18) d6 d7 1 := + exec_exactBin (by simp +decide [d6, hd5_0]; omega) hfit_base + have e6 := er1.seq er2 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq e6)))) + have hmem_hdr : ∀ j < 8, d7.mem (base + j) = d4.mem (base + j) := by + intro j hj + exact hd5_out (base + j) (Or.inl (by omega)) + refine ⟨d7, ⟨_, ?_, hall⟩, ?_, by simp +decide [d7], ?_⟩ + · dsimp only [initTableSteps, b] + omega + · refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · have := hmem_hdr 0 (by decide) + rw [Nat.add_zero] at this + rw [this, hd4_out base (Or.inl (by omega)), hm0] + · rw [hmem_hdr 1 (by decide), hd4_out (base + 1) (Or.inl (by omega)), hm1] + · rw [hmem_hdr 2 (by decide), hd4_out (base + 2) (Or.inl (by omega)), hm2] + · rw [hmem_hdr 3 (by decide), hd4_out (base + 3) (Or.inl (by omega)), hm3] + · rw [hmem_hdr 4 (by decide), hd4_out (base + 4) (Or.inl (by omega)), hm4] + · rw [hmem_hdr 5 (by decide), hd4_out (base + 5) (Or.inl (by omega)), hm5] + · rw [hmem_hdr 6 (by decide), hm6] + · rw [hmem_hdr 7 (by decide), hm7] + · intro idx hidx + exact htab idx hidx + · intro a ha + have ha_eq : a < base ∨ base + (8 + k + 2 ^ (2 * b)) ≤ a := by + simpa only [space, spaceFromB, b] using ha + have ha_d5 : a < base + 8 + k + 0 ∨ base + 8 + k + 2 ^ (2 * b) ≤ a := by omega + have ha_d4 : a < base + 6 ∨ base + 8 ≤ a := by omega + have ha_d3 : a < base ∨ base + 6 ≤ a := by omega + calc d7.mem a = d5.mem a := rfl + _ = d4.mem a := hd5_out a ha_d5 + _ = d3.mem a := hd4_out a ha_d4 + _ = d2.mem a := hd3_out a ha_d3 + _ = x.mem a := by rw [hd2_mem, hd1_mem] + +/-- Setup prefix for `resampleCode k`: loads `2 ^ q - 1` from `base + 3` into register `1` and +initializes `ptr = base + 8` (`reg 2`), `end = base + 8 + k` (`reg 3`), and `one = 1` (`reg 4`). -/ +@[expose] def resampleSetupCode (k : ℕ) : Code .exact 18 := + .seq (.op (.const 1 3)) + (.seq (.op (.exactBin .add 1 0 1)) + (.seq (.op (.load 1 1)) + (.seq (.op (.const 2 8)) + (.seq (.op (.exactBin .add 2 0 2)) + (.seq (.op (.const 3 k)) + (.seq (.op (.exactBin .add 3 2 3)) + (.op (.const 4 1)))))))) + +/-- Closed-form state after `resampleSetupCode k`. -/ +@[expose] def resampleSetupData (base U M k : ℕ) (x : Data 18) : Data 18 := + ((((x.set 1 (2 ^ trinomialDeg U M - 1)).set 2 (base + 8)).set 3 (base + 8 + k)).set 4 1) + +/-- Injective register map embedding `randFillLoop` into `Data 18` so `regExtra` (`5`) maps to +`regBase` (`0`). -/ +@[expose] def resampleMap : Fin 6 → Fin 18 := + ![2, 3, 1, 4, 5, 0] + +theorem injective_resampleMap : Function.Injective resampleMap := by + decide + +/-- Seed-resampling program `resampleCode k`: samples the `k` polynomial coefficients +`seed[0 .. k - 1]` uniformly in `Fin (2 ^ trinomialDeg U M)` using `k` `Code.rand` calls in +`5 * k + 10` instructions while preserving `TableRep` and `regBase = base`. -/ +@[expose] def resampleCode (k : ℕ) : Code .exact 18 := + .seq (resampleSetupCode k) (randFillLoop.rename resampleMap) + +/-- Exact `ProbComp` semantics of `resampleCode k`. -/ +@[expose] noncomputable def resampleComp (base U M k : ℕ) (x : Data 18) : ProbComp (Data 18 × ℕ) := + let d0 := resampleSetupData base U M k x + (fun z : Data 6 × ℕ => (z.1.extend resampleMap d0, 8 + z.2)) <$> + randFillComp (trinomialDeg U M) (base + 8) k (d0.comap resampleMap) + +/-- `resampleSetupCode k` executes deterministically in `8` steps. -/ +theorem resampleSetupCode_exec {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) : + Exec w (resampleSetupCode k) x (resampleSetupData base U M k x) 8 := by + have hfit_3 : Fits w 3 := hfit_end.of_le (by omega) + have hfit_b3 : Fits w (base + 3) := hfit_end.of_le (by omega) + have hfit_8 : Fits w 8 := hfit_end.of_le (by omega) + have hfit_b8 : Fits w (base + 8) := hfit_end.of_le (by omega) + have hfit_k : Fits w k := hfit_end.of_le (by omega) + have hfit_1 : Fits w 1 := hfit_end.of_le (by omega) + have hm3 : x.mem (base + 3) = 2 ^ trinomialDeg U M - 1 := htab.2.2.2.1 + set s1 := x.set 1 3 + have e1 : Exec w (.op (.const 1 3) : Code .exact 18) x s1 1 := Exec.const hfit_3 + set s2 := s1.set 1 (base + 3) + have e2 : Exec w (.op (.exactBin .add 1 0 1) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h0]) hfit_b3 + set s3 := s2.set 1 (2 ^ trinomialDeg U M - 1) + have e3 : Exec w (.op (.load 1 1) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, hm3] using + (Exec.load (B := .exact) (w := w) (d := 1) (a := 1) (x := s2)) + set s4 := s3.set 2 8 + have e4 : Exec w (.op (.const 2 8) : Code .exact 18) s3 s4 1 := Exec.const hfit_8 + set s5 := s4.set 2 (base + 8) + have e5 : Exec w (.op (.exactBin .add 2 0 2) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h0]) hfit_b8 + set s6 := s5.set 3 k + have e6 : Exec w (.op (.const 3 k) : Code .exact 18) s5 s6 1 := Exec.const hfit_k + set s7 := s6.set 3 (base + 8 + k) + have e7 : Exec w (.op (.exactBin .add 3 2 3) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, Nat.add_assoc]) hfit_end + set s8 := s7.set 4 1 + have e8 : Exec w (.op (.const 4 1) : Code .exact 18) s7 s8 1 := Exec.const hfit_1 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq e8)))))) + have hd : s8 = resampleSetupData base U M k x := by + ext i : 2 + · fin_cases i <;> rfl + · rfl + rwa [hd] at hall + +/-- **`KWise.resampleCode k` exact randomized execution**: executes with exact `ProbComp` +semantics `resampleComp base U M k x`. -/ +theorem resampleCode_execRand {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ExecRand w (resampleCode k) x (resampleComp base U M k x) := by + have hsetup := resampleSetupCode_exec htab h0 hfit_end + set d0 := resampleSetupData base U M k x + have hfit_mask : Fits w (2 ^ trinomialDeg U M - 1) := hfit_q.of_le (Nat.sub_le _ 1) + have hloop0 := randFillLoop_execRand (w := w) (bits := trinomialDeg U M) (arrBase := base + 8) + k (x := d0.comap resampleMap) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regPtr]) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regEnd]) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regMask]) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regOne]) + hfit_end hfit_mask + have hloop := hloop0.rename injective_resampleMap + have hseq := hsetup.seqRand hloop + simpa only [resampleCode, resampleComp, d0, Functor.map_map] using hseq + +/-- **Uniformity of resampled seeds**: under `(resampleComp base U M k x).eval`, the extracted seed +tuple `extractSeeds base U M k z.1.mem` is uniformly distributed over +`Fin k → Fin (2 ^ trinomialDeg U M)`. -/ +theorem map_extractSeeds_eval_resampleComp (base U M k : ℕ) (x : Data 18) : + ((resampleComp base U M k x).eval).map (fun z => extractSeeds base U M k z.1.mem) = + PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M)) := by + dsimp only [resampleComp, extractSeeds] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractSlice_eval_randFillComp (trinomialDeg U M) (base + 8) k _ + +/-- **`KWise.resampleCode k` Hoare-style specification**: halts in `5 * k + 10 = O(k)` steps and `k` +`rand` queries, preserving `TableRep base b U M k`, establishing +`Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem`, preserving `reg 0 = base`, and +modifying memory only within `[base + 8, base + 8 + k)`. -/ +theorem resampleCode_execRandWithin {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ExecRandWithin w (resampleCode k) x + (fun y => + Rep base b U M k (extractSeeds base U M k y.mem) y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → y.mem a = x.mem a)) + (5 * k + 10) k := by + refine ⟨resampleComp base U M k x, resampleCode_execRand htab h0 hfit_end hfit_q, ?_, ?_⟩ + · dsimp only [resampleComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp (trinomialDeg U M) (base + 8) k _) + · intro z hz + dsimp only [resampleComp] at hz + obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hout⟩ := mem_support_randFillComp hz0 + have htab' : TableRep base b U M k z0.1.mem := by + refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · rw [hout base (Or.inl (by omega))]; exact htab.1 + · rw [hout (base + 1) (Or.inl (by omega))]; exact htab.2.1 + · rw [hout (base + 2) (Or.inl (by omega))]; exact htab.2.2.1 + · rw [hout (base + 3) (Or.inl (by omega))]; exact htab.2.2.2.1 + · rw [hout (base + 4) (Or.inl (by omega))]; exact htab.2.2.2.2.1 + · rw [hout (base + 5) (Or.inl (by omega))]; exact htab.2.2.2.2.2.1 + · rw [hout (base + 6) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.1 + · rw [hout (base + 7) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.2.1 + · intro idx hidx + rw [hout (base + 8 + k + idx) (Or.inr (by omega))] + exact htab.2.2.2.2.2.2.2.2 idx hidx + have hseed : SeedRep base U M k (extractSeeds base U M k z0.1.mem) z0.1.mem := by + intro j + simp [extractSeeds, extractSlice, hlt j.val j.isLt] + have hbase' : (z0.1.extend resampleMap (resampleSetupData base U M k x)).regs 0 = base := by + change ((z0.1.extend resampleMap (resampleSetupData base U M k x)).comap + resampleMap).regs regExtra = base + rw [Data.comap_extend injective_resampleMap, hextra] + simpa +decide [resampleSetupData, resampleMap, Data.comap, regExtra] using h0 + exact ⟨⟨⟨htab', hseed⟩, hbase', hout⟩, by omega⟩ + +/-- Full one-shot initialization program `initCode k := .seq (initTableCode k) (resampleCode k)`. -/ +@[expose] def initCode (k : ℕ) : Code .exact 18 := + .seq (initTableCode k) (resampleCode k) + +/-- **`KWise.initCode k` Hoare-style specification**: runs `initTableCode k` followed by +`resampleCode k` in `initTableSteps n_max U M + (5 * k + 10)` steps and `k` `rand` calls. -/ +theorem initCode_execRandWithin {w base n_max U M k : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = n_max) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (hfit_space : Fits w (base + space n_max k)) + (hfit_n : Fits w (max n_max 2)) (hfit_U : Fits w (max U 2 + 1)) + (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ExecRandWithin w (initCode k) x + (fun y => + Rep base (chunkBits n_max) U M k (extractSeeds base U M k y.mem) y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base ∨ base + space n_max k ≤ a) → y.mem a = x.mem a)) + (initTableSteps n_max U M + (5 * k + 10)) k := by + obtain ⟨dTab, ⟨s1, hs1, he1⟩, htab, hdTab_0, hdTab_out⟩ := + initTableCode_execWithin h0 h1 h2 h3 hfit_space hfit_n hfit_U hfit_q + have hsp : space n_max k = 8 + k + 2 ^ (2 * chunkBits n_max) := rfl + have hpow_pos : 0 < 2 ^ (2 * chunkBits n_max) := by positivity + have hfit_end : Fits w (base + 8 + k) := hfit_space.of_le (by omega) + obtain ⟨m2, he2, hqb, hsupp⟩ := resampleCode_execRandWithin htab hdTab_0 hfit_end hfit_q + refine ⟨(fun z : Data 18 × ℕ => (z.1, s1 + z.2)) <$> m2, he1.seqRand he2, + (OracleComp.isCostBounded_map_iff _ _ _).mpr hqb, ?_⟩ + intro z hz + obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨⟨hrep, hz0_0, hz0_out⟩, hz0_steps⟩ := hsupp z0 hz0 + refine ⟨⟨hrep, hz0_0, ?_⟩, by omega⟩ + intro a ha + have ha' : a < base + 8 ∨ base + 8 + k ≤ a := by omega + rw [hz0_out a ha', hdTab_out a ha] + +/-! ## Section 4: Deterministic Horner evaluation (`evalCode`) and `k`-wise independence -/ + +/-- Register `1`: holds the input key `u < U` on entry to `evalCode`. -/ +abbrev regKey : Fin 18 := 1 + +/-- Register `2`: holds the output hash value `< 2 ^ M` on exit from `evalCode`. -/ +abbrev regOut : Fin 18 := 2 + +private theorem and_xor_shiftRight_shiftLeft (a b : ℕ) : + (a &&& (2 ^ b - 1)) ^^^ ((a >>> b) <<< b) = a := by + apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_xor, Nat.testBit_and, Nat.testBit_two_pow_sub_one, + Nat.testBit_shiftLeft, Nat.testBit_shiftRight] + by_cases h : i < b + · have hge : ¬ b ≤ i := not_le.mpr h + simp [h, hge] + · have hge : b ≤ i := le_of_not_gt h + simp [h, hge, Nat.add_sub_of_le hge] + +private theorem clmul_split_right (x y b : ℕ) : + clmul x y = clmul x (y &&& (2 ^ b - 1)) ^^^ (clmul x (y >>> b) <<< b) := by + conv_lhs => rw [← and_xor_shiftRight_shiftLeft y b] + rw [clmul_xor_right, clmul_shiftLeft_right] + +private theorem clmul_split_left (x y b : ℕ) : + clmul x y = clmul (x &&& (2 ^ b - 1)) y ^^^ (clmul (x >>> b) y <<< b) := by + conv_lhs => rw [← and_xor_shiftRight_shiftLeft x b] + rw [clmul_xor_left, clmul_shiftLeft_left] + +private theorem shiftLeft_add_eq (a m n : ℕ) : + a <<< (m + n) = (a <<< m) <<< n := by + rw [Nat.shiftLeft_eq, Nat.shiftLeft_eq, Nat.shiftLeft_eq, pow_add, ← mul_assoc] + +private theorem xor_shiftLeft_eq (a b k : ℕ) : + (a ^^^ b) <<< k = (a <<< k) ^^^ (b <<< k) := by + apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_shiftLeft, Nat.testBit_xor, Nat.testBit_xor, + Nat.testBit_shiftLeft, Nat.testBit_shiftLeft] + by_cases h : i ≥ k <;> simp [h] + +private theorem shiftLeft_xor_parts {u v b : ℕ} (hu : u < 2 ^ b) (hv : v < 2 ^ b) : + (u <<< b) ^^^ v < 2 ^ (2 * b) ∧ + ((u <<< b) ^^^ v) >>> b = u ∧ + ((u <<< b) ^^^ v) &&& (2 ^ b - 1) = v := by + have hush : u <<< b < 2 ^ (2 * b) := by + rw [Nat.shiftLeft_eq, two_mul, pow_add] + exact Nat.mul_lt_mul_of_pos_right hu (by positivity) + have hv2 : v < 2 ^ (2 * b) := + lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) (by omega)) + refine ⟨Nat.xor_lt_two_pow hush hv2, ?_, ?_⟩ + · apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_shiftRight, Nat.testBit_xor, Nat.testBit_shiftLeft] + have hge : b + i ≥ b := Nat.le_add_right b i + have hsub : b + i - b = i := Nat.add_sub_cancel_left b i + have hv_false : v.testBit (b + i) = false := + Nat.testBit_lt_two_pow (lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) hge)) + simp [hge, hsub, hv_false] + · apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_and, Nat.testBit_two_pow_sub_one, Nat.testBit_xor, Nat.testBit_shiftLeft] + by_cases hi : i < b + · have hge : ¬ i ≥ b := not_le.mpr hi + simp [hi, hge] + · have hge : b ≤ i := le_of_not_gt hi + have hv_false : v.testBit i = false := + Nat.testBit_lt_two_pow (lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) hge)) + simp [hi, hv_false] + +private theorem shiftRight_mul_two_pow_add_le (a b s : ℕ) : + (a >>> b) * 2 ^ (s + b) ≤ a * 2 ^ s := by + rw [Nat.shiftRight_eq_div_pow, pow_add, ← mul_assoc, mul_comm (a / 2 ^ b) (2 ^ s), mul_assoc, + mul_comm a (2 ^ s)] + exact Nat.mul_le_mul_left (2 ^ s) (Nat.div_mul_le_self a (2 ^ b)) + +private theorem shiftRight_lt_two_pow_mul {a rem b : ℕ} (ha : a < 2 ^ ((rem + 1) * b)) : + a >>> b < 2 ^ (rem * b) := by + rw [Nat.shiftRight_eq_div_pow] + apply Nat.div_lt_of_lt_mul + rwa [← pow_add, show b + rem * b = (rem + 1) * b by ring] + +private theorem lt_two_pow_sub_of_mul_two_pow_lt {c s q : ℕ} (hc : 0 < c) + (h : c * 2 ^ s < 2 ^ q) : s ≤ q ∧ c < 2 ^ (q - s) := by + have hs : s ≤ q := by + by_contra hlt + have hle : 2 ^ q ≤ c * 2 ^ s := + calc 2 ^ q ≤ 2 ^ s := Nat.pow_le_pow_right (by decide) (by omega) + _ ≤ c * 2 ^ s := Nat.le_mul_of_pos_left _ hc + omega + refine ⟨hs, ?_⟩ + apply Nat.lt_of_mul_lt_mul_right (a := 2 ^ s) + rwa [← pow_add, Nat.sub_add_cancel hs] + +private theorem clmul_and_shiftLeft_lt {x_rem y_rem sh_x sh_y b q : ℕ} + (hx : x_rem * 2 ^ sh_x < 2 ^ q) (hy : y_rem * 2 ^ sh_y < 2 ^ q) : + clmul (x_rem &&& (2 ^ b - 1)) (y_rem &&& (2 ^ b - 1)) <<< (sh_x + sh_y) < 2 ^ (2 * q) := by + set xc := x_rem &&& (2 ^ b - 1) + set yc := y_rem &&& (2 ^ b - 1) + rcases eq_or_ne xc 0 with hxc | hxc + · simp [hxc] + · rcases eq_or_ne yc 0 with hyc | hyc + · simp [hyc] + · have hxc_lt : xc * 2 ^ sh_x < 2 ^ q := + (Nat.mul_le_mul_right (2 ^ sh_x) Nat.and_le_left).trans_lt hx + have hyc_lt : yc * 2 ^ sh_y < 2 ^ q := + (Nat.mul_le_mul_right (2 ^ sh_y) Nat.and_le_left).trans_lt hy + obtain ⟨hsx, hxc'⟩ := lt_two_pow_sub_of_mul_two_pow_lt (Nat.pos_of_ne_zero hxc) hxc_lt + obtain ⟨hsy, hyc'⟩ := lt_two_pow_sub_of_mul_two_pow_lt (Nat.pos_of_ne_zero hyc) hyc_lt + have hcl := clmul_lt_two_pow hxc' hyc' + rw [Nat.shiftLeft_eq] + calc clmul xc yc * 2 ^ (sh_x + sh_y) + < 2 ^ (q - sh_x + (q - sh_y)) * 2 ^ (sh_x + sh_y) := + Nat.mul_lt_mul_of_pos_right hcl (by positivity) + _ = 2 ^ (2 * q) := by + rw [← pow_add] + congr 1 + omega + +private theorem fits_of_lt_two_pow_two_mul {w q v : ℕ} (hw : 2 * q ≤ w) + (hv : v < 2 ^ (2 * q)) : Fits w v := + lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) hw) + +/-- Body of `innerClmulLoop`: looks up `clmul (x_rem &&& mask_b) (y_rem &&& mask_b)` in `clmulTab`, +shifts left by `shift_j`, XORs into `acc` (`reg 2`), shifts `y_rem` right by `b`, advances +`shift_j` by `b`, and decrements `rem_j`. -/ +@[expose] def innerClmulBody : Code .exact 18 := + .seq (.op (.exactBin .and 17 15 5)) + (.seq (.op (.exactBin .xor 17 13 17)) + (.seq (.op (.exactBin .add 17 7 17)) + (.seq (.op (.load 17 17)) + (.seq (.op (.exactBin .shiftLeft 17 17 16)) + (.seq (.op (.exactBin .xor 2 2 17)) + (.seq (.op (.exactBin .shiftRight 15 15 4)) + (.seq (.op (.exactBin .add 16 16 4)) + (.op (.exactBin .sub 14 14 8))))))))) + +/-- Inner Four-Russians loop `while 0 < rem_j do innerClmulBody`. -/ +@[expose] def innerClmulLoop : Code .exact 18 := + .loop (.lt 9 14) innerClmulBody + +/-- `innerClmulLoop` executes deterministically in `11 * rem_j + 2` steps, XORing +`clmul (x_rem &&& (2 ^ b - 1)) y_rem <<< (sh_x + sh_y)` into `reg 2` (`acc`). -/ +theorem innerClmulLoop_exec {w tabBase b q x_rem sh_x : ℕ} + (hb : 0 < b) (hw : 2 * q ≤ w) + (hfit_tabEnd : Fits w (tabBase + 2 ^ (2 * b))) + (hx_bound : x_rem * 2 ^ sh_x < 2 ^ q) : + ∀ rem_j y_rem sh_y acc : ℕ, ∀ s : Data 18, + y_rem < 2 ^ (rem_j * b) → + y_rem * 2 ^ sh_y < 2 ^ q → + acc < 2 ^ (2 * q) → + Fits w (sh_x + sh_y + rem_j * b) → + s.regs 2 = acc → + s.regs 4 = b → + s.regs 5 = 2 ^ b - 1 → + s.regs 7 = tabBase → + s.regs 8 = 1 → + s.regs 9 = 0 → + s.regs 13 = (x_rem &&& (2 ^ b - 1)) <<< b → + s.regs 14 = rem_j → + s.regs 15 = y_rem → + s.regs 16 = sh_x + sh_y → + (∀ i < 2 ^ (2 * b), s.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) → + ∃ y : Data 18, + Exec w innerClmulLoop s y (11 * rem_j + 2) ∧ + y.regs 2 = acc ^^^ (clmul (x_rem &&& (2 ^ b - 1)) y_rem <<< (sh_x + sh_y)) ∧ + (∀ r : Fin 18, r ≠ 2 ∧ r ≠ 14 ∧ r ≠ 15 ∧ r ≠ 16 ∧ r ≠ 17 → y.regs r = s.regs r) ∧ + y.mem = s.mem := by + intro rem_j + induction rem_j with + | zero => + intro y_rem sh_y acc s hy_lt _ _ _ h2 _ _ _ _ h9 _ h14 _ _ _ + have hy0 : y_rem = 0 := by + rw [Nat.zero_mul, pow_zero] at hy_lt + omega + have hc : ¬(Cond.lt (9 : Fin 18) 14).Holds s := by + simp [Cond.Holds, h9, h14] + refine ⟨s, Exec.loop_exit hc, ?_, fun _ _ => rfl, rfl⟩ + simp [h2, hy0] + | succ rem_j ih => + intro y_rem sh_y acc s hy_lt hy_bound hacc_lt hfit_sh h2 h4 h5 h7 h8 h9 h13 h14 h15 h16 htab + have hc : (Cond.lt (9 : Fin 18) 14).Holds s := by + simp [Cond.Holds, h9, h14] + set xc := x_rem &&& (2 ^ b - 1) + set yc := y_rem &&& (2 ^ b - 1) + set idx := (xc <<< b) ^^^ yc + have hxc_lt : xc < 2 ^ b := by + dsimp only [xc] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hyc_lt : yc < 2 ^ b := by + dsimp only [yc] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + obtain ⟨hidx_lt, hidx_u, hidx_v⟩ := shiftLeft_xor_parts hxc_lt hyc_lt + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hyc_lt2 : yc < 2 ^ (2 * b) := lt_of_lt_of_le hyc_lt hpow_le + have hidx_lt' : idx < 2 ^ (2 * b) := hidx_lt + have hfit_yc : Fits w yc := hfit_tabEnd.of_le (by omega) + have hfit_idx : Fits w idx := hfit_tabEnd.of_le (by omega) + have hfit_addr : Fits w (tabBase + idx) := hfit_tabEnd.of_le (by omega) + have hval_sh_lt : clmul xc yc <<< (sh_x + sh_y) < 2 ^ (2 * q) := + clmul_and_shiftLeft_lt hx_bound hy_bound + have hfit_val_sh : Fits w (clmul xc yc <<< (sh_x + sh_y)) := + fits_of_lt_two_pow_two_mul hw hval_sh_lt + have hacc'_lt : acc ^^^ (clmul xc yc <<< (sh_x + sh_y)) < 2 ^ (2 * q) := + Nat.xor_lt_two_pow hacc_lt hval_sh_lt + have hfit_acc' : Fits w (acc ^^^ (clmul xc yc <<< (sh_x + sh_y))) := + fits_of_lt_two_pow_two_mul hw hacc'_lt + have hy_shr_lt : y_rem >>> b < 2 ^ (rem_j * b) := shiftRight_lt_two_pow_mul hy_lt + have hy_shr_bound : (y_rem >>> b) * 2 ^ (sh_y + b) < 2 ^ q := + (shiftRight_mul_two_pow_add_le y_rem b sh_y).trans_lt hy_bound + have hy_shr_lt_2q : y_rem >>> b < 2 ^ (2 * q) := by + have hy_lt_q : y_rem < 2 ^ q := + (Nat.le_mul_of_pos_right _ (by positivity)).trans_lt hy_bound + exact (Nat.shiftRight_le y_rem b).trans_lt + (lt_of_lt_of_le hy_lt_q (Nat.pow_le_pow_right (by decide) (by omega))) + have hfit_y_shr : Fits w (y_rem >>> b) := fits_of_lt_two_pow_two_mul hw hy_shr_lt_2q + have hsh_le : sh_x + sh_y + b ≤ sh_x + sh_y + (rem_j + 1) * b := by + rw [Nat.succ_mul] + omega + have hfit_sh_next : Fits w (sh_x + sh_y + b) := hfit_sh.of_le hsh_le + have hfit_rem : Fits w rem_j := by + have : rem_j * 1 ≤ (rem_j + 1) * b := Nat.mul_le_mul (by omega) hb + exact hfit_sh.of_le (by omega) + set s1 := s.set 17 yc + have e1 : Exec w (.op (.exactBin .and 17 15 5) : Code .exact 18) s s1 1 := + exec_exactBin (by simp [h15, h5, yc]) hfit_yc + set s2 := s1.set 17 idx + have e2 : Exec w (.op (.exactBin .xor 17 13 17) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h13, xc, idx]) hfit_idx + set s3 := s2.set 17 (tabBase + idx) + have e3 : Exec w (.op (.exactBin .add 17 7 17) : Code .exact 18) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h7]) hfit_addr + set s4 := s3.set 17 (clmul xc yc) + have hmem_idx : s3.mem (tabBase + idx) = clmul xc yc := by + change s.mem (tabBase + idx) = clmul xc yc + rw [htab idx hidx_lt, hidx_u, hidx_v] + have e4 : Exec w (.op (.load 17 17) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, hmem_idx] using + (Exec.load (B := .exact) (w := w) (d := 17) (a := 17) (x := s3)) + set s5 := s4.set 17 (clmul xc yc <<< (sh_x + sh_y)) + have e5 : Exec w (.op (.exactBin .shiftLeft 17 17 16) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h16]) hfit_val_sh + set s6 := s5.set 2 (acc ^^^ (clmul xc yc <<< (sh_x + sh_y))) + have e6 : Exec w (.op (.exactBin .xor 2 2 17) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h2]) hfit_acc' + set s7 := s6.set 15 (y_rem >>> b) + have e7 : Exec w (.op (.exactBin .shiftRight 15 15 4) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, s4, s3, s2, s1, h15, h4]) hfit_y_shr + set s8 := s7.set 16 (sh_x + sh_y + b) + have e8 : Exec w (.op (.exactBin .add 16 16 4) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, h16, h4]) hfit_sh_next + set s9 := s8.set 14 rem_j + have e9 : Exec w (.op (.exactBin .sub 14 14 8) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h14, h8]) hfit_rem + have hbody := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq e9))))))) + have hfit_sh' : Fits w (sh_x + (sh_y + b) + rem_j * b) := by + rwa [show sh_x + (sh_y + b) + rem_j * b = sh_x + sh_y + (rem_j + 1) * b by ring] + obtain ⟨y, hrec, hy2, hy_regs, hy_mem⟩ := + ih (y_rem >>> b) (sh_y + b) (acc ^^^ (clmul xc yc <<< (sh_x + sh_y))) s9 + hy_shr_lt hy_shr_bound hacc'_lt hfit_sh' + (by simp +decide [s9, s8, s7, s6]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h4]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h5]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h7]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h8]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h9]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h13, xc]) + (by simp +decide [s9]) + (by simp +decide [s9, s8, s7]) + (by simp +decide [s9, s8, Nat.add_assoc]) + htab + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, ?_, ?_, hy_mem⟩ + · simpa only [innerClmulLoop, innerClmulBody, + show 1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + 1))))))) + (11 * rem_j + 2) + 2 = + 11 * (rem_j + 1) + 2 by omega] using hiter + · rw [hy2, Nat.xor_assoc] + congr 1 + calc (clmul xc yc <<< (sh_x + sh_y)) ^^^ (clmul xc (y_rem >>> b) <<< (sh_x + (sh_y + b))) + = (clmul xc yc <<< (sh_x + sh_y)) ^^^ + ((clmul xc (y_rem >>> b) <<< b) <<< (sh_x + sh_y)) := by + rw [show sh_x + (sh_y + b) = b + (sh_x + sh_y) by omega, + shiftLeft_add_eq _ b (sh_x + sh_y)] + _ = (clmul xc yc ^^^ (clmul xc (y_rem >>> b) <<< b)) <<< (sh_x + sh_y) := by + rw [← xor_shiftLeft_eq] + _ = clmul xc y_rem <<< (sh_x + sh_y) := by + rw [← clmul_split_right] + · intro r hr + rcases hr with ⟨hr2, hr14, hr15, hr16, hr17⟩ + rw [hy_regs r ⟨hr2, hr14, hr15, hr16, hr17⟩] + simp only [s9, s8, s7, s6, s5, s4, s3, s2, s1, Data.set, + Function.update_of_ne hr2, Function.update_of_ne hr14, + Function.update_of_ne hr15, Function.update_of_ne hr16, + Function.update_of_ne hr17] + +/-- Body of `outerClmulLoop`: computes `x_chunk_sh = (x_rem &&& mask_b) <<< b`, initializes the +inner loop over `L` limbs of `u` starting at shift `shift_i`, runs `innerClmulLoop`, and advances +`x_rem`, `shift_i`, and `rem_i`. -/ +@[expose] def outerClmulBody : Code .exact 18 := + .seq (.op (.exactBin .and 13 11 5)) + (.seq (.op (.exactBin .shiftLeft 13 13 4)) + (.seq (.op (.copy 14 6)) + (.seq (.op (.copy 15 1)) + (.seq (.op (.copy 16 12)) + (.seq innerClmulLoop + (.seq (.op (.exactBin .shiftRight 11 11 4)) + (.seq (.op (.exactBin .add 12 12 4)) + (.op (.exactBin .sub 10 10 8))))))))) + +/-- Outer Four-Russians loop `while 0 < rem_i do outerClmulBody`. -/ +@[expose] def outerClmulLoop : Code .exact 18 := + .loop (.lt 9 10) outerClmulBody + +/-- `outerClmulLoop` executes deterministically in `(11 * L + 12) * rem_i + 2` steps, XORing +`clmul x_rem u <<< sh_x` into `reg 2` (`acc`). -/ +theorem outerClmulLoop_exec {w tabBase b q L u : ℕ} + (hb : 0 < b) (hw : 2 * q ≤ w) + (hfit_tabEnd : Fits w (tabBase + 2 ^ (2 * b))) + (hu_limbs : u < 2 ^ (L * b)) (hu_bound : u < 2 ^ q) : + ∀ rem_i x_rem sh_x acc : ℕ, ∀ s : Data 18, + x_rem < 2 ^ (rem_i * b) → + x_rem * 2 ^ sh_x < 2 ^ q → + acc < 2 ^ (2 * q) → + Fits w (sh_x + rem_i * b + L * b) → + s.regs 1 = u → + s.regs 2 = acc → + s.regs 4 = b → + s.regs 5 = 2 ^ b - 1 → + s.regs 6 = L → + s.regs 7 = tabBase → + s.regs 8 = 1 → + s.regs 9 = 0 → + s.regs 10 = rem_i → + s.regs 11 = x_rem → + s.regs 12 = sh_x → + (∀ i < 2 ^ (2 * b), s.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) → + ∃ y : Data 18, + Exec w outerClmulLoop s y ((11 * L + 12) * rem_i + 2) ∧ + y.regs 2 = acc ^^^ (clmul x_rem u <<< sh_x) ∧ + (∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → y.regs r = s.regs r) ∧ + y.mem = s.mem := by + intro rem_i + induction rem_i with + | zero => + intro x_rem sh_x acc s hx_lt _ _ _ _ h2 _ _ _ _ _ h9 h10 _ _ _ + have hx0 : x_rem = 0 := by + rw [Nat.zero_mul, pow_zero] at hx_lt + omega + have hc : ¬(Cond.lt (9 : Fin 18) 10).Holds s := by + simp [Cond.Holds, h9, h10] + refine ⟨s, Exec.loop_exit hc, ?_, fun _ _ => rfl, rfl⟩ + simp [h2, hx0] + | succ rem_i ih => + intro x_rem sh_x acc s hx_lt hx_bound hacc_lt hfit_sh h1 h2 h4 h5 h6 h7 h8 h9 h10 h11 h12 htab + have hc : (Cond.lt (9 : Fin 18) 10).Holds s := by + simp [Cond.Holds, h9, h10] + set xc := x_rem &&& (2 ^ b - 1) + have hxc_lt : xc < 2 ^ b := by + dsimp only [xc] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hxc_sh_lt : xc <<< b < 2 ^ (2 * b) := by + rw [Nat.shiftLeft_eq, two_mul, pow_add] + exact Nat.mul_lt_mul_of_pos_right hxc_lt (by positivity) + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hxc_lt2 : xc < 2 ^ (2 * b) := lt_of_lt_of_le hxc_lt hpow_le + have hfit_xc : Fits w xc := hfit_tabEnd.of_le (by omega) + have hfit_xc_sh : Fits w (xc <<< b) := hfit_tabEnd.of_le (by omega) + set s1 := s.set 13 xc + have e1 : Exec w (.op (.exactBin .and 13 11 5) : Code .exact 18) s s1 1 := + exec_exactBin (by simp [h11, h5, xc]) hfit_xc + set s2 := s1.set 13 (xc <<< b) + have e2 : Exec w (.op (.exactBin .shiftLeft 13 13 4) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h4]) hfit_xc_sh + set s3 := s2.set 14 L + have e3 : Exec w (.op (.copy 14 6) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, h6] using + (Exec.copy (B := .exact) (w := w) (d := 14) (s := 6) (x := s2)) + set s4 := s3.set 15 u + have e4 : Exec w (.op (.copy 15 1) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, h1] using + (Exec.copy (B := .exact) (w := w) (d := 15) (s := 1) (x := s3)) + set s5 := s4.set 16 sh_x + have e5 : Exec w (.op (.copy 16 12) : Code .exact 18) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, h12] using + (Exec.copy (B := .exact) (w := w) (d := 16) (s := 12) (x := s4)) + have hfit_inner_sh : Fits w (sh_x + 0 + L * b) := + hfit_sh.of_le (by omega) + obtain ⟨s6, e6, hs6_2, hs6_regs, hs6_mem⟩ := + innerClmulLoop_exec hb hw hfit_tabEnd hx_bound L u 0 acc s5 + hu_limbs (by simpa using hu_bound) hacc_lt hfit_inner_sh + (by simp +decide [s5, s4, s3, s2, s1, h2]) + (by simp +decide [s5, s4, s3, s2, s1, h4]) + (by simp +decide [s5, s4, s3, s2, s1, h5]) + (by simp +decide [s5, s4, s3, s2, s1, h7]) + (by simp +decide [s5, s4, s3, s2, s1, h8]) + (by simp +decide [s5, s4, s3, s2, s1, h9]) + (by simp +decide [s5, s4, s3, s2, xc]) + (by simp +decide [s5, s4, s3]) + (by simp +decide [s5, s4]) + (by simp +decide [s5]) + htab + rw [Nat.add_zero] at hs6_2 + have hs6_4 : s6.regs 4 = b := by + rw [hs6_regs 4 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h4] + have hs6_8 : s6.regs 8 = 1 := by + rw [hs6_regs 8 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h8] + have hs6_10 : s6.regs 10 = rem_i + 1 := by + rw [hs6_regs 10 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h10] + have hs6_11 : s6.regs 11 = x_rem := by + rw [hs6_regs 11 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h11] + have hs6_12 : s6.regs 12 = sh_x := by + rw [hs6_regs 12 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h12] + have hx_shr_lt : x_rem >>> b < 2 ^ (rem_i * b) := shiftRight_lt_two_pow_mul hx_lt + have hx_shr_bound : (x_rem >>> b) * 2 ^ (sh_x + b) < 2 ^ q := + (shiftRight_mul_two_pow_add_le x_rem b sh_x).trans_lt hx_bound + have hx_shr_lt_2q : x_rem >>> b < 2 ^ (2 * q) := by + have hx_lt_q : x_rem < 2 ^ q := + (Nat.le_mul_of_pos_right _ (by positivity)).trans_lt hx_bound + exact (Nat.shiftRight_le x_rem b).trans_lt + (lt_of_lt_of_le hx_lt_q (Nat.pow_le_pow_right (by decide) (by omega))) + have hfit_x_shr : Fits w (x_rem >>> b) := fits_of_lt_two_pow_two_mul hw hx_shr_lt_2q + have hfit_sh_next : Fits w (sh_x + b) := + hfit_sh.of_le (by rw [Nat.succ_mul]; omega) + have hfit_rem_i : Fits w rem_i := by + have : rem_i * 1 ≤ (rem_i + 1) * b := Nat.mul_le_mul (by omega) hb + exact hfit_sh.of_le (by omega) + set s7 := s6.set 11 (x_rem >>> b) + have e7 : Exec w (.op (.exactBin .shiftRight 11 11 4) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [hs6_11, hs6_4]) hfit_x_shr + set s8 := s7.set 12 (sh_x + b) + have e8 : Exec w (.op (.exactBin .add 12 12 4) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, hs6_12, hs6_4]) hfit_sh_next + set s9 := s8.set 10 rem_i + have e9 : Exec w (.op (.exactBin .sub 10 10 8) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, hs6_10, hs6_8]) hfit_rem_i + have hbody := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq e9))))))) + have hcl_xc_u_lt : clmul xc u <<< sh_x < 2 ^ (2 * q) := by + rcases eq_or_ne xc 0 with hxc0 | hxc0 + · simp [hxc0] + · have hxc_q : xc * 2 ^ sh_x < 2 ^ q := + (Nat.mul_le_mul_right (2 ^ sh_x) Nat.and_le_left).trans_lt hx_bound + obtain ⟨hsx, hxc'⟩ := lt_two_pow_sub_of_mul_two_pow_lt (Nat.pos_of_ne_zero hxc0) hxc_q + have hcl := clmul_lt_two_pow hxc' hu_bound + rw [Nat.shiftLeft_eq] + calc clmul xc u * 2 ^ sh_x + < 2 ^ (q - sh_x + q) * 2 ^ sh_x := Nat.mul_lt_mul_of_pos_right hcl (by positivity) + _ = 2 ^ (2 * q) := by + rw [← pow_add] + congr 1 + omega + have hacc'_lt : acc ^^^ (clmul xc u <<< sh_x) < 2 ^ (2 * q) := + Nat.xor_lt_two_pow hacc_lt hcl_xc_u_lt + have hfit_sh' : Fits w (sh_x + b + rem_i * b + L * b) := by + rwa [show sh_x + b + rem_i * b + L * b = sh_x + (rem_i + 1) * b + L * b by ring] + have hs9_low : ∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → s9.regs r = s.regs r := by + intro r ⟨hr_lt, hr2⟩ + have hr10 : r ≠ 10 := by omega + have hr11 : r ≠ 11 := by omega + have hr12 : r ≠ 12 := by omega + have hr13 : r ≠ 13 := by omega + have hr14 : r ≠ 14 := by omega + have hr15 : r ≠ 15 := by omega + have hr16 : r ≠ 16 := by omega + have hr17 : r ≠ 17 := by omega + calc s9.regs r = s6.regs r := by + simp only [s9, s8, s7, Data.set, + Function.update_of_ne hr10, Function.update_of_ne hr11, Function.update_of_ne hr12] + _ = s5.regs r := hs6_regs r ⟨hr2, hr14, hr15, hr16, hr17⟩ + _ = s.regs r := by + simp only [s5, s4, s3, s2, s1, Data.set, + Function.update_of_ne hr13, Function.update_of_ne hr14, + Function.update_of_ne hr15, Function.update_of_ne hr16] + have hs9_2 : s9.regs 2 = acc ^^^ (clmul xc u <<< sh_x) := by + change s6.regs 2 = acc ^^^ (clmul xc u <<< sh_x) + exact hs6_2 + obtain ⟨y, hrec, hy2, hy_regs, hy_mem⟩ := + ih (x_rem >>> b) (sh_x + b) (acc ^^^ (clmul xc u <<< sh_x)) s9 + hx_shr_lt hx_shr_bound hacc'_lt hfit_sh' + (by rw [hs9_low 1 ⟨by decide, by decide⟩, h1]) + hs9_2 + (by rw [hs9_low 4 ⟨by decide, by decide⟩, h4]) + (by rw [hs9_low 5 ⟨by decide, by decide⟩, h5]) + (by rw [hs9_low 6 ⟨by decide, by decide⟩, h6]) + (by rw [hs9_low 7 ⟨by decide, by decide⟩, h7]) + (by rw [hs9_low 8 ⟨by decide, by decide⟩, h8]) + (by rw [hs9_low 9 ⟨by decide, by decide⟩, h9]) + (by simp +decide [s9]) + (by simp +decide [s9, s8, s7]) + (by simp +decide [s9, s8]) + (by rw [show s9.mem = s.mem from hs6_mem]; exact htab) + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, ?_, ?_, by rw [hy_mem]; exact hs6_mem⟩ + · simpa only [outerClmulLoop, outerClmulBody, + show 1 + (1 + (1 + (1 + (1 + ((11 * L + 2) + (1 + (1 + 1))))))) + + ((11 * L + 12) * rem_i + 2) + 2 = (11 * L + 12) * (rem_i + 1) + 2 by ring] using hiter + · rw [hy2, Nat.xor_assoc] + congr 1 + calc (clmul xc u <<< sh_x) ^^^ (clmul (x_rem >>> b) u <<< (sh_x + b)) + = (clmul xc u <<< sh_x) ^^^ ((clmul (x_rem >>> b) u <<< b) <<< sh_x) := by + rw [show sh_x + b = b + sh_x by omega, shiftLeft_add_eq _ b sh_x] + _ = (clmul xc u ^^^ (clmul (x_rem >>> b) u <<< b)) <<< sh_x := by + rw [← xor_shiftLeft_eq] + _ = clmul x_rem u <<< sh_x := by + rw [← clmul_split_left] + · intro r hr + rw [hy_regs r hr, hs9_low r hr] + +/-- 16-instruction straight-line reduction block on `Data 18`: loads `r`, `q = 2 * r`, +`2 ^ r - 1`, and `2 ^ q - 1` from `base + 0 .. base + 3` into registers `10..13` and replaces `c` +in `reg 2` with `reduceTrinomialCode (trinomialStep U M) c`, preserving registers `0, 1, 3..9` and +memory. -/ +@[expose] def reduceStepCode : Code .exact 18 := + .seq (.op (.load 10 0)) + (.seq (.op (.exactBin .add 15 0 8)) + (.seq (.op (.load 11 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 12 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 13 15)) + (.seq (.op (.exactBin .add 14 11 10)) + (.seq (.op (.exactBin .shiftRight 15 2 14)) + (.seq (.op (.exactBin .shiftRight 16 2 11)) + (.seq (.op (.exactBin .and 16 16 12)) + (.seq (.op (.exactBin .and 2 2 13)) + (.seq (.op (.exactBin .xor 2 2 15)) + (.seq (.op (.exactBin .xor 2 2 16)) + (.seq (.op (.exactBin .shiftLeft 16 16 10)) + (.op (.exactBin .xor 2 2 16)))))))))))))))) + +/-- `reduceStepCode` executes deterministically in `16` instructions, replacing `c` in `reg 2` +with `reduceTrinomialCode (trinomialStep U M) c`. -/ +theorem reduceStepCode_exec {w base b U M k c : ℕ} {s : Data 18} + (htab : TableRep base b U M k s.mem) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_end8 : Fits w (base + 8)) + (hc : c < 2 ^ (2 * trinomialDeg U M)) + (h0 : s.regs 0 = base) (h2 : s.regs 2 = c) (h8 : s.regs 8 = 1) : + ∃ y : Data 18, + Exec w reduceStepCode s y 16 ∧ + y.regs 2 = reduceTrinomialCode (trinomialStep U M) c ∧ + (∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → y.regs r = s.regs r) ∧ + y.mem = s.mem := by + set t := trinomialStep U M + set r := 3 ^ t + set q := trinomialDeg U M + have hq_eq : q = 2 * r := rfl + have hm0 : s.mem base = r := htab.1 + have hm1 : s.mem (base + 1) = q := htab.2.1 + have hm2 : s.mem (base + 2) = 2 ^ r - 1 := htab.2.2.1 + have hm3 : s.mem (base + 3) = 2 ^ q - 1 := htab.2.2.2.1 + have hfit_b1 : Fits w (base + 1) := hfit_end8.of_le (by omega) + have hfit_b2 : Fits w (base + 2) := hfit_end8.of_le (by omega) + have hfit_b3 : Fits w (base + 3) := hfit_end8.of_le (by omega) + have h3r_lt : q + r < 2 ^ (2 * q) := by + have : 1 ≤ r := Nat.one_le_pow t 3 (by decide) + have h2q : q + r ≤ 2 * q := by omega + exact lt_of_le_of_lt h2q Nat.lt_two_pow_self + have hfit_3r : Fits w (q + r) := fits_of_lt_two_pow_two_mul hw h3r_lt + set c3 := c >>> (q + r) + set c2_raw := c >>> q + set c2 := c2_raw &&& (2 ^ r - 1) + set cLow := c &&& (2 ^ q - 1) + have hfit_c : Fits w c := fits_of_lt_two_pow_two_mul hw hc + have hfit_c3 : Fits w c3 := hfit_c.of_le (Nat.shiftRight_le _ _) + have hfit_c2_raw : Fits w c2_raw := hfit_c.of_le (Nat.shiftRight_le _ _) + have hfit_c2 : Fits w c2 := hfit_c2_raw.of_le Nat.and_le_left + have hfit_cLow : Fits w cLow := hfit_c.of_le Nat.and_le_left + have hcLow_lt : cLow < 2 ^ q := by + dsimp only [cLow] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hc3_lt : c3 < 2 ^ q := by + have hlt : c >>> (q + r) < 2 ^ r := by + rw [Nat.shiftRight_eq_div_pow] + apply Nat.div_lt_of_lt_mul + rw [← pow_add 2 (q + r) r, show q + r + r = 2 * q by omega] + exact hc + exact lt_of_lt_of_le hlt (Nat.pow_le_pow_right (by decide) (by omega)) + have hc2_r_lt : c2 < 2 ^ r := by + dsimp only [c2] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hc2_lt : c2 < 2 ^ q := + lt_of_lt_of_le hc2_r_lt (Nat.pow_le_pow_right (by decide) (by omega)) + have hc2_sh_lt : c2 <<< r < 2 ^ q := by + rw [Nat.shiftLeft_eq, hq_eq, two_mul, pow_add] + exact Nat.mul_lt_mul_of_pos_right hc2_r_lt (by positivity) + have hpow_q_le : 2 ^ q ≤ 2 ^ (2 * q) := Nat.pow_le_pow_right (by decide) (by omega) + have hfit_xor1 : Fits w (cLow ^^^ c3) := + fits_of_lt_two_pow_two_mul hw ((Nat.xor_lt_two_pow hcLow_lt hc3_lt).trans_le hpow_q_le) + have hfit_xor2 : Fits w ((cLow ^^^ c3) ^^^ c2) := + fits_of_lt_two_pow_two_mul hw + ((Nat.xor_lt_two_pow (Nat.xor_lt_two_pow hcLow_lt hc3_lt) hc2_lt).trans_le hpow_q_le) + have hfit_c2_sh : Fits w (c2 <<< r) := + fits_of_lt_two_pow_two_mul hw (hc2_sh_lt.trans_le hpow_q_le) + have hfit_xor3 : Fits w (((cLow ^^^ c3) ^^^ c2) ^^^ (c2 <<< r)) := + fits_of_lt_two_pow_two_mul hw + ((Nat.xor_lt_two_pow + (Nat.xor_lt_two_pow (Nat.xor_lt_two_pow hcLow_lt hc3_lt) hc2_lt) hc2_sh_lt).trans_le + hpow_q_le) + set s1 := s.set 10 r + have e1 : Exec w (.op (.load 10 0) : Code .exact 18) s s1 1 := by + simpa +decide [s1, h0, hm0] using (Exec.load (B := .exact) (w := w) (d := 10) (a := 0) (x := s)) + set s2 := s1.set 15 (base + 1) + have e2 : Exec w (.op (.exactBin .add 15 0 8) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h0, h8]) hfit_b1 + set s3 := s2.set 11 q + have e3 : Exec w (.op (.load 11 15) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, hm1] using + (Exec.load (B := .exact) (w := w) (d := 11) (a := 15) (x := s2)) + set s4 := s3.set 15 (base + 2) + have e4 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, h8, Nat.add_assoc]) hfit_b2 + set s5 := s4.set 12 (2 ^ r - 1) + have e5 : Exec w (.op (.load 12 15) : Code .exact 18) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, hm2] using + (Exec.load (B := .exact) (w := w) (d := 12) (a := 15) (x := s4)) + set s6 := s5.set 15 (base + 3) + have e6 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h8, Nat.add_assoc]) hfit_b3 + set s7 := s6.set 13 (2 ^ q - 1) + have e7 : Exec w (.op (.load 13 15) : Code .exact 18) s6 s7 1 := by + simpa +decide [s7, s6, s5, s4, s3, s2, s1, hm3] using + (Exec.load (B := .exact) (w := w) (d := 13) (a := 15) (x := s6)) + set s8 := s7.set 14 (q + r) + have e8 : Exec w (.op (.exactBin .add 14 11 10) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1]) hfit_3r + set s9 := s8.set 15 c3 + have e9 : Exec w (.op (.exactBin .shiftRight 15 2 14) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h2, c3]) hfit_c3 + set s10 := s9.set 16 c2_raw + have e10 : Exec w (.op (.exactBin .shiftRight 16 2 11) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h2, c2_raw]) hfit_c2_raw + set s11 := s10.set 16 c2 + have e11 : Exec w (.op (.exactBin .and 16 16 12) : Code .exact 18) s10 s11 1 := + exec_exactBin (by simp +decide [s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, c2]) hfit_c2 + set s12 := s11.set 2 cLow + have e12 : Exec w (.op (.exactBin .and 2 2 13) : Code .exact 18) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h2, cLow]) + hfit_cLow + set s13 := s12.set 2 (cLow ^^^ c3) + have e13 : Exec w (.op (.exactBin .xor 2 2 15) : Code .exact 18) s12 s13 1 := + exec_exactBin (by simp +decide [s12, s11, s10, s9]) hfit_xor1 + set s14 := s13.set 2 ((cLow ^^^ c3) ^^^ c2) + have e14 : Exec w (.op (.exactBin .xor 2 2 16) : Code .exact 18) s13 s14 1 := + exec_exactBin (by simp +decide [s13, s12, s11]) hfit_xor2 + set s15 := s14.set 16 (c2 <<< r) + have e15 : Exec w (.op (.exactBin .shiftLeft 16 16 10) : Code .exact 18) s14 s15 1 := + exec_exactBin (by simp +decide [s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1]) + hfit_c2_sh + set s16 := s15.set 2 (((cLow ^^^ c3) ^^^ c2) ^^^ (c2 <<< r)) + have e16 : Exec w (.op (.exactBin .xor 2 2 16) : Code .exact 18) s15 s16 1 := + exec_exactBin (by simp +decide [s15, s14]) hfit_xor3 + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq e16)))))))))))))) + refine ⟨s16, hall, ?_, ?_, rfl⟩ + · simp only [s16, Data.set, Function.update_self, reduceTrinomialCode, + cLow, c3, c2, c2_raw, q, trinomialDeg, r, t, + show 2 * 3 ^ trinomialStep U M + 3 ^ trinomialStep U M = 3 * 3 ^ trinomialStep U M by ring] + · intro reg ⟨hreg_lt, hreg2⟩ + have hr10 : reg ≠ 10 := by omega + have hr11 : reg ≠ 11 := by omega + have hr12 : reg ≠ 12 := by omega + have hr13 : reg ≠ 13 := by omega + have hr14 : reg ≠ 14 := by omega + have hr15 : reg ≠ 15 := by omega + have hr16 : reg ≠ 16 := by omega + simp only [s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, Data.set, + Function.update_of_ne hreg2, Function.update_of_ne hr10, Function.update_of_ne hr11, + Function.update_of_ne hr12, Function.update_of_ne hr13, Function.update_of_ne hr14, + Function.update_of_ne hr15, Function.update_of_ne hr16] + +/-- Body of `hornerLoop` on `Data 18`: decrements `j` (`reg 3`), multiplies `acc` (`reg 2`) by +`u` (`reg 1`) in `TrinomialField (trinomialStep U M)` via `outerClmulLoop` and `reduceStepCode`, +and adds `seed[j]` from `mem[base + 8 + j]`. -/ +@[expose] def hornerBody : Code .exact 18 := + .seq (.op (.exactBin .sub 3 3 8)) + (.seq (.op (.copy 11 2)) + (.seq (.op (.copy 2 9)) + (.seq (.op (.copy 10 6)) + (.seq (.op (.copy 12 9)) + (.seq outerClmulLoop + (.seq reduceStepCode + (.seq (.op (.const 15 8)) + (.seq (.op (.exactBin .add 15 0 15)) + (.seq (.op (.exactBin .add 15 15 3)) + (.seq (.op (.load 15 15)) + (.op (.exactBin .xor 2 2 15)))))))))))) + +/-- Horner evaluation loop `while 0 < j do hornerBody`. -/ +@[expose] def hornerLoop : Code .exact 18 := + .loop (.lt 9 3) hornerBody + +/-- `hornerLoop` executes deterministically in `((11 * L + 12) * L + 30) * m + 2` instructions, +updating `reg 2` from `(equivFin t acc).val` to +`(equivFin t (acc * x ^ m + ∑ i : Fin m, c_field (Fin.castLE hm i) * x ^ (i : ℕ))).val`. -/ +theorem hornerLoop_exec {w base b U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} (u : Fin U) + (hb : 0 < b) (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + spaceFromB b k)) : + let t := trinomialStep U M + let q := trinomialDeg U M + let L := limbCountFromB b q + let c_field : Fin k → TrinomialField t := fun j => (TrinomialField.equivFin t).symm (c j) + let x_field : TrinomialField t := AdjoinRoot.mk (trinomial t) (polyOfCode u.val) + ∀ m : ℕ, ∀ hm : m ≤ k, ∀ acc : TrinomialField t, ∀ s : Data 18, + Rep base b U M k c s.mem → + s.regs 0 = base → + s.regs 1 = u.val → + s.regs 2 = (TrinomialField.equivFin t acc).val → + s.regs 3 = m → + s.regs 4 = b → + s.regs 5 = 2 ^ b - 1 → + s.regs 6 = L → + s.regs 7 = base + 8 + k → + s.regs 8 = 1 → + s.regs 9 = 0 → + ∃ y : Data 18, + Exec w hornerLoop s y (((11 * L + 12) * L + 30) * m + 2) ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = (TrinomialField.equivFin t + (acc * x_field ^ m + ∑ i : Fin m, c_field (Fin.castLE hm i) * x_field ^ (i : ℕ))).val ∧ + y.mem = s.mem := by + intro t q L c_field x_field m + induction m with + | zero => + intro hm acc s _ h0 h1 h2 h3 _ _ _ _ _ h9 + have hc : ¬(Cond.lt (9 : Fin 18) 3).Holds s := by + simp [Cond.Holds, h9, h3] + refine ⟨s, Exec.loop_exit hc, h0, h1, ?_, rfl⟩ + simp [h2] + | succ m ih => + intro hm acc s hrep h0 h1 h2 h3 h4 h5 h6 h7 h8 h9 + have hc : (Cond.lt (9 : Fin 18) 3).Holds s := by + simp [Cond.Holds, h9, h3] + obtain ⟨htab, hseed⟩ := hrep + have hpow_pos : 0 < 2 ^ (2 * b) := by positivity + have hfit_tabEnd : Fits w (base + 8 + k + 2 ^ (2 * b)) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + have hfit_end8 : Fits w (base + 8) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + have hfit_end_k : Fits w (base + 8 + k) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + have hfit_m : Fits w m := hfit_end_k.of_le (by omega) + have hq_le_Lb : q ≤ L * b := le_limbCountFromB_mul hb q + have hpow_q_Lb : 2 ^ q ≤ 2 ^ (L * b) := Nat.pow_le_pow_right (by decide) hq_le_Lb + have hu_bound : u.val < 2 ^ q := + lt_of_lt_of_le u.isLt (le_two_pow_trinomialDeg U M) + have hu_limbs : u.val < 2 ^ (L * b) := lt_of_lt_of_le hu_bound hpow_q_Lb + set acc_val := (TrinomialField.equivFin t acc).val + have hacc_bound : acc_val < 2 ^ q := (TrinomialField.equivFin t acc).isLt + have hacc_limbs : acc_val < 2 ^ (L * b) := lt_of_lt_of_le hacc_bound hpow_q_Lb + have hLb_le : L * b ≤ q + b := by + dsimp only [L, limbCountFromB] + have := Nat.div_mul_le_self (q + b - 1) b + omega + have h2b_lt_w : 2 * b < w := by + apply (Nat.pow_lt_pow_iff_right one_lt_two).mp + exact lt_of_le_of_lt (by dsimp only [spaceFromB]; omega) hfit_space + have hq_ge_2 : 2 ≤ q := by + dsimp only [q, trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + omega + have hw_pos : 1 ≤ w := by omega + have hw_le_pow : w ≤ 2 ^ (w - 1) := by + have hlt : w - 1 < 2 ^ (w - 1) := Nat.lt_two_pow_self + omega + have h2b_lt_pow : 2 * b < 2 ^ (w - 1) := + calc 2 * b < 2 ^ (2 * b) := Nat.lt_two_pow_self + _ ≤ 2 ^ (w - 1) := Nat.pow_le_pow_right (by decide) (by omega) + have hpow_w_eq : 2 ^ (w - 1) + 2 ^ (w - 1) = 2 ^ w := by + conv_rhs => rw [← Nat.sub_add_cancel hw_pos, pow_succ] + omega + have hfit_2Lb : Fits w (0 + L * b + L * b) := by + dsimp only [Fits] + omega + set s1 := s.set 3 m + have e1 : Exec w (.op (.exactBin .sub 3 3 8) : Code .exact 18) s s1 1 := + exec_exactBin (by simp +decide [h3, h8]) hfit_m + set s2 := s1.set 11 acc_val + have e2 : Exec w (.op (.copy 11 2) : Code .exact 18) s1 s2 1 := by + simpa +decide [s2, s1, h2, acc_val] using + (Exec.copy (B := .exact) (w := w) (d := 11) (s := 2) (x := s1)) + set s3 := s2.set 2 0 + have e3 : Exec w (.op (.copy 2 9) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, h9] using + (Exec.copy (B := .exact) (w := w) (d := 2) (s := 9) (x := s2)) + set s4 := s3.set 10 L + have e4 : Exec w (.op (.copy 10 6) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, h6] using + (Exec.copy (B := .exact) (w := w) (d := 10) (s := 6) (x := s3)) + set s5 := s4.set 12 0 + have e5 : Exec w (.op (.copy 12 9) : Code .exact 18) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, h9] using + (Exec.copy (B := .exact) (w := w) (d := 12) (s := 9) (x := s4)) + obtain ⟨s6, e6, hs6_2, hs6_low, hs6_mem⟩ := + outerClmulLoop_exec hb hw hfit_tabEnd hu_limbs hu_bound L acc_val 0 0 s5 + hacc_limbs (by simpa using hacc_bound) (by positivity) hfit_2Lb + (by simp +decide [s5, s4, s3, s2, s1, h1]) + (by simp +decide [s5, s4, s3]) + (by simp +decide [s5, s4, s3, s2, s1, h4]) + (by simp +decide [s5, s4, s3, s2, s1, h5]) + (by simp +decide [s5, s4, s3, s2, s1, h6]) + (by simp +decide [s5, s4, s3, s2, s1, h7]) + (by simp +decide [s5, s4, s3, s2, s1, h8]) + (by simp +decide [s5, s4, s3, s2, s1, h9]) + (by simp +decide [s5, s4]) + (by simp +decide [s5, s4, s3, s2]) + (by simp +decide [s5]) + htab.2.2.2.2.2.2.2.2 + simp only [Nat.shiftLeft_zero, Nat.zero_xor] at hs6_2 + have hcl_lt : clmul acc_val u.val < 2 ^ (2 * q) := + clmul_lt_two_pow_two_mul hacc_bound hu_bound + have hs6_0 : s6.regs 0 = base := by + rw [hs6_low 0 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1, h0] + have hs6_8 : s6.regs 8 = 1 := by + rw [hs6_low 8 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1, h8] + have htab_s6 : TableRep base b U M k s6.mem := by + rw [hs6_mem] + exact htab + obtain ⟨s7, e7, hs7_2, hs7_low, hs7_mem⟩ := + reduceStepCode_exec htab_s6 hw hfit_end8 hcl_lt hs6_0 hs6_2 hs6_8 + have hs7_low_eq : ∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → s7.regs r = s5.regs r := by + intro r hr + rw [hs7_low r hr, hs6_low r hr] + have hs7_0 : s7.regs 0 = base := by + rw [hs7_low_eq 0 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1, h0] + have hs7_3 : s7.regs 3 = m := by + rw [hs7_low_eq 3 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1] + have hfit_8 : Fits w 8 := hfit_end8.of_le (by omega) + have hfit_b8m : Fits w (base + 8 + m) := hfit_end_k.of_le (by omega) + have hm_lt : m < k := by omega + have hmem_seed : s7.mem (base + 8 + m) = (c ⟨m, hm_lt⟩).val := by + rw [hs7_mem, hs6_mem] + exact hseed ⟨m, hm_lt⟩ + have hx_val : (TrinomialField.equivFin t x_field).val = u.val := + TrinomialField.val_equivFin_mk_polyOfCode t hu_bound + have hmul_val : reduceTrinomialCode t (clmul acc_val u.val) = + (TrinomialField.equivFin t (acc * x_field)).val := by + rw [TrinomialField.val_equivFin_mul, hx_val] + have hseed_val : (c ⟨m, hm_lt⟩).val = + (TrinomialField.equivFin t (c_field ⟨m, hm_lt⟩)).val := by + have h : TrinomialField.equivFin t (c_field ⟨m, hm_lt⟩) = c ⟨m, hm_lt⟩ := + (TrinomialField.equivFin t).apply_symm_apply (c ⟨m, hm_lt⟩) + exact (congrArg Fin.val h).symm + set acc_next := acc * x_field + c_field ⟨m, hm_lt⟩ + have hxor_val : reduceTrinomialCode t (clmul acc_val u.val) ^^^ (c ⟨m, hm_lt⟩).val = + (TrinomialField.equivFin t acc_next).val := by + rw [hmul_val, hseed_val, ← TrinomialField.val_equivFin_add] + have hacc_next_lt_q : (TrinomialField.equivFin t acc_next).val < 2 ^ q := + (TrinomialField.equivFin t acc_next).isLt + have hfit_acc_next : Fits w (TrinomialField.equivFin t acc_next).val := + fits_of_lt_two_pow_two_mul hw + (lt_of_lt_of_le hacc_next_lt_q + (Nat.pow_le_pow_right (by decide) (by omega))) + set s8 := s7.set 15 8 + have e8 : Exec w (.op (.const 15 8) : Code .exact 18) s7 s8 1 := Exec.const hfit_8 + set s9 := s8.set 15 (base + 8) + have e9 : Exec w (.op (.exactBin .add 15 0 15) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, hs7_0]) hfit_end8 + set s10 := s9.set 15 (base + 8 + m) + have e10 : Exec w (.op (.exactBin .add 15 15 3) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, hs7_3]) hfit_b8m + set s11 := s10.set 15 (c ⟨m, hm_lt⟩).val + have e11 : Exec w (.op (.load 15 15) : Code .exact 18) s10 s11 1 := by + simpa +decide [s11, s10, s9, s8, hmem_seed] using + (Exec.load (B := .exact) (w := w) (d := 15) (a := 15) (x := s10)) + set s12 := s11.set 2 (TrinomialField.equivFin t acc_next).val + have e12 : Exec w (.op (.exactBin .xor 2 2 15) : Code .exact 18) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, hs7_2, t, hxor_val]) hfit_acc_next + have hbody := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq e12)))))))))) + have hs12_low : ∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → s12.regs r = s5.regs r := by + intro r ⟨hr_lt, hr2⟩ + have hr15 : r ≠ 15 := by omega + calc s12.regs r = s7.regs r := by + simp only [s12, s11, s10, s9, s8, Data.set, + Function.update_of_ne hr2, Function.update_of_ne hr15] + _ = s5.regs r := hs7_low_eq r ⟨hr_lt, hr2⟩ + have hs12_mem : s12.mem = s.mem := by + change s7.mem = s.mem + rw [hs7_mem, hs6_mem] + rfl + have hm_le : m ≤ k := Nat.le_of_succ_le hm + obtain ⟨y, hrec, hy0, hy1, hy2, hy_mem⟩ := + ih hm_le acc_next s12 + (by rw [hs12_mem]; exact ⟨htab, hseed⟩) + (by rw [hs12_low 0 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h0]) + (by rw [hs12_low 1 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h1]) + (by simp only [s12, Data.set, Function.update_self]) + (by rw [hs12_low 3 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1]) + (by rw [hs12_low 4 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h4]) + (by rw [hs12_low 5 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h5]) + (by rw [hs12_low 6 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h6]) + (by rw [hs12_low 7 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h7]) + (by rw [hs12_low 8 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h8]) + (by rw [hs12_low 9 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h9]) + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, hy0, hy1, ?_, by rw [hy_mem, hs12_mem]⟩ + · simpa only [hornerLoop, hornerBody, + show 1 + (1 + (1 + (1 + (1 + (((11 * L + 12) * L + 2) + + (16 + (1 + (1 + (1 + (1 + 1)))))))))) + (((11 * L + 12) * L + 30) * m + 2) + 2 = + ((11 * L + 12) * L + 30) * (m + 1) + 2 by ring] using hiter + · rw [hy2] + congr 2 + rw [Fin.sum_univ_castSucc] + simp only [acc_next, Fin.val_castSucc, Fin.val_last, pow_succ] + have hcast : ∀ i : Fin m, Fin.castLE hm i.castSucc = Fin.castLE hm_le i := + fun _ => rfl + have hlast : Fin.castLE hm (Fin.last m) = ⟨m, hm_lt⟩ := rfl + simp only [hcast, hlast] + ring + +/-- 13-instruction setup block for `evalCode k`: initializes `reg 8 = 1`, `reg 9 = 0`, +`reg 2 = 0`, `reg 3 = k`, loads `b`, `2 ^ b - 1`, `L` from `base + 5 .. base + 7` into `reg 4..6`, +and sets `reg 7 = base + 8 + k`. -/ +@[expose] def evalSetupCode (k : ℕ) : Code .exact 18 := + .seq (.op (.const 8 1)) + (.seq (.op (.const 9 0)) + (.seq (.op (.copy 2 9)) + (.seq (.op (.const 3 k)) + (.seq (.op (.const 15 5)) + (.seq (.op (.exactBin .add 15 0 15)) + (.seq (.op (.load 4 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 5 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 6 15)) + (.seq (.op (.const 7 (8 + k))) + (.op (.exactBin .add 7 0 7))))))))))))) + +/-- 4-instruction suffix for `evalCode k`: loads `2 ^ M - 1` from `base + 4` and masks `reg 2`. -/ +@[expose] def evalMaskCode : Code .exact 18 := + .seq (.op (.const 15 4)) + (.seq (.op (.exactBin .add 15 0 15)) + (.seq (.op (.load 15 15)) + (.op (.exactBin .and 2 2 15)))) + +/-- Deterministic evaluation program `evalCode k`: given `reg 0 = base` and `reg 1 = u.val`, +evaluates `trinomialPolyHash (trinomialStep U M) k U M c u` into `reg 2` (`regOut`) in +`evalSteps b U M k` instructions while preserving `reg 0 = base`, `reg 1 = u.val`, and +all memory. -/ +@[expose] def evalCode (k : ℕ) : Code .exact 18 := + .seq (evalSetupCode k) (.seq hornerLoop evalMaskCode) + +/-- Exact instruction count of `evalCode k` for chunk width `b`, universe `U`, output bits `M`, +and independence parameter `k`. -/ +@[expose] def evalStepsFromB (b U M k : ℕ) : ℕ := + let L := limbCountFromB b (trinomialDeg U M) + ((11 * L + 12) * L + 30) * k + 19 + +/-- Exact instruction count of `evalCode k` for capacity `n_max`, universe `U`, output bits `M`, +and independence parameter `k`. -/ +@[expose] def evalSteps (n_max U M k : ℕ) : ℕ := + evalStepsFromB (chunkBits n_max) U M k + +/-- `evalSetupCode k` executes deterministically in `13` steps. -/ +theorem evalSetupCode_exec {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) : + ∃ y : Data 18, + Exec w (evalSetupCode k) x y 13 ∧ + y.regs 0 = base ∧ + y.regs 1 = x.regs 1 ∧ + y.regs 2 = 0 ∧ + y.regs 3 = k ∧ + y.regs 4 = b ∧ + y.regs 5 = 2 ^ b - 1 ∧ + y.regs 6 = limbCountFromB b (trinomialDeg U M) ∧ + y.regs 7 = base + 8 + k ∧ + y.regs 8 = 1 ∧ + y.regs 9 = 0 ∧ + y.mem = x.mem := by + have hfit_1 : Fits w 1 := hfit_end.of_le (by omega) + have hfit_0 : Fits w 0 := hfit_end.of_le (by omega) + have hfit_k : Fits w k := hfit_end.of_le (by omega) + have hfit_5 : Fits w 5 := hfit_end.of_le (by omega) + have hfit_b5 : Fits w (base + 5) := hfit_end.of_le (by omega) + have hfit_b6 : Fits w (base + 6) := hfit_end.of_le (by omega) + have hfit_b7 : Fits w (base + 7) := hfit_end.of_le (by omega) + have hfit_8k : Fits w (8 + k) := hfit_end.of_le (by omega) + have hm5 : x.mem (base + 5) = b := htab.2.2.2.2.2.1 + have hm6 : x.mem (base + 6) = 2 ^ b - 1 := htab.2.2.2.2.2.2.1 + have hm7 : x.mem (base + 7) = limbCountFromB b (trinomialDeg U M) := htab.2.2.2.2.2.2.2.1 + set s1 := x.set 8 1 + have e1 : Exec w (.op (.const 8 1) : Code .exact 18) x s1 1 := Exec.const hfit_1 + set s2 := s1.set 9 0 + have e2 : Exec w (.op (.const 9 0) : Code .exact 18) s1 s2 1 := Exec.const hfit_0 + set s3 := s2.set 2 0 + have e3 : Exec w (.op (.copy 2 9) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2] using (Exec.copy (B := .exact) (w := w) (d := 2) (s := 9) (x := s2)) + set s4 := s3.set 3 k + have e4 : Exec w (.op (.const 3 k) : Code .exact 18) s3 s4 1 := Exec.const hfit_k + set s5 := s4.set 15 5 + have e5 : Exec w (.op (.const 15 5) : Code .exact 18) s4 s5 1 := Exec.const hfit_5 + set s6 := s5.set 15 (base + 5) + have e6 : Exec w (.op (.exactBin .add 15 0 15) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h0]) hfit_b5 + set s7 := s6.set 4 b + have e7 : Exec w (.op (.load 4 15) : Code .exact 18) s6 s7 1 := by + simpa +decide [s7, s6, s5, s4, s3, s2, s1, hm5] using + (Exec.load (B := .exact) (w := w) (d := 4) (a := 15) (x := s6)) + set s8 := s7.set 15 (base + 6) + have e8 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, Nat.add_assoc]) hfit_b6 + set s9 := s8.set 5 (2 ^ b - 1) + have e9 : Exec w (.op (.load 5 15) : Code .exact 18) s8 s9 1 := by + simpa +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, hm6] using + (Exec.load (B := .exact) (w := w) (d := 5) (a := 15) (x := s8)) + set s10 := s9.set 15 (base + 7) + have e10 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, Nat.add_assoc]) hfit_b7 + set s11 := s10.set 6 (limbCountFromB b (trinomialDeg U M)) + have e11 : Exec w (.op (.load 6 15) : Code .exact 18) s10 s11 1 := by + simpa +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, hm7] using + (Exec.load (B := .exact) (w := w) (d := 6) (a := 15) (x := s10)) + set s12 := s11.set 7 (8 + k) + have e12 : Exec w (.op (.const 7 (8 + k)) : Code .exact 18) s11 s12 1 := Exec.const hfit_8k + set s13 := s12.set 7 (base + 8 + k) + have e13 : Exec w (.op (.exactBin .add 7 0 7) : Code .exact 18) s12 s13 1 := + exec_exactBin + (by simp +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0, Nat.add_assoc]) + hfit_end + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq e13))))))))))) + refine ⟨s13, hall, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, rfl⟩ + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4] + · simp +decide [s13, s12, s11, s10, s9, s8, s7] + · simp +decide [s13, s12, s11, s10, s9] + · simp +decide [s13, s12, s11] + · simp +decide [s13] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2] + +/-- **`KWise.evalCode k` correctness and complexity (general `b`)**: from +`Rep base b U M k c x.mem`, `x.regs 0 = base`, and `x.regs 1 = u.val` (`u : Fin U`), `evalCode k` +halts deterministically in `evalStepsFromB b U M k = O(k * L ^ 2 + 1)` instructions, writing +`(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `reg 2` (`regOut`) and preserving +`reg 0 = base`, `reg 1 = u.val`, and all memory (`y.mem = x.mem`). -/ +theorem evalCode_execWithin_fromB {w base b U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} (u : Fin U) {x : Data 18} + (hb : 0 < b) (hrep : Rep base b U M k c x.mem) + (h0 : x.regs 0 = base) (h1 : x.regs 1 = u.val) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + spaceFromB b k)) : + ∃ y : Data 18, + ExecWithin w (evalCode k) x y (evalStepsFromB b U M k) ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = (trinomialPolyHash (trinomialStep U M) k U M c u).val ∧ + y.mem = x.mem := by + have hpow_pos : 0 < 2 ^ (2 * b) := by positivity + have hfit_end : Fits w (base + 8 + k) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + obtain ⟨d1, e1, hd1_0, hd1_1, hd1_2, hd1_3, hd1_4, hd1_5, hd1_6, hd1_7, + hd1_8, hd1_9, hd1_mem⟩ := + evalSetupCode_exec hrep.1 h0 hfit_end + rw [h1] at hd1_1 + set t := trinomialStep U M + have hzero_val : (TrinomialField.equivFin t 0).val = 0 := by + have h0_eq : (0 : TrinomialField t) = AdjoinRoot.mk (trinomial t) (polyOfCode 0) := by + simp + rw [h0_eq] + exact TrinomialField.val_equivFin_mk_polyOfCode t (by positivity) + have hd1_2' : d1.regs 2 = (TrinomialField.equivFin t 0).val := by + rw [hd1_2, hzero_val] + obtain ⟨d2, e2, hd2_0, hd2_1, hd2_2, hd2_mem⟩ := + hornerLoop_exec u hb hw hfit_space k le_rfl 0 d1 + (by rw [hd1_mem]; exact hrep) + hd1_0 hd1_1 hd1_2' hd1_3 hd1_4 hd1_5 hd1_6 hd1_7 hd1_8 hd1_9 + have hpoly_eq : (0 : TrinomialField t) * + (AdjoinRoot.mk (trinomial t) (polyOfCode u.val)) ^ k + + ∑ i : Fin k, (TrinomialField.equivFin t).symm (c (Fin.castLE le_rfl i)) * + (AdjoinRoot.mk (trinomial t) (polyOfCode u.val)) ^ (i : ℕ) = + polynomialHash (fun j => (TrinomialField.equivFin t).symm (c j)) + (AdjoinRoot.mk (trinomial t) (polyOfCode u.val)) := by + rw [zero_mul, zero_add] + rfl + have hd2_eval : d2.regs 2 = (trinomialPolyEval t k U c u).val := by + rw [hd2_2, hpoly_eq] + rfl + have hfit_4 : Fits w 4 := hfit_end.of_le (by omega) + have hfit_b4 : Fits w (base + 4) := hfit_end.of_le (by omega) + have hm4 : d2.mem (base + 4) = 2 ^ M - 1 := by + rw [hd2_mem, hd1_mem] + exact hrep.1.2.2.2.2.1 + have heval_lt : + (trinomialPolyEval t k U c u).val &&& (2 ^ M - 1) < 2 ^ (2 * trinomialDeg U M) := + lt_of_le_of_lt Nat.and_le_left + (lt_of_lt_of_le (trinomialPolyEval t k U c u).isLt + (Nat.pow_le_pow_right (by decide) (by dsimp only [trinomialDeg, t]; omega))) + have hfit_out : Fits w ((trinomialPolyEval t k U c u).val &&& (2 ^ M - 1)) := + fits_of_lt_two_pow_two_mul hw heval_lt + set s1 := d2.set 15 4 + have em1 : Exec w (.op (.const 15 4) : Code .exact 18) d2 s1 1 := Exec.const hfit_4 + set s2 := s1.set 15 (base + 4) + have em2 : Exec w (.op (.exactBin .add 15 0 15) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, hd2_0]) hfit_b4 + set s3 := s2.set 15 (2 ^ M - 1) + have em3 : Exec w (.op (.load 15 15) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, hm4] using + (Exec.load (B := .exact) (w := w) (d := 15) (a := 15) (x := s2)) + set s4 := s3.set 2 ((trinomialPolyEval t k U c u).val &&& (2 ^ M - 1)) + have em4 : Exec w (.op (.exactBin .and 2 2 15) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, hd2_eval]) hfit_out + have e3 := em1.seq (em2.seq (em3.seq em4)) + have hall := e1.seq (e2.seq e3) + refine ⟨s4, ⟨_, ?_, hall⟩, ?_, ?_, ?_, ?_⟩ + · dsimp only [evalStepsFromB] + omega + · simp +decide [s4, s3, s2, s1, hd2_0] + · simp +decide [s4, s3, s2, s1, hd2_1] + · simp only [s4, Data.set, Function.update_self, t] + exact (val_trinomialPolyHash (trinomialStep U M) k U M c u).symm + · change d2.mem = x.mem + rw [hd2_mem, hd1_mem] + +/-- **`KWise.evalCode k` correctness and complexity**: from +`Rep base (chunkBits n_max) U M k c x.mem`, `x.regs 0 = base`, and `x.regs 1 = u.val` +(`u : Fin U`), `evalCode k` halts deterministically in `evalSteps n_max U M k = O(k * L ^ 2 + 1)` +instructions, writing `(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `reg 2` +(`regOut`) and preserving `reg 0 = base`, `reg 1 = u.val`, and all memory (`y.mem = x.mem`). -/ +theorem evalCode_execWithin {w base n_max U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} (u : Fin U) {x : Data 18} + (hrep : Rep base (chunkBits n_max) U M k c x.mem) + (h0 : x.regs 0 = base) (h1 : x.regs 1 = u.val) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) : + ∃ y : Data 18, + ExecWithin w (evalCode k) x y (evalSteps n_max U M k) ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = (trinomialPolyHash (trinomialStep U M) k U M c u).val ∧ + y.mem = x.mem := + evalCode_execWithin_fromB u (chunkBits_pos n_max) hrep h0 h1 hw hfit_space + +/-- **End-to-end Word-RAM `k`-wise independent hash theorem (`isKWiseIndependentHash_eval`)**: +1. The hash family `trinomialPolyHash (trinomialStep U M) k U M` evaluated by `evalCode k` is + `k`-wise independent onto `Fin (2 ^ M)` (`IsKWiseIndependentHash k`). +2. From any machine state `x` satisfying `TableRep base (chunkBits n_max) U M k x.mem` and + `x.regs 0 = base`, `resampleCode k` runs in `5 * k + 10` instructions and `k` `Code.rand` + queries with distribution `resampleComp base U M k x`, whose extracted seed + `extractSeeds base U M k z.1.mem` is uniformly distributed over + `Fin k → Fin (2 ^ trinomialDeg U M)` (`PMF.uniformOfFintype`). +3. Every resampled state `y` produced by `resampleCode k` satisfies `Rep`, so for every key + `u : Fin U`, running `evalCode k` from any state `x_eval` with `x_eval.mem = y.mem`, + `x_eval.regs 0 = base`, and `x_eval.regs 1 = u.val` halts deterministically in + `evalSteps n_max U M k` instructions, preserving all memory and writing + `(trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val` to + `reg 2`. -/ +theorem isKWiseIndependentHash_eval {w base n_max U M k : ℕ} {x : Data 18} + (htab : TableRep base (chunkBits n_max) U M k x.mem) + (h0 : x.regs 0 = base) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) : + IsKWiseIndependentHash k (trinomialPolyHash (trinomialStep U M) k U M) ∧ + ExecRand w (resampleCode k) x (resampleComp base U M k x) ∧ + ((resampleComp base U M k x).eval).map (fun z => extractSeeds base U M k z.1.mem) = + PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M)) ∧ + ExecRandWithin w (resampleCode k) x + (fun y => + Rep base (chunkBits n_max) U M k (extractSeeds base U M k y.mem) y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → y.mem a = x.mem a) ∧ + (∀ u : Fin U, ∀ x_eval : Data 18, + x_eval.mem = y.mem → + x_eval.regs 0 = base → + x_eval.regs 1 = u.val → + ∃ y_eval : Data 18, + ExecWithin w (evalCode k) x_eval y_eval (evalSteps n_max U M k) ∧ + y_eval.regs 0 = base ∧ + y_eval.regs 1 = u.val ∧ + y_eval.regs 2 = + (trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val ∧ + y_eval.mem = y.mem)) + (5 * k + 10) k := by + have hpow_pos : 0 < 2 ^ (2 * chunkBits n_max) := by positivity + have hsp : space n_max k = 8 + k + 2 ^ (2 * chunkBits n_max) := rfl + have hfit_end : Fits w (base + 8 + k) := hfit_space.of_le (by omega) + have hfit_q : Fits w (2 ^ trinomialDeg U M) := by + have hlt : 2 ^ trinomialDeg U M < 2 ^ (2 * trinomialDeg U M) := by + apply (Nat.pow_lt_pow_iff_right one_lt_two).mpr + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + dsimp only [trinomialDeg] + omega + exact fits_of_lt_two_pow_two_mul hw hlt + obtain ⟨m, he_rand, hqb, hsupp⟩ := resampleCode_execRandWithin htab h0 hfit_end hfit_q + refine ⟨isKWiseIndependentHash_trinomialPolyHash + (le_two_pow_trinomialDeg U M) (le_trinomialDeg U M), + resampleCode_execRand htab h0 hfit_end hfit_q, + map_extractSeeds_eval_resampleComp base U M k x, + ⟨m, he_rand, hqb, ?_⟩⟩ + intro z hz + obtain ⟨⟨hrep, hz0, hzout⟩, hsteps⟩ := hsupp z hz + refine ⟨⟨hrep, hz0, hzout, ?_⟩, hsteps⟩ + intro u x_eval hmem he0 he1 + have hrep_eval : Rep base (chunkBits n_max) U M k (extractSeeds base U M k z.1.mem) x_eval.mem := + hmem ▸ hrep + obtain ⟨y_eval, heval, hy0, hy1, hy2, hymem⟩ := + evalCode_execWithin u hrep_eval he0 he1 hw hfit_space + exact ⟨y_eval, heval, hy0, hy1, hy2, hymem.trans hmem⟩ + +end Cslib.Algorithms.WordRAM.Hashing.KWise diff --git a/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean new file mode 100644 index 000000000..d6aba6086 --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean @@ -0,0 +1,1565 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Computability.WordRAM.Randomized.Structured +public import Complexity.Probability.PairwiseIndependentHash.Shift +public import Complexity.Probability.Product +public import Complexity.Probability.Uniform +public import Mathlib.Data.Nat.Log + +/-! +# Self-contained Word-RAM random tables and simple tabulation hashing + +This file provides two self-contained Word-RAM hashing primitives over `Cslib.WordRAM.Basis.exact`, +built on a shared uniform word-slice sampler (`randFillLoop`): + +1. **`RandomTable`**: initializes or resamples a table `Z[0 .. 2 ^ M - 1]` of `m = 2 ^ M` + independent uniform `M`-bit words in `O(2 ^ M)` instructions and `2 ^ M` calls to `Code.rand` + (`PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`), and supports `O(1)` deterministic table + lookup `Z[idx]` as well as XOR-shifted evaluation `f(u) ^^^ Z[g(u)]` + [Carter and Wegman][CarterWegman1979]; [Mitzenmacher and Upfal][MitzenmacherUpfal2017]. +2. **`TabulatedOddStride`**: `d`-character simple tabulation with character bit-width + `b = max 1 (Nat.log 2 (max n_max 2) / 2)` (`2 ^ b ≤ max 2 n_max` words per character table), + initialized via `Code.rand` in `O(d * 2 ^ b + log n_max) = O(n_max)` steps and evaluated in + `O(d)` deterministic instructions, producing an odd stride + `(2 * simpleTab(u) + 1) &&& (2 ^ M - 1)` coprime to `2 ^ M` whose linear probe sequence + `(h₁(u) + i * h₂(u)) % 2 ^ M` is a full-cycle permutation of `Fin (2 ^ M)` + [Cormen, Leiserson, Rivest, and Stein, Section 11.4][CLRS2009]. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.randFillLoop`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.randFillComp`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.extractSlice`: generic loop filling a contiguous + memory slice with independent uniform `bits`-bit words. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.resampleCode`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.lookupCode`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.xorLookupCode`: 1-D random table + initialization/resampling and `O(1)` deterministic lookup. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.charBits`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.simpleTab`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.oddStride`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.probeSeq`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.initCode`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.evalCode`: `d`-character simple + tabulation and odd-stride permutation generator on the Word RAM. + +## Main results + +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.randFillLoop_execRand`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.map_extractSlice_eval_randFillComp`: exact execution + and product uniformity of `randFillLoop`. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.resampleCode_execRand`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.map_extractTable_eval_resampleComp`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.lookupCode_exec`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.xorLookupCode_exec`: exact execution and + uniformity of `RandomTable`. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.bijective_probeSeq`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.initCode_execRandWithin`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.evalCode_execWithin`: full-cycle + permutation property, `O(d * 2 ^ b)` initialization, and `O(d)` deterministic evaluation of + `TabulatedOddStride`. + +## References + +* [J. L. Carter and M. N. Wegman, *Universal Classes of Hash Functions*][CarterWegman1979] +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, + *Introduction to Algorithms*][CLRS2009] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.Hashing.Tabulation + +open Cslib.WordRAM Cslib.Probability +open scoped ENNReal + +/-! ## Exact-basis instruction helpers -/ + +/-- Execute a single `.exactBin` operation when `evalExact` returns `some v` and `Fits w v`. -/ +theorem exec_exactBin {w r : ℕ} {f : BinOp} {dst i j : Fin r} {d : Data r} {v : ℕ} + (hev : f.evalExact (d.regs i) (d.regs j) = some v) (hfit : Fits w v) : + Exec w (.op (.exactBin f dst i j) : Code .exact r) d (d.set dst v) 1 := by + apply Exec.op + rw [Op.eval_exactBin, hev] + exact Data.put_of_fits hfit + +/-- 3-instruction equal-cost branch setting register `dst` to `max (x.regs a) (x.regs b)`. -/ +@[expose] def maxCode {n : ℕ} (dst a b : Fin n) : Code .exact n := + .branch (.lt a b) (.seq (.op (.copy dst b)) (.op (.copy dst dst))) (.op (.copy dst a)) + +/-- `maxCode dst a b` executes deterministically in 3 instructions and sets `dst` to +`max (x.regs a) (x.regs b)`. -/ +theorem maxCode_exec {w n : ℕ} (dst a b : Fin n) (x : Data n) : + Exec w (maxCode dst a b) x (x.set dst (max (x.regs a) (x.regs b))) 3 := by + by_cases hlt : x.regs a < x.regs b + · have hc : (Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs b := Nat.max_eq_right (le_of_lt hlt) + rw [hmax] + set d1 := x.set dst (x.regs b) + have s1 : Exec w (.op (.copy dst b) : Code .exact n) x d1 1 := Exec.copy + have hd1 : d1.set dst (d1.regs dst) = d1 := by + ext i : 2 <;> simp [d1, Data.set] + have s2 : Exec w (.op (.copy dst dst) : Code .exact n) d1 d1 1 := by + simpa only [hd1] using (Exec.copy (B := .exact) (w := w) (d := dst) (s := dst) (x := d1)) + exact Exec.branch_pos hc (s1.seq s2) + · have hc : ¬(Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs a := Nat.max_eq_left (not_lt.mp hlt) + rw [hmax] + exact Exec.branch_neg hc (Exec.copy (B := .exact)) + +/-! ## Section 1: Uniform random word-slice filler (`randFillLoop`) -/ + +/-- Register `0` of `randFillLoop`: write pointer `ptr`. -/ +abbrev regPtr : Fin 6 := 0 + +/-- Register `1` of `randFillLoop`: end pointer `arrBase + len`. -/ +abbrev regEnd : Fin 6 := 1 + +/-- Register `2` of `randFillLoop`: random bitmask `2 ^ bits - 1`. -/ +abbrev regMask : Fin 6 := 2 + +/-- Register `3` of `randFillLoop`: constant `1`. -/ +abbrev regOne : Fin 6 := 3 + +/-- Register `4` of `randFillLoop`: sampled value scratch register. -/ +abbrev regVal : Fin 6 := 4 + +/-- Register `5` of `randFillLoop`: caller-preserved register. -/ +abbrev regExtra : Fin 6 := 5 + +/-- Body of `randFillLoop`: samples `regVal ← rand regMask`, stores `*regPtr ← regVal`, and +advances `regPtr ← regPtr + regOne`. -/ +@[expose] def randFillBody : Code .exact 6 := + .seq (.rand regVal regMask) + (.seq (.op (.store regPtr regVal)) + (.op (.exactBin .add regPtr regPtr regOne))) + +/-- Structured Word-RAM loop that fills `[regPtr, regEnd)` with independent uniform draws in +`[0, regMask]`. -/ +@[expose] def randFillLoop : Code .exact 6 := + .loop (.lt regPtr regEnd) randFillBody + +/-- Single-iteration state update of `randFillBody` when `Code.rand` draws `v`. -/ +@[expose] def randFillStepData (arrBase v : ℕ) (x : Data 6) : Data 6 := + ((x.set regVal v).write arrBase v).set regPtr (arrBase + 1) + +/-- Explicit `ProbComp (Data 6 × ℕ)` induced by `randFillLoop` over `len` words starting at +`arrBase`. -/ +@[expose] def randFillComp (bits arrBase : ℕ) : ℕ → Data 6 → ProbComp (Data 6 × ℕ) + | 0, x => pure (x, 2) + | len + 1, x => do + let k ← ProbComp.unifFin (2 ^ bits - 1) + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) + pure (z.1, 3 + z.2 + 2) + +/-- Extract a `< 2 ^ bits` word as an element of `Fin (2 ^ bits)`, defaulting to `0` out of +range. -/ +@[expose] def extractWord (bits v : ℕ) : Fin (2 ^ bits) := + if h : v < 2 ^ bits then ⟨v, h⟩ else ⟨0, by positivity⟩ + +/-- Extract a contiguous slice of `len` words at `arrBase` as a tuple `Fin len → Fin (2 ^ bits)`. -/ +@[expose] def extractSlice (arrBase bits len : ℕ) (mem : ℕ → ℕ) : Fin len → Fin (2 ^ bits) := + fun i => extractWord bits (mem (arrBase + i.val)) + +@[simp] +theorem extractWord_val_of_lt {bits v : ℕ} (hv : v < 2 ^ bits) : + (extractWord bits v).val = v := by + simp [extractWord, hv] + +@[simp] +theorem extractWord_unifFin (bits : ℕ) (k : Fin (2 ^ bits - 1 + 1)) : + extractWord bits k.val = finCongr (Nat.sub_add_cancel Nat.one_le_two_pow) k := by + have hlt : k.val < 2 ^ bits := by + have hpos : 1 ≤ 2 ^ bits := Nat.one_le_two_pow + have hk := k.isLt + omega + apply Fin.ext + simp [extractWord, hlt] + +/-- One iteration of `randFillBody` executes in `3` instructions and `1` `rand` call. -/ +theorem randFillBody_execRand {w bits arrBase : ℕ} {x : Data 6} + (hptr : x.regs regPtr = arrBase) (hmask : x.regs regMask = 2 ^ bits - 1) + (hone : x.regs regOne = 1) (hfit_ptr : Fits w (arrBase + 1)) + (hfit_mask : Fits w (2 ^ bits - 1)) : + ExecRand w randFillBody x (do + let k ← ProbComp.unifFin (2 ^ bits - 1) + pure (randFillStepData arrBase k.val x, 3)) := by + have hfit_src : Fits w (x.regs regMask) := by rwa [hmask] + have h1 := ExecRand.rand (B := .exact) (w := w) (dst := regVal) (src := regMask) hfit_src + rw [hmask] at h1 + have h23 : ∀ z ∈ (do + let k ← ProbComp.unifFin (2 ^ bits - 1) + pure (x.set regVal k.val, 1) : ProbComp (Data 6 × ℕ)).support, + ExecRand w (.seq (.op (.store regPtr regVal)) + (.op (.exactBin .add regPtr regPtr regOne))) z.1 + (pure (randFillStepData arrBase (z.1.regs regVal) x, 2)) := by + intro z hz + obtain ⟨k, -, hk⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + cases OracleComp.eq_of_mem_supportWith_pure hk + set d1 := x.set regVal k.val + have hd1_ptr : d1.regs regPtr = arrBase := by simp [d1, hptr, regPtr, regVal] + have hd1_val : d1.regs regVal = k.val := by simp [d1] + have s2 : Exec w (.op (.store regPtr regVal) : Code .exact 6) + d1 (d1.write arrBase k.val) 1 := by + simpa [hd1_ptr, hd1_val] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regVal) (x := d1)) + set d2 := d1.write arrBase k.val + have s3 : Exec w (.op (.exactBin .add regPtr regPtr regOne) : Code .exact 6) + d2 (d2.set regPtr (arrBase + 1)) 1 := + exec_exactBin (by simp [d2, d1, hptr, hone, regPtr, regOne, regVal]) hfit_ptr + simpa [randFillStepData, d2, d1, hd1_val] using (s2.seq s3).execRand + have hseq := ExecRand.seq + (m₂ := fun d => pure (randFillStepData arrBase (d.regs regVal) x, 2)) h1 h23 + simpa only [randFillBody, bind_assoc, pure_bind, Data.set_regs, Function.update_self] using hseq + +/-- **Exact `ExecRand` semantics of `randFillLoop`** over `len` words starting at `arrBase`. -/ +theorem randFillLoop_execRand {w bits arrBase : ℕ} (len : ℕ) {x : Data 6} + (hptr : x.regs regPtr = arrBase) (hend : x.regs regEnd = arrBase + len) + (hmask : x.regs regMask = 2 ^ bits - 1) (hone : x.regs regOne = 1) + (hfit_end : Fits w (arrBase + len)) (hfit_mask : Fits w (2 ^ bits - 1)) : + ExecRand w randFillLoop x (randFillComp bits arrBase len x) := by + induction len generalizing arrBase x with + | zero => + have hc : ¬(Cond.lt regPtr regEnd).Holds x := by + simp [Cond.Holds, hptr, hend] + exact ExecRand.loop_exit hc + | succ len ih => + have hc : (Cond.lt regPtr regEnd).Holds x := by + simp [Cond.Holds, hptr, hend] + have hfit_ptr1 : Fits w (arrBase + 1) := hfit_end.of_le (by omega) + have hb := randFillBody_execRand hptr hmask hone hfit_ptr1 hfit_mask + have hcont : ∀ z ∈ (do + let k ← ProbComp.unifFin (2 ^ bits - 1) + pure (randFillStepData arrBase k.val x, 3) : ProbComp (Data 6 × ℕ)).support, + ExecRand w randFillLoop z.1 (randFillComp bits (arrBase + 1) len z.1) := by + intro z hz + obtain ⟨k, -, hk⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + cases OracleComp.eq_of_mem_supportWith_pure hk + refine ih (arrBase := arrBase + 1) + (by simp +decide [randFillStepData, regPtr, regVal]) + (by simp +decide [randFillStepData, regEnd, regPtr, regVal, hend]; omega) + (by simp +decide [randFillStepData, regMask, regPtr, regVal, hmask]) + (by simp +decide [randFillStepData, regOne, regPtr, regVal, hone]) + (by rwa [show arrBase + 1 + len = arrBase + (len + 1) by omega]) + have hiter := ExecRand.loop_iter hc hb hcont + simpa only [randFillLoop, randFillComp, bind_assoc, pure_bind] using hiter + +/-- `randFillComp bits arrBase len x` makes at most `len` `rand` calls. -/ +theorem isQueryBounded_randFillComp (bits arrBase len : ℕ) (x : Data 6) : + (randFillComp bits arrBase len x).IsQueryBounded len := by + induction len generalizing arrBase x with + | zero => + exact .pure _ 0 + | succ len ih => + dsimp only [randFillComp] + rw [Nat.add_comm len 1] + refine (ProbComp.isQueryBounded_unifFin (2 ^ bits - 1)).bind fun k => ?_ + simpa only [Nat.add_zero] using + (ih (arrBase + 1) (randFillStepData arrBase k.val x)).bind fun _ => .pure _ 0 + +/-- Pathwise postcondition of `randFillComp`: takes `5 * len + 2` instructions, preserves registers +`regEnd`, `regMask`, `regOne`, `regExtra`, fills `[arrBase, arrBase + len)` with `< 2 ^ bits` +words, and preserves all memory outside `[arrBase, arrBase + len)`. -/ +theorem mem_support_randFillComp {bits arrBase len : ℕ} {x : Data 6} {z : Data 6 × ℕ} + (hz : z ∈ (randFillComp bits arrBase len x).support) : + z.2 = 5 * len + 2 ∧ + z.1.regs regPtr = (if len = 0 then x.regs regPtr else arrBase + len) ∧ + z.1.regs regEnd = x.regs regEnd ∧ + z.1.regs regMask = x.regs regMask ∧ + z.1.regs regOne = x.regs regOne ∧ + z.1.regs regExtra = x.regs regExtra ∧ + (∀ i < len, z.1.mem (arrBase + i) < 2 ^ bits) ∧ + (∀ a, (a < arrBase ∨ arrBase + len ≤ a) → z.1.mem a = x.mem a) := by + induction len generalizing arrBase x z with + | zero => + dsimp only [randFillComp] at hz + cases OracleComp.eq_of_mem_supportWith_pure hz + refine ⟨rfl, by simp, rfl, rfl, rfl, rfl, fun _ hi => by omega, fun _ _ => rfl⟩ + | succ len ih => + dsimp only [randFillComp] at hz + obtain ⟨k, -, hrest⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + obtain ⟨z', hz', hpure⟩ := OracleComp.mem_supportWith_bind_iff.mp hrest + cases OracleComp.eq_of_mem_supportWith_pure hpure + obtain ⟨hsteps, hptr, hend, hmask, hone, hextra, hlt, hframe⟩ := ih hz' + have hk_lt : k.val < 2 ^ bits := by + have hpos : 1 ≤ 2 ^ bits := Nat.one_le_two_pow + have hk := k.isLt + omega + refine ⟨by omega, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simp only [Nat.succ_ne_zero, ↓reduceIte] + rcases eq_or_ne len 0 with rfl | hlen + · simp [hptr, randFillStepData] + · rw [hptr, ite_eq_right hlen] + omega + · simpa [randFillStepData, regEnd, regPtr, regVal] using hend + · simpa [randFillStepData, regMask, regPtr, regVal] using hmask + · simpa [randFillStepData, regOne, regPtr, regVal] using hone + · simpa [randFillStepData, regExtra, regPtr, regVal] using hextra + · intro i hi + rcases eq_or_ne i 0 with rfl | hi0 + · have hmem0 := hframe arrBase (Or.inl (by omega)) + simpa [randFillStepData, hmem0] using hk_lt + · have hlt' := hlt (i - 1) (by omega) + rwa [show arrBase + 1 + (i - 1) = arrBase + i by omega] at hlt' + · intro a ha + have h1 := hframe a (by omega) + have hne : a ≠ arrBase := by omega + simpa [randFillStepData, Function.update_of_ne hne] using h1 + +/-- Extracting `len + 1` words at `arrBase` after one `randFillStepData` step and the remaining +`len` steps equals `Fin.cons` of the first draw and the remaining `len`-word slice. -/ +private theorem extractSlice_succ_of_support {bits arrBase len : ℕ} {x : Data 6} + (k : Fin (2 ^ bits - 1 + 1)) {z : Data 6 × ℕ} + (hz : z ∈ (randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x)).support) : + extractSlice arrBase bits (len + 1) z.1.mem = + Fin.cons (finCongr (Nat.sub_add_cancel Nat.one_le_two_pow) k) + (extractSlice (arrBase + 1) bits len z.1.mem) := by + obtain ⟨-, -, -, -, -, -, -, hframe⟩ := mem_support_randFillComp hz + funext j + refine Fin.cases ?_ ?_ j + · have h0 := hframe arrBase (Or.inl (by omega)) + simp [extractSlice, randFillStepData, h0] + · intro i + simp only [extractSlice, Fin.cons_succ, Fin.val_succ] + congr 2 + omega + +/-- **Uniformity of `randFillComp`**: the slice `extractSlice arrBase bits len z.1.mem` produced by +`randFillComp bits arrBase len x` has the exact product uniform distribution +`PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))`. -/ +theorem map_extractSlice_eval_randFillComp (bits arrBase len : ℕ) (x : Data 6) : + ((randFillComp bits arrBase len x).eval).map (fun z => extractSlice arrBase bits len z.1.mem) = + PMF.uniformOfFintype (Fin len → Fin (2 ^ bits)) := by + induction len generalizing arrBase x with + | zero => + ext f + have heq : f = extractSlice arrBase bits 0 x.mem := Subsingleton.elim _ _ + simp [randFillComp, ProbComp.eval_pure, PMF.map, PMF.pure_apply, heq, + PMF.uniformOfFintype_apply] + | succ len ih => + let e := finCongr (Nat.sub_add_cancel (Nat.one_le_two_pow (n := bits))) + have hstep : ((randFillComp bits arrBase (len + 1) x).eval).map + (fun z => extractSlice arrBase bits (len + 1) z.1.mem) = + (PMF.uniformOfFintype (Fin (2 ^ bits))).bind (fun first => + (PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))).map (Fin.cons first)) := by + dsimp only [randFillComp] + rw [ProbComp.eval_bind, ProbComp.eval_unifFin, PMF.map_bind] + have hbody : ∀ k : Fin (2 ^ bits - 1 + 1), + ((do + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) + pure (z.1, 3 + z.2 + 2)).eval).map + (fun z : Data 6 × ℕ => extractSlice arrBase bits (len + 1) z.1.mem) = + (PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))).map (Fin.cons (e k)) := by + intro k + have hd : (do + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) + pure (z.1, 3 + z.2 + 2)) = + (fun z : Data 6 × ℕ => (z.1, 3 + z.2 + 2)) <$> + randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) := + bind_pure_comp _ _ + rw [hd, ProbComp.eval_map, PMF.map_comp] + have hcongr : + ((randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x)).eval).map + ((fun z : Data 6 × ℕ => extractSlice arrBase bits (len + 1) z.1.mem) ∘ + fun z : Data 6 × ℕ => (z.1, 3 + z.2 + 2)) = + ((randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x)).eval).map + (Fin.cons (e k) ∘ + fun z : Data 6 × ℕ => extractSlice (arrBase + 1) bits len z.1.mem) := by + apply PMF.map_congr_on_support + intro z hz + rw [ProbComp.support_eval] at hz + exact extractSlice_succ_of_support (z := z) k hz + rw [hcongr, ← PMF.map_comp, ih (arrBase + 1) (randFillStepData arrBase k.val x)] + have hbind : (PMF.uniformOfFintype (Fin (2 ^ bits - 1 + 1))).bind + (fun a => + PMF.map (fun z : Data 6 × ℕ => extractSlice arrBase bits (len + 1) z.1.mem) + (do + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase a.val x) + pure (z.1, 3 + z.2 + 2)).eval) = + (PMF.uniformOfFintype (Fin (2 ^ bits - 1 + 1))).bind + (fun a => (PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))).map (Fin.cons (e a))) := by + congr 1 + funext k + exact hbody k + rw [hbind] + have heq := PMF.uniformOfFintype_map_equiv e + rw [← heq, PMF.bind_map] + rfl + rw [hstep, ← PMF.pi_uniformOfFintype, ← PMF.pi_fin_succ, PMF.pi_uniformOfFintype] + +/-! ## Section 2: `RandomTable` — 1-D random table for Cuckoo Hashing -/ + +namespace RandomTable + +/-- Footprint size of `RandomTable M`: `2 ^ M` words at `[base, base + 2 ^ M)`. -/ +@[expose] def space (M : ℕ) : ℕ := 2 ^ M + +/-- Register `0`: holds the base memory address `base` of the table. -/ +abbrev regBase : Fin 6 := 0 + +/-- Register `1`: holds `M` during `resampleCode`, or index `idx < 2 ^ M` during `lookupCode`. -/ +abbrev regM : Fin 6 := 1 + +/-- Alias for register `1` when used as the lookup index `idx < 2 ^ M`. -/ +abbrev regIdx : Fin 6 := 1 + +/-- Register `2`: receives `Z[idx]` in `lookupCode` or `fx ^^^ Z[idx]` in `xorLookupCode`. -/ +abbrev regOut : Fin 6 := 2 + +/-- Register `3`: scratch register 1. -/ +abbrev regT1 : Fin 6 := 3 + +/-- Register `4`: scratch register 2. -/ +abbrev regT2 : Fin 6 := 4 + +/-- Register `5`: scratch register 3. -/ +abbrev regT3 : Fin 6 := 5 + +/-- Memory representation invariant: `[base, base + 2 ^ M)` holds the entries of +`Z : Fin (2 ^ M) → Fin (2 ^ M)`. -/ +def Rep (base M : ℕ) (Z : Fin (2 ^ M) → Fin (2 ^ M)) (mem : ℕ → ℕ) : Prop := + ∀ i : Fin (2 ^ M), mem (base + i.val) = (Z i).val + +/-- Extract the random table `Z : Fin (2 ^ M) → Fin (2 ^ M)` from memory at `base`. -/ +@[expose] def extractTable (base M : ℕ) (mem : ℕ → ℕ) : Fin (2 ^ M) → Fin (2 ^ M) := + extractSlice base M (2 ^ M) mem + +/-- If every word in `[base, base + 2 ^ M)` is `< 2 ^ M`, then +`Rep base M (extractTable base M mem) mem` holds. -/ +theorem rep_extractTable {base M : ℕ} {mem : ℕ → ℕ} + (h : ∀ i < 2 ^ M, mem (base + i) < 2 ^ M) : + Rep base M (extractTable base M mem) mem := by + intro i + simp [extractTable, extractSlice, h i.val i.isLt] + +/-- Injective register permutation mapping `randFillLoop`'s registers into `RandomTable` so that +`regExtra` maps to `regBase` (preserving `regBase`). -/ +@[expose] def fillMap : Fin 6 → Fin 6 + | 0 => regT1 + | 1 => regT2 + | 2 => regT3 + | 3 => regOut + | 4 => regM + | 5 => regBase + +theorem injective_fillMap : Function.Injective fillMap := by + decide + +/-- Deterministic setup before `randFillLoop` in `resampleCode`: computes `2 ^ M`, `2 ^ M - 1`, +`base + 2 ^ M`, and initializes the loop registers. -/ +@[expose] def setupCode : Code .exact 6 := + .seq (.op (.const regOut 1)) + (.seq (.op (.exactBin .shiftLeft regT2 regOut regM)) + (.seq (.op (.exactBin .sub regT3 regT2 regOut)) + (.seq (.op (.copy regT1 regBase)) + (.op (.exactBin .add regT2 regBase regT2))))) + +/-- Closed-form machine data after `setupCode`. -/ +@[expose] def setupData (base M : ℕ) (x : Data 6) : Data 6 := + ((((x.set regOut 1).set regT2 (2 ^ M)).set regT3 (2 ^ M - 1)).set regT1 base).set + regT2 (base + 2 ^ M) + +/-- Structured Word-RAM program that initializes or resamples `Z[0 .. 2 ^ M - 1]` at `base` using +`2 ^ M` calls to `Code.rand` on `2 ^ M - 1`. -/ +@[expose] def resampleCode : Code .exact 6 := + .seq setupCode (randFillLoop.rename fillMap) + +/-- Alias for `resampleCode` when used for initial table construction. -/ +abbrev initCode : Code .exact 6 := resampleCode + +/-- Explicit `ProbComp (Data 6 × ℕ)` induced by `resampleCode` at `(base, M)` from `x`. -/ +@[expose] noncomputable def resampleComp (base M : ℕ) (x : Data 6) : ProbComp (Data 6 × ℕ) := + let s := setupData base M x + (fun z => (z.1.extend fillMap s, 5 + z.2)) <$> + randFillComp M base (2 ^ M) (s.comap fillMap) + +/-- `setupCode` executes deterministically in `5` instructions when `Fits w (base + 2 ^ M)` and +`Fits w (2 ^ M)`. -/ +theorem setupCode_exec {w base M : ℕ} {x : Data 6} + (hbase : x.regs regBase = base) (hM : x.regs regM = M) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + Exec w setupCode x (setupData base M x) 5 := by + have hpos : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hfit_1 : Fits w 1 := hfit_pow.of_le hpos + have hfit_mask : Fits w (2 ^ M - 1) := hfit_pow.of_le (Nat.sub_le _ _) + set d1 := x.set regOut 1 + have s1 : Exec w (.op (.const regOut 1) : Code .exact 6) x d1 1 := Exec.const hfit_1 + set d2 := d1.set regT2 (2 ^ M) + have s2 : Exec w (.op (.exactBin .shiftLeft regT2 regOut regM) : Code .exact 6) d1 d2 1 := + exec_exactBin (by simp [d1, hM, regOut, regM, Nat.shiftLeft_eq]) hfit_pow + set d3 := d2.set regT3 (2 ^ M - 1) + have s3 : Exec w (.op (.exactBin .sub regT3 regT2 regOut) : Code .exact 6) d2 d3 1 := + exec_exactBin (by simp [d2, d1, hpos, regT2, regOut]) hfit_mask + set d4 := d3.set regT1 base + have s4 : Exec w (.op (.copy regT1 regBase) : Code .exact 6) d3 d4 1 := by + simpa [d3, d2, d1, hbase, regBase, regT1, regT2, regT3, regOut] using + (Exec.copy (B := .exact) (w := w) (d := regT1) (s := regBase) (x := d3)) + set d5 := d4.set regT2 (base + 2 ^ M) + have s5 : Exec w (.op (.exactBin .add regT2 regBase regT2) : Code .exact 6) d4 d5 1 := + exec_exactBin (by simp [d4, d3, d2, d1, hbase, regBase, regT1, regT2, regT3, regOut]) hfit_end + exact s1.seq (s2.seq (s3.seq (s4.seq s5))) + +/-- **Exact `ExecRand` execution of `RandomTable.resampleCode`**: runs in `5 * 2 ^ M + 7` +instructions and `2 ^ M` `rand` calls. -/ +theorem resampleCode_execRand {w base M : ℕ} {x : Data 6} + (hbase : x.regs regBase = base) (hM : x.regs regM = M) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + ExecRand w resampleCode x (resampleComp base M x) := by + have h1 := setupCode_exec hbase hM hfit_end hfit_pow + set s := setupData base M x + have hfit_mask : Fits w (2 ^ M - 1) := hfit_pow.of_le (Nat.sub_le _ _) + have hloop := randFillLoop_execRand (w := w) (bits := M) (arrBase := base) (len := 2 ^ M) + (x := s.comap fillMap) + (by simp +decide [s, setupData, fillMap, Data.comap]) + (by simp +decide [s, setupData, fillMap, Data.comap]) + (by simp +decide [s, setupData, fillMap, Data.comap]) + (by simp +decide [s, setupData, fillMap, Data.comap]) + hfit_end hfit_mask + have hren := hloop.rename injective_fillMap + have hseq := h1.seqRand hren + rw [resampleCode] + simpa only [resampleComp, s, Functor.map_map] using hseq + +/-- **Uniformity of `RandomTable.resampleComp`**: the extracted table `extractTable base M z.1.mem` +is distributed according to `PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`. -/ +theorem map_extractTable_eval_resampleComp (base M : ℕ) (x : Data 6) : + ((resampleComp base M x).eval).map (fun z => extractTable base M z.1.mem) = + PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M)) := by + dsimp only [resampleComp, extractTable] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractSlice_eval_randFillComp M base (2 ^ M) ((setupData base M x).comap fillMap) + +/-- Hoare-style `ExecRandWithin` specification for `RandomTable.resampleCode`: runs in +`5 * 2 ^ M + 7` steps and `2 ^ M` `rand` calls, establishing `Rep`, preserving `regBase`, and +modifying only `[base, base + 2 ^ M)`. -/ +theorem resampleCode_execRandWithin {w base M : ℕ} {x : Data 6} + (hbase : x.regs regBase = base) (hM : x.regs regM = M) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + ExecRandWithin w resampleCode x + (fun y => Rep base M (extractTable base M y.mem) y.mem ∧ + y.regs regBase = base ∧ + (∀ a, (a < base ∨ base + 2 ^ M ≤ a) → y.mem a = x.mem a)) + (5 * 2 ^ M + 7) (2 ^ M) := by + refine ⟨resampleComp base M x, resampleCode_execRand hbase hM hfit_end hfit_pow, ?_, ?_⟩ + · dsimp only [resampleComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp M base (2 ^ M) _) + · intro z hz + dsimp only [resampleComp] at hz + obtain ⟨z₀, hz₀, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hframe⟩ := mem_support_randFillComp hz₀ + have hbase' : (z₀.1.extend fillMap (setupData base M x)).regs regBase = base := by + change ((z₀.1.extend fillMap (setupData base M x)).comap fillMap).regs regExtra = base + rw [Data.comap_extend injective_fillMap, hextra] + simpa +decide [setupData, fillMap, Data.comap] using hbase + exact ⟨⟨rep_extractTable hlt, hbase', hframe⟩, by omega⟩ + +/-- Structured Word-RAM program that reads `Z[regIdx]` at `regBase + regIdx` into `regOut` in `2` +deterministic instructions. -/ +@[expose] def lookupCode : Code .exact 6 := + .seq (.op (.exactBin .add regT1 regBase regIdx)) + (.op (.load regOut regT1)) + +/-- Final machine data after `lookupCode`. -/ +@[expose] def lookupData (x : Data 6) : Data 6 := + let addr := x.regs regBase + x.regs regIdx + (x.set regT1 addr).set regOut (x.mem addr) + +/-- `lookupCode` executes in `2` deterministic instructions, preserving memory and `regBase` and +writing `(Z idx).val` into `regOut`. -/ +theorem lookupCode_exec {w base M : ℕ} {Z : Fin (2 ^ M) → Fin (2 ^ M)} {x : Data 6} + (hrep : Rep base M Z x.mem) (idx : Fin (2 ^ M)) + (hbase : x.regs regBase = base) (hidx : x.regs regIdx = idx.val) + (hfit_end : Fits w (base + 2 ^ M)) : + Exec w lookupCode x (lookupData x) 2 ∧ + (lookupData x).regs regOut = (Z idx).val ∧ + (lookupData x).regs regBase = base ∧ + (lookupData x).mem = x.mem := by + have hfit_addr : Fits w (base + idx.val) := hfit_end.of_le (by have := idx.isLt; omega) + set d1 := x.set regT1 (base + idx.val) + have s1 : Exec w (.op (.exactBin .add regT1 regBase regIdx) : Code .exact 6) x d1 1 := + exec_exactBin (by simp [hbase, hidx]) hfit_addr + have s2 : Exec w (.op (.load regOut regT1) : Code .exact 6) d1 + (d1.set regOut (x.mem (base + idx.val))) 1 := by + simpa [d1] using (Exec.load (B := .exact) (w := w) (d := regOut) (a := regT1) (x := d1)) + have hd : d1.set regOut (x.mem (base + idx.val)) = lookupData x := by + simp [d1, lookupData, hbase, hidx] + rw [hd] at s2 + refine ⟨s1.seq s2, ?_, by simp [lookupData, hbase, regBase, regT1, regOut], rfl⟩ + simp [lookupData, hbase, hidx, hrep idx] + +/-- Structured Word-RAM program that computes `regOut ← regOut ^^^ Z[regIdx]` in `3` deterministic +instructions (computing `f(u) ^^^ Z[g(u)]` when `regOut` holds `f(u)` and `regIdx` holds +`g(u)`). -/ +@[expose] def xorLookupCode : Code .exact 6 := + .seq (.op (.exactBin .add regT1 regBase regIdx)) + (.seq (.op (.load regT2 regT1)) + (.op (.exactBin .xor regOut regOut regT2))) + +/-- Final machine data after `xorLookupCode`. -/ +@[expose] def xorLookupData (x : Data 6) : Data 6 := + let addr := x.regs regBase + x.regs regIdx + let zval := x.mem addr + ((x.set regT1 addr).set regT2 zval).set regOut (x.regs regOut ^^^ zval) + +/-- `xorLookupCode` executes in `3` deterministic instructions, preserving memory and `regBase` and +writing `fx.val ^^^ (Z idx).val < 2 ^ M` into `regOut`. -/ +theorem xorLookupCode_exec {w base M : ℕ} {Z : Fin (2 ^ M) → Fin (2 ^ M)} {x : Data 6} + (hrep : Rep base M Z x.mem) (idx fx : Fin (2 ^ M)) + (hbase : x.regs regBase = base) (hidx : x.regs regIdx = idx.val) + (hfx : x.regs regOut = fx.val) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + Exec w xorLookupCode x (xorLookupData x) 3 ∧ + (xorLookupData x).regs regOut = fx.val ^^^ (Z idx).val ∧ + (xorLookupData x).regs regOut < 2 ^ M ∧ + (xorLookupData x).regs regBase = base ∧ + (xorLookupData x).mem = x.mem := by + have hfit_addr : Fits w (base + idx.val) := hfit_end.of_le (by have := idx.isLt; omega) + have hxor_lt : fx.val ^^^ (Z idx).val < 2 ^ M := Nat.xor_lt_two_pow fx.isLt (Z idx).isLt + have hfit_xor : Fits w (fx.val ^^^ (Z idx).val) := hfit_pow.of_le (le_of_lt hxor_lt) + set d1 := x.set regT1 (base + idx.val) + have s1 : Exec w (.op (.exactBin .add regT1 regBase regIdx) : Code .exact 6) x d1 1 := + exec_exactBin (by simp [hbase, hidx]) hfit_addr + set d2 := d1.set regT2 (Z idx).val + have s2 : Exec w (.op (.load regT2 regT1) : Code .exact 6) d1 d2 1 := by + simpa [d1, d2, hrep idx] using + (Exec.load (B := .exact) (w := w) (d := regT2) (a := regT1) (x := d1)) + set d3 := d2.set regOut (fx.val ^^^ (Z idx).val) + have s3 : Exec w (.op (.exactBin .xor regOut regOut regT2) : Code .exact 6) d2 d3 1 := + exec_exactBin (by simp [d2, d1, hfx, regOut, regT1, regT2]) hfit_xor + have hd3 : d3 = xorLookupData x := by + simp [d3, d2, d1, xorLookupData, hbase, hidx, hfx, hrep idx] + rw [hd3] at s3 + refine ⟨s1.seq (s2.seq s3), ?_, ?_, + by simp [xorLookupData, hbase, regBase, regT1, regT2, regOut], rfl⟩ + · simp [xorLookupData, hbase, hidx, hfx, hrep idx] + · simpa [xorLookupData, hbase, hidx, hfx, hrep idx] using hxor_lt + +end RandomTable + +/-! ## Section 3: Shared `Nat.log 2` loop and `TabulatedOddStride` -/ + +/-- One iteration of the floor-`log₂` loop on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, +`reg 2 = 1`): shifts `rem` right by `1` (`rem / 2`) and increments `acc`. -/ +@[expose] def log2Body : Code .exact 3 := + .seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2)) + +/-- Floor-`log₂` loop `while 1 < rem do [ rem ← rem >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def log2Loop : Code .exact 3 := + .loop (.lt 2 0) log2Body + +/-- Closed-form final state of `log2Loop` starting from `rem` in register `0` and `acc` in +register `1`. -/ +@[expose] def log2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.log 2 rem) + +/-- `log2Loop` executes deterministically in `4 * Nat.log 2 rem + 2` instructions, accumulating +`acc + Nat.log 2 rem` into register `1` and preserving all memory. -/ +theorem log2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem : Fits w rem) (hfit_acc : Fits w (acc + Nat.log 2 rem)) : + Exec w log2Loop x (log2LoopData rem acc x) (4 * Nat.log 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : rem >>> 1 = rem / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hlog : Nat.log 2 rem = Nat.log 2 (rem / 2) + 1 := + Nat.log_of_one_lt_of_le (by decide) (by omega) + have hfit_div : Fits w (rem / 2) := hfit_rem.of_le (Nat.div_le_self rem 2) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem / 2) + have s1 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone, hshr]) hfit_div + set d2 := d1.set 1 (acc + 1) + have s2 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hacc, hone]) hfit_acc1 + have hdiv_lt : rem / 2 < rem := Nat.div_lt_self (by omega) (by decide) + have hrec := ih (rem / 2) hdiv_lt (acc := acc + 1) (x := d2) + (by simp +decide [d2, d1]) + (by simp +decide [d2]) + (by simp +decide [d2, d1, hone]) + hfit_div + (by rwa [show acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem by omega]) + have hd : log2LoopData (rem / 2) (acc + 1) d2 = log2LoopData rem acc x := by + have hdiv_ne : rem / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem + omega + · simp +decide [log2LoopData, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq s2) hrec + simpa only [log2Loop, log2Body, hlog, show 1 + 1 + (4 * Nat.log 2 (rem / 2) + 2) + 2 = + 4 * (Nat.log 2 (rem / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hlog0 : Nat.log 2 rem = 0 := Nat.log_of_lt (by omega) + have hd : log2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by + split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, hrem_eq, hrem] + · simp +decide [log2LoopData, hlog0, hacc] + · simp +decide [log2LoopData] + · rfl + rw [hd, hlog0] + exact Exec.loop_exit hc + +namespace TabulatedOddStride + +/-- Character bit-width `b = max 1 (⌊log₂ (max n_max 2)⌋ / 2)` for `d`-character simple tabulation, +ensuring `1 ≤ b` and `2 ^ b ≤ max 2 n_max` (`2 ^ (2 * b) ≤ max 4 n_max`). -/ +@[expose] def charBits (n_max : ℕ) : ℕ := + max 1 (Nat.log 2 (max n_max 2) / 2) + +/-- `charBits n_max` is always at least `1`. -/ +theorem one_le_charBits (n_max : ℕ) : 1 ≤ charBits n_max := + Nat.le_max_left 1 _ + +/-- `charBits n_max` is always positive. -/ +theorem charBits_pos (n_max : ℕ) : 0 < charBits n_max := + one_le_charBits n_max + +/-- The character alphabet size `2 ^ charBits n_max` is at most `max 2 n_max`. -/ +theorem two_pow_charBits_le (n_max : ℕ) : 2 ^ charBits n_max ≤ max 2 n_max := by + have hlog_ge : 1 ≤ Nat.log 2 (max n_max 2) := + Nat.le_log_of_pow_le (by decide) (by omega) + have hle_log : charBits n_max ≤ Nat.log 2 (max n_max 2) := by + simp only [charBits] + omega + calc + 2 ^ charBits n_max ≤ 2 ^ Nat.log 2 (max n_max 2) := + Nat.pow_le_pow_right (by decide) hle_log + _ ≤ max n_max 2 := Nat.pow_log_le_self 2 (by omega) + _ = max 2 n_max := Nat.max_comm n_max 2 + +/-- Odd stride `(2 * tab + 1) &&& (2 ^ M - 1) = (2 * tab + 1) % 2 ^ M` in `Fin (2 ^ M)`. -/ +@[expose] def oddStride (M tab : ℕ) : Fin (2 ^ M) := + ⟨(2 * tab + 1) &&& (2 ^ M - 1), by + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (Nat.two_pow_pos M)⟩ + +@[simp] +theorem val_oddStride (M tab : ℕ) : (oddStride M tab).val = (2 * tab + 1) % 2 ^ M := + Nat.and_two_pow_sub_one_eq_mod (2 * tab + 1) M + +/-- When `0 < M`, `(oddStride M tab).val` is odd (`% 2 = 1`). -/ +theorem odd_oddStride {M : ℕ} (hM : 0 < M) (tab : ℕ) : (oddStride M tab).val % 2 = 1 := by + rw [val_oddStride] + have hdvd : 2 ∣ 2 ^ M := by + cases M with + | zero => omega + | succ m => exact ⟨2 ^ m, by ring⟩ + rw [Nat.mod_mod_of_dvd _ hdvd] + omega + +/-- `(oddStride M tab).val` is always coprime to `2 ^ M`. -/ +theorem coprime_oddStride_two_pow (M tab : ℕ) : Nat.Coprime (oddStride M tab).val (2 ^ M) := by + rw [val_oddStride, Nat.Coprime, ← Nat.gcd_rec] + have hcop2 : Nat.Coprime 2 (2 * tab + 1) := by + rw [Nat.coprime_two_left, Nat.odd_iff] + omega + exact hcop2.pow_left M + +/-- Double-hashing probe sequence `probeSeq M h₁ tab i = (h₁ + i * oddStride M tab) mod 2 ^ M` +on `Fin (2 ^ M)`. -/ +@[expose] def probeSeq (M h₁ tab : ℕ) (i : Fin (2 ^ M)) : Fin (2 ^ M) := + ⟨(h₁ + i.val * (oddStride M tab).val) % 2 ^ M, Nat.mod_lt _ (Nat.two_pow_pos M)⟩ + +/-- Because `oddStride M tab` is coprime to `2 ^ M`, `probeSeq M h₁ tab` is a full-cycle +bijection (permutation) of `Fin (2 ^ M)` for every `M`, initial probe `h₁`, and raw table sum +`tab`. -/ +theorem bijective_probeSeq (M h₁ tab : ℕ) : Function.Bijective (probeSeq M h₁ tab) := by + rw [Fintype.bijective_iff_injective_and_card] + refine ⟨?_, rfl⟩ + have hinj : ∀ i j : Fin (2 ^ M), j.val ≤ i.val → + probeSeq M h₁ tab i = probeSeq M h₁ tab j → i = j := by + intro i j hij heq + have hmod : (h₁ + j.val * (oddStride M tab).val) ≡ + (h₁ + i.val * (oddStride M tab).val) [MOD 2 ^ M] := by + simpa [probeSeq, Fin.ext_iff, Nat.ModEq] using heq.symm + have hmul : j.val * (oddStride M tab).val ≡ + i.val * (oddStride M tab).val [MOD 2 ^ M] := + Nat.ModEq.add_left_cancel' h₁ hmod + have hsub : 0 + j.val * (oddStride M tab).val ≡ + (i.val - j.val) * (oddStride M tab).val + j.val * (oddStride M tab).val [MOD 2 ^ M] := by + simpa [Nat.zero_add, ← Nat.add_mul, Nat.sub_add_cancel hij] using hmul + have hzero : 0 ≡ (i.val - j.val) * (oddStride M tab).val [MOD 2 ^ M] := + Nat.ModEq.add_right_cancel' (j.val * (oddStride M tab).val) hsub + have hdvd_mul : 2 ^ M ∣ (i.val - j.val) * (oddStride M tab).val := + (Nat.modEq_zero_iff_dvd.mp hzero.symm) + have hdvd : 2 ^ M ∣ i.val - j.val := + (coprime_oddStride_two_pow M tab).symm.dvd_of_dvd_mul_right hdvd_mul + have hlt : i.val - j.val < 2 ^ M := by + have := i.isLt + omega + have hdiff0 : i.val - j.val = 0 := Nat.eq_zero_of_dvd_of_lt hdvd hlt + exact Fin.ext (by omega) + intro i j heq + rcases le_total j.val i.val with hij | hij + · exact hinj i j hij heq + · exact (hinj j i hij heq.symm).symm + +/-- Tail-recursive `d`-character simple tabulation accumulator from character offset `j` across +`rem` remaining characters: XORs `T[(j + k) * 2 ^ b + ((u >>> (k * b)) &&& (2 ^ b - 1))]` into +`acc`. -/ +@[expose] def simpleTabAux (b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) : + ℕ → ℕ → ℕ → Fin (2 ^ M) → Fin (2 ^ M) + | 0, _, _, acc => acc + | rem + 1, j, u, acc => + let ch := u &&& (2 ^ b - 1) + let idx := j * 2 ^ b + ch + let entry : Fin (2 ^ M) := + if h : idx < d * 2 ^ b then T ⟨idx, h⟩ else ⟨0, Nat.two_pow_pos M⟩ + simpleTabAux b d M T rem (j + 1) (u >>> b) + ⟨acc.val ^^^ entry.val, Nat.xor_lt_two_pow acc.isLt entry.isLt⟩ + +/-- `d`-character simple tabulation hash `simpleTab b d M T u : Fin (2 ^ M)`, XORing the table +entries for the first `d` base-`2 ^ b` characters of `u`. -/ +@[expose] def simpleTab (b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) (u : ℕ) : Fin (2 ^ M) := + simpleTabAux b d M T d 0 u ⟨0, Nat.two_pow_pos M⟩ + +/-- Tabulated odd stride `tabulatedOddStride b d M T u : Fin (2 ^ M)` obtained by mapping the +`d`-character simple tabulation hash through `oddStride M`. -/ +@[expose] def tabulatedOddStride (b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) (u : ℕ) : + Fin (2 ^ M) := + oddStride M (simpleTab b d M T u).val + +/-- Memory footprint `4 + d * 2 ^ b` words for a `d`-character simple tabulation table with +character bit-width `b` (4 header words `(b, 2 ^ b, 2 ^ b - 1, 2 ^ M - 1)` followed by +`d * 2 ^ b` random `< 2 ^ M` words). -/ +@[expose] def spaceFromB (b d : ℕ) : ℕ := 4 + d * 2 ^ b + +/-- Memory footprint `4 + d * 2 ^ (charBits n_max)` words at capacity parameter `n_max`. -/ +@[expose] def space (n_max d : ℕ) : ℕ := spaceFromB (charBits n_max) d + +/-- Memory representation predicate for `TabulatedOddStride` at `[base, base + 4 + d * 2 ^ b)`: +header words `b`, `2 ^ b`, `2 ^ b - 1`, `2 ^ M - 1` at `base + 0 .. base + 3`, and table entries +`(T i).val` at `base + 4 + i.val` for `i : Fin (d * 2 ^ b)`. -/ +@[expose] def Rep (base b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) (mem : ℕ → ℕ) : Prop := + mem base = b ∧ + mem (base + 1) = 2 ^ b ∧ + mem (base + 2) = 2 ^ b - 1 ∧ + mem (base + 3) = 2 ^ M - 1 ∧ + ∀ i : Fin (d * 2 ^ b), mem (base + 4 + i.val) = (T i).val + +/-- Extract the flat character table `Fin (d * 2 ^ b) → Fin (2 ^ M)` from `[base + 4, ...)`. -/ +@[expose] def extractTable (base b d M : ℕ) (mem : ℕ → ℕ) : Fin (d * 2 ^ b) → Fin (2 ^ M) := + extractSlice (base + 4) M (d * 2 ^ b) mem + +/-- Register `0` of `TabulatedOddStride`: base memory address `base`. -/ +abbrev regBase : Fin 9 := 0 + +/-- Register `1` of `TabulatedOddStride`: holds `n_max` in `initCode` or input key `u` in +`evalCode`. -/ +abbrev regKey : Fin 9 := 1 + +/-- Register `2` of `TabulatedOddStride`: holds `M` in `initCode` or receives the odd stride in +`evalCode`. -/ +abbrev regOut : Fin 9 := 2 + +/-- Register `3` of `TabulatedOddStride`: table pointer scratch register. -/ +abbrev regPtr : Fin 9 := 3 + +/-- Register `4` of `TabulatedOddStride`: end pointer / mask scratch register. -/ +abbrev regEnd : Fin 9 := 4 + +/-- Register `5` of `TabulatedOddStride`: character bit-width `b` / loop bound register. -/ +abbrev regB : Fin 9 := 5 + +/-- Register `6` of `TabulatedOddStride`: character alphabet size `2 ^ b` / sampled value +register. -/ +abbrev regPowB : Fin 9 := 6 + +/-- Register `7` of `TabulatedOddStride`: character mask `2 ^ b - 1` register. -/ +abbrev regMaskB : Fin 9 := 7 + +/-- Register `8` of `TabulatedOddStride`: general scratch register. -/ +abbrev regTmp : Fin 9 := 8 + +/-- Register injection `Fin 3 → Fin 9` mapping `log2Loop`'s `(rem, acc, one)` to +`(regPtr, regB, regTmp)`. -/ +@[expose] def logMap9 : Fin 3 → Fin 9 + | ⟨0, _⟩ => regPtr + | ⟨1, _⟩ => regB + | ⟨2, _⟩ => regTmp + +theorem injective_logMap9 : Function.Injective logMap9 := by + decide + +/-- Register injection `Fin 6 → Fin 9` mapping `randFillLoop`'s `(ptr, end, mask, one, val, extra)` +to `(regPtr, regB, regEnd, regTmp, regPowB, regBase)`. -/ +@[expose] def fillMap9 : Fin 6 → Fin 9 + | ⟨0, _⟩ => regPtr + | ⟨1, _⟩ => regB + | ⟨2, _⟩ => regEnd + | ⟨3, _⟩ => regTmp + | ⟨4, _⟩ => regPowB + | ⟨5, _⟩ => regBase + +theorem injective_fillMap9 : Function.Injective fillMap9 := by + decide + +/-- Structured Word-RAM program that computes `regB ← charBits n_max` +(`= max 1 (⌊log₂ (max n_max 2)⌋ / 2)`) at runtime from `n_max` in `regKey`, using `regPtr`, +`regB`, `regTmp` as scratch registers and preserving `regBase`, `regOut`, and all memory. -/ +@[expose] def charBitsCode : Code .exact 9 := + .seq (.op (.const regTmp 2)) + (.seq (maxCode regPtr regKey regTmp) + (.seq (.op (.const regB 0)) + (.seq (.op (.const regTmp 1)) + (.seq (log2Loop.rename logMap9) + (.seq (.op (.exactBin .shiftRight regB regB regTmp)) + (maxCode regB regB regTmp)))))) + +/-- State after `charBitsCode` starting from `n_max` in `regKey`. -/ +noncomputable def charBitsData (n_max : ℕ) (x : Data 9) : Data 9 := + let dPre := (((x.set regTmp 2).set regPtr (max n_max 2)).set regB 0).set regTmp 1 + let dLog := (log2LoopData (max n_max 2) 0 (dPre.comap logMap9)).extend logMap9 dPre + (dLog.set regB (Nat.log 2 (max n_max 2) / 2)).set regB (charBits n_max) + +/-- `charBitsCode` executes deterministically in `4 * Nat.log 2 (max n_max 2) + 12` instructions, +setting `regB = charBits n_max` while preserving `regBase`, `regOut`, and all memory. -/ +theorem charBitsCode_exec {w n_max : ℕ} {x : Data 9} + (hn : x.regs regKey = n_max) (hfit_n : Fits w (max n_max 2)) : + Exec w charBitsCode x (charBitsData n_max x) (4 * Nat.log 2 (max n_max 2) + 12) ∧ + (charBitsData n_max x).regs regB = charBits n_max ∧ + (charBitsData n_max x).regs regBase = x.regs regBase ∧ + (charBitsData n_max x).regs regOut = x.regs regOut ∧ + (charBitsData n_max x).mem = x.mem := by + have hfit_2 : Fits w 2 := hfit_n.of_le (Nat.le_max_right n_max 2) + have hfit_1 : Fits w 1 := hfit_2.of_le (by decide) + have hfit_0 : Fits w 0 := hfit_1.of_le (by decide) + have hlog_le : Nat.log 2 (max n_max 2) ≤ max n_max 2 := Nat.log_le_self 2 (max n_max 2) + have hfit_log : Fits w (Nat.log 2 (max n_max 2)) := hfit_n.of_le hlog_le + set d1 := x.set regTmp 2 + have s1 : Exec w (.op (.const regTmp 2) : Code .exact 9) x d1 1 := Exec.const hfit_2 + set d2 := d1.set regPtr (max n_max 2) + have s2 : Exec w (maxCode regPtr regKey regTmp) d1 d2 3 := by + simpa +decide [d2, d1, hn] using maxCode_exec (w := w) regPtr regKey regTmp d1 + set d3 := d2.set regB 0 + have s3 : Exec w (.op (.const regB 0) : Code .exact 9) d2 d3 1 := Exec.const hfit_0 + set d4 := d3.set regTmp 1 + have s4 : Exec w (.op (.const regTmp 1) : Code .exact 9) d3 d4 1 := Exec.const hfit_1 + have slog0 := log2Loop_exec (w := w) (max n_max 2) (acc := 0) (x := d4.comap logMap9) + (by simp +decide [d4, d3, d2, logMap9, Data.comap]) + (by simp +decide [d4, d3, logMap9, Data.comap]) + (by simp +decide [d4, logMap9, Data.comap]) + hfit_n (by simpa using hfit_log) + have slog := slog0.rename injective_logMap9 + set d5 := (log2LoopData (max n_max 2) 0 (d4.comap logMap9)).extend logMap9 d4 + have hd5_b : d5.regs regB = Nat.log 2 (max n_max 2) := by + change (d5.comap logMap9).regs 1 = Nat.log 2 (max n_max 2) + rw [Data.comap_extend injective_logMap9] + simp [log2LoopData] + have hd5_tmp : d5.regs regTmp = 1 := by + change (d5.comap logMap9).regs 2 = 1 + rw [Data.comap_extend injective_logMap9] + simp +decide [log2LoopData, d4, logMap9, Data.comap] + have hshr : Nat.log 2 (max n_max 2) >>> 1 = Nat.log 2 (max n_max 2) / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hfit_div : Fits w (Nat.log 2 (max n_max 2) / 2) := + hfit_log.of_le (Nat.div_le_self _ 2) + set d6 := d5.set regB (Nat.log 2 (max n_max 2) / 2) + have s6 : Exec w (.op (.exactBin .shiftRight regB regB regTmp) : Code .exact 9) d5 d6 1 := + exec_exactBin (by simp [hd5_b, hd5_tmp, hshr]) hfit_div + set d7 := d6.set regB (charBits n_max) + have hcb : max (Nat.log 2 (max n_max 2) / 2) 1 = charBits n_max := Nat.max_comm _ _ + have s7 : Exec w (maxCode regB regB regTmp) d6 d7 3 := by + simpa +decide [d7, d6, hd5_tmp, hcb] using maxCode_exec (w := w) regB regB regTmp d6 + have hall := s1.seq (s2.seq (s3.seq (s4.seq (slog.seq (s6.seq s7))))) + have hsteps : 1 + (3 + (1 + (1 + (4 * Nat.log 2 (max n_max 2) + 2 + (1 + 3))))) = + 4 * Nat.log 2 (max n_max 2) + 12 := by omega + rw [hsteps] at hall + have hbase_eq : d7.regs regBase = x.regs regBase := by + simp +decide [d7, d6, d5, Data.extend, d4, d3, d2, d1] + have hout_eq : d7.regs regOut = x.regs regOut := by + simp +decide [d7, d6, d5, Data.extend, d4, d3, d2, d1] + refine ⟨hall, by simp [charBitsData], hbase_eq, hout_eq, rfl⟩ + +/-- Deterministic 16-instruction setup prefix of `initFromBCode d`: writes the 4 header words +`(b, 2 ^ b, 2 ^ b - 1, 2 ^ M - 1)` to `base + 0 .. base + 3` and sets registers for +`randFillLoop.rename fillMap9` on `[base + 4, base + 4 + d * 2 ^ b)`. -/ +@[expose] def setupFromBCode (d : ℕ) : Code .exact 9 := + .seq (.op (.store regBase regB)) + (.seq (.op (.const regTmp 1)) + (.seq (.op (.exactBin .shiftLeft regPowB regTmp regB)) + (.seq (.op (.exactBin .add regPtr regBase regTmp)) + (.seq (.op (.store regPtr regPowB)) + (.seq (.op (.exactBin .sub regMaskB regPowB regTmp)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.store regPtr regMaskB)) + (.seq (.op (.exactBin .shiftLeft regEnd regTmp regOut)) + (.seq (.op (.exactBin .sub regEnd regEnd regTmp)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.store regPtr regEnd)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.const regB d)) + (.seq (.op (.exactBin .mul regB regB regPowB)) + (.op (.exactBin .add regB regPtr regB)))))))))))))))) + +/-- State after `setupFromBCode d`. -/ +@[expose] def setupFromBData (base b d M : ℕ) (x : Data 9) : Data 9 := + let m0 := (((x.write base b).write (base + 1) (2 ^ b)).write + (base + 2) (2 ^ b - 1)).write (base + 3) (2 ^ M - 1) + ((((((m0.set regTmp 1).set regPowB (2 ^ b)).set regPtr (base + 1)).set + regMaskB (2 ^ b - 1)).set regPtr (base + 2)).set regEnd (2 ^ M)).set + regEnd (2 ^ M - 1) |>.set regPtr (base + 3) |>.set regPtr (base + 4) |>.set + regB d |>.set regB (d * 2 ^ b) |>.set regB (base + 4 + d * 2 ^ b) + +/-- `setupFromBCode d` executes in `16` deterministic steps. -/ +theorem setupFromBCode_exec {w base b d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hb : x.regs regB = b) (hM : x.regs regOut = M) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powB : Fits w (2 ^ b)) (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + Exec w (setupFromBCode d) x (setupFromBData base b d M x) 16 := by + have h1_le_b : 1 ≤ 2 ^ b := Nat.one_le_two_pow + have h1_le_M : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hfit_1 : Fits w 1 := hfit_powB.of_le h1_le_b + have hfit_maskB : Fits w (2 ^ b - 1) := hfit_powB.of_le (Nat.sub_le _ 1) + have hfit_maskM : Fits w (2 ^ M - 1) := hfit_powM.of_le (Nat.sub_le _ 1) + have hfit_b1 : Fits w (base + 1) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b2 : Fits w (base + 2) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b3 : Fits w (base + 3) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b4 : Fits w (base + 4) := hfit_space.of_le (by simp [spaceFromB]) + have hfit_dmul : Fits w (d * 2 ^ b) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_end : Fits w (base + 4 + d * 2 ^ b) := + hfit_space.of_le (by simp [spaceFromB]; omega) + set s0 := x.write base b + have e1 : Exec w (.op (.store regBase regB) : Code .exact 9) x s0 1 := by + simpa [s0, hbase, hb] using + (Exec.store (B := .exact) (w := w) (a := regBase) (s := regB) (x := x)) + set s1 := s0.set regTmp 1 + have e2 : Exec w (.op (.const regTmp 1) : Code .exact 9) s0 s1 1 := Exec.const hfit_1 + set s2 := s1.set regPowB (2 ^ b) + have e3 : Exec w (.op (.exactBin .shiftLeft regPowB regTmp regB) : Code .exact 9) s1 s2 1 := + exec_exactBin (by simp +decide [s1, s0, hb, Nat.shiftLeft_eq]) hfit_powB + set s3 := s2.set regPtr (base + 1) + have e4 : Exec w (.op (.exactBin .add regPtr regBase regTmp) : Code .exact 9) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, s0, hbase]) hfit_b1 + set s4 := s3.write (base + 1) (2 ^ b) + have e5 : Exec w (.op (.store regPtr regPowB) : Code .exact 9) s3 s4 1 := by + simpa +decide [s4, s3, s2] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regPowB) (x := s3)) + set s5 := s4.set regMaskB (2 ^ b - 1) + have e6 : Exec w (.op (.exactBin .sub regMaskB regPowB regTmp) : Code .exact 9) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h1_le_b]) hfit_maskB + set s6 := s5.set regPtr (base + 2) + have e7 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1]) hfit_b2 + set s7 := s6.write (base + 2) (2 ^ b - 1) + have e8 : Exec w (.op (.store regPtr regMaskB) : Code .exact 9) s6 s7 1 := by + simpa +decide [s7, s6, s5] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regMaskB) (x := s6)) + set s8 := s7.set regEnd (2 ^ M) + have e9 : Exec w (.op (.exactBin .shiftLeft regEnd regTmp regOut) : Code .exact 9) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, s0, hM, Nat.shiftLeft_eq]) hfit_powM + set s9 := s8.set regEnd (2 ^ M - 1) + have e10 : Exec w (.op (.exactBin .sub regEnd regEnd regTmp) : Code .exact 9) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h1_le_M]) hfit_maskM + set s10 := s9.set regPtr (base + 3) + have e11 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1]) hfit_b3 + set s11 := s10.write (base + 3) (2 ^ M - 1) + have e12 : Exec w (.op (.store regPtr regEnd) : Code .exact 9) s10 s11 1 := by + simpa +decide [s11, s10, s9] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regEnd) (x := s10)) + set s12 := s11.set regPtr (base + 4) + have e13 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1]) hfit_b4 + set s13 := s12.set regB d + have e14 : Exec w (.op (.const regB d) : Code .exact 9) s12 s13 1 := Exec.const hfit_d + set s14 := s13.set regB (d * 2 ^ b) + have e15 : Exec w (.op (.exactBin .mul regB regB regPowB) : Code .exact 9) s13 s14 1 := + exec_exactBin (by simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2]) hfit_dmul + set s15 := s14.set regB (base + 4 + d * 2 ^ b) + have e16 : Exec w (.op (.exactBin .add regB regPtr regB) : Code .exact 9) s14 s15 1 := + exec_exactBin (by simp +decide [s14, s13, s12]) hfit_end + have hs15 : s15 = setupFromBData base b d M x := by + ext i : 2 + · fin_cases i <;> rfl + · rfl + rw [hs15] at e16 + exact e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq + (e9.seq (e10.seq (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq e16)))))))))))))) + +/-- Structured Word-RAM program that initializes a `d`-character simple tabulation table from +`(base, M, b)` in `(regBase, regOut, regB)`. -/ +@[expose] def initFromBCode (d : ℕ) : Code .exact 9 := + .seq (setupFromBCode d) (randFillLoop.rename fillMap9) + +/-- Explicit `ProbComp (Data 9 × ℕ)` produced by `initFromBCode d`. -/ +@[expose] noncomputable def initFromBComp (base b d M : ℕ) (x : Data 9) : ProbComp (Data 9 × ℕ) := + let s := setupFromBData base b d M x + (fun z => (z.1.extend fillMap9 s, 16 + z.2)) <$> + randFillComp M (base + 4) (d * 2 ^ b) (s.comap fillMap9) + +/-- Exact `ExecRand` semantics of `initFromBCode d`. -/ +theorem initFromBCode_execRand {w base b d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hb : x.regs regB = b) (hM : x.regs regOut = M) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powB : Fits w (2 ^ b)) (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRand w (initFromBCode d) x (initFromBComp base b d M x) := by + have hsetup := setupFromBCode_exec hbase hb hM hfit_space hfit_powB hfit_powM hfit_d + set s := setupFromBData base b d M x + have hloop0 := randFillLoop_execRand (w := w) (bits := M) (arrBase := base + 4) + (d * 2 ^ b) (x := s.comap fillMap9) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (hfit_space.of_le (by simp [spaceFromB]; omega)) + (hfit_powM.of_le (Nat.sub_le _ 1)) + have hloop := hloop0.rename injective_fillMap9 + have hseq := hsetup.seqRand hloop + have heq : ((fun z => (z.1, 16 + z.2)) <$> + ((fun z => (z.1.extend fillMap9 s, z.2)) <$> + randFillComp M (base + 4) (d * 2 ^ b) (s.comap fillMap9))) = + initFromBComp base b d M x := by + simp [initFromBComp, s, Functor.map_map] + rwa [heq] at hseq + +/-- The character table `extractTable base b d M z.1.mem` produced by `initFromBComp` is uniformly +distributed over `Fin (d * 2 ^ b) → Fin (2 ^ M)`. -/ +theorem map_extractTable_eval_initFromBComp (base b d M : ℕ) (x : Data 9) : + ((initFromBComp base b d M x).eval).map (fun z => extractTable base b d M z.1.mem) = + PMF.uniformOfFintype (Fin (d * 2 ^ b) → Fin (2 ^ M)) := by + dsimp only [initFromBComp, extractTable] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractSlice_eval_randFillComp M (base + 4) (d * 2 ^ b) + ((setupFromBData base b d M x).comap fillMap9) + +/-- Self-contained `initCode d` that computes `b = charBits n_max` from `n_max` in `regKey` and +initializes the `d`-character simple tabulation table at `base` (`regBase`) with `M`-bit entries +(`regOut = M`). -/ +@[expose] def initCode (d : ℕ) : Code .exact 9 := + .seq charBitsCode (initFromBCode d) + +/-- Pathwise postcondition of `initFromBComp`: runs in `5 * (d * 2 ^ b) + 18` instructions and +establishes `Rep base b d M (extractTable base b d M z.1.mem) z.1.mem`, `z.1.regs regBase = base`, +and frame preservation outside `[base, base + spaceFromB b d)`. -/ +theorem mem_support_initFromBComp {base b d M : ℕ} {x : Data 9} {z : Data 9 × ℕ} + (hbase : x.regs regBase = base) + (hz : z ∈ (initFromBComp base b d M x).support) : + z.2 = 5 * (d * 2 ^ b) + 18 ∧ + Rep base b d M (extractTable base b d M z.1.mem) z.1.mem ∧ + z.1.regs regBase = base ∧ + (∀ a, (a < base ∨ base + spaceFromB b d ≤ a) → z.1.mem a = x.mem a) := by + dsimp only [initFromBComp] at hz + obtain ⟨z₀, hz₀, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hframe⟩ := mem_support_randFillComp hz₀ + set s := setupFromBData base b d M x + have hmem0 : z₀.1.mem base = b := by + rw [hframe base (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write, Function.update_of_ne] + have hmem1 : z₀.1.mem (base + 1) = 2 ^ b := by + rw [hframe (base + 1) (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write, Function.update_of_ne] + have hmem2 : z₀.1.mem (base + 2) = 2 ^ b - 1 := by + rw [hframe (base + 2) (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write, Function.update_of_ne] + have hmem3 : z₀.1.mem (base + 3) = 2 ^ M - 1 := by + rw [hframe (base + 3) (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write] + have hrep : Rep base b d M (extractTable base b d M z₀.1.mem) z₀.1.mem := by + refine ⟨hmem0, hmem1, hmem2, hmem3, fun i => ?_⟩ + simp [extractTable, extractSlice, hlt i.val i.isLt] + have hbase' : (z₀.1.extend fillMap9 s).regs regBase = base := by + change ((z₀.1.extend fillMap9 s).comap fillMap9).regs regExtra = base + rw [Data.comap_extend injective_fillMap9, hextra] + simpa +decide [s, setupFromBData, fillMap9, Data.comap] using hbase + have hframe' : ∀ a, (a < base ∨ base + spaceFromB b d ≤ a) → z₀.1.mem a = x.mem a := by + intro a ha + have ha4 : a < base + 4 ∨ base + 4 + d * 2 ^ b ≤ a := by + simp only [spaceFromB] at ha + omega + rw [hframe a ha4] + have hne0 : a ≠ base := by simp only [spaceFromB] at ha; omega + have hne1 : a ≠ base + 1 := by simp only [spaceFromB] at ha; omega + have hne2 : a ≠ base + 2 := by simp only [spaceFromB] at ha; omega + have hne3 : a ≠ base + 3 := by simp only [spaceFromB] at ha; omega + simp [s, setupFromBData, Data.comap, Data.write, + Function.update_of_ne hne0, Function.update_of_ne hne1, + Function.update_of_ne hne2, Function.update_of_ne hne3] + exact ⟨by omega, hrep, hbase', hframe'⟩ + +/-- Hoare-style `ExecRandWithin` specification for `initFromBCode d`. -/ +theorem initFromBCode_execRandWithin {w base b d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hb : x.regs regB = b) (hM : x.regs regOut = M) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powB : Fits w (2 ^ b)) (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRandWithin w (initFromBCode d) x + (fun y => Rep base b d M (extractTable base b d M y.mem) y.mem ∧ + y.regs regBase = base ∧ + (∀ a, (a < base ∨ base + spaceFromB b d ≤ a) → y.mem a = x.mem a)) + (5 * (d * 2 ^ b) + 18) (d * 2 ^ b) := by + refine ⟨initFromBComp base b d M x, + initFromBCode_execRand hbase hb hM hfit_space hfit_powB hfit_powM hfit_d, ?_, ?_⟩ + · dsimp only [initFromBComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp M (base + 4) (d * 2 ^ b) _) + · intro z hz + obtain ⟨hsteps, hrep, hbase', hframe'⟩ := mem_support_initFromBComp hbase hz + exact ⟨⟨hrep, hbase', hframe'⟩, by omega⟩ + +/-- Explicit `ProbComp (Data 9 × ℕ)` produced by `initCode d` from `(base, n_max, M)`. -/ +@[expose] noncomputable def initComp (base n_max d M : ℕ) (x : Data 9) : ProbComp (Data 9 × ℕ) := + (fun z => (z.1, (4 * Nat.log 2 (max n_max 2) + 12) + z.2)) <$> + initFromBComp base (charBits n_max) d M (charBitsData n_max x) + +/-- Exact `ExecRand` semantics of `initCode d`. -/ +theorem initCode_execRand {w base n_max d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hn : x.regs regKey = n_max) (hM : x.regs regOut = M) + (hfit_n : Fits w (max n_max 2)) + (hfit_space : Fits w (base + space n_max d)) + (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRand w (initCode d) x (initComp base n_max d M x) := by + obtain ⟨hcb, hb_eq, hbase_eq, hout_eq, _⟩ := charBitsCode_exec (w := w) hn hfit_n + have hfit_powB : Fits w (2 ^ charBits n_max) := by + have hle := two_pow_charBits_le n_max + rw [Nat.max_comm 2 n_max] at hle + exact hfit_n.of_le hle + have hinit := initFromBCode_execRand + (hbase_eq.trans hbase) hb_eq (hout_eq.trans hM) hfit_space hfit_powB hfit_powM hfit_d + exact hcb.seqRand hinit + +/-- **Uniformity of `TabulatedOddStride.initComp`**: the character table extracted from final +memory has the exact uniform distribution over `Fin (d * 2 ^ (charBits n_max)) → Fin (2 ^ M)`. -/ +theorem map_extractTable_eval_initComp (base n_max d M : ℕ) (x : Data 9) : + ((initComp base n_max d M x).eval).map + (fun z => extractTable base (charBits n_max) d M z.1.mem) = + PMF.uniformOfFintype (Fin (d * 2 ^ (charBits n_max)) → Fin (2 ^ M)) := by + dsimp only [initComp] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractTable_eval_initFromBComp base (charBits n_max) d M (charBitsData n_max x) + +/-- Hoare-style `ExecRandWithin` specification for `TabulatedOddStride.initCode d`. -/ +theorem initCode_execRandWithin {w base n_max d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hn : x.regs regKey = n_max) (hM : x.regs regOut = M) + (hfit_n : Fits w (max n_max 2)) + (hfit_space : Fits w (base + space n_max d)) + (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRandWithin w (initCode d) x + (fun y => + Rep base (charBits n_max) d M (extractTable base (charBits n_max) d M y.mem) y.mem ∧ + y.regs regBase = base ∧ + (∀ a, (a < base ∨ base + space n_max d ≤ a) → y.mem a = x.mem a)) + (4 * Nat.log 2 (max n_max 2) + 5 * (d * 2 ^ (charBits n_max)) + 30) + (d * 2 ^ (charBits n_max)) := by + refine ⟨initComp base n_max d M x, + initCode_execRand hbase hn hM hfit_n hfit_space hfit_powM hfit_d, ?_, ?_⟩ + · dsimp only [initComp, initFromBComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + ((OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp M (base + 4) (d * 2 ^ (charBits n_max)) _)) + · intro z hz + dsimp only [initComp] at hz + obtain ⟨z₁, hz₁, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨-, -, hbase_cb, -, hmem_cb⟩ := charBitsCode_exec (w := w) hn hfit_n + obtain ⟨hsteps, hrep, hbase', hframe'⟩ := + mem_support_initFromBComp (hbase_cb.trans hbase) hz₁ + refine ⟨⟨hrep, hbase', fun a ha => ?_⟩, by omega⟩ + rw [hframe' a ha, hmem_cb] + +/-- 12-instruction deterministic setup before `tabLoop` in `evalCode d`: loads `b`, `2 ^ b`, +`2 ^ b - 1` from `base + 0 .. base + 2`, sets `regPtr = base + 4` and +`regEnd = base + 4 + d * 2 ^ b`, and zeros `regOut`. -/ +@[expose] def evalSetupCode (d : ℕ) : Code .exact 9 := + .seq (.op (.load regB regBase)) + (.seq (.op (.const regTmp 1)) + (.seq (.op (.exactBin .add regPtr regBase regTmp)) + (.seq (.op (.load regPowB regPtr)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.load regMaskB regPtr)) + (.seq (.op (.const regOut 2)) + (.seq (.op (.exactBin .add regPtr regPtr regOut)) + (.seq (.op (.const regOut d)) + (.seq (.op (.exactBin .mul regEnd regOut regPowB)) + (.seq (.op (.exactBin .add regEnd regPtr regEnd)) + (.op (.const regOut 0)))))))))))) + +/-- State after `evalSetupCode d`. -/ +@[expose] def evalSetupData (base b d : ℕ) (x : Data 9) : Data 9 := + (((((((((((x.set regB b).set regTmp 1).set regPtr (base + 1)).set + regPowB (2 ^ b)).set regPtr (base + 2)).set regMaskB (2 ^ b - 1)).set + regOut 2).set regPtr (base + 4)).set regOut d).set regEnd (d * 2 ^ b)).set + regEnd (base + 4 + d * 2 ^ b)).set regOut 0 + +/-- `evalSetupCode d` executes in `12` deterministic steps when `Rep base b d M T x.mem` holds. -/ +theorem evalSetupCode_exec {w base b d M : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 ^ M)} {x : Data 9} + (hrep : Rep base b d M T x.mem) (hbase : x.regs regBase = base) + (hfit_space : Fits w (base + spaceFromB b d)) (hfit_d : Fits w d) : + Exec w (evalSetupCode d) x (evalSetupData base b d x) 12 := by + obtain ⟨hm0, hm1, hm2, -, -⟩ := hrep + have hfit_2 : Fits w 2 := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_1 : Fits w 1 := hfit_2.of_le (by decide) + have hfit_0 : Fits w 0 := hfit_1.of_le (by decide) + have hfit_b1 : Fits w (base + 1) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b2 : Fits w (base + 2) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b4 : Fits w (base + 4) := hfit_space.of_le (by simp [spaceFromB]) + have hfit_dmul : Fits w (d * 2 ^ b) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_end : Fits w (base + 4 + d * 2 ^ b) := hfit_space.of_le (by simp [spaceFromB]; omega) + set s0 := x.set regB b + have e1 : Exec w (.op (.load regB regBase) : Code .exact 9) x s0 1 := by + simpa [s0, hbase, hm0] using + (Exec.load (B := .exact) (w := w) (d := regB) (a := regBase) (x := x)) + set s1 := s0.set regTmp 1 + have e2 : Exec w (.op (.const regTmp 1) : Code .exact 9) s0 s1 1 := Exec.const hfit_1 + set s2 := s1.set regPtr (base + 1) + have e3 : Exec w (.op (.exactBin .add regPtr regBase regTmp) : Code .exact 9) s1 s2 1 := + exec_exactBin (by simp +decide [s1, s0, hbase]) hfit_b1 + set s3 := s2.set regPowB (2 ^ b) + have e4 : Exec w (.op (.load regPowB regPtr) : Code .exact 9) s2 s3 1 := by + simpa +decide [s3, s2, s1, s0, hm1] using + (Exec.load (B := .exact) (w := w) (d := regPowB) (a := regPtr) (x := s2)) + set s4 := s3.set regPtr (base + 2) + have e5 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1]) hfit_b2 + set s5 := s4.set regMaskB (2 ^ b - 1) + have e6 : Exec w (.op (.load regMaskB regPtr) : Code .exact 9) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, s0, hm2] using + (Exec.load (B := .exact) (w := w) (d := regMaskB) (a := regPtr) (x := s4)) + set s6 := s5.set regOut 2 + have e7 : Exec w (.op (.const regOut 2) : Code .exact 9) s5 s6 1 := Exec.const hfit_2 + set s7 := s6.set regPtr (base + 4) + have e8 : Exec w (.op (.exactBin .add regPtr regPtr regOut) : Code .exact 9) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, s4]) hfit_b4 + set s8 := s7.set regOut d + have e9 : Exec w (.op (.const regOut d) : Code .exact 9) s7 s8 1 := Exec.const hfit_d + set s9 := s8.set regEnd (d * 2 ^ b) + have e10 : Exec w (.op (.exactBin .mul regEnd regOut regPowB) : Code .exact 9) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3]) hfit_dmul + set s10 := s9.set regEnd (base + 4 + d * 2 ^ b) + have e11 : Exec w (.op (.exactBin .add regEnd regPtr regEnd) : Code .exact 9) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7]) hfit_end + set s11 := s10.set regOut 0 + have e12 : Exec w (.op (.const regOut 0) : Code .exact 9) s10 s11 1 := Exec.const hfit_0 + exact e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq + (e7.seq (e8.seq (e9.seq (e10.seq (e11.seq e12)))))))))) + +/-- Body of `tabLoop` (6 instructions): extracts `ch = regKey &&& regMaskB`, loads +`T[regPtr + ch]`, XORs into `regOut`, shifts `regKey` right by `regB`, and advances `regPtr` by +`regPowB`. -/ +@[expose] def tabBody : Code .exact 9 := + .seq (.op (.exactBin .and regTmp regKey regMaskB)) + (.seq (.op (.exactBin .add regTmp regPtr regTmp)) + (.seq (.op (.load regTmp regTmp)) + (.seq (.op (.exactBin .xor regOut regOut regTmp)) + (.seq (.op (.exactBin .shiftRight regKey regKey regB)) + (.op (.exactBin .add regPtr regPtr regPowB)))))) + +/-- Loop over the `d` character tables `[base + 4, base + 4 + d * 2 ^ b)`. -/ +@[expose] def tabLoop : Code .exact 9 := + .loop (.lt regPtr regEnd) tabBody + +/-- State after one iteration of `tabBody`. -/ +@[expose] def tabStepData (ptr ch entry newAcc newKey nextPtr : ℕ) (x : Data 9) : Data 9 := + (((((x.set regTmp ch).set regTmp (ptr + ch)).set regTmp entry).set + regOut newAcc).set regKey newKey).set regPtr nextPtr + +/-- `tabLoop` executes deterministically in `8 * rem + 2` instructions from character offset `j` +with `j + rem = d`, accumulating `(simpleTabAux b d M T rem j u acc).val` into `regOut` and +preserving `regBase` and all memory. -/ +theorem tabLoop_exec {w base b d M : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 ^ M)} {x : Data 9} + (hrep : Rep base b d M T x.mem) + (hfit_space : Fits w (base + spaceFromB b d)) (hfit_powM : Fits w (2 ^ M)) : + ∀ (rem j u : ℕ) (acc : Fin (2 ^ M)) (d0 : Data 9), + j + rem = d → + d0.mem = x.mem → + d0.regs regBase = base → + d0.regs regPtr = base + 4 + j * 2 ^ b → + d0.regs regEnd = base + 4 + d * 2 ^ b → + d0.regs regB = b → + d0.regs regPowB = 2 ^ b → + d0.regs regMaskB = 2 ^ b - 1 → + d0.regs regKey = u → + d0.regs regOut = acc.val → + Fits w u → + ∃ y : Data 9, + Exec w tabLoop d0 y (8 * rem + 2) ∧ + y.regs regOut = (simpleTabAux b d M T rem j u acc).val ∧ + y.regs regBase = base ∧ + y.mem = x.mem := by + intro rem + induction rem with + | zero => + intro j u acc d0 hj hmem hbase hptr hend _ _ _ _ hout _ + have hj_eq : j = d := by omega + have hc : ¬(Cond.lt regPtr regEnd).Holds d0 := by + simp [Cond.Holds, hptr, hend, hj_eq] + exact ⟨d0, Exec.loop_exit hc, by simp [simpleTabAux, hout], hbase, hmem⟩ + | succ rem ih => + intro j u acc d0 hj hmem hbase hptr hend hb hpowB hmaskB hkey hout hfit_u + have hj_lt : j < d := by omega + have hpow_pos : 0 < 2 ^ b := Nat.two_pow_pos b + have hj_mul_lt : j * 2 ^ b < d * 2 ^ b := Nat.mul_lt_mul_of_pos_right hj_lt hpow_pos + have hc : (Cond.lt regPtr regEnd).Holds d0 := by + simp only [Cond.Holds, hptr, hend] + omega + set ch := u &&& (2 ^ b - 1) + have hch_lt : ch < 2 ^ b := by + dsimp only [ch] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt u hpow_pos + set idx := j * 2 ^ b + ch + have hidx_lt : idx < d * 2 ^ b := by + have : (j + 1) * 2 ^ b ≤ d * 2 ^ b := Nat.mul_le_mul_right (2 ^ b) (by omega) + calc + idx < j * 2 ^ b + 2 ^ b := by omega + _ = (j + 1) * 2 ^ b := by ring + _ ≤ d * 2 ^ b := this + set entry := (T ⟨idx, hidx_lt⟩).val + have hentry_mem : d0.mem (base + 4 + j * 2 ^ b + ch) = entry := by + rw [hmem, show base + 4 + j * 2 ^ b + ch = base + 4 + idx by omega] + exact hrep.2.2.2.2 ⟨idx, hidx_lt⟩ + set newAcc : Fin (2 ^ M) := + ⟨acc.val ^^^ entry, Nat.xor_lt_two_pow acc.isLt (T ⟨idx, hidx_lt⟩).isLt⟩ + have hfit_ch : Fits w ch := hfit_u.of_le Nat.and_le_left + have hfit_addr : Fits w (base + 4 + j * 2 ^ b + ch) := + hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_xor : Fits w newAcc.val := hfit_powM.of_le (le_of_lt newAcc.isLt) + have hfit_shr : Fits w (u >>> b) := hfit_u.of_le (Nat.shiftRight_le u b) + have hnext_eq : base + 4 + j * 2 ^ b + 2 ^ b = base + 4 + (j + 1) * 2 ^ b := by ring + have hfit_next : Fits w (base + 4 + j * 2 ^ b + 2 ^ b) := by + have : (j + 1) * 2 ^ b ≤ d * 2 ^ b := Nat.mul_le_mul_right (2 ^ b) (by omega) + exact hfit_space.of_le (by simp [spaceFromB]; omega) + set s1 := d0.set regTmp ch + have e1 : Exec w (.op (.exactBin .and regTmp regKey regMaskB) : Code .exact 9) d0 s1 1 := + exec_exactBin (by simp [hkey, hmaskB, ch]) hfit_ch + set s2 := s1.set regTmp (base + 4 + j * 2 ^ b + ch) + have e2 : Exec w (.op (.exactBin .add regTmp regPtr regTmp) : Code .exact 9) s1 s2 1 := + exec_exactBin (by simp +decide [s1, hptr]) hfit_addr + set s3 := s2.set regTmp entry + have e3 : Exec w (.op (.load regTmp regTmp) : Code .exact 9) s2 s3 1 := by + simpa +decide [s3, s2, s1, hentry_mem] using + (Exec.load (B := .exact) (w := w) (d := regTmp) (a := regTmp) (x := s2)) + set s4 := s3.set regOut newAcc.val + have e4 : Exec w (.op (.exactBin .xor regOut regOut regTmp) : Code .exact 9) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, hout, newAcc]) hfit_xor + set s5 := s4.set regKey (u >>> b) + have e5 : Exec w (.op (.exactBin .shiftRight regKey regKey regB) : Code .exact 9) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, hkey, hb]) hfit_shr + set s6 := s5.set regPtr (base + 4 + j * 2 ^ b + 2 ^ b) + have e6 : Exec w (.op (.exactBin .add regPtr regPtr regPowB) : Code .exact 9) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, hptr, hpowB]) hfit_next + have hbody := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq e6)))) + obtain ⟨y, hrec, hy_out, hy_base, hy_mem⟩ := + ih (j + 1) (u >>> b) newAcc s6 (by omega) hmem + (by simp +decide [s6, s5, s4, s3, s2, s1, hbase]) + (by simp +decide [s6, hnext_eq]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hend]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hb]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hpowB]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hmaskB]) + (by simp +decide [s6, s5]) + (by simp +decide [s6, s5, s4]) + hfit_shr + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, ?_, hy_base, hy_mem⟩ + · simpa only [tabLoop, tabBody, + show 1 + (1 + (1 + (1 + (1 + 1)))) + (8 * rem + 2) + 2 = 8 * (rem + 1) + 2 by omega] + using hiter + · rw [hy_out] + simp only [simpleTabAux, ch, idx, hidx_lt, ↓reduceDIte, entry, newAcc] + +/-- 7-instruction finishing suffix of `evalCode d`: loads `2 ^ M - 1` from `base + 3` and maps +`acc` in `regOut` to `(2 * acc + 1) &&& (2 ^ M - 1)`. -/ +@[expose] def evalFinishCode : Code .exact 9 := + .seq (.op (.const regTmp 3)) + (.seq (.op (.exactBin .add regTmp regBase regTmp)) + (.seq (.op (.load regTmp regTmp)) + (.seq (.op (.exactBin .add regOut regOut regOut)) + (.seq (.op (.const regB 1)) + (.seq (.op (.exactBin .add regOut regOut regB)) + (.op (.exactBin .and regOut regOut regTmp))))))) + +/-- Complete `d`-character simple tabulation odd-stride evaluation program `evalCode d`. -/ +@[expose] def evalCode (d : ℕ) : Code .exact 9 := + .seq (evalSetupCode d) (.seq tabLoop evalFinishCode) + +/-- **`TabulatedOddStride.evalCode d` correctness and complexity**: for any character table +`T : Fin (d * 2 ^ b) → Fin (2 ^ M)` represented at `base` and key `u` with `Fits w u`, +`evalCode d` halts deterministically in `8 * d + 21 = O(d)` instructions, preserving `regBase` +and all memory and writing `(tabulatedOddStride b d M T u).val` into `regOut`. -/ +theorem evalCode_execWithin {w base b d M u : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 ^ M)} {x : Data 9} + (hrep : Rep base b d M T x.mem) + (hbase : x.regs regBase = base) (hu : x.regs regKey = u) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powM : Fits w (2 * 2 ^ M)) (hfit_d : Fits w d) (hfit_u : Fits w u) : + ∃ y : Data 9, + ExecWithin w (evalCode d) x y (8 * d + 21) ∧ + y.regs regOut = (tabulatedOddStride b d M T u).val ∧ + y.regs regBase = base ∧ + y.mem = x.mem := by + have hfit_M : Fits w (2 ^ M) := hfit_powM.of_le (by omega) + have hsetup := evalSetupCode_exec hrep hbase hfit_space hfit_d + set d0 := evalSetupData base b d x + obtain ⟨yLoop, hloop, hy_out, hy_base, hy_mem⟩ := + tabLoop_exec hrep hfit_space hfit_M d 0 u ⟨0, Nat.two_pow_pos M⟩ d0 + (by omega) rfl + (by simp +decide [d0, evalSetupData, hbase]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData, hu]) + (by simp +decide [d0, evalSetupData]) + hfit_u + set tab := simpleTab b d M T u + have htab_eq : yLoop.regs regOut = tab.val := hy_out + have hfit_3 : Fits w 3 := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b3 : Fits w (base + 3) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_1 : Fits w 1 := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_2tab : Fits w (tab.val + tab.val) := + hfit_powM.of_le (by have := tab.isLt; omega) + have hfit_2tab1 : Fits w (tab.val + tab.val + 1) := + hfit_powM.of_le (by have := tab.isLt; omega) + have hfit_and : Fits w ((tab.val + tab.val + 1) &&& (2 ^ M - 1)) := + hfit_2tab1.of_le Nat.and_le_left + have hm3 : yLoop.mem (base + 3) = 2 ^ M - 1 := by rw [hy_mem]; exact hrep.2.2.2.1 + set f1 := yLoop.set regTmp 3 + have e1 : Exec w (.op (.const regTmp 3) : Code .exact 9) yLoop f1 1 := Exec.const hfit_3 + set f2 := f1.set regTmp (base + 3) + have e2 : Exec w (.op (.exactBin .add regTmp regBase regTmp) : Code .exact 9) f1 f2 1 := + exec_exactBin (by simp +decide [f1, hy_base]) hfit_b3 + set f3 := f2.set regTmp (2 ^ M - 1) + have e3 : Exec w (.op (.load regTmp regTmp) : Code .exact 9) f2 f3 1 := by + simpa +decide [f3, f2, f1, hm3] using + (Exec.load (B := .exact) (w := w) (d := regTmp) (a := regTmp) (x := f2)) + set f4 := f3.set regOut (tab.val + tab.val) + have e4 : Exec w (.op (.exactBin .add regOut regOut regOut) : Code .exact 9) f3 f4 1 := + exec_exactBin (by simp +decide [f3, f2, f1, htab_eq]) hfit_2tab + set f5 := f4.set regB 1 + have e5 : Exec w (.op (.const regB 1) : Code .exact 9) f4 f5 1 := Exec.const hfit_1 + set f6 := f5.set regOut (tab.val + tab.val + 1) + have e6 : Exec w (.op (.exactBin .add regOut regOut regB) : Code .exact 9) f5 f6 1 := + exec_exactBin (by simp +decide [f5, f4]) hfit_2tab1 + set f7 := f6.set regOut ((tab.val + tab.val + 1) &&& (2 ^ M - 1)) + have e7 : Exec w (.op (.exactBin .and regOut regOut regTmp) : Code .exact 9) f6 f7 1 := + exec_exactBin (by simp +decide [f6, f5, f4, f3]) hfit_and + have hfin := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq e7))))) + have hall := hsetup.seq (hloop.seq hfin) + refine ⟨f7, ⟨12 + (8 * d + 2 + (1 + (1 + (1 + (1 + (1 + (1 + 1))))))), by omega, hall⟩, + ?_, by simp +decide [f7, f6, f5, f4, f3, f2, f1, hy_base], hy_mem⟩ + simp +decide [f7, tabulatedOddStride, oddStride, tab, Nat.two_mul] + +end TabulatedOddStride + +end Cslib.Algorithms.WordRAM.Hashing.Tabulation diff --git a/ComplexityTests/WordRAMKWiseHash.lean b/ComplexityTests/WordRAMKWiseHash.lean new file mode 100644 index 000000000..77b8a398d --- /dev/null +++ b/ComplexityTests/WordRAMKWiseHash.lean @@ -0,0 +1,431 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Algorithms.WordRAM.Hashing.KWise +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation + +/-! +# Concrete interpreter and non-vacuity tests for `Hashing.KWise` and `Hashing.Tabulation` + +Verifies both concrete `Cslib.WordRAM.Code.eval` execution and theorem-level non-vacuity of: +1. `Cslib.Algorithms.WordRAM.Hashing.KWise`: + - `initTableCode` and `evalCode` executed via `Code.eval` at `k = 5` for both `b = 1, L = 2` + (`n_max = 8`, `U = 4`, `M = 2`) and `b = 2, L = 1` (`n_max = 64`, `U = 4`, `M = 2`), and at + `t = 1` (`q = 6`, `b = 2`, `L = 3`, `n_max = 64`, `U = 16`, `M = 3`, `k = 5`), agreeing on + every tested key with direct unchunked `reduceTrinomialCode`/`clmul` Horner evaluation while + preserving boundary sentinels; + - non-vacuous instantiations of `initTableCode_execWithin`, `resampleCode_execRand`, + `initCode_execRandWithin`, `evalCode_execWithin`, and `isKWiseIndependentHash_eval` at `k = 5`, + `U = 1000`, `M = 10`, `n_max = 64`, `w = 64`. +2. `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable`: + - `lookupCode` and `xorLookupCode` executed via `Code.eval` on a concrete table of size + `2 ^ 3 = 8`; + - non-vacuous instantiations of `resampleCode_execRand`, `map_extractTable_eval_resampleComp`, + `resampleCode_execRandWithin`, `lookupCode_exec`, and `xorLookupCode_exec`. +3. `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride`: + - `evalCode 3` executed via `Code.eval` on concrete character tables (`d = 3`, `b = 1`, `M = 3`), + matching `tabulatedOddStride` and producing odd strides whose linear probe sequence `probeSeq` + visits all `2 ^ M = 8` table slots; + - non-vacuous instantiations of `initCode_execRand`, `map_extractTable_eval_initComp`, + `initCode_execRandWithin`, `evalCode_execWithin`, and `bijective_probeSeq`. +-/ + +@[expose] public section + +namespace ComplexityTests.WordRAMKWiseHash + +open Cslib Cslib.Probability Cslib.WordRAM +open Cslib.Algorithms.WordRAM.Hashing +open Cslib.Algorithms.WordRAM.Hashing.Tabulation + +/-! ## Helper for compacting interpreter state between `Code.eval` calls -/ + +/-- Rebuild a natural number from `0` and `Nat.succ` so that `Function.update` closures in its +originating expression are eliminated during kernel reduction. -/ +def forceNat : ℕ → Option ℕ + | 0 => some 0 + | n + 1 => (forceNat n).map Nat.succ + +/-- Eagerly force every element of a list of natural numbers via `forceNat`. -/ +def forceList : List ℕ → Option (List ℕ) + | [] => some [] + | x :: xs => + match forceNat x, forceList xs with + | some y, some ys => some (y :: ys) + | _, _ => none + +/-- Tabulate registers `0..r - 1` and memory `0..memCap - 1` into flat lists between interpreter +calls so `Function.update` closures do not accumulate across steps. -/ +def compactData {r : ℕ} (memCap : ℕ) (d : Data r) : Option (Data r) := do + let rs ← forceList ((List.finRange r).map d.regs) + let ms ← forceList ((List.range memCap).map d.mem) + let tail ← forceNat (d.mem memCap) + pure + { regs := fun i => rs.getD i.val 0 + mem := fun a => if a < memCap then ms.getD a 0 else tail } + +/-! ## 1. `KWise`: concrete `Code.eval` execution matching unchunked trinomial Horner evaluation -/ + +/-- Structurally recursive fuel-bounded carryless multiplication for kernel `decide` reduction. -/ +def clmulFuel : ℕ → ℕ → ℕ → ℕ + | 0, _, _ => 0 + | fuel + 1, a, b => + if b = 0 then 0 + else (if b % 2 = 1 then a else 0) ^^^ (clmulFuel fuel a (b / 2) <<< 1) + +/-- `clmulFuel fuel a b` agrees with `clmul a b` whenever `b < 2 ^ fuel`. -/ +theorem clmulFuel_eq_clmul : ∀ (fuel a b : ℕ), b < 2 ^ fuel → clmulFuel fuel a b = clmul a b + | 0, a, b, hb => by + have hb0 : b = 0 := by omega + subst hb0 + simp [clmulFuel] + | fuel + 1, a, b, hb => by + rcases eq_or_ne b 0 with rfl | hne + · simp [clmulFuel] + · have hb_div : b / 2 < 2 ^ fuel := by + rw [Nat.pow_succ] at hb + omega + rw [clmulFuel, clmul, ite_eq_right hne, ite_eq_right hne, + clmulFuel_eq_clmul fuel a (b / 2) hb_div] + +/-- Sample `5`-tuple of coefficients in `Fin 4` (`t = 0`, `q = 2`, `2 ^ q = 4`). -/ +def sampleCoeffs5 : Fin 5 → Fin 4 + | ⟨0, _⟩ => ⟨3, by decide⟩ + | ⟨1, _⟩ => ⟨1, by decide⟩ + | ⟨2, _⟩ => ⟨2, by decide⟩ + | ⟨3, _⟩ => ⟨3, by decide⟩ + | ⟨_, _⟩ => ⟨2, by decide⟩ + +/-- Computable unchunked reference Horner evaluator for `k = 5` over `TrinomialField t` using +`clmulFuel 6` and `reduceTrinomialCode` directly (without limb splitting), masked to `M` bits. -/ +def trinomialHornerRef5 (t M : ℕ) (c : Fin 5 → Fin (2 ^ (2 * 3 ^ t))) (u : ℕ) : ℕ := + let a4 := (c ⟨4, by decide⟩).val + let a3 := reduceTrinomialCode t (clmulFuel 6 a4 u) ^^^ (c ⟨3, by decide⟩).val + let a2 := reduceTrinomialCode t (clmulFuel 6 a3 u) ^^^ (c ⟨2, by decide⟩).val + let a1 := reduceTrinomialCode t (clmulFuel 6 a2 u) ^^^ (c ⟨1, by decide⟩).val + let a0 := reduceTrinomialCode t (clmulFuel 6 a1 u) ^^^ (c ⟨0, by decide⟩).val + a0 &&& (2 ^ M - 1) + +/-- Write the `5` coefficients of `sampleCoeffs5` into `[base + 8, base + 13)` of `d.mem`. -/ +def writeSampleCoeffs5 (base : ℕ) (d : Data 18) : Data 18 := + ((((d.write (base + 8) 3).write (base + 9) 1).write (base + 10) 2).write + (base + 11) 3).write (base + 12) 2 + +/-- Run `KWise.initTableCode 5` with `(base, n_max, U, M)` in registers `0..3`, populate +`sampleCoeffs5`, and evaluate `KWise.evalCode 5` via `Code.eval` on a list of keys `us`, returning +the list of output hash values together with the final values at boundary sentinels `base - 1` and +`base + KWise.space n_max 5`. -/ +def runKWiseEval5 (w fuel n_max U M base : ℕ) (us : List ℕ) : + Option (List ℕ × ℕ × ℕ) := do + let sp := KWise.space n_max 5 + let d0 : Data 18 := + ((((((Data.init [] : Data 18).set 0 base).set 1 n_max).set 2 U).set 3 M).write + (base - 1) 77).write (base + sp) 88 + let (d1, _) ← (KWise.initTableCode 5).eval w fuel d0 + let d2 ← compactData (base + sp + 2) (writeSampleCoeffs5 base d1) + let outs ← us.mapM (fun u => do + let dIn := (d2.set 0 base).set KWise.regKey u + let (dOut, _) ← (KWise.evalCode 5).eval w fuel dIn + forceNat (dOut.regs KWise.regOut)) + let sLo ← forceNat (d2.mem (base - 1)) + let sHi ← forceNat (d2.mem (base + sp)) + pure (outs, sLo, sHi) + +/-- Concrete `Code.eval` test at `k = 5, b = 1, L = 2` (`n_max = 8`, `U = 4`, `M = 2`, `base = 10`): +`KWise.initTableCode 5` builds the `4`-entry carryless multiplication table and metadata in +`[10, 27)`, and `KWise.evalCode 5` (performing `2 × 2` multi-limb chunk multiplication at each of +the `5` Horner steps) matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys +`u ∈ [0, 1, 2, 3]` while preserving sentinels at addresses `9` (`77`) and `27` (`88`). -/ +theorem kwise_eval5_b1_concrete : + runKWiseEval5 16 16 8 4 2 10 [0, 1, 2, 3] = + some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by + decide +kernel + +/-- Concrete `Code.eval` test at `k = 5, b = 2, L = 1` (`n_max = 64`, `U = 4`, `M = 2`, `base = 4`): +`KWise.initTableCode 5` builds the `16`-entry carryless multiplication table in `[4, 33)`, and +`KWise.evalCode 5` matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys `u ∈ [0, 1, 2, 3]` +while preserving sentinels at addresses `3` (`77`) and `33` (`88`). -/ +theorem kwise_eval5_b2_concrete : + runKWiseEval5 16 32 64 4 2 4 [0, 1, 2, 3] = + some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by + decide +kernel + +/-- Sample `5`-tuple of coefficients in `Fin 64` (`t = 1`, `q = 6`, `2 ^ q = 64`) for testing +`k = 5` with `3 × 3` multi-limb multiplication over `TrinomialField 1` (`X ^ 6 + X ^ 3 + 1`). -/ +def sampleCoeffs5T1 : Fin 5 → Fin 64 + | ⟨0, _⟩ => ⟨13, by decide⟩ + | ⟨1, _⟩ => ⟨42, by decide⟩ + | ⟨2, _⟩ => ⟨7, by decide⟩ + | ⟨3, _⟩ => ⟨59, by decide⟩ + | ⟨_, _⟩ => ⟨25, by decide⟩ + +/-- Run `KWise.initTableCode 5` and `KWise.evalCode 5` at `t = 1, q = 6, b = 2, L = 3` +(`n_max = 64, U = 16, M = 3, base = 4`). -/ +def runKWiseEval5T1 (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := do + let base := 4 + let sp := KWise.space 64 5 + let d0 : Data 18 := + ((((((Data.init [] : Data 18).set 0 base).set 1 64).set 2 16).set 3 3).write + (base - 1) 77).write (base + sp) 88 + let (d1, _) ← (KWise.initTableCode 5).eval 16 32 d0 + let d2 ← compactData (base + sp + 2) + (((((d1.write (base + 8) 13).write (base + 9) 42).write (base + 10) 7).write + (base + 11) 59).write (base + 12) 25) + let outs ← us.mapM (fun u => do + let dIn := (d2.set 0 base).set KWise.regKey u + let (dOut, _) ← (KWise.evalCode 5).eval 16 32 dIn + forceNat (dOut.regs KWise.regOut)) + let sLo ← forceNat (d2.mem (base - 1)) + let sHi ← forceNat (d2.mem (base + sp)) + pure (outs, sLo, sHi) + +/-- Concrete `Code.eval` test at `k = 5, t = 1, q = 6, b = 2, L = 3` (`3 × 3` limb multiplication +over `X ^ 6 + X ^ 3 + 1`): `KWise.evalCode 5` matches `trinomialHornerRef5 1 3 sampleCoeffs5T1` on +keys `u ∈ [0, 5, 11, 15]`. -/ +theorem kwise_eval5_t1_b2_L3_concrete : + runKWiseEval5T1 [0, 5, 11, 15] = + some ([0, 5, 11, 15].map (trinomialHornerRef5 1 3 sampleCoeffs5T1), 77, 88) := by + decide +kernel + +/-- Non-vacuous theorem-level instantiation of `KWise.initTableCode_execWithin`, +`KWise.resampleCode_execRand`, `KWise.initCode_execRandWithin`, `KWise.evalCode_execWithin`, and +`KWise.isKWiseIndependentHash_eval` at `k = 5`, `U = 1000`, `M = 10`, `n_max = 64`, `w = 64`, +`base = 0`. -/ +theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) + (h0 : x.regs 0 = 0) (h1 : x.regs 1 = 64) (h2 : x.regs 2 = 1000) (h3 : x.regs 3 = 10) : + (∃ y : Data 18, + ExecWithin 64 (KWise.initTableCode 5) x y (KWise.initTableSteps 64 1000 10) ∧ + KWise.TableRep 0 (KWise.chunkBits 64) 1000 10 5 y.mem) ∧ + ExecRandWithin 64 (KWise.initCode 5) x + (fun y => + KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 + (KWise.extractSeeds 0 1000 10 5 y.mem) y.mem ∧ + y.regs 0 = 0 ∧ + (∀ a, (a < 0 ∨ 0 + KWise.space 64 5 ≤ a) → y.mem a = x.mem a)) + (KWise.initTableSteps 64 1000 10 + 35) 5 ∧ + (∀ y : Data 18, + KWise.TableRep 0 (KWise.chunkBits 64) 1000 10 5 y.mem → + y.regs 0 = 0 → + IsKWiseIndependentHash 5 (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10) ∧ + ExecRand 64 (KWise.resampleCode 5) y (KWise.resampleComp 0 1000 10 5 y) ∧ + ((KWise.resampleComp 0 1000 10 5 y).eval).map + (fun z => KWise.extractSeeds 0 1000 10 5 z.1.mem) = + PMF.uniformOfFintype (Fin 5 → Fin (2 ^ trinomialDeg 1000 10)) ∧ + ExecRandWithin 64 (KWise.resampleCode 5) y + (fun z => + KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 + (KWise.extractSeeds 0 1000 10 5 z.mem) z.mem ∧ + z.regs 0 = 0 ∧ + (∀ a, (a < 0 + 8 ∨ 0 + 8 + 5 ≤ a) → z.mem a = y.mem a) ∧ + (∀ u : Fin 1000, ∀ x_eval : Data 18, + x_eval.mem = z.mem → + x_eval.regs 0 = 0 → + x_eval.regs 1 = u.val → + ∃ y_eval : Data 18, + ExecWithin 64 (KWise.evalCode 5) x_eval y_eval (KWise.evalSteps 64 1000 10 5) ∧ + y_eval.regs 0 = 0 ∧ + y_eval.regs 1 = u.val ∧ + y_eval.regs 2 = + (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10 + (KWise.extractSeeds 0 1000 10 5 z.mem) u).val ∧ + y_eval.mem = z.mem)) + 35 5) := by + have htr : trinomialDeg 1000 10 = 18 := by + have hclog : Nat.clog 2 (max 1000 2) ≤ 10 := + (Nat.clog_le_iff_le_pow one_lt_two).mpr (by decide) + have ht : trinomialTargetBits 1000 10 = 10 := by + rw [trinomialTargetBits]; omega + have hs : trinomialStep 1000 10 = 2 := by + rw [trinomialStep, ht, trinomialStepAux, trinomialStepAux, trinomialStepAux]; decide + rw [trinomialDeg, hs]; decide + have hw : 2 * trinomialDeg 1000 10 ≤ 64 := by omega + have hfit_sp : Fits 64 (0 + KWise.space 64 5) := by + rw [fits_iff] + dsimp [KWise.space, KWise.spaceFromB, KWise.chunkBits] + decide + have hfit_n : Fits 64 (max 64 2) := by rw [fits_iff]; decide + have hfit_U : Fits 64 (max 1000 2 + 1) := by rw [fits_iff]; decide + have hfit_q : Fits 64 (2 ^ trinomialDeg 1000 10) := by + rw [htr, fits_iff]; decide + obtain ⟨y, hy_exec, hy_tab, -, -⟩ := + KWise.initTableCode_execWithin h0 h1 h2 h3 hfit_sp hfit_n hfit_U hfit_q + refine ⟨⟨y, hy_exec, hy_tab⟩, + KWise.initCode_execRandWithin h0 h1 h2 h3 hfit_sp hfit_n hfit_U hfit_q, + fun y' htab' h0' => + KWise.isKWiseIndependentHash_eval htab' h0' hw hfit_sp⟩ + +/-! ## 2. `RandomTable`: concrete `Code.eval` execution and theorem instantiations -/ + +/-- Populate `Z = [6, 3, 7, 1, 5, 0, 4, 2]` at `base = 4` (`M = 3`) and run +`RandomTable.lookupCode` and `RandomTable.xorLookupCode` via `Code.eval`. -/ +def runRandomTableLookups : Option (List ℕ × List ℕ) := do + let base := 4 + let d0 : Data 6 := + (((((((((Data.init [] : Data 6).write 4 6).write 5 3).write 6 7).write 7 1).write + 8 5).write 9 0).write 10 4).write 11 2) + let lookups ← ([0, 2, 4, 7] : List ℕ).mapM (fun idx => do + let dIn := (d0.set RandomTable.regBase base).set RandomTable.regIdx idx + let (dOut, _) ← RandomTable.lookupCode.eval 16 8 dIn + forceNat (dOut.regs RandomTable.regOut)) + let xorLookups ← ([(5, 0), (5, 2), (3, 4), (7, 7)] : List (ℕ × ℕ)).mapM (fun (f, g) => do + let dIn := ((d0.set RandomTable.regBase base).set RandomTable.regIdx g).set + RandomTable.regOut f + let (dOut, _) ← RandomTable.xorLookupCode.eval 16 8 dIn + forceNat (dOut.regs RandomTable.regOut)) + pure (lookups, xorLookups) + +/-- Concrete `Code.eval` verification of `RandomTable.lookupCode` (`Z[idx]`) and +`RandomTable.xorLookupCode` (`f ^^^ Z[g]`). -/ +theorem randomTable_lookup_and_xorLookup_concrete : + runRandomTableLookups = + some ([6, 7, 5, 2], [5 ^^^ 6, 5 ^^^ 7, 3 ^^^ 5, 7 ^^^ 2]) := by + decide + +/-- Non-vacuous theorem-level instantiation of `RandomTable.resampleCode_execRand`, +`RandomTable.map_extractTable_eval_resampleComp`, `RandomTable.resampleCode_execRandWithin`, +`RandomTable.lookupCode_exec`, and `RandomTable.xorLookupCode_exec` at `M = 4` (`2 ^ M = 16`), +`base = 10`, `w = 16`. -/ +theorem randomTable_wordram_nonvacuous (x : Data 6) + (h0 : x.regs RandomTable.regBase = 10) (h1 : x.regs RandomTable.regM = 4) : + ExecRand 16 RandomTable.resampleCode x (RandomTable.resampleComp 10 4 x) ∧ + ((RandomTable.resampleComp 10 4 x).eval).map (fun z => RandomTable.extractTable 10 4 z.1.mem) = + PMF.uniformOfFintype (Fin 16 → Fin 16) ∧ + ExecRandWithin 16 RandomTable.resampleCode x + (fun y => + RandomTable.Rep 10 4 (RandomTable.extractTable 10 4 y.mem) y.mem ∧ + y.regs RandomTable.regBase = 10 ∧ + (∀ a, (a < 10 ∨ 10 + 16 ≤ a) → y.mem a = x.mem a)) + 87 16 ∧ + (∀ (Z : Fin 16 → Fin 16) (idx : Fin 16) (y : Data 6), + RandomTable.Rep 10 4 Z y.mem → + y.regs RandomTable.regBase = 10 → y.regs RandomTable.regIdx = idx.val → + Exec 16 RandomTable.lookupCode y (RandomTable.lookupData y) 2 ∧ + (RandomTable.lookupData y).regs RandomTable.regOut = (Z idx).val ∧ + (RandomTable.lookupData y).mem = y.mem) ∧ + (∀ (Z : Fin 16 → Fin 16) (idx fx : Fin 16) (y : Data 6), + RandomTable.Rep 10 4 Z y.mem → + y.regs RandomTable.regBase = 10 → y.regs RandomTable.regIdx = idx.val → + y.regs RandomTable.regOut = fx.val → + Exec 16 RandomTable.xorLookupCode y (RandomTable.xorLookupData y) 3 ∧ + (RandomTable.xorLookupData y).regs RandomTable.regOut = fx.val ^^^ (Z idx).val ∧ + (RandomTable.xorLookupData y).mem = y.mem) := by + have hfit_end : Fits 16 (10 + 2 ^ 4) := by rw [fits_iff]; decide + have hfit_pow : Fits 16 (2 ^ 4) := by rw [fits_iff]; decide + refine ⟨RandomTable.resampleCode_execRand h0 h1 hfit_end hfit_pow, + RandomTable.map_extractTable_eval_resampleComp 10 4 x, + RandomTable.resampleCode_execRandWithin h0 h1 hfit_end hfit_pow, + fun Z idx y hrep hy0 hy1 => ?_, + fun Z idx fx y hrep hy0 hy1 hy2 => ?_⟩ + · obtain ⟨he, ho, -, hm⟩ := RandomTable.lookupCode_exec hrep idx hy0 hy1 hfit_end + exact ⟨he, ho, hm⟩ + · obtain ⟨he, ho, -, -, hm⟩ := + RandomTable.xorLookupCode_exec hrep idx fx hy0 hy1 hy2 hfit_end hfit_pow + exact ⟨he, ho, hm⟩ + +/-! ## 3. `TabulatedOddStride`: concrete `Code.eval` execution and full-cycle permutation test -/ + +/-- Sample flat `3 * 2 ^ 1 = 6`-entry character table `T : Fin 6 → Fin 8` (`d = 3`, `b = 1`, +`M = 3`). -/ +def sampleFlatCharTable : Fin 6 → Fin 8 + | ⟨0, _⟩ => ⟨1, by decide⟩ + | ⟨1, _⟩ => ⟨3, by decide⟩ + | ⟨2, _⟩ => ⟨2, by decide⟩ + | ⟨3, _⟩ => ⟨0, by decide⟩ + | ⟨4, _⟩ => ⟨3, by decide⟩ + | ⟨_, _⟩ => ⟨2, by decide⟩ + +/-- Populate the 4 header words `(b = 1, 2 ^ b = 2, 2 ^ b - 1 = 1, 2 ^ M - 1 = 7)` and the 6 +entries of `sampleFlatCharTable` starting at `base = 2` (addresses `2..11`) and run +`TabulatedOddStride.evalCode 3` via `Code.eval` on all keys `u ∈ 0..7`. -/ +def runTabulatedOddStrideEval : Option (List ℕ) := do + let base := 2 + let d0 : Data 9 := + (((((((((((Data.init [] : Data 9).write 2 1).write 3 2).write 4 1).write 5 7).write + 6 1).write 7 3).write 8 2).write 9 0).write 10 3).write 11 2) + (List.range 8).mapM (fun u => do + let dIn := (d0.set TabulatedOddStride.regBase base).set TabulatedOddStride.regKey u + let (dOut, _) ← (TabulatedOddStride.evalCode 3).eval 16 16 dIn + forceNat (dOut.regs TabulatedOddStride.regOut)) + +/-- Concrete `Code.eval` verification of `TabulatedOddStride.evalCode 3`: on all `8` keys +`u ∈ 0..7`, `evalCode 3` matches +`(TabulatedOddStride.tabulatedOddStride 1 3 3 sampleFlatCharTable u).val` and always produces an odd +stride in `{1, 3, 5, 7}`. -/ +theorem tabulatedOddStride_evalCode_matches_spec : + runTabulatedOddStrideEval = + some ((List.range 8).map (fun u => + (TabulatedOddStride.tabulatedOddStride 1 3 3 sampleFlatCharTable u).val)) ∧ + ∀ u : ℕ, + (TabulatedOddStride.tabulatedOddStride 1 3 3 sampleFlatCharTable u).val % 2 = 1 := by + refine ⟨by decide, fun u => TabulatedOddStride.odd_oddStride (by decide) _⟩ + +/-- Full-cycle permutation property on `Fin 8` (`M = 3`): for every starting slot `h₁ : ℕ` and +every raw tabulation sum `tab : ℕ`, the linear probe sequence `i ↦ probeSeq 3 h₁ tab i` is a +bijection on `Fin 8` (and concretely visits all `8` slots without repetition). -/ +theorem tabulatedOddStride_probeSeq_full_cycle (h₁ tab : ℕ) : + Function.Bijective (TabulatedOddStride.probeSeq 3 h₁ tab) ∧ + ((List.finRange 8).map (fun i => + (TabulatedOddStride.probeSeq 3 3 + (TabulatedOddStride.simpleTab 1 3 3 sampleFlatCharTable 5).val i).val)).Nodup := + ⟨TabulatedOddStride.bijective_probeSeq 3 h₁ tab, by decide⟩ + +/-- Non-vacuous theorem-level instantiation of `TabulatedOddStride.initCode_execRand`, +`TabulatedOddStride.map_extractTable_eval_initComp`, `TabulatedOddStride.initCode_execRandWithin`, +and `TabulatedOddStride.evalCode_execWithin` at `n_max = 16`, `d = 4`, `M = 5` (`2 ^ M = 32`), +`base = 0`, `w = 16`. -/ +theorem tabulatedOddStride_wordram_nonvacuous (x : Data 9) + (h0 : x.regs TabulatedOddStride.regBase = 0) + (h1 : x.regs TabulatedOddStride.regKey = 16) + (h2 : x.regs TabulatedOddStride.regOut = 5) : + ExecRand 16 (TabulatedOddStride.initCode 4) x + (TabulatedOddStride.initComp 0 16 4 5 x) ∧ + ((TabulatedOddStride.initComp 0 16 4 5 x).eval).map + (fun z => TabulatedOddStride.extractTable 0 (TabulatedOddStride.charBits 16) 4 5 z.1.mem) = + PMF.uniformOfFintype (Fin (4 * 2 ^ TabulatedOddStride.charBits 16) → Fin 32) ∧ + ExecRandWithin 16 (TabulatedOddStride.initCode 4) x + (fun y => + TabulatedOddStride.Rep 0 (TabulatedOddStride.charBits 16) 4 5 + (TabulatedOddStride.extractTable 0 (TabulatedOddStride.charBits 16) 4 5 y.mem) y.mem ∧ + y.regs TabulatedOddStride.regBase = 0 ∧ + (∀ a, (a < 0 ∨ 0 + TabulatedOddStride.space 16 4 ≤ a) → y.mem a = x.mem a)) + (4 * Nat.log 2 (max 16 2) + 5 * (4 * 2 ^ TabulatedOddStride.charBits 16) + 30) + (4 * 2 ^ TabulatedOddStride.charBits 16) ∧ + (∀ (T : Fin (4 * 2 ^ TabulatedOddStride.charBits 16) → Fin 32) (u : Fin 256) (y : Data 9), + TabulatedOddStride.Rep 0 (TabulatedOddStride.charBits 16) 4 5 T y.mem → + y.regs TabulatedOddStride.regBase = 0 → + y.regs TabulatedOddStride.regKey = u.val → + ∃ z : Data 9, + ExecWithin 16 (TabulatedOddStride.evalCode 4) y z 53 ∧ + z.regs TabulatedOddStride.regOut = + (TabulatedOddStride.tabulatedOddStride + (TabulatedOddStride.charBits 16) 4 5 T u.val).val ∧ + z.regs TabulatedOddStride.regBase = 0 ∧ + z.mem = y.mem) := by + have hfit_n : Fits 16 (max 16 2) := by rw [fits_iff]; decide + have hfit_sp : Fits 16 (0 + TabulatedOddStride.space 16 4) := by + rw [fits_iff] + dsimp [TabulatedOddStride.space, TabulatedOddStride.spaceFromB, TabulatedOddStride.charBits] + decide + have hfit_powM : Fits 16 (2 ^ 5) := by rw [fits_iff]; decide + have hfit_2powM : Fits 16 (2 * 2 ^ 5) := by rw [fits_iff]; decide + have hfit_d : Fits 16 4 := by rw [fits_iff]; decide + refine ⟨TabulatedOddStride.initCode_execRand h0 h1 h2 hfit_n hfit_sp hfit_powM hfit_d, + TabulatedOddStride.map_extractTable_eval_initComp 0 16 4 5 x, + TabulatedOddStride.initCode_execRandWithin h0 h1 h2 hfit_n hfit_sp hfit_powM hfit_d, + fun T u y hrep hy0 hy1 => ?_⟩ + have hfit_u : Fits 16 u.val := by + rw [fits_iff] + have := u.isLt + omega + exact TabulatedOddStride.evalCode_execWithin hrep hy0 hy1 hfit_sp hfit_2powM hfit_d hfit_u + +end ComplexityTests.WordRAMKWiseHash + +end From 8f23b1138509fb3422afa61008c99d590a3e81cd Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Fri, 30 Oct 2026 00:33:43 +0000 Subject: [PATCH 16/18] fix(WordRAM/Hashing): compose isKWiseIndependentHash_eval and extract ExactLog --- Complexity.lean | 1 + Complexity/Algorithms/WordRAM/ExactLog.lean | 239 +++++++++++ .../Algorithms/WordRAM/Hashing/KWise.lean | 384 +++++++++--------- .../WordRAM/Hashing/Tabulation.lean | 155 ++----- ComplexityTests/WordRAMKWiseHash.lean | 99 +++-- 5 files changed, 503 insertions(+), 375 deletions(-) create mode 100644 Complexity/Algorithms/WordRAM/ExactLog.lean diff --git a/Complexity.lean b/Complexity.lean index 34622b270..5cc6274c0 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -89,6 +89,7 @@ public import Complexity.Algorithms.WordRAM.Dijkstra.Statement public import Complexity.Algorithms.WordRAM.DynamicArray public import Complexity.Algorithms.WordRAM.ElementDistinctness public import Complexity.Algorithms.WordRAM.ElementDistinctnessRAM +public import Complexity.Algorithms.WordRAM.ExactLog public import Complexity.Algorithms.WordRAM.ExactTriangle.Catalogue public import Complexity.Algorithms.WordRAM.ExactTriangle.ChunkCatalogue public import Complexity.Algorithms.WordRAM.ExactTriangle.Chunks diff --git a/Complexity/Algorithms/WordRAM/ExactLog.lean b/Complexity/Algorithms/WordRAM/ExactLog.lean new file mode 100644 index 000000000..adbfb2226 --- /dev/null +++ b/Complexity/Algorithms/WordRAM/ExactLog.lean @@ -0,0 +1,239 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Computability.WordRAM.Structured +public import Mathlib.Data.Nat.Log +import Mathlib.Tactic.FinCases + +/-! +# Exact-basis Word-RAM binary logarithms and arithmetic helpers + +This file provides structured subprograms over `Cslib.WordRAM.Basis.exact` for floor and ceiling +base-`2` logarithms (`log2Loop` and `clog2Loop`), an equal-cost register-maximum gadget (`maxCode`), +and an `.exactBin` execution helper (`exec_exactBin`). + +Whereas `Cslib.Algorithms.WordRAM.NatLog` and `Cslib.Algorithms.WordRAM.NatClog` are +`Code .standard 4` programs with `ExecWithin` upper bounds, the subprograms here operate in the +`.exact` basis using right-shift division by `2` (`>>> 1`) and carry exact closed-form `Exec` step +counts (`4 * Nat.log 2 rem + 2` and `5 * Nat.clog 2 rem + 2`) together with explicit final-state +formulas (`log2LoopData` and `clog2LoopData`) for embedding via `Code.rename`. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.ExactLog.maxCode`: 3-instruction equal-cost branch computing the + maximum of two registers. +* `Cslib.Algorithms.WordRAM.ExactLog.log2Body`, + `Cslib.Algorithms.WordRAM.ExactLog.log2Loop`, + `Cslib.Algorithms.WordRAM.ExactLog.log2LoopData`: floor-`log₂` loop over `Basis.exact`. +* `Cslib.Algorithms.WordRAM.ExactLog.clog2Body`, + `Cslib.Algorithms.WordRAM.ExactLog.clog2Loop`, + `Cslib.Algorithms.WordRAM.ExactLog.clog2LoopData`: ceiling-`log₂` loop over `Basis.exact`. + +## Main results + +* `Cslib.Algorithms.WordRAM.ExactLog.exec_exactBin`: single-instruction execution of `.exactBin`. +* `Cslib.Algorithms.WordRAM.ExactLog.maxCode_exec`: 3-instruction execution of `maxCode`. +* `Cslib.Algorithms.WordRAM.ExactLog.log2Loop_exec`: exact `4 * Nat.log 2 rem + 2`-instruction + execution of `log2Loop`. +* `Cslib.Algorithms.WordRAM.ExactLog.clog2Loop_exec`: exact `5 * Nat.clog 2 rem + 2`-instruction + execution of `clog2Loop`. +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.ExactLog + +open Cslib.WordRAM + +/-- Execute a single `.exactBin` operation when `evalExact` returns `some v` and `Fits w v`. -/ +theorem exec_exactBin {w r : ℕ} {f : BinOp} {dst i j : Fin r} {d : Data r} {v : ℕ} + (hev : f.evalExact (d.regs i) (d.regs j) = some v) (hfit : Fits w v) : + Exec w (.op (.exactBin f dst i j) : Code .exact r) d (d.set dst v) 1 := by + apply Exec.op + rw [Op.eval_exactBin, hev] + exact Data.put_of_fits hfit + +/-- 3-instruction equal-cost branch setting register `dst` to `max (x.regs a) (x.regs b)`. -/ +@[expose] def maxCode {n : ℕ} (dst a b : Fin n) : Code .exact n := + .branch (.lt a b) (.seq (.op (.copy dst b)) (.op (.copy dst dst))) (.op (.copy dst a)) + +/-- `maxCode dst a b` executes deterministically in 3 instructions and sets `dst` to +`max (x.regs a) (x.regs b)`. -/ +theorem maxCode_exec {w n : ℕ} (dst a b : Fin n) (x : Data n) : + Exec w (maxCode dst a b) x (x.set dst (max (x.regs a) (x.regs b))) 3 := by + by_cases hlt : x.regs a < x.regs b + · have hc : (Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs b := Nat.max_eq_right (le_of_lt hlt) + rw [hmax] + set d1 := x.set dst (x.regs b) + have s1 : Exec w (.op (.copy dst b) : Code .exact n) x d1 1 := Exec.copy + have hd1 : d1.set dst (d1.regs dst) = d1 := by + ext i : 2 <;> simp [d1, Data.set] + have s2 : Exec w (.op (.copy dst dst) : Code .exact n) d1 d1 1 := by + simpa only [hd1] using (Exec.copy (B := .exact) (w := w) (d := dst) (s := dst) (x := d1)) + exact Exec.branch_pos hc (s1.seq s2) + · have hc : ¬(Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs a := Nat.max_eq_left (not_lt.mp hlt) + rw [hmax] + exact Exec.branch_neg hc (Exec.copy (B := .exact)) + +/-- One iteration of the floor-`log₂` loop on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, +`reg 2 = 1`): shifts `rem` right by `1` (`rem / 2`) and increments `acc`. -/ +@[expose] def log2Body : Code .exact 3 := + .seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2)) + +/-- Floor-`log₂` loop `while 1 < rem do [ rem ← rem >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def log2Loop : Code .exact 3 := + .loop (.lt 2 0) log2Body + +/-- Closed-form final state of `log2Loop` starting from `rem` in register `0` and `acc` in +register `1`. -/ +@[expose] def log2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.log 2 rem) + +/-- `log2Loop` executes deterministically in `4 * Nat.log 2 rem + 2` instructions, accumulating +`acc + Nat.log 2 rem` into register `1` and preserving all memory. -/ +theorem log2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem : Fits w rem) (hfit_acc : Fits w (acc + Nat.log 2 rem)) : + Exec w log2Loop x (log2LoopData rem acc x) (4 * Nat.log 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : rem >>> 1 = rem / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hlog : Nat.log 2 rem = Nat.log 2 (rem / 2) + 1 := + Nat.log_of_one_lt_of_le (by decide) (by omega) + have hfit_div : Fits w (rem / 2) := hfit_rem.of_le (Nat.div_le_self rem 2) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem / 2) + have s1 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone, hshr]) hfit_div + set d2 := d1.set 1 (acc + 1) + have s2 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hacc, hone]) hfit_acc1 + have hdiv_lt : rem / 2 < rem := Nat.div_lt_self (by omega) (by decide) + have hrec := ih (rem / 2) hdiv_lt (acc := acc + 1) (x := d2) + (by simp +decide [d2, d1]) + (by simp +decide [d2]) + (by simp +decide [d2, d1, hone]) + hfit_div + (by rwa [show acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem by omega]) + have hd : log2LoopData (rem / 2) (acc + 1) d2 = log2LoopData rem acc x := by + have hdiv_ne : rem / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem + omega + · simp +decide [log2LoopData, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq s2) hrec + simpa only [log2Loop, log2Body, hlog, show 1 + 1 + (4 * Nat.log 2 (rem / 2) + 2) + 2 = + 4 * (Nat.log 2 (rem / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hlog0 : Nat.log 2 rem = 0 := Nat.log_of_lt (by omega) + have hd : log2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by + split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, hrem_eq, hrem] + · simp +decide [log2LoopData, hlog0, hacc] + · simp +decide [log2LoopData] + · rfl + rw [hd, hlog0] + exact Exec.loop_exit hc + +/-- Body of `clog2Loop` on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, `reg 2 = 1`): updates +`rem ← (rem + 1) >>> 1` (`= (rem + 1) / 2`) and `acc ← acc + 1`. -/ +@[expose] def clog2Body : Code .exact 3 := + .seq (.op (.exactBin .add 0 0 2)) + (.seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2))) + +/-- Ceiling-`log₂` loop `while 1 < rem do [ rem ← (rem + 1) >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def clog2Loop : Code .exact 3 := + .loop (.lt 2 0) clog2Body + +/-- Closed-form final state of `clog2Loop`. -/ +@[expose] def clog2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.clog 2 rem) + +/-- `clog2Loop` executes deterministically in `5 * Nat.clog 2 rem + 2` instructions, accumulating +`acc + Nat.clog 2 rem` into register `1` and preserving all memory. -/ +theorem clog2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem1 : Fits w (rem + 1)) (hfit_acc : Fits w (acc + Nat.clog 2 rem)) : + Exec w clog2Loop x (clog2LoopData rem acc x) (5 * Nat.clog 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : (rem + 1) >>> 1 = (rem + 1) / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hclog : Nat.clog 2 rem = Nat.clog 2 ((rem + 1) / 2) + 1 := + Nat.clog_of_one_lt (by decide) hlt + have hdiv_lt : (rem + 1) / 2 < rem := by omega + have hfit_div1 : Fits w ((rem + 1) / 2 + 1) := hfit_rem1.of_le (by omega) + have hfit_div : Fits w ((rem + 1) / 2) := hfit_div1.of_le (by omega) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem + 1) + have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone]) hfit_rem1 + set d2 := d1.set 0 ((rem + 1) / 2) + have s2 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hone, hshr]) hfit_div + set d3 := d2.set 1 (acc + 1) + have s3 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d2 d3 1 := + exec_exactBin (by simp +decide [d2, d1, hacc, hone]) hfit_acc1 + have hrec := ih ((rem + 1) / 2) hdiv_lt (acc := acc + 1) (x := d3) + (by simp +decide [d3, d2]) + (by simp +decide [d3]) + (by simp +decide [d3, d2, d1, hone]) + hfit_div1 + (by rwa [show acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem by omega]) + have hd : clog2LoopData ((rem + 1) / 2) (acc + 1) d3 = clog2LoopData rem acc x := by + have hdiv_ne : (rem + 1) / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, d3, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem + omega + · simp +decide [clog2LoopData, d3, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec + simpa only [clog2Loop, clog2Body, hclog, + show 1 + (1 + 1) + (5 * Nat.clog 2 ((rem + 1) / 2) + 2) + 2 = + 5 * (Nat.clog 2 ((rem + 1) / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hclog0 : Nat.clog 2 rem = 0 := Nat.clog_of_right_le_one (by omega) 2 + have hd : clog2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, hrem_eq, hrem] + · simp +decide [clog2LoopData, hclog0, hacc] + · simp +decide [clog2LoopData] + · rfl + rw [hd, hclog0] + exact Exec.loop_exit hc + +end Cslib.Algorithms.WordRAM.ExactLog diff --git a/Complexity/Algorithms/WordRAM/Hashing/KWise.lean b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean index 3697a633c..95df6c383 100644 --- a/Complexity/Algorithms/WordRAM/Hashing/KWise.lean +++ b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean @@ -7,9 +7,11 @@ Authors: Renfei Zhou module -- upstream: candidate +public import Complexity.Algorithms.WordRAM.ExactLog public import Complexity.Algorithms.WordRAM.Hashing.Tabulation public import Complexity.Foundations.FieldTheory.Finite.Trinomial public import Complexity.Probability.KWiseIndependent.Trinomial +import Complexity.Probability.PMF.Map /-! # Self-contained Word-RAM `k`-wise independent hashing @@ -19,29 +21,35 @@ This file implements self-contained Word-RAM `k`-wise independent hashing over finite field $\mathbb{F}_{2^{2 \cdot 3^t}} = \mathbb{F}_2[X] / (X^{2 \cdot 3^t} + X^{3^t} + 1)$ (`Cslib.TrinomialField t`, where `t = trinomialStep U M` is chosen at runtime so that $q = 2 \cdot 3^t \ge \max(\lceil \log_2 \max(U, 2) \rceil, \max(M, 1))$) and projecting to the low -`M` bits (`Cslib.Probability.trinomialPolyHash`) [Carter and Wegman][CarterWegman1979]; -[Wegman and Carter][WegmanCarter1981]; [Lidl and Niederreiter][LidlNiederreiter1997]. - -To achieve `O(1)` Word-RAM multiplication in $\mathbb{F}_{2^q}$ without dedicated carryless -multiplication hardware when $U \le n_{\max}^c$, we precompute the Four-Russians carryless -multiplication lookup table `clmulTab[0 .. 2 ^ (2 * b) - 1]` for $b$-bit chunks -(`b = chunkBits n_max = max 1 (⌊log₂ (max n_max 2)⌋ / 3)`, so $2^{2b} \le \max(4, n_{\max}) = -O(n_{\max}^{2/3}) \le O(n_{\max})$ words), decompose each $q$-bit field element into -$L = \lceil q / b \rceil$ limbs of $b$ bits, accumulate the $L \times L$ chunk carryless products, -and fold the resulting $< 2^{2q}$ polynomial mod $X^{2r} + X^r + 1$ in 8 straight-line bit -operations (`reduceTrinomialCode`). +`M` bits (`Cslib.Probability.trinomialPolyHash`). The binary-field polynomial construction and +bit-prefix projection follow [Vadhan, Section 3.5.5, Construction 3.32 / Proposition 3.33 and +Corollary 3.34][Vadhan2012] and [Wegman and Carter][WegmanCarter1981], building on the universal +hashing framework of [Carter and Wegman, pp. 144–145][CarterWegman1979] and finite-field arithmetic +in [Lidl and Niederreiter][LidlNiederreiter1997]. + +Because `Basis.exact` lacks a hardware carryless-multiplication instruction, `initTableCode k` +precomputes a Four-Russians carryless multiplication table `clmulTab[0 .. 2 ^ (2 * b) - 1]` for +$b$-bit chunks (`b = chunkBits n_max = max 1 (⌊log₂ (max n_max 2)⌋ / 3)`). This table uses at most +$\max(4, n_{\max})$ words for all $n_{\max}$ (`two_pow_two_mul_chunkBits_le`) and satisfies the +sublinear cube bound $(2^{2b})^3 \le n_{\max}^2$, i.e., $2^{2b} \le n_{\max}^{2/3}$, whenever +$n_{\max} \ge 8$ (`two_pow_two_mul_chunkBits_pow_three_le`). Each $q$-bit field multiplication +splits its operands into $L = \lceil q / b \rceil$ limbs of $b$ bits, accumulates the $L \times L$ +chunk products from `clmulTab`, and reduces the resulting $< 2^{2q}$ polynomial modulo +$X^{2r} + X^r + 1$ in 8 bit operations (`reduceTrinomialCode`). Under the closed-form bound +`evalSteps n_max U M k = ((11 * L + 12) * L + 30) * k + 19`, each field multiplication takes +$O(L^2)$ instructions, which is $O(1)$ instructions when both $U \le n_{\max}^c$ and +$M \le c \log_2 n_{\max}$ for a fixed constant $c$. The initialization is factored into two reusable stages: 1. **`initTableCode k`**: deterministically computes $r = 3^t$, $q = 2r$, $b$, masks, and populates - `clmulTab[0 .. 2 ^ (2 * b) - 1]` in $O(n_{\max})$ deterministic instructions from - `(base, n_max, U, M)` alone (zero external primes or lookup hints). + `clmulTab[0 .. 2 ^ (2 * b) - 1]` in `initTableSteps n_max U M` + ($= O(n_{\max} + \log U + \log M)$) deterministic instructions from `(base, n_max, U, M)` alone. 2. **`resampleCode k`**: samples the $k$ polynomial coefficients `seed[0 .. k - 1]` uniformly and - independently in `Fin (2 ^ q)` using $k$ `Code.rand` calls in $O(k)$ instructions while - preserving `clmulTab`, so Cuckoo Hashing can rehash in $O(k) = O(1)$ instructions without - rebuilding `clmulTab`. + independently in `Fin (2 ^ q)` using $k$ `Code.rand` calls in $5k + 10 = O(k)$ instructions while + preserving `clmulTab`, so rehashing takes $O(k)$ instructions without rebuilding `clmulTab`. 3. **`initCode k := .seq (initTableCode k) (resampleCode k)`**: full one-shot initialization. -4. **`evalCode k`**: evaluates Horner's rule for `trinomialPolyHash` in $O(k \cdot L^2 + 1)$ - deterministic instructions, preserving all memory and writing +4. **`evalCode k`**: evaluates Horner's rule for `trinomialPolyHash` in `evalSteps n_max U M k` + ($= O(k \cdot L^2 + 1)$) deterministic instructions, preserving all memory and writing `(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `regOut`. ## Main definitions @@ -56,25 +64,33 @@ The initialization is factored into two reusable stages: * `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode`, `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode`, `Cslib.Algorithms.WordRAM.Hashing.KWise.initCode`, - `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode`: structured Word-RAM programs over - `Basis.exact`. + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalHashOut`: structured Word-RAM programs over + `Basis.exact` and the induced deterministic hash output map. ## Main results -* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode_execWithin`: deterministic $O(n_{\max})$ - table construction establishing `TableRep`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.two_pow_two_mul_chunkBits_le`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.two_pow_two_mul_chunkBits_pow_three_le`: coarse + $\max(4, n_{\max})$ and sublinear $n_{\max}^{2/3}$ bounds on the Four-Russians table size. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode_execWithin`: deterministic + `initTableSteps n_max U M` ($= O(n_{\max} + \log U + \log M)$) table construction establishing + `TableRep`. * `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRand`, `Cslib.Algorithms.WordRAM.Hashing.KWise.map_extractSeeds_eval_resampleComp`, - `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRandWithin`: $O(k)$-step, $k$-query seed - resampling preserving `TableRep` and inducing `PMF.uniformOfFintype (Fin k → Fin (2 ^ q))`. -* `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode_execWithin`: deterministic $O(k \cdot L^2 + 1)$ - Horner evaluation matching `trinomialPolyHash`. -* `Cslib.Algorithms.WordRAM.Hashing.KWise.isKWiseIndependentHash_eval`: end-to-end Word-RAM - $k$-wise independence theorem combining `resampleComp`, `evalCode_execWithin`, and - `isKWiseIndependentHash_trinomialPolyHash`. + `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRandWithin`: $(5k + 10)$-step, $k$-query + seed resampling preserving `TableRep` and inducing `PMF.uniformOfFintype (Fin k → Fin (2 ^ q))`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode_execWithin`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalHashOut_eq_trinomialPolyHash`: deterministic + `evalSteps n_max U M k` ($= O(k \cdot L^2 + 1)$) Horner evaluation matching `trinomialPolyHash`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.map_evalHashOut_eval_resampleComp`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.isKWiseIndependentHash_eval`: composed Word-RAM $k$-wise + independence theorem proving that for every set $T$ of at most $k$ keys, the restricted tuple of + `evalCode k` outputs after `resampleCode k` is uniformly distributed on $T \to \text{Fin}(2^M)$. ## References +* [S. P. Vadhan, *Pseudorandomness*, Section 3.5.5][Vadhan2012] * [J. L. Carter and M. N. Wegman, *Universal Classes of Hash Functions*][CarterWegman1979] * [M. N. Wegman and J. L. Carter, *New Hash Functions and Their Use in Authentication and Set Equality*][WegmanCarter1981] @@ -87,6 +103,7 @@ public section namespace Cslib.Algorithms.WordRAM.Hashing.KWise open Cslib.WordRAM Cslib.Probability +open Cslib.Algorithms.WordRAM.ExactLog open Cslib.Algorithms.WordRAM.Hashing.Tabulation open scoped ENNReal @@ -106,7 +123,7 @@ theorem chunkBits_pos (n_max : ℕ) : 0 < chunkBits n_max := one_le_chunkBits n_max /-- The Four-Russians carryless multiplication table size `2 ^ (2 * chunkBits n_max)` is at most -`max 4 n_max` (`= O(n_max^(2/3)) ≤ O(n_max)` words). -/ +`max 4 n_max` for all `n_max`. -/ theorem two_pow_two_mul_chunkBits_le (n_max : ℕ) : 2 ^ (2 * chunkBits n_max) ≤ max 4 n_max := by by_cases hdiv : Nat.log 2 (max n_max 2) / 3 ≤ 1 @@ -121,6 +138,24 @@ theorem two_pow_two_mul_chunkBits_le (n_max : ℕ) : _ ≤ max n_max 2 := Nat.pow_log_le_self 2 (by omega) _ ≤ max 4 n_max := by omega +/-- For `8 ≤ n_max`, the Four-Russians table size `2 ^ (2 * chunkBits n_max)` satisfies the +sublinear cube bound `(2 ^ (2 * chunkBits n_max)) ^ 3 ≤ n_max ^ 2` (i.e., +`2 ^ (2 * chunkBits n_max) ≤ n_max ^ (2 / 3)`). -/ +theorem two_pow_two_mul_chunkBits_pow_three_le {n_max : ℕ} (hn : 8 ≤ n_max) : + (2 ^ (2 * chunkBits n_max)) ^ 3 ≤ n_max ^ 2 := by + have hmax : max n_max 2 = n_max := Nat.max_eq_left (by omega) + have hlog3 : 3 ≤ Nat.log 2 n_max := + Nat.le_log_of_pow_le (by decide) (by omega) + have hcb : chunkBits n_max = Nat.log 2 n_max / 3 := by + simp only [chunkBits, hmax] + omega + have hexp : 2 * chunkBits n_max * 3 ≤ Nat.log 2 n_max * 2 := by omega + calc + (2 ^ (2 * chunkBits n_max)) ^ 3 = 2 ^ (2 * chunkBits n_max * 3) := (pow_mul 2 _ 3).symm + _ ≤ 2 ^ (Nat.log 2 n_max * 2) := Nat.pow_le_pow_right (by decide) hexp + _ = (2 ^ Nat.log 2 n_max) ^ 2 := pow_mul 2 _ 2 + _ ≤ n_max ^ 2 := Nat.pow_le_pow_left (Nat.pow_log_le_self 2 (by omega)) 2 + /-- Number of `b`-bit limbs `L = ⌈q / b⌉ = (q + b - 1) / b` needed to cover a `q`-bit field element. -/ @[expose] def limbCountFromB (b q : ℕ) : ℕ := @@ -196,84 +231,6 @@ coefficients `(c j).val < 2 ^ (trinomialDeg U M)` at `base + 8 + j.val`. -/ /-! ## Section 2: Helper loops for `initTableCode` -/ -/-- Body of `clog2Loop` on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, `reg 2 = 1`): updates -`rem ← (rem + 1) >>> 1` (`= (rem + 1) / 2`) and `acc ← acc + 1`. -/ -@[expose] def clog2Body : Code .exact 3 := - .seq (.op (.exactBin .add 0 0 2)) - (.seq (.op (.exactBin .shiftRight 0 0 2)) - (.op (.exactBin .add 1 1 2))) - -/-- Ceiling-`log₂` loop `while 1 < rem do [ rem ← (rem + 1) >>> 1; acc ← acc + 1 ]`. -/ -@[expose] def clog2Loop : Code .exact 3 := - .loop (.lt 2 0) clog2Body - -/-- Closed-form final state of `clog2Loop`. -/ -@[expose] def clog2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := - (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.clog 2 rem) - -/-- `clog2Loop` executes deterministically in `5 * Nat.clog 2 rem + 2` instructions, accumulating -`acc + Nat.clog 2 rem` into register `1` and preserving all memory. -/ -theorem clog2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} - (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) - (hfit_rem1 : Fits w (rem + 1)) (hfit_acc : Fits w (acc + Nat.clog 2 rem)) : - Exec w clog2Loop x (clog2LoopData rem acc x) (5 * Nat.clog 2 rem + 2) := by - induction rem using Nat.strong_induction_on generalizing acc x with - | h rem ih => - by_cases hlt : 1 < rem - · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hshr : (rem + 1) >>> 1 = (rem + 1) / 2 := by - rw [Nat.shiftRight_eq_div_pow] - have hclog : Nat.clog 2 rem = Nat.clog 2 ((rem + 1) / 2) + 1 := - Nat.clog_of_one_lt (by decide) hlt - have hdiv_lt : (rem + 1) / 2 < rem := by omega - have hfit_div1 : Fits w ((rem + 1) / 2 + 1) := hfit_rem1.of_le (by omega) - have hfit_div : Fits w ((rem + 1) / 2) := hfit_div1.of_le (by omega) - have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) - set d1 := x.set 0 (rem + 1) - have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 3) x d1 1 := - exec_exactBin (by simp [hrem, hone]) hfit_rem1 - set d2 := d1.set 0 ((rem + 1) / 2) - have s2 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) d1 d2 1 := - exec_exactBin (by simp +decide [d1, hone, hshr]) hfit_div - set d3 := d2.set 1 (acc + 1) - have s3 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d2 d3 1 := - exec_exactBin (by simp +decide [d2, d1, hacc, hone]) hfit_acc1 - have hrec := ih ((rem + 1) / 2) hdiv_lt (acc := acc + 1) (x := d3) - (by simp +decide [d3, d2]) - (by simp +decide [d3]) - (by simp +decide [d3, d2, d1, hone]) - hfit_div1 - (by rwa [show acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem by omega]) - have hd : clog2LoopData ((rem + 1) / 2) (acc + 1) d3 = clog2LoopData rem acc x := by - have hdiv_ne : (rem + 1) / 2 ≠ 0 := by omega - have hrem_ne : rem ≠ 0 := by omega - ext i : 2 - · fin_cases i - · simp +decide [clog2LoopData, d3, d2, d1, hdiv_ne, hrem_ne] - · change acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem - omega - · simp +decide [clog2LoopData, d3, d2, d1] - · rfl - rw [hd] at hrec - have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec - simpa only [clog2Loop, clog2Body, hclog, - show 1 + (1 + 1) + (5 * Nat.clog 2 ((rem + 1) / 2) + 2) + 2 = - 5 * (Nat.clog 2 ((rem + 1) / 2) + 1) + 2 by omega] using hiter - · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hclog0 : Nat.clog 2 rem = 0 := Nat.clog_of_right_le_one (by omega) 2 - have hd : clog2LoopData rem acc x = x := by - have hrem_eq : (if rem = 0 then 0 else 1) = rem := by split_ifs <;> omega - ext i : 2 - · fin_cases i - · simp +decide [clog2LoopData, hrem_eq, hrem] - · simp +decide [clog2LoopData, hclog0, hacc] - · simp +decide [clog2LoopData] - · rfl - rw [hd, hclog0] - exact Exec.loop_exit hc - /-- Final remainder reached by `trinomialStepAux` when repeatedly mapping `B ↦ (B + 2) / 3` until `≤ 2`. -/ @[expose] def trinomialStepRem (B : ℕ) : ℕ := @@ -1428,6 +1385,42 @@ theorem map_extractSeeds_eval_resampleComp (base U M k : ℕ) (x : Data 18) : rw [ProbComp.eval_map, PMF.map_comp] exact map_extractSlice_eval_randFillComp (trinomialDeg U M) (base + 8) k _ +/-- Every outcome `z ∈ (resampleComp base U M k x).support` preserves `TableRep base b U M k`, +establishes `Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem`, sets `z.1.regs 0 = base`, +modifies memory only within `[base + 8, base + 8 + k)`, and takes at most `5 * k + 10` steps. -/ +theorem mem_support_resampleComp {base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + {z : Data 18 × ℕ} (hz : z ∈ (resampleComp base U M k x).support) : + (Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem ∧ + z.1.regs 0 = base ∧ + (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → z.1.mem a = x.mem a)) ∧ + z.2 ≤ 5 * k + 10 := by + dsimp only [resampleComp] at hz + obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hout⟩ := mem_support_randFillComp hz0 + have htab' : TableRep base b U M k z0.1.mem := by + refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · rw [hout base (Or.inl (by omega))]; exact htab.1 + · rw [hout (base + 1) (Or.inl (by omega))]; exact htab.2.1 + · rw [hout (base + 2) (Or.inl (by omega))]; exact htab.2.2.1 + · rw [hout (base + 3) (Or.inl (by omega))]; exact htab.2.2.2.1 + · rw [hout (base + 4) (Or.inl (by omega))]; exact htab.2.2.2.2.1 + · rw [hout (base + 5) (Or.inl (by omega))]; exact htab.2.2.2.2.2.1 + · rw [hout (base + 6) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.1 + · rw [hout (base + 7) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.2.1 + · intro idx hidx + rw [hout (base + 8 + k + idx) (Or.inr (by omega))] + exact htab.2.2.2.2.2.2.2.2 idx hidx + have hseed : SeedRep base U M k (extractSeeds base U M k z0.1.mem) z0.1.mem := by + intro j + simp [extractSeeds, extractSlice, hlt j.val j.isLt] + have hbase' : (z0.1.extend resampleMap (resampleSetupData base U M k x)).regs 0 = base := by + change ((z0.1.extend resampleMap (resampleSetupData base U M k x)).comap + resampleMap).regs regExtra = base + rw [Data.comap_extend injective_resampleMap, hextra] + simpa +decide [resampleSetupData, resampleMap, Data.comap, regExtra] using h0 + exact ⟨⟨⟨htab', hseed⟩, hbase', hout⟩, by omega⟩ + /-- **`KWise.resampleCode k` Hoare-style specification**: halts in `5 * k + 10 = O(k)` steps and `k` `rand` queries, preserving `TableRep base b U M k`, establishing `Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem`, preserving `reg 0 = base`, and @@ -1446,31 +1439,7 @@ theorem resampleCode_execRandWithin {w base b U M k : ℕ} {x : Data 18} exact (OracleComp.isCostBounded_map_iff _ _ _).mpr (isQueryBounded_randFillComp (trinomialDeg U M) (base + 8) k _) · intro z hz - dsimp only [resampleComp] at hz - obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz - obtain ⟨hsteps, -, -, -, -, hextra, hlt, hout⟩ := mem_support_randFillComp hz0 - have htab' : TableRep base b U M k z0.1.mem := by - refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ - · rw [hout base (Or.inl (by omega))]; exact htab.1 - · rw [hout (base + 1) (Or.inl (by omega))]; exact htab.2.1 - · rw [hout (base + 2) (Or.inl (by omega))]; exact htab.2.2.1 - · rw [hout (base + 3) (Or.inl (by omega))]; exact htab.2.2.2.1 - · rw [hout (base + 4) (Or.inl (by omega))]; exact htab.2.2.2.2.1 - · rw [hout (base + 5) (Or.inl (by omega))]; exact htab.2.2.2.2.2.1 - · rw [hout (base + 6) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.1 - · rw [hout (base + 7) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.2.1 - · intro idx hidx - rw [hout (base + 8 + k + idx) (Or.inr (by omega))] - exact htab.2.2.2.2.2.2.2.2 idx hidx - have hseed : SeedRep base U M k (extractSeeds base U M k z0.1.mem) z0.1.mem := by - intro j - simp [extractSeeds, extractSlice, hlt j.val j.isLt] - have hbase' : (z0.1.extend resampleMap (resampleSetupData base U M k x)).regs 0 = base := by - change ((z0.1.extend resampleMap (resampleSetupData base U M k x)).comap - resampleMap).regs regExtra = base - rw [Data.comap_extend injective_resampleMap, hextra] - simpa +decide [resampleSetupData, resampleMap, Data.comap, regExtra] using h0 - exact ⟨⟨⟨htab', hseed⟩, hbase', hout⟩, by omega⟩ + exact mem_support_resampleComp htab h0 hz /-- Full one-shot initialization program `initCode k := .seq (initTableCode k) (resampleCode k)`. -/ @[expose] def initCode (k : ℕ) : Code .exact 18 := @@ -2461,7 +2430,7 @@ and sets `reg 7 = base + 8 + k`. -/ /-- Deterministic evaluation program `evalCode k`: given `reg 0 = base` and `reg 1 = u.val`, evaluates `trinomialPolyHash (trinomialStep U M) k U M c u` into `reg 2` (`regOut`) in -`evalSteps b U M k` instructions while preserving `reg 0 = base`, `reg 1 = u.val`, and +`evalStepsFromB b U M k` instructions while preserving `reg 0 = base`, `reg 1 = u.val`, and all memory. -/ @[expose] def evalCode (k : ℕ) : Code .exact 18 := .seq (evalSetupCode k) (.seq hornerLoop evalMaskCode) @@ -2661,70 +2630,95 @@ theorem evalCode_execWithin {w base n_max U M k : ℕ} y.mem = x.mem := evalCode_execWithin_fromB u (chunkBits_pos n_max) hrep h0 h1 hw hfit_space -/-- **End-to-end Word-RAM `k`-wise independent hash theorem (`isKWiseIndependentHash_eval`)**: -1. The hash family `trinomialPolyHash (trinomialStep U M) k U M` evaluated by `evalCode k` is - `k`-wise independent onto `Fin (2 ^ M)` (`IsKWiseIndependentHash k`). -2. From any machine state `x` satisfying `TableRep base (chunkBits n_max) U M k x.mem` and - `x.regs 0 = base`, `resampleCode k` runs in `5 * k + 10` instructions and `k` `Code.rand` - queries with distribution `resampleComp base U M k x`, whose extracted seed - `extractSeeds base U M k z.1.mem` is uniformly distributed over - `Fin k → Fin (2 ^ trinomialDeg U M)` (`PMF.uniformOfFintype`). -3. Every resampled state `y` produced by `resampleCode k` satisfies `Rep`, so for every key - `u : Fin U`, running `evalCode k` from any state `x_eval` with `x_eval.mem = y.mem`, - `x_eval.regs 0 = base`, and `x_eval.regs 1 = u.val` halts deterministically in - `evalSteps n_max U M k` instructions, preserving all memory and writing - `(trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val` to - `reg 2`. -/ -theorem isKWiseIndependentHash_eval {w base n_max U M k : ℕ} {x : Data 18} +/-- Output of deterministic `evalCode k` execution at word size `w` from state `y` with +`reg 0 ← base` and `reg 1 ← u.val`, reduced modulo `2 ^ M` into `Fin (2 ^ M)`. -/ +noncomputable def evalHashOut (w base k U M : ℕ) (y : Data 18) (u : Fin U) : Fin (2 ^ M) := + open Classical in + let x_u := (y.set 0 base).set 1 u.val + if h : ∃ z n, Exec w (evalCode k) x_u z n then + ⟨(Classical.choose h).regs 2 % 2 ^ M, Nat.mod_lt _ (Nat.two_pow_pos M)⟩ + else + ⟨0, Nat.two_pow_pos M⟩ + +/-- On any machine state `y` satisfying `Rep base (chunkBits n_max) U M k c y.mem`, the +deterministic `evalCode k` output `evalHashOut w base k U M y u` equals +`trinomialPolyHash (trinomialStep U M) k U M c u`. -/ +theorem evalHashOut_eq_trinomialPolyHash {w base n_max U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} {y : Data 18} + (hrep : Rep base (chunkBits n_max) U M k c y.mem) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) + (u : Fin U) : + evalHashOut w base k U M y u = trinomialPolyHash (trinomialStep U M) k U M c u := by + let x_u := (y.set 0 base).set 1 u.val + have hrep_u : Rep base (chunkBits n_max) U M k c x_u.mem := hrep + have h0_u : x_u.regs 0 = base := by simp +decide [x_u] + have h1_u : x_u.regs 1 = u.val := by simp +decide [x_u] + obtain ⟨z, ⟨n, _, hexec⟩, -, -, hz2, -⟩ := + evalCode_execWithin u hrep_u h0_u h1_u hw hfit_space + have hex : ∃ z n, Exec w (evalCode k) x_u z n := ⟨z, n, hexec⟩ + obtain ⟨n', hexec'⟩ := Classical.choose_spec hex + have huniq : Classical.choose hex = z := (hexec'.unique hexec).1 + ext + simp only [evalHashOut, x_u, hex, ↓reduceDIte, huniq, hz2, + Nat.mod_eq_of_lt (trinomialPolyHash _ k U M c u).isLt] + +/-- Under `resampleComp base U M k x`, the Word-RAM hash function `evalHashOut w base k U M z.1` +has the exact distribution of `trinomialPolyHash (trinomialStep U M) k U M c` for a uniform seed +`c : Fin k → Fin (2 ^ trinomialDeg U M)`. -/ +theorem map_evalHashOut_eval_resampleComp {w base n_max U M k : ℕ} {x : Data 18} (htab : TableRep base (chunkBits n_max) U M k x.mem) (h0 : x.regs 0 = base) (hw : 2 * trinomialDeg U M ≤ w) (hfit_space : Fits w (base + space n_max k)) : - IsKWiseIndependentHash k (trinomialPolyHash (trinomialStep U M) k U M) ∧ - ExecRand w (resampleCode k) x (resampleComp base U M k x) ∧ - ((resampleComp base U M k x).eval).map (fun z => extractSeeds base U M k z.1.mem) = - PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M)) ∧ - ExecRandWithin w (resampleCode k) x - (fun y => - Rep base (chunkBits n_max) U M k (extractSeeds base U M k y.mem) y.mem ∧ - y.regs 0 = base ∧ - (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → y.mem a = x.mem a) ∧ - (∀ u : Fin U, ∀ x_eval : Data 18, - x_eval.mem = y.mem → - x_eval.regs 0 = base → - x_eval.regs 1 = u.val → - ∃ y_eval : Data 18, - ExecWithin w (evalCode k) x_eval y_eval (evalSteps n_max U M k) ∧ - y_eval.regs 0 = base ∧ - y_eval.regs 1 = u.val ∧ - y_eval.regs 2 = - (trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val ∧ - y_eval.mem = y.mem)) - (5 * k + 10) k := by - have hpow_pos : 0 < 2 ^ (2 * chunkBits n_max) := by positivity - have hsp : space n_max k = 8 + k + 2 ^ (2 * chunkBits n_max) := rfl - have hfit_end : Fits w (base + 8 + k) := hfit_space.of_le (by omega) - have hfit_q : Fits w (2 ^ trinomialDeg U M) := by - have hlt : 2 ^ trinomialDeg U M < 2 ^ (2 * trinomialDeg U M) := by - apply (Nat.pow_lt_pow_iff_right one_lt_two).mpr - have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) - dsimp only [trinomialDeg] - omega - exact fits_of_lt_two_pow_two_mul hw hlt - obtain ⟨m, he_rand, hqb, hsupp⟩ := resampleCode_execRandWithin htab h0 hfit_end hfit_q - refine ⟨isKWiseIndependentHash_trinomialPolyHash - (le_two_pow_trinomialDeg U M) (le_trinomialDeg U M), - resampleCode_execRand htab h0 hfit_end hfit_q, - map_extractSeeds_eval_resampleComp base U M k x, - ⟨m, he_rand, hqb, ?_⟩⟩ - intro z hz - obtain ⟨⟨hrep, hz0, hzout⟩, hsteps⟩ := hsupp z hz - refine ⟨⟨hrep, hz0, hzout, ?_⟩, hsteps⟩ - intro u x_eval hmem he0 he1 - have hrep_eval : Rep base (chunkBits n_max) U M k (extractSeeds base U M k z.1.mem) x_eval.mem := - hmem ▸ hrep - obtain ⟨y_eval, heval, hy0, hy1, hy2, hymem⟩ := - evalCode_execWithin u hrep_eval he0 he1 hw hfit_space - exact ⟨y_eval, heval, hy0, hy1, hy2, hymem.trans hmem⟩ + ((resampleComp base U M k x).eval).map (fun z => evalHashOut w base k U M z.1) = + (PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M))).map + (trinomialPolyHash (trinomialStep U M) k U M) := by + rw [← map_extractSeeds_eval_resampleComp base U M k x] + dsimp only [trinomialDeg, extractSeeds, extractSlice] + rw [PMF.map_comp] + refine Cslib.Probability.PMF.map_congr_on_support _ _ _ fun z hz => ?_ + rw [ProbComp.support_eval] at hz + have hrep := (mem_support_resampleComp htab h0 hz).1.1 + funext u + exact evalHashOut_eq_trinomialPolyHash hrep hw hfit_space u + +/-- For any `k`-wise independent hash family `h : ι → X → Y` and any set `T : Finset X` of at most +`k` points, restricting `h i` to `T` under a uniform seed `i ← PMF.uniformOfFintype ι` yields the +uniform distribution `PMF.uniformOfFintype (T → Y)`. -/ +theorem uniformOfFintype_map_restrict_of_isKWiseIndependentHash + {ι X Y : Type*} [Fintype ι] [Nonempty ι] [DecidableEq X] + [Fintype Y] [DecidableEq Y] [Nonempty Y] + {k : ℕ} {h : ι → X → Y} (hh : IsKWiseIndependentHash k h) + {T : Finset X} (hT : Finset.card T ≤ k) : + (PMF.uniformOfFintype ι).map (fun i => T.restrict (h i)) = + PMF.uniformOfFintype (T → Y) := by + ext y + apply (ENNReal.toReal_eq_toReal_iff' (PMF.apply_ne_top _ _) (PMF.apply_ne_top _ _)).mp + rw [Cslib.Probability.PMF.toReal_uniformOfFintype_map_apply, hh.dens_filter_restrict_eq hT y] + simp + +/-- **Composed Word-RAM `k`-wise independence of `resampleCode` and `evalCode` +(`isKWiseIndependentHash_eval`)**: +from any machine state `x` satisfying `TableRep base (chunkBits n_max) U M k x.mem` and +`x.regs 0 = base`, under the resampling distribution `(resampleComp base U M k x).eval` of +`resampleCode k`, for every set `T : Finset (Fin U)` of at most `k` keys (`T.card ≤ k`), the +restricted tuple of deterministic `evalCode k` outputs `T.restrict (evalHashOut w base k U M z.1)` +is uniformly distributed on `T → Fin (2 ^ M)` (`PMF.uniformOfFintype (T → Fin (2 ^ M))`). -/ +theorem isKWiseIndependentHash_eval {w base n_max U M k : ℕ} {x : Data 18} + (htab : TableRep base (chunkBits n_max) U M k x.mem) + (h0 : x.regs 0 = base) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) + {T : Finset (Fin U)} (hT : Finset.card T ≤ k) : + ((resampleComp base U M k x).eval).map (fun z => T.restrict (evalHashOut w base k U M z.1)) = + PMF.uniformOfFintype (T → Fin (2 ^ M)) := by + have hmap := congrArg (PMF.map T.restrict) + (map_evalHashOut_eval_resampleComp htab h0 hw hfit_space) + dsimp only [trinomialDeg] at hmap + rw [PMF.map_comp, PMF.map_comp] at hmap + exact hmap.trans (uniformOfFintype_map_restrict_of_isKWiseIndependentHash + (isKWiseIndependentHash_trinomialPolyHash (le_two_pow_trinomialDeg U M) (le_trinomialDeg U M)) + hT) end Cslib.Algorithms.WordRAM.Hashing.KWise diff --git a/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean index d6aba6086..cabfafe4d 100644 --- a/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean +++ b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean @@ -7,6 +7,7 @@ Authors: Renfei Zhou module -- upstream: candidate +public import Complexity.Algorithms.WordRAM.ExactLog public import Complexity.Computability.WordRAM.Randomized.Structured public import Complexity.Probability.PairwiseIndependentHash.Shift public import Complexity.Probability.Product @@ -14,23 +15,31 @@ public import Complexity.Probability.Uniform public import Mathlib.Data.Nat.Log /-! -# Self-contained Word-RAM random tables and simple tabulation hashing +# Self-contained Word-RAM random tables and tabulated odd-stride generation This file provides two self-contained Word-RAM hashing primitives over `Cslib.WordRAM.Basis.exact`, built on a shared uniform word-slice sampler (`randFillLoop`): -1. **`RandomTable`**: initializes or resamples a table `Z[0 .. 2 ^ M - 1]` of `m = 2 ^ M` - independent uniform `M`-bit words in `O(2 ^ M)` instructions and `2 ^ M` calls to `Code.rand` - (`PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`), and supports `O(1)` deterministic table - lookup `Z[idx]` as well as XOR-shifted evaluation `f(u) ^^^ Z[g(u)]` - [Carter and Wegman][CarterWegman1979]; [Mitzenmacher and Upfal][MitzenmacherUpfal2017]. -2. **`TabulatedOddStride`**: `d`-character simple tabulation with character bit-width - `b = max 1 (Nat.log 2 (max n_max 2) / 2)` (`2 ^ b ≤ max 2 n_max` words per character table), - initialized via `Code.rand` in `O(d * 2 ^ b + log n_max) = O(n_max)` steps and evaluated in - `O(d)` deterministic instructions, producing an odd stride - `(2 * simpleTab(u) + 1) &&& (2 ^ M - 1)` coprime to `2 ^ M` whose linear probe sequence - `(h₁(u) + i * h₂(u)) % 2 ^ M` is a full-cycle permutation of `Fin (2 ^ M)` - [Cormen, Leiserson, Rivest, and Stein, Section 11.4][CLRS2009]. +1. **`RandomTable`**: `resampleCode` populates an array `Z[0 .. 2 ^ M - 1]` of `m = 2 ^ M` + independent uniform `M`-bit words in `5 * 2 ^ M + 7` instructions and `2 ^ M` calls to + `Code.rand`, inducing `PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`. `lookupCode` reads + `Z[idx]` in `2` deterministic instructions, and `xorLookupCode` evaluates the XOR-shifted hash + `f(u) ^^^ Z[g(u)]` in `3` deterministic instructions [Carter and Wegman][CarterWegman1979]; + [Mitzenmacher and Upfal][MitzenmacherUpfal2017]. +2. **`TabulatedOddStride`**: `simpleTab` splits the low `d * b` bits of an input key `u` into `d` + characters of `b = max 1 (Nat.log 2 (max n_max 2) / 2)` bits (`2 ^ b ≤ max 2 n_max` words per + character table) and XORs their table entries (following the simple tabulation scheme of Zobrist + 1970 and Pătraşcu and Thorup 2012). `initCode d` samples the `d` character tables via + `Code.rand` in `4 * Nat.log 2 (max n_max 2) + 5 * (d * 2 ^ b) + 30` instructions + (`O(d * 2 ^ b + log n_max)`, which is `O(n_max)` for fixed `d`), and `evalCode d` evaluates + `oddStride M (simpleTab b d M T u).val = (2 * (simpleTab b d M T u).val + 1) &&& (2 ^ M - 1)` in + `8 * d + 21` deterministic instructions (`O(d)`). When `M > 0`, this stride `h₂(u)` is odd (at + `M = 0` it is `0` on `Fin 1`), and for every `M` it is coprime to `2 ^ M`, so for any initial + slot `h₁(u) : Fin (2 ^ M)` the linear probe sequence `probeSeq M h₁(u) tab i = + (h₁(u) + i * h₂(u)) % 2 ^ M` is a bijection on `Fin (2 ^ M)` (`bijective_probeSeq`) + [Cormen, Leiserson, Rivest, and Stein, Section 11.4][CLRS2009]. Only this full-cycle permutation + property is proved here: no `k`-wise or 3-wise independence theorem for simple tabulation is + claimed in this module, and `simpleTab` inspects only the low `d * b` bits of `u`. ## Main definitions @@ -63,8 +72,8 @@ built on a shared uniform word-slice sampler (`randFillLoop`): * `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.bijective_probeSeq`, `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.initCode_execRandWithin`, `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.evalCode_execWithin`: full-cycle - permutation property, `O(d * 2 ^ b)` initialization, and `O(d)` deterministic evaluation of - `TabulatedOddStride`. + permutation property, `O(d * 2 ^ b + log n_max)` initialization, and `O(d)` deterministic + evaluation of `TabulatedOddStride`. ## References @@ -79,42 +88,9 @@ public section namespace Cslib.Algorithms.WordRAM.Hashing.Tabulation open Cslib.WordRAM Cslib.Probability +open Cslib.Algorithms.WordRAM.ExactLog open scoped ENNReal -/-! ## Exact-basis instruction helpers -/ - -/-- Execute a single `.exactBin` operation when `evalExact` returns `some v` and `Fits w v`. -/ -theorem exec_exactBin {w r : ℕ} {f : BinOp} {dst i j : Fin r} {d : Data r} {v : ℕ} - (hev : f.evalExact (d.regs i) (d.regs j) = some v) (hfit : Fits w v) : - Exec w (.op (.exactBin f dst i j) : Code .exact r) d (d.set dst v) 1 := by - apply Exec.op - rw [Op.eval_exactBin, hev] - exact Data.put_of_fits hfit - -/-- 3-instruction equal-cost branch setting register `dst` to `max (x.regs a) (x.regs b)`. -/ -@[expose] def maxCode {n : ℕ} (dst a b : Fin n) : Code .exact n := - .branch (.lt a b) (.seq (.op (.copy dst b)) (.op (.copy dst dst))) (.op (.copy dst a)) - -/-- `maxCode dst a b` executes deterministically in 3 instructions and sets `dst` to -`max (x.regs a) (x.regs b)`. -/ -theorem maxCode_exec {w n : ℕ} (dst a b : Fin n) (x : Data n) : - Exec w (maxCode dst a b) x (x.set dst (max (x.regs a) (x.regs b))) 3 := by - by_cases hlt : x.regs a < x.regs b - · have hc : (Cond.lt a b).Holds x := hlt - have hmax : max (x.regs a) (x.regs b) = x.regs b := Nat.max_eq_right (le_of_lt hlt) - rw [hmax] - set d1 := x.set dst (x.regs b) - have s1 : Exec w (.op (.copy dst b) : Code .exact n) x d1 1 := Exec.copy - have hd1 : d1.set dst (d1.regs dst) = d1 := by - ext i : 2 <;> simp [d1, Data.set] - have s2 : Exec w (.op (.copy dst dst) : Code .exact n) d1 d1 1 := by - simpa only [hd1] using (Exec.copy (B := .exact) (w := w) (d := dst) (s := dst) (x := d1)) - exact Exec.branch_pos hc (s1.seq s2) - · have hc : ¬(Cond.lt a b).Holds x := hlt - have hmax : max (x.regs a) (x.regs b) = x.regs a := Nat.max_eq_left (not_lt.mp hlt) - rw [hmax] - exact Exec.branch_neg hc (Exec.copy (B := .exact)) - /-! ## Section 1: Uniform random word-slice filler (`randFillLoop`) -/ /-- Register `0` of `randFillLoop`: write pointer `ptr`. -/ @@ -645,81 +621,7 @@ theorem xorLookupCode_exec {w base M : ℕ} {Z : Fin (2 ^ M) → Fin (2 ^ M)} {x end RandomTable -/-! ## Section 3: Shared `Nat.log 2` loop and `TabulatedOddStride` -/ - -/-- One iteration of the floor-`log₂` loop on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, -`reg 2 = 1`): shifts `rem` right by `1` (`rem / 2`) and increments `acc`. -/ -@[expose] def log2Body : Code .exact 3 := - .seq (.op (.exactBin .shiftRight 0 0 2)) - (.op (.exactBin .add 1 1 2)) - -/-- Floor-`log₂` loop `while 1 < rem do [ rem ← rem >>> 1; acc ← acc + 1 ]`. -/ -@[expose] def log2Loop : Code .exact 3 := - .loop (.lt 2 0) log2Body - -/-- Closed-form final state of `log2Loop` starting from `rem` in register `0` and `acc` in -register `1`. -/ -@[expose] def log2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := - (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.log 2 rem) - -/-- `log2Loop` executes deterministically in `4 * Nat.log 2 rem + 2` instructions, accumulating -`acc + Nat.log 2 rem` into register `1` and preserving all memory. -/ -theorem log2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} - (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) - (hfit_rem : Fits w rem) (hfit_acc : Fits w (acc + Nat.log 2 rem)) : - Exec w log2Loop x (log2LoopData rem acc x) (4 * Nat.log 2 rem + 2) := by - induction rem using Nat.strong_induction_on generalizing acc x with - | h rem ih => - by_cases hlt : 1 < rem - · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hshr : rem >>> 1 = rem / 2 := by - rw [Nat.shiftRight_eq_div_pow] - have hlog : Nat.log 2 rem = Nat.log 2 (rem / 2) + 1 := - Nat.log_of_one_lt_of_le (by decide) (by omega) - have hfit_div : Fits w (rem / 2) := hfit_rem.of_le (Nat.div_le_self rem 2) - have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) - set d1 := x.set 0 (rem / 2) - have s1 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) x d1 1 := - exec_exactBin (by simp [hrem, hone, hshr]) hfit_div - set d2 := d1.set 1 (acc + 1) - have s2 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d1 d2 1 := - exec_exactBin (by simp +decide [d1, hacc, hone]) hfit_acc1 - have hdiv_lt : rem / 2 < rem := Nat.div_lt_self (by omega) (by decide) - have hrec := ih (rem / 2) hdiv_lt (acc := acc + 1) (x := d2) - (by simp +decide [d2, d1]) - (by simp +decide [d2]) - (by simp +decide [d2, d1, hone]) - hfit_div - (by rwa [show acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem by omega]) - have hd : log2LoopData (rem / 2) (acc + 1) d2 = log2LoopData rem acc x := by - have hdiv_ne : rem / 2 ≠ 0 := by omega - have hrem_ne : rem ≠ 0 := by omega - ext i : 2 - · fin_cases i - · simp +decide [log2LoopData, d2, d1, hdiv_ne, hrem_ne] - · change acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem - omega - · simp +decide [log2LoopData, d2, d1] - · rfl - rw [hd] at hrec - have hiter := Exec.loop_iter hc (s1.seq s2) hrec - simpa only [log2Loop, log2Body, hlog, show 1 + 1 + (4 * Nat.log 2 (rem / 2) + 2) + 2 = - 4 * (Nat.log 2 (rem / 2) + 1) + 2 by omega] using hiter - · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hlog0 : Nat.log 2 rem = 0 := Nat.log_of_lt (by omega) - have hd : log2LoopData rem acc x = x := by - have hrem_eq : (if rem = 0 then 0 else 1) = rem := by - split_ifs <;> omega - ext i : 2 - · fin_cases i - · simp +decide [log2LoopData, hrem_eq, hrem] - · simp +decide [log2LoopData, hlog0, hacc] - · simp +decide [log2LoopData] - · rfl - rw [hd, hlog0] - exact Exec.loop_exit hc +/-! ## Section 3: `TabulatedOddStride` -/ namespace TabulatedOddStride @@ -1368,11 +1270,6 @@ theorem evalSetupCode_exec {w base b d M : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 @[expose] def tabLoop : Code .exact 9 := .loop (.lt regPtr regEnd) tabBody -/-- State after one iteration of `tabBody`. -/ -@[expose] def tabStepData (ptr ch entry newAcc newKey nextPtr : ℕ) (x : Data 9) : Data 9 := - (((((x.set regTmp ch).set regTmp (ptr + ch)).set regTmp entry).set - regOut newAcc).set regKey newKey).set regPtr nextPtr - /-- `tabLoop` executes deterministically in `8 * rem + 2` instructions from character offset `j` with `j + rem = d`, accumulating `(simpleTabAux b d M T rem j u acc).val` into `regOut` and preserving `regBase` and all memory. -/ diff --git a/ComplexityTests/WordRAMKWiseHash.lean b/ComplexityTests/WordRAMKWiseHash.lean index 77b8a398d..ec6055db0 100644 --- a/ComplexityTests/WordRAMKWiseHash.lean +++ b/ComplexityTests/WordRAMKWiseHash.lean @@ -113,29 +113,33 @@ def trinomialHornerRef5 (t M : ℕ) (c : Fin 5 → Fin (2 ^ (2 * 3 ^ t))) (u : let a0 := reduceTrinomialCode t (clmulFuel 6 a1 u) ^^^ (c ⟨0, by decide⟩).val a0 &&& (2 ^ M - 1) -/-- Write the `5` coefficients of `sampleCoeffs5` into `[base + 8, base + 13)` of `d.mem`. -/ -def writeSampleCoeffs5 (base : ℕ) (d : Data 18) : Data 18 := - ((((d.write (base + 8) 3).write (base + 9) 1).write (base + 10) 2).write - (base + 11) 3).write (base + 12) 2 - -/-- Run `KWise.initTableCode 5` with `(base, n_max, U, M)` in registers `0..3`, populate -`sampleCoeffs5`, and evaluate `KWise.evalCode 5` via `Code.eval` on a list of keys `us`, returning -the list of output hash values together with the final values at boundary sentinels `base - 1` and -`base + KWise.space n_max 5`. -/ -def runKWiseEval5 (w fuel n_max U M base : ℕ) (us : List ℕ) : +/-- Write `5` coefficients `c : Fin 5 → Fin n` into `[base + 8, base + 13)` of `d.mem`. -/ +def writeSampleCoeffs5 {n : ℕ} (base : ℕ) (c : Fin 5 → Fin n) (d : Data 18) : Data 18 := + ((((d.write (base + 8) (c ⟨0, by decide⟩).val).write (base + 9) (c ⟨1, by decide⟩).val).write + (base + 10) (c ⟨2, by decide⟩).val).write (base + 11) (c ⟨3, by decide⟩).val).write + (base + 12) (c ⟨4, by decide⟩).val + +/-- Run `KWise.initTableCode 5` with `(base, n_max, U, M)` in registers `0..3`, populate `c` via +`writeSampleCoeffs5`, and evaluate `KWise.evalCode 5` via `Code.eval` on a list of keys `us`, +verifying that boundary sentinels at `base - 1` and `base + KWise.space n_max 5` are preserved both +after `initTableCode 5` and after each `evalCode 5` invocation. -/ +def runKWiseEval5 {n : ℕ} (w fuel n_max U M base : ℕ) (c : Fin 5 → Fin n) (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := do let sp := KWise.space n_max 5 let d0 : Data 18 := ((((((Data.init [] : Data 18).set 0 base).set 1 n_max).set 2 U).set 3 M).write (base - 1) 77).write (base + sp) 88 let (d1, _) ← (KWise.initTableCode 5).eval w fuel d0 - let d2 ← compactData (base + sp + 2) (writeSampleCoeffs5 base d1) + let d2 ← compactData (base + sp + 2) (writeSampleCoeffs5 base c d1) + let sLo ← forceNat (d2.mem (base - 1)) + let sHi ← forceNat (d2.mem (base + sp)) let outs ← us.mapM (fun u => do let dIn := (d2.set 0 base).set KWise.regKey u let (dOut, _) ← (KWise.evalCode 5).eval w fuel dIn - forceNat (dOut.regs KWise.regOut)) - let sLo ← forceNat (d2.mem (base - 1)) - let sHi ← forceNat (d2.mem (base + sp)) + if dOut.mem (base - 1) = sLo ∧ dOut.mem (base + sp) = sHi then + forceNat (dOut.regs KWise.regOut) + else + none) pure (outs, sLo, sHi) /-- Concrete `Code.eval` test at `k = 5, b = 1, L = 2` (`n_max = 8`, `U = 4`, `M = 2`, `base = 10`): @@ -144,7 +148,7 @@ def runKWiseEval5 (w fuel n_max U M base : ℕ) (us : List ℕ) : the `5` Horner steps) matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys `u ∈ [0, 1, 2, 3]` while preserving sentinels at addresses `9` (`77`) and `27` (`88`). -/ theorem kwise_eval5_b1_concrete : - runKWiseEval5 16 16 8 4 2 10 [0, 1, 2, 3] = + runKWiseEval5 16 16 8 4 2 10 sampleCoeffs5 [0, 1, 2, 3] = some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by decide +kernel @@ -153,7 +157,7 @@ theorem kwise_eval5_b1_concrete : `KWise.evalCode 5` matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys `u ∈ [0, 1, 2, 3]` while preserving sentinels at addresses `3` (`77`) and `33` (`88`). -/ theorem kwise_eval5_b2_concrete : - runKWiseEval5 16 32 64 4 2 4 [0, 1, 2, 3] = + runKWiseEval5 16 32 64 4 2 4 sampleCoeffs5 [0, 1, 2, 3] = some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by decide +kernel @@ -168,23 +172,8 @@ def sampleCoeffs5T1 : Fin 5 → Fin 64 /-- Run `KWise.initTableCode 5` and `KWise.evalCode 5` at `t = 1, q = 6, b = 2, L = 3` (`n_max = 64, U = 16, M = 3, base = 4`). -/ -def runKWiseEval5T1 (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := do - let base := 4 - let sp := KWise.space 64 5 - let d0 : Data 18 := - ((((((Data.init [] : Data 18).set 0 base).set 1 64).set 2 16).set 3 3).write - (base - 1) 77).write (base + sp) 88 - let (d1, _) ← (KWise.initTableCode 5).eval 16 32 d0 - let d2 ← compactData (base + sp + 2) - (((((d1.write (base + 8) 13).write (base + 9) 42).write (base + 10) 7).write - (base + 11) 59).write (base + 12) 25) - let outs ← us.mapM (fun u => do - let dIn := (d2.set 0 base).set KWise.regKey u - let (dOut, _) ← (KWise.evalCode 5).eval 16 32 dIn - forceNat (dOut.regs KWise.regOut)) - let sLo ← forceNat (d2.mem (base - 1)) - let sHi ← forceNat (d2.mem (base + sp)) - pure (outs, sLo, sHi) +def runKWiseEval5T1 (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := + runKWiseEval5 16 32 64 16 3 4 sampleCoeffs5T1 us /-- Concrete `Code.eval` test at `k = 5, t = 1, q = 6, b = 2, L = 3` (`3 × 3` limb multiplication over `X ^ 6 + X ^ 3 + 1`): `KWise.evalCode 5` matches `trinomialHornerRef5 1 3 sampleCoeffs5T1` on @@ -196,8 +185,8 @@ theorem kwise_eval5_t1_b2_L3_concrete : /-- Non-vacuous theorem-level instantiation of `KWise.initTableCode_execWithin`, `KWise.resampleCode_execRand`, `KWise.initCode_execRandWithin`, `KWise.evalCode_execWithin`, and -`KWise.isKWiseIndependentHash_eval` at `k = 5`, `U = 1000`, `M = 10`, `n_max = 64`, `w = 64`, -`base = 0`. -/ +the composed Word-RAM `k`-wise independence theorem `KWise.isKWiseIndependentHash_eval` at `k = 5`, +`U = 1000`, `M = 10`, `n_max = 64`, `w = 64`, `base = 0`. -/ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) (h0 : x.regs 0 = 0) (h1 : x.regs 1 = 64) (h2 : x.regs 2 = 1000) (h3 : x.regs 3 = 10) : (∃ y : Data 18, @@ -213,7 +202,10 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) (∀ y : Data 18, KWise.TableRep 0 (KWise.chunkBits 64) 1000 10 5 y.mem → y.regs 0 = 0 → - IsKWiseIndependentHash 5 (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10) ∧ + (∀ (T : Finset (Fin 1000)), Finset.card T ≤ 5 → + ((KWise.resampleComp 0 1000 10 5 y).eval).map + (fun z => T.restrict (KWise.evalHashOut 64 0 5 1000 10 z.1)) = + PMF.uniformOfFintype (T → Fin (2 ^ 10))) ∧ ExecRand 64 (KWise.resampleCode 5) y (KWise.resampleComp 0 1000 10 5 y) ∧ ((KWise.resampleComp 0 1000 10 5 y).eval).map (fun z => KWise.extractSeeds 0 1000 10 5 z.1.mem) = @@ -223,20 +215,19 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 (KWise.extractSeeds 0 1000 10 5 z.mem) z.mem ∧ z.regs 0 = 0 ∧ - (∀ a, (a < 0 + 8 ∨ 0 + 8 + 5 ≤ a) → z.mem a = y.mem a) ∧ - (∀ u : Fin 1000, ∀ x_eval : Data 18, - x_eval.mem = z.mem → - x_eval.regs 0 = 0 → - x_eval.regs 1 = u.val → - ∃ y_eval : Data 18, - ExecWithin 64 (KWise.evalCode 5) x_eval y_eval (KWise.evalSteps 64 1000 10 5) ∧ - y_eval.regs 0 = 0 ∧ - y_eval.regs 1 = u.val ∧ - y_eval.regs 2 = - (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10 - (KWise.extractSeeds 0 1000 10 5 z.mem) u).val ∧ - y_eval.mem = z.mem)) - 35 5) := by + (∀ a, (a < 0 + 8 ∨ 0 + 8 + 5 ≤ a) → z.mem a = y.mem a)) + 35 5 ∧ + (∀ (c : Fin 5 → Fin (2 ^ trinomialDeg 1000 10)) (u : Fin 1000) (x_eval : Data 18), + KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 c x_eval.mem → + x_eval.regs 0 = 0 → + x_eval.regs 1 = u.val → + ∃ y_eval : Data 18, + ExecWithin 64 (KWise.evalCode 5) x_eval y_eval (KWise.evalSteps 64 1000 10 5) ∧ + y_eval.regs 0 = 0 ∧ + y_eval.regs 1 = u.val ∧ + y_eval.regs 2 = + (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10 c u).val ∧ + y_eval.mem = x_eval.mem)) := by have htr : trinomialDeg 1000 10 = 18 := by have hclog : Nat.clog 2 (max 1000 2) ≤ 10 := (Nat.clog_le_iff_le_pow one_lt_two).mpr (by decide) @@ -250,6 +241,7 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) rw [fits_iff] dsimp [KWise.space, KWise.spaceFromB, KWise.chunkBits] decide + have hfit_end : Fits 64 (0 + 8 + 5) := by rw [fits_iff]; decide have hfit_n : Fits 64 (max 64 2) := by rw [fits_iff]; decide have hfit_U : Fits 64 (max 1000 2 + 1) := by rw [fits_iff]; decide have hfit_q : Fits 64 (2 ^ trinomialDeg 1000 10) := by @@ -259,7 +251,12 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) refine ⟨⟨y, hy_exec, hy_tab⟩, KWise.initCode_execRandWithin h0 h1 h2 h3 hfit_sp hfit_n hfit_U hfit_q, fun y' htab' h0' => - KWise.isKWiseIndependentHash_eval htab' h0' hw hfit_sp⟩ + ⟨fun T hT => KWise.isKWiseIndependentHash_eval htab' h0' hw hfit_sp hT, + KWise.resampleCode_execRand htab' h0' hfit_end hfit_q, + KWise.map_extractSeeds_eval_resampleComp 0 1000 10 5 y', + KWise.resampleCode_execRandWithin htab' h0' hfit_end hfit_q, + fun c u x_eval hrep hx0 hx1 => + KWise.evalCode_execWithin u hrep hx0 hx1 hw hfit_sp⟩⟩ /-! ## 2. `RandomTable`: concrete `Code.eval` execution and theorem instantiations -/ From a31eaf98b029522c8a665af8cb10820e493d5797 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Fri, 30 Oct 2026 02:19:17 +0000 Subject: [PATCH 17/18] feat(WordRAM/Dictionary): add uniform probing model and Word-RAM dictionary --- Complexity.lean | 2 + .../Algorithms/Lean/UniformProbing.lean | 1393 +++++++++++ .../WordRAM/Dictionary/UniformProbing.lean | 2206 +++++++++++++++++ ComplexityTests/WordRAMUniformProbing.lean | 219 ++ 4 files changed, 3820 insertions(+) create mode 100644 Complexity/Algorithms/Lean/UniformProbing.lean create mode 100644 Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean create mode 100644 ComplexityTests/WordRAMUniformProbing.lean diff --git a/Complexity.lean b/Complexity.lean index 976f7af96..6df9d6582 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -26,6 +26,7 @@ public import Complexity.Algorithms.Lean.Sort.Bitonic.Basic public import Complexity.Algorithms.Lean.Sort.Bitonic.Correctness public import Complexity.Algorithms.Lean.Sort.Bitonic.Trace public import Complexity.Algorithms.Lean.Sort.Bitonic.ZeroOne +public import Complexity.Algorithms.Lean.UniformProbing public import Complexity.Algorithms.Lean.VanEmdeBoas.Basic public import Complexity.Algorithms.Lean.VanEmdeBoas.Elements public import Complexity.Algorithms.Lean.VanEmdeBoas.Insert @@ -78,6 +79,7 @@ public import Complexity.Algorithms.ThreeSum.ThreeSets public import Complexity.Algorithms.WordRAM.Deduplicate.Count public import Complexity.Algorithms.WordRAM.Deduplicate.Model public import Complexity.Algorithms.WordRAM.Deduplicate.Program +public import Complexity.Algorithms.WordRAM.Dictionary.UniformProbing public import Complexity.Algorithms.WordRAM.Dijkstra.Fibonacci public import Complexity.Algorithms.WordRAM.Dijkstra.FibonacciStatement public import Complexity.Algorithms.WordRAM.Dijkstra.GenericPQ diff --git a/Complexity/Algorithms/Lean/UniformProbing.lean b/Complexity/Algorithms/Lean/UniformProbing.lean new file mode 100644 index 000000000..3f7c56a26 --- /dev/null +++ b/Complexity/Algorithms/Lean/UniformProbing.lean @@ -0,0 +1,1393 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Algorithms.Lean.Dictionary +public import Complexity.Languages.Probabilistic.RandomOracle +public import Complexity.Probability.PMF.Expect +public import Complexity.Probability.PMF.Uniform +public import Complexity.Probability.Product +import Mathlib.Data.Fintype.Perm +import Mathlib.Topology.Instances.ENNReal.Lemmas + +/-! +# Uniform Probing in Open-Addressed Hash Tables + +This file formalizes the classical uniform-probing model of open-addressed hashing +([Cormen, Leiserson, Rivest, and Stein, Section 11.4, Theorems 11.6 and 11.8][CLRS2009]), +realizing the bounded-capacity dictionary specification `Cslib.Algorithms.Lean.Dictionary.spec` +over `m` table slots (`2 * n_max ≤ m`). + +In the textbook uniform-probing model, each key `u : Fin U` is assigned an independent uniform +random permutation `π u : Equiv.Perm (Fin m)` (`PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`), +which may equivalently be sampled lazily on demand via +`Cslib.RandomOracle.uniform_interpret_eq_lazy` +(`Complexity/Languages/Probabilistic/RandomOracle.lean`). + +## Main definitions + +* `Cslib.Algorithms.Lean.UniformProbing.allOccupied`, + `Cslib.Algorithms.Lean.UniformProbing.missedProbesFrom`, + `Cslib.Algorithms.Lean.UniformProbing.missedProbes`, + `Cslib.Algorithms.Lean.UniformProbing.unsuccessfulProbes`, + `Cslib.Algorithms.Lean.UniformProbing.probeSlotIndex`: slot-probe predicates, probe counts, and + first-available slot index along a probe permutation `σ : Equiv.Perm (Fin m)`. +* `Cslib.Algorithms.Lean.UniformProbing.Table`, + `Cslib.Algorithms.Lean.UniformProbing.emptyTable`, + `Cslib.Algorithms.Lean.UniformProbing.occupiedSlots`, + `Cslib.Algorithms.Lean.UniformProbing.otherOccupiedSlots`, + `Cslib.Algorithms.Lean.UniformProbing.probeCount`: pure open-addressed table of `m` slots and + per-key probe count. +* `Cslib.Algorithms.Lean.UniformProbing.tableGet`, + `Cslib.Algorithms.Lean.UniformProbing.tableInsert`, + `Cslib.Algorithms.Lean.UniformProbing.tableStep`, + `Cslib.Algorithms.Lean.UniformProbing.runTable`, + `Cslib.Algorithms.Lean.UniformProbing.TableRep`: open-addressed `get`/`insert` operations and + representation invariant linking `Table U m` to `Dictionary.State U`. + +## Main results + +* `Cslib.Algorithms.Lean.UniformProbing.toOuterMeasure_allOccupied_eq_prod`, + `Cslib.Algorithms.Lean.UniformProbing.toOuterMeasure_allOccupied_le_pow`: on a table with + `n = O.card ≤ m` occupied slots, the probability over + `σ ← PMF.uniformOfFintype (Equiv.Perm (Fin m))` + that the first `k ≤ m` probes all hit `O` (`k < unsuccessfulProbes O σ`) is + `∏ i ∈ Finset.range k, ((n - i : ℝ≥0∞) / (m - i : ℝ≥0∞)) ≤ ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k`. +* `Cslib.Algorithms.Lean.UniformProbing.expect_unsuccessfulProbes_le`, + `Cslib.Algorithms.Lean.UniformProbing.expect_unsuccessfulProbes_le_two`, + `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_eq_none_le`: Theorem 11.6 and + Corollary 11.7 of [CLRS2009] — an unsuccessful lookup or fresh-key insertion into a table holding + `n < m` keys makes at most `m / (m - n)` probes in expectation (`≤ 2` when `2 * n ≤ m`). +* `Cslib.Algorithms.Lean.UniformProbing.TableRep.probeCount_tableInsert_of_isSome`, + `Cslib.Algorithms.Lean.UniformProbing.probeCount_runTable_eq_firstInsert`, + `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_mem_dom_le`: Theorem 11.8 of + [CLRS2009] — a successful lookup of a present key `u ∈ dom s` makes the exact same number of + probes as the operation that first inserted `u`, which is at most `m / (m - n) ≤ 2` in + expectation. +* `Cslib.Algorithms.Lean.UniformProbing.uniform_interpret_eq_lazy_perm`: eager sampling of the + independent permutation family `Fin U → Equiv.Perm (Fin m)` agrees in distribution with lazy + on-demand oracle evaluation (`RandomOracle.uniform_interpret_eq_lazy`). + +## Representation space note + +Storing an explicit table of independent permutations in `(Equiv.Perm (Fin m)) ^ U` requires +`U * log₂ (m!)` bits (`Ω(U)` machine words when `U = poly(n_max) ≫ n_max`). To achieve an +`O(n_max)`-word self-contained Word-RAM dictionary, +`Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean` pairs a 2-universal multiply-shift +primary hash `h₁(u) ∈ Fin (2 ^ M)` with a tabulated odd stride `h₂(u) ∈ Fin (2 ^ M)` +([CLRS2009], Section 11.4, "Double hashing"), reusing the open-addressed table invariants proved +here. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, + *Introduction to Algorithms*, Section 11.4][CLRS2009] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] +-/ + +@[expose] public section + +namespace Cslib.Algorithms.Lean.UniformProbing + +open Cslib.Algorithms.Lean +open Cslib.Probability +open scoped BigOperators ENNReal + +/-! ## Section 1: Permutation probe counts and stopping indices -/ + +/-- Event that the first `k` slots `σ ⟨0, _⟩, ..., σ ⟨k - 1, _⟩` of permutation `σ` all belong to +the occupied slot set `O`. -/ +def allOccupied {m : ℕ} (O : Finset (Fin m)) (k : ℕ) (σ : Equiv.Perm (Fin m)) : Prop := + ∀ i : Fin m, i.val < k → σ i ∈ O + +instance instDecidableAllOccupied {m : ℕ} (O : Finset (Fin m)) (k : ℕ) + (σ : Equiv.Perm (Fin m)) : Decidable (allOccupied O k σ) := by + unfold allOccupied + infer_instance + +@[simp] +theorem allOccupied_zero {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) : + allOccupied O 0 σ := by + intro i hi + omega + +theorem allOccupied_succ_iff {m : ℕ} {O : Finset (Fin m)} {k : ℕ} (hk : k < m) + (σ : Equiv.Perm (Fin m)) : + allOccupied O (k + 1) σ ↔ allOccupied O k σ ∧ σ ⟨k, hk⟩ ∈ O := by + constructor + · intro h + refine ⟨fun i hi => h i (by omega), h ⟨k, hk⟩ (by simp)⟩ + · rintro ⟨h, hk_mem⟩ i hi + rcases Nat.lt_succ_iff_lt_or_eq.mp hi with hlt | heq + · exact h i hlt + · have : i = ⟨k, hk⟩ := Fin.ext heq + rwa [this] + +theorem allOccupied_mono {m : ℕ} {O : Finset (Fin m)} {j k : ℕ} (hjk : j ≤ k) + {σ : Equiv.Perm (Fin m)} (h : allOccupied O k σ) : allOccupied O j σ := + fun i hi => h i (lt_of_lt_of_le hi hjk) + +/-- Count how many consecutive slots `σ ⟨idx, _⟩, σ ⟨idx + 1, _⟩, ...` belong to `O`, for up to +`rem` steps starting at `idx`. -/ +def missedProbesFrom {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) : + ℕ → ℕ → ℕ + | 0, _ => 0 + | rem + 1, idx => + if h : idx < m then + if σ ⟨idx, h⟩ ∈ O then 1 + missedProbesFrom O σ rem (idx + 1) else 0 + else 0 + +/-- Number of consecutive probes `σ 0, σ 1, ...` that hit `O` before the first slot outside `O`. -/ +def missedProbes {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) : ℕ := + missedProbesFrom O σ m 0 + +/-- Total number of slot probes `1 + missedProbes O σ` made by an unsuccessful search (or fresh-key +insertion) against occupied slot set `O` along permutation `σ`. -/ +def unsuccessfulProbes {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) : ℕ := + 1 + missedProbes O σ + +theorem missedProbesFrom_le {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) + (rem idx : ℕ) : missedProbesFrom O σ rem idx ≤ rem := by + induction rem generalizing idx with + | zero => simp [missedProbesFrom] + | succ rem ih => + simp only [missedProbesFrom] + split_ifs + · have := ih (idx + 1) + omega + · omega + · omega + +theorem missedProbes_le {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) : + missedProbes O σ ≤ m := + missedProbesFrom_le O σ m 0 + +theorem mem_of_lt_missedProbesFrom {m : ℕ} {O : Finset (Fin m)} {σ : Equiv.Perm (Fin m)} + {rem idx t : ℕ} (hrem : idx + rem ≤ m) (ht : t < missedProbesFrom O σ rem idx) : + ∃ h : idx + t < m, σ ⟨idx + t, h⟩ ∈ O := by + induction rem generalizing idx t with + | zero => + simp [missedProbesFrom] at ht + | succ rem ih => + have hidx : idx < m := by omega + simp only [missedProbesFrom, hidx, ↓reduceDIte] at ht + split_ifs at ht with hmem + · cases t with + | zero => + exact ⟨hidx, by simpa using hmem⟩ + | succ t => + have ht' : t < missedProbesFrom O σ rem (idx + 1) := by omega + obtain ⟨h', hmem'⟩ := ih (idx := idx + 1) (by omega) ht' + refine ⟨by omega, ?_⟩ + have heq : (⟨idx + (t + 1), by omega⟩ : Fin m) = ⟨idx + 1 + t, h'⟩ := + Fin.ext (show idx + (t + 1) = idx + 1 + t by omega) + rwa [heq] + · omega + +theorem notMem_at_missedProbesFrom {m : ℕ} {O : Finset (Fin m)} {σ : Equiv.Perm (Fin m)} + {rem idx : ℕ} (hrem : idx + rem ≤ m) (hlt : missedProbesFrom O σ rem idx < rem) : + ∃ h : idx + missedProbesFrom O σ rem idx < m, + σ ⟨idx + missedProbesFrom O σ rem idx, h⟩ ∉ O := by + induction rem generalizing idx with + | zero => omega + | succ rem ih => + have hidx : idx < m := by omega + simp only [missedProbesFrom, hidx, ↓reduceDIte] at hlt ⊢ + split_ifs with hmem + · simp only [hmem, ↓reduceIte] at hlt + have hlt' : missedProbesFrom O σ rem (idx + 1) < rem := by omega + obtain ⟨h', hnot⟩ := ih (idx := idx + 1) (by omega) hlt' + refine ⟨by omega, ?_⟩ + have heq : + (⟨idx + (1 + missedProbesFrom O σ rem (idx + 1)), by omega⟩ : Fin m) = + ⟨idx + 1 + missedProbesFrom O σ rem (idx + 1), h'⟩ := + Fin.ext (show idx + (1 + missedProbesFrom O σ rem (idx + 1)) = + idx + 1 + missedProbesFrom O σ rem (idx + 1) by omega) + rwa [heq] + · exact ⟨hidx, by simpa using hmem⟩ + +/-- For any `k ≤ m`, at least `k` probes miss (`k ≤ missedProbes O σ`) if and only if the first `k` +slots of `σ` all belong to `O`. -/ +theorem le_missedProbes_iff_allOccupied {m : ℕ} {O : Finset (Fin m)} {k : ℕ} (hk : k ≤ m) + (σ : Equiv.Perm (Fin m)) : + k ≤ missedProbes O σ ↔ allOccupied O k σ := by + constructor + · intro hle i hi + have hit : i.val < missedProbesFrom O σ m 0 := lt_of_lt_of_le hi hle + obtain ⟨_, hmem⟩ := mem_of_lt_missedProbesFrom (by omega) hit + simpa using hmem + · intro hall + by_contra hnot + have hlt : missedProbesFrom O σ m 0 < k := not_le.mp hnot + obtain ⟨hidx, hnotMem⟩ := notMem_at_missedProbesFrom (by omega) (lt_of_lt_of_le hlt hk) + have hidx' : missedProbesFrom O σ m 0 < m := by omega + have hnotMem' : σ ⟨missedProbesFrom O σ m 0, hidx'⟩ ∉ O := by + convert hnotMem using 2 + ext + simp + exact hnotMem' (hall ⟨missedProbesFrom O σ m 0, hidx'⟩ hlt) + +/-- For any `k ≤ m`, an unsuccessful search inspects more than `k` slots +(`k < unsuccessfulProbes O σ`) if and only if the first `k` slots of `σ` all belong to `O`. -/ +theorem lt_unsuccessfulProbes_iff_allOccupied {m : ℕ} {O : Finset (Fin m)} {k : ℕ} (hk : k ≤ m) + (σ : Equiv.Perm (Fin m)) : + k < unsuccessfulProbes O σ ↔ allOccupied O k σ := by + rw [unsuccessfulProbes, ← le_missedProbes_iff_allOccupied hk] + omega + +/-- When `O.card < m`, not all `m` slots of `σ` can belong to `O`. -/ +theorem not_allOccupied_self_of_card_lt {m : ℕ} {O : Finset (Fin m)} (hO : O.card < m) + (σ : Equiv.Perm (Fin m)) : ¬ allOccupied O m σ := by + intro hall + have hsub : Finset.univ.map σ.toEmbedding ⊆ O := by + intro x hx + rcases Finset.mem_map.mp hx with ⟨i, -, rfl⟩ + exact hall i i.isLt + have hcard := Finset.card_le_card hsub + simp only [Finset.card_map, Finset.card_univ, Fintype.card_fin] at hcard + omega + +/-- When `O.card < m`, `missedProbes O σ < m` for every permutation `σ`. -/ +theorem missedProbes_lt_of_card_lt {m : ℕ} {O : Finset (Fin m)} (hO : O.card < m) + (σ : Equiv.Perm (Fin m)) : missedProbes O σ < m := by + have hle := missedProbes_le O σ + have hne : missedProbes O σ ≠ m := by + intro heq + exact not_allOccupied_self_of_card_lt hO σ + ((le_missedProbes_iff_allOccupied le_rfl σ).mp (by omega)) + omega + +/-- The probe index `⟨missedProbes O σ, _⟩ : Fin m` of the first slot along `σ` outside `O` when +`O.card < m`. -/ +def probeSlotIndex {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) + (hO : O.card < m) : Fin m := + ⟨missedProbes O σ, missedProbes_lt_of_card_lt hO σ⟩ + +@[simp] +theorem val_probeSlotIndex {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) + (hO : O.card < m) : (probeSlotIndex O σ hO).val = missedProbes O σ := + rfl + +theorem mem_of_lt_probeSlotIndex {m : ℕ} {O : Finset (Fin m)} {σ : Equiv.Perm (Fin m)} + (hO : O.card < m) {i : Fin m} (hi : i.val < (probeSlotIndex O σ hO).val) : + σ i ∈ O := + (le_missedProbes_iff_allOccupied (le_of_lt (missedProbes_lt_of_card_lt hO σ)) σ).mp le_rfl i hi + +theorem notMem_apply_probeSlotIndex {m : ℕ} {O : Finset (Fin m)} {σ : Equiv.Perm (Fin m)} + (hO : O.card < m) : σ (probeSlotIndex O σ hO) ∉ O := by + obtain ⟨_, hnot⟩ := notMem_at_missedProbesFrom (O := O) (σ := σ) (by omega) + (missedProbes_lt_of_card_lt hO σ) + simpa [probeSlotIndex, missedProbes] using hnot + +/-- Exact characterization of `missedProbes O σ`: for any `i₀ : Fin m`, `missedProbes O σ = i₀.val` +if and only if all earlier probes `i < i₀` lie in `O` and `σ i₀ ∉ O`. -/ +theorem missedProbes_eq_iff {m : ℕ} {O : Finset (Fin m)} {σ : Equiv.Perm (Fin m)} (i₀ : Fin m) : + missedProbes O σ = i₀.val ↔ allOccupied O i₀.val σ ∧ σ i₀ ∉ O := by + constructor + · intro heq + have hlt : missedProbes O σ < m := heq ▸ i₀.isLt + have hall : allOccupied O i₀.val σ := + (le_missedProbes_iff_allOccupied (le_of_lt i₀.isLt) σ).mp (by omega) + obtain ⟨hidx, hnot⟩ := notMem_at_missedProbesFrom (O := O) (σ := σ) (by omega) hlt + have hi₀ : (⟨0 + missedProbesFrom O σ m 0, hidx⟩ : Fin m) = i₀ := by + ext + simpa [missedProbes] using heq + rw [hi₀] at hnot + exact ⟨hall, hnot⟩ + · rintro ⟨hall, hnot⟩ + have hge : i₀.val ≤ missedProbes O σ := + (le_missedProbes_iff_allOccupied (le_of_lt i₀.isLt) σ).mpr hall + have hnot_succ : ¬ (i₀.val + 1 ≤ missedProbes O σ) := by + intro hle + have hall' := (le_missedProbes_iff_allOccupied i₀.isLt σ).mp hle + exact hnot (hall' i₀ (by omega)) + omega + +theorem probeSlotIndex_eq_iff {m : ℕ} {O : Finset (Fin m)} {σ : Equiv.Perm (Fin m)} + (hO : O.card < m) (i₀ : Fin m) : + probeSlotIndex O σ hO = i₀ ↔ allOccupied O i₀.val σ ∧ σ i₀ ∉ O := by + rw [Fin.ext_iff, val_probeSlotIndex, missedProbes_eq_iff] + +/-- Tail-sum identity on `Fin m`: when `O.card < m`, the total number of probes +`unsuccessfulProbes O σ` equals the number of `k ∈ Finset.range m` for which `allOccupied O k σ` +holds. -/ +theorem unsuccessfulProbes_eq_sum_boole {m : ℕ} {O : Finset (Fin m)} (hO : O.card < m) + (σ : Equiv.Perm (Fin m)) : + unsuccessfulProbes O σ = + ∑ k ∈ Finset.range m, if allOccupied O k σ then 1 else 0 := by + have hlt : missedProbes O σ < m := missedProbes_lt_of_card_lt hO σ + have hcongr : ∀ k ∈ Finset.range m, + (if allOccupied O k σ then 1 else 0) = + (if k ∈ Finset.range (missedProbes O σ + 1) then 1 else 0) := by + intro k hk + have hk_le : k ≤ m := le_of_lt (Finset.mem_range.mp hk) + simp only [Finset.mem_range, Nat.lt_succ_iff, le_missedProbes_iff_allOccupied hk_le] + rw [Finset.sum_congr rfl hcongr, ← Finset.sum_filter, + Finset.filter_mem_eq_inter, Finset.inter_eq_right.mpr (by + intro k hk + rw [Finset.mem_range] at hk ⊢ + omega), + Finset.sum_const, smul_eq_mul, mul_one, Finset.card_range, unsuccessfulProbes, Nat.add_comm] + +/-! ## Section 2: Textbook Uniform-Probing Expected-Probe Bounds ([CLRS2009], Theorem 11.6) -/ + +/-- Set of permutations `σ : Equiv.Perm (Fin m)` whose first `k` probes all belong to `O`. -/ +def occPermSet {m : ℕ} (O : Finset (Fin m)) (k : ℕ) : Finset (Equiv.Perm (Fin m)) := + Finset.univ.filter fun σ => allOccupied O k σ + +@[simp] +theorem mem_occPermSet {m : ℕ} {O : Finset (Fin m)} {k : ℕ} {σ : Equiv.Perm (Fin m)} : + σ ∈ occPermSet O k ↔ allOccupied O k σ := by + simp [occPermSet] + +@[simp] +theorem occPermSet_zero {m : ℕ} (O : Finset (Fin m)) : + occPermSet O 0 = Finset.univ := by + ext σ + simp [occPermSet] + +/-- Right-multiplying `σ` by `Equiv.swap ⟨k, hk⟩ j` (for `k ≤ j.val`) preserves `allOccupied O k`. +-/ +private theorem allOccupied_mul_swap_iff {m : ℕ} {O : Finset (Fin m)} {k : ℕ} (hk : k < m) + {j : Fin m} (hj : k ≤ j.val) (σ : Equiv.Perm (Fin m)) : + allOccupied O k (σ * Equiv.swap ⟨k, hk⟩ j) ↔ allOccupied O k σ := by + have hfix : ∀ i : Fin m, i.val < k → (Equiv.swap ⟨k, hk⟩ j) i = i := by + intro i hi + apply Equiv.swap_apply_of_ne_of_ne + · intro h + have : i.val = k := congrArg Fin.val h + omega + · intro h + have : i.val = j.val := congrArg Fin.val h + omega + refine ⟨fun h i hi => ?_, fun h i hi => ?_⟩ + · have := h i hi + rwa [Equiv.Perm.mul_apply, hfix i hi] at this + · rw [Equiv.Perm.mul_apply, hfix i hi] + exact h i hi + +/-- By transposition symmetry `σ ↦ σ * Equiv.swap ⟨k, hk⟩ j`, for every `j : Fin m` with +`k ≤ j.val`, the number of permutations in `occPermSet O k` with `σ j ∈ O` equals +`(occPermSet O (k + 1)).card`. -/ +private theorem card_filter_occPermSet_eq_succ {m : ℕ} (O : Finset (Fin m)) {k : ℕ} (hk : k < m) + {j : Fin m} (hj : k ≤ j.val) : + ((occPermSet O k).filter fun σ => σ j ∈ O).card = (occPermSet O (k + 1)).card := by + classical + let τ : Equiv.Perm (Fin m) := Equiv.swap ⟨k, hk⟩ j + refine Finset.card_bij (fun σ _ => σ * τ) ?_ ?_ ?_ + · intro σ hσ + rcases Finset.mem_filter.mp hσ with ⟨hσ_occ, hσj⟩ + rw [mem_occPermSet] at hσ_occ ⊢ + rw [allOccupied_succ_iff hk] + refine ⟨(allOccupied_mul_swap_iff hk hj σ).mpr hσ_occ, ?_⟩ + simpa [τ] using hσj + · intro σ₁ _ σ₂ _ heq + exact mul_right_cancel heq + · intro σ hσ + rw [mem_occPermSet, allOccupied_succ_iff hk] at hσ + refine ⟨σ * τ, ?_, ?_⟩ + · rw [Finset.mem_filter, mem_occPermSet] + refine ⟨(allOccupied_mul_swap_iff hk hj σ).mpr hσ.1, ?_⟩ + simpa [τ] using hσ.2 + · ext x + simp [τ] + +/-- For any permutation `σ : Equiv.Perm (Fin m)`, the number of indices `i : Fin m` with `σ i ∈ O` +equals `O.card`. -/ +private theorem card_filter_apply_mem_eq {m : ℕ} (O : Finset (Fin m)) (σ : Equiv.Perm (Fin m)) : + (Finset.univ.filter fun i : Fin m => σ i ∈ O).card = O.card := by + refine Finset.card_bij (fun i _ => σ i) ?_ ?_ ?_ + · intro i hi + exact (Finset.mem_filter.mp hi).2 + · intro i₁ _ i₂ _ h + exact σ.injective h + · intro y hy + exact ⟨σ.symm y, Finset.mem_filter.mpr ⟨Finset.mem_univ _, by simp [hy]⟩, by simp⟩ + +/-- Partition of a `Finset` into elements satisfying `p` and `¬ p`. -/ +private theorem card_filter_add_card_filter_neg {α : Type*} (s : Finset α) + (p : α → Prop) [DecidablePred p] : + (s.filter p).card + (s.filter fun x => ¬ p x).card = s.card := by + classical + induction s using Finset.induction_on with + | empty => simp + | @insert a s ha ih => + by_cases hp : p a <;> simp [Finset.filter_insert, hp, ha] <;> omega + +/-- Cardinality of `{i : Fin m | i.val < k}` when `k ≤ m`. -/ +private theorem card_filter_val_lt {m k : ℕ} (hk : k ≤ m) : + (Finset.univ.filter fun i : Fin m => i.val < k).card = k := by + refine (Finset.card_bij (fun i _ => i.val) ?_ ?_ ?_).trans (Finset.card_range k) + · intro i hi + exact Finset.mem_range.mpr (Finset.mem_filter.mp hi).2 + · intro i₁ _ i₂ _ h + exact Fin.ext h + · intro j hj + have hj_lt : j < k := Finset.mem_range.mp hj + exact ⟨⟨j, lt_of_lt_of_le hj_lt hk⟩, + Finset.mem_filter.mpr ⟨Finset.mem_univ _, hj_lt⟩, rfl⟩ + +/-- Cardinality of `{i : Fin m | k ≤ i.val}` when `k ≤ m`. -/ +private theorem card_filter_val_ge {m k : ℕ} (hk : k ≤ m) : + (Finset.univ.filter fun i : Fin m => k ≤ i.val).card = m - k := by + have hpart := card_filter_add_card_filter_neg + (Finset.univ : Finset (Fin m)) (fun i : Fin m => i.val < k) + have hneg : (Finset.univ.filter fun i : Fin m => ¬ i.val < k) = + Finset.univ.filter fun i : Fin m => k ≤ i.val := by + ext i + simp + rw [hneg, card_filter_val_lt hk, Finset.card_univ, Fintype.card_fin] at hpart + omega + +/-- For any `σ ∈ occPermSet O k` with `k ≤ m`, the number of remaining indices `j ≥ k` satisfying +`σ j ∈ O` is `O.card - k` (and `k ≤ O.card`). -/ +private theorem card_filter_ge_and_mem {m : ℕ} {O : Finset (Fin m)} {k : ℕ} (hk : k ≤ m) + {σ : Equiv.Perm (Fin m)} (hσ : σ ∈ occPermSet O k) : + ((Finset.univ.filter fun j : Fin m => k ≤ j.val).filter fun j => σ j ∈ O).card = + O.card - k := by + rw [mem_occPermSet] at hσ + set S := Finset.univ.filter fun j : Fin m => σ j ∈ O + have hpart := card_filter_add_card_filter_neg S (fun j : Fin m => j.val < k) + have hlt_eq : S.filter (fun j : Fin m => j.val < k) = + Finset.univ.filter fun j : Fin m => j.val < k := by + ext j + simp only [S, Finset.mem_filter, Finset.mem_univ, true_and] + exact ⟨fun h => h.2, fun h => ⟨hσ j h, h⟩⟩ + have hge_eq : S.filter (fun j : Fin m => ¬ j.val < k) = + (Finset.univ.filter fun j : Fin m => k ≤ j.val).filter fun j => σ j ∈ O := by + ext j + simp only [S, Finset.mem_filter, Finset.mem_univ, true_and, not_lt] + exact and_comm + rw [hlt_eq, hge_eq, card_filter_val_lt hk, card_filter_apply_mem_eq O σ] at hpart + omega + +/-- **Exact step recurrence for `occPermSet`**: for any `k < m`, +`(m - k) * (occPermSet O (k + 1)).card = (O.card - k) * (occPermSet O k).card`. -/ +theorem mul_card_occPermSet_succ {m : ℕ} (O : Finset (Fin m)) {k : ℕ} (hk : k < m) : + (m - k) * (occPermSet O (k + 1)).card = (O.card - k) * (occPermSet O k).card := by + classical + set J_ge := Finset.univ.filter fun j : Fin m => k ≤ j.val + have hsum_j : + ∑ j ∈ J_ge, ((occPermSet O k).filter fun σ => σ j ∈ O).card = + (m - k) * (occPermSet O (k + 1)).card := by + have hcongr : ∀ j ∈ J_ge, + ((occPermSet O k).filter fun σ => σ j ∈ O).card = (occPermSet O (k + 1)).card := by + intro j hj + exact card_filter_occPermSet_eq_succ O hk (Finset.mem_filter.mp hj).2 + rw [Finset.sum_congr rfl hcongr, Finset.sum_const, smul_eq_mul, + card_filter_val_ge (le_of_lt hk)] + have hsum_σ : + ∑ j ∈ J_ge, ((occPermSet O k).filter fun σ => σ j ∈ O).card = + (O.card - k) * (occPermSet O k).card := by + simp only [Finset.card_filter] + rw [Finset.sum_comm] + have hcongr : ∀ σ ∈ occPermSet O k, + (∑ j ∈ J_ge, if σ j ∈ O then 1 else 0) = O.card - k := by + intro σ hσ + rw [← Finset.card_filter] + exact card_filter_ge_and_mem (le_of_lt hk) hσ + rw [Finset.sum_congr rfl hcongr, Finset.sum_const, smul_eq_mul, Nat.mul_comm] + omega + +/-- **Exact textbook product formula ([CLRS2009], Proof of Theorem 11.6)**: +for any occupied slot set `O : Finset (Fin m)` and `k ≤ m`, the probability over a uniform random +permutation `σ ← PMF.uniformOfFintype (Equiv.Perm (Fin m))` that the first `k` probes all hit `O` +(`k < unsuccessfulProbes O σ`) is `∏ i ∈ Finset.range k, ((O.card - i : ℝ≥0∞) / (m - i : ℝ≥0∞))`. +-/ +theorem toOuterMeasure_allOccupied_eq_prod {m : ℕ} (O : Finset (Fin m)) {k : ℕ} (hk : k ≤ m) : + (PMF.uniformOfFintype (Equiv.Perm (Fin m))).toOuterMeasure {σ | allOccupied O k σ} = + ∏ i ∈ Finset.range k, (((O.card - i : ℕ) : ℝ≥0∞) / ((m - i : ℕ) : ℝ≥0∞)) := by + induction k with + | zero => + rw [PMF.toOuterMeasure_uniformOfFintype] + change ((occPermSet O 0).card : ℝ≥0∞) * (Fintype.card (Equiv.Perm (Fin m)) : ℝ≥0∞)⁻¹ = _ + rw [occPermSet_zero, Finset.card_univ, Finset.prod_range_zero] + have hcard_pos : 0 < Fintype.card (Equiv.Perm (Fin m)) := Fintype.card_pos_iff.mpr ⟨1⟩ + exact ENNReal.mul_inv_cancel (by exact_mod_cast hcard_pos.ne') (ENNReal.natCast_ne_top _) + | succ k ih => + have hk_lt : k < m := hk + rw [Finset.prod_range_succ, ← ih (le_of_lt hk_lt)] + rw [PMF.toOuterMeasure_uniformOfFintype, PMF.toOuterMeasure_uniformOfFintype] + change ((occPermSet O (k + 1)).card : ℝ≥0∞) * (Fintype.card (Equiv.Perm (Fin m)) : ℝ≥0∞)⁻¹ = + ((occPermSet O k).card : ℝ≥0∞) * (Fintype.card (Equiv.Perm (Fin m)) : ℝ≥0∞)⁻¹ * + (((O.card - k : ℕ) : ℝ≥0∞) / ((m - k : ℕ) : ℝ≥0∞)) + have hrec : (((m - k : ℕ) : ℝ≥0∞) * ((occPermSet O (k + 1)).card : ℝ≥0∞)) = + ((O.card - k : ℕ) : ℝ≥0∞) * ((occPermSet O k).card : ℝ≥0∞) := by + exact_mod_cast mul_card_occPermSet_succ O hk_lt + have hmk_ne0 : ((m - k : ℕ) : ℝ≥0∞) ≠ 0 := by + exact_mod_cast (by omega : m - k ≠ 0) + have hmk_top : ((m - k : ℕ) : ℝ≥0∞) ≠ ⊤ := ENNReal.natCast_ne_top _ + have hcard_eq : ((occPermSet O (k + 1)).card : ℝ≥0∞) = + ((occPermSet O k).card : ℝ≥0∞) * (((O.card - k : ℕ) : ℝ≥0∞) / ((m - k : ℕ) : ℝ≥0∞)) := by + rw [div_eq_mul_inv, ← mul_assoc, mul_comm ((occPermSet O k).card : ℝ≥0∞), ← hrec, + mul_comm ((m - k : ℕ) : ℝ≥0∞), mul_assoc, ENNReal.mul_inv_cancel hmk_ne0 hmk_top, mul_one] + rw [hcard_eq] + ring + +/-- Each factor `(n - i) / (m - i)` for `i < m` and `n ≤ m` is at most `n / m`. -/ +theorem sub_div_sub_le_div {n m i : ℕ} (hnm : n ≤ m) (hi : i < m) : + (((n - i : ℕ) : ℝ≥0∞) / ((m - i : ℕ) : ℝ≥0∞)) ≤ (n : ℝ≥0∞) / (m : ℝ≥0∞) := by + have hm_ne0 : (m : ℝ≥0∞) ≠ 0 := by exact_mod_cast (by omega : m ≠ 0) + have hm_top : (m : ℝ≥0∞) ≠ ⊤ := ENNReal.natCast_ne_top m + have hmi_ne0 : ((m - i : ℕ) : ℝ≥0∞) ≠ 0 := by exact_mod_cast (by omega : m - i ≠ 0) + have hmi_top : ((m - i : ℕ) : ℝ≥0∞) ≠ ⊤ := ENNReal.natCast_ne_top (m - i) + have hnat : (n - i) * m ≤ n * (m - i) := by + rw [Nat.sub_mul, Nat.mul_sub, Nat.mul_comm n i] + exact Nat.sub_le_sub_left (Nat.mul_le_mul_left i hnm) (n * m) + have hcast : ((n - i : ℕ) : ℝ≥0∞) * (m : ℝ≥0∞) ≤ (n : ℝ≥0∞) * ((m - i : ℕ) : ℝ≥0∞) := by + exact_mod_cast hnat + calc + (((n - i : ℕ) : ℝ≥0∞) / ((m - i : ℕ) : ℝ≥0∞)) + = (((n - i : ℕ) : ℝ≥0∞) * (m : ℝ≥0∞)) * ((m : ℝ≥0∞)⁻¹ * ((m - i : ℕ) : ℝ≥0∞)⁻¹) := by + rw [div_eq_mul_inv, mul_assoc, ← mul_assoc (m : ℝ≥0∞), + ENNReal.mul_inv_cancel hm_ne0 hm_top, one_mul] + _ ≤ ((n : ℝ≥0∞) * ((m - i : ℕ) : ℝ≥0∞)) * ((m : ℝ≥0∞)⁻¹ * ((m - i : ℕ) : ℝ≥0∞)⁻¹) := by + gcongr + _ = (n : ℝ≥0∞) / (m : ℝ≥0∞) := by + rw [div_eq_mul_inv, mul_assoc, mul_left_comm ((m - i : ℕ) : ℝ≥0∞), + ENNReal.mul_inv_cancel hmi_ne0 hmi_top, mul_one] + +/-- **Upper bound `(n / m) ^ k` on the `k`-probe tail probability ([CLRS2009], Proof of Theorem +11.6)**: for any occupied slot set `O : Finset (Fin m)` and `k ≤ m`, the probability over a uniform +random permutation `σ` that the first `k` probes all hit `O` is at most +`((O.card : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k`. -/ +theorem toOuterMeasure_allOccupied_le_pow {m : ℕ} (O : Finset (Fin m)) {k : ℕ} (hk : k ≤ m) : + (PMF.uniformOfFintype (Equiv.Perm (Fin m))).toOuterMeasure {σ | allOccupied O k σ} ≤ + ((O.card : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k := by + have hO_le : O.card ≤ m := by + simpa using Finset.card_le_univ O + rw [toOuterMeasure_allOccupied_eq_prod O hk] + calc + (∏ i ∈ Finset.range k, (((O.card - i : ℕ) : ℝ≥0∞) / ((m - i : ℕ) : ℝ≥0∞))) + ≤ ∏ _i ∈ Finset.range k, ((O.card : ℝ≥0∞) / (m : ℝ≥0∞)) := + Finset.prod_le_prod fun i hi => + sub_div_sub_le_div hO_le (lt_of_lt_of_le (Finset.mem_range.mp hi) hk) + _ = ((O.card : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k := by + simp [Finset.prod_const] + +/-- Finite geometric sum identity in `ℝ≥0∞` without subtraction: for `n < m`, +`∑ k ∈ Finset.range j, (n / m) ^ k + (m / (m - n)) * (n / m) ^ j = m / (m - n)`. -/ +theorem sum_div_pow_add_tail {n m : ℕ} (hnm : n < m) (j : ℕ) : + (∑ k ∈ Finset.range j, ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k) + + ((m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞)) * ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ j = + (m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞) := by + have hm_ne0 : (m : ℝ≥0∞) ≠ 0 := by exact_mod_cast (by omega : m ≠ 0) + have hm_top : (m : ℝ≥0∞) ≠ ⊤ := ENNReal.natCast_ne_top m + have hmn_ne0 : ((m - n : ℕ) : ℝ≥0∞) ≠ 0 := by exact_mod_cast (by omega : m - n ≠ 0) + have hmn_top : ((m - n : ℕ) : ℝ≥0∞) ≠ ⊤ := ENNReal.natCast_ne_top (m - n) + have hstep : 1 + ((m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞)) * ((n : ℝ≥0∞) / (m : ℝ≥0∞)) = + (m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞) := by + have hmul : ((m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞)) * ((n : ℝ≥0∞) / (m : ℝ≥0∞)) = + (n : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞) := by + rw [div_eq_mul_inv, div_eq_mul_inv, div_eq_mul_inv, + mul_comm (n : ℝ≥0∞), ← mul_assoc, mul_right_comm (m : ℝ≥0∞), + ENNReal.mul_inv_cancel hm_ne0 hm_top, one_mul, mul_comm] + rw [hmul, ← ENNReal.div_self hmn_ne0 hmn_top, ← ENNReal.add_div] + congr 1 + exact_mod_cast Nat.sub_add_cancel (le_of_lt hnm) + induction j with + | zero => simp + | succ j ih => + have htail : ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ j + + ((m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞)) * ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ (j + 1) = + ((m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞)) * ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ j := by + rw [pow_succ, ← mul_assoc, + mul_comm ((m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞)) (((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ j), + mul_assoc] + nth_rw 1 [← mul_one (((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ j)] + rw [← mul_add, hstep, mul_comm] + rw [Finset.sum_range_succ, add_assoc, htail, ih] + +/-- For `n < m`, any partial geometric sum `∑ k ∈ Finset.range j, (n / m) ^ k` is at most +`m / (m - n)`. -/ +theorem sum_div_pow_le {n m : ℕ} (hnm : n < m) (j : ℕ) : + (∑ k ∈ Finset.range j, ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k) ≤ + (m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞) := by + rw [← sum_div_pow_add_tail hnm j] + exact le_self_add + +/-- When `2 * n ≤ m` and `0 < m`, `m / (m - n) ≤ 2` (`1 / (1 - α) ≤ 2` for load factor `α ≤ 1 / 2`). +-/ +theorem div_sub_le_two {n m : ℕ} (hm : 0 < m) (h2n : 2 * n ≤ m) : + (m : ℝ≥0∞) / ((m - n : ℕ) : ℝ≥0∞) ≤ 2 := by + have hmn_ne0 : ((m - n : ℕ) : ℝ≥0∞) ≠ 0 := by exact_mod_cast (by omega : m - n ≠ 0) + have hmn_top : ((m - n : ℕ) : ℝ≥0∞) ≠ ⊤ := ENNReal.natCast_ne_top (m - n) + rw [ENNReal.div_le_iff_le_mul (Or.inl hmn_ne0) (Or.inl hmn_top)] + exact_mod_cast (by omega : m ≤ 2 * (m - n)) + +/-- **Theorem 11.6 of [CLRS2009] (Expected probes in an unsuccessful search)**: +for any occupied slot set `O : Finset (Fin m)` with `O.card < m`, the expected number of probes +`unsuccessfulProbes O σ` over a uniform random permutation +`σ ← PMF.uniformOfFintype (Equiv.Perm (Fin m))` is at most `m / (m - O.card)`. -/ +theorem expect_unsuccessfulProbes_le {m : ℕ} {O : Finset (Fin m)} (hO : O.card < m) : + (∑' σ : Equiv.Perm (Fin m), + PMF.uniformOfFintype (Equiv.Perm (Fin m)) σ * (unsuccessfulProbes O σ : ℝ≥0∞)) ≤ + (m : ℝ≥0∞) / ((m - O.card : ℕ) : ℝ≥0∞) := by + classical + let μ := PMF.uniformOfFintype (Equiv.Perm (Fin m)) + have hdecomp : ∀ σ : Equiv.Perm (Fin m), + μ σ * (unsuccessfulProbes O σ : ℝ≥0∞) = + ∑ k ∈ Finset.range m, μ σ * (if allOccupied O k σ then 1 else 0) := by + intro σ + rw [← Finset.mul_sum] + congr 1 + exact_mod_cast unsuccessfulProbes_eq_sum_boole hO σ + change (∑' σ : Equiv.Perm (Fin m), μ σ * (unsuccessfulProbes O σ : ℝ≥0∞)) ≤ _ + simp_rw [hdecomp] + rw [tsum_fintype, Finset.sum_comm] + have hterm : ∀ k ∈ Finset.range m, + (∑ σ : Equiv.Perm (Fin m), μ σ * (if allOccupied O k σ then (1 : ℝ≥0∞) else 0)) = + μ.toOuterMeasure {σ | allOccupied O k σ} := by + intro k _ + rw [PMF.toOuterMeasure_apply, tsum_fintype] + refine Finset.sum_congr rfl fun σ _ => ?_ + unfold Set.indicator + by_cases h : allOccupied O k σ + · have hmem : σ ∈ {σ | allOccupied O k σ} := h + simp [h, hmem] + · have hmem : σ ∉ {σ | allOccupied O k σ} := h + simp [h, hmem] + rw [Finset.sum_congr rfl hterm] + calc + (∑ k ∈ Finset.range m, μ.toOuterMeasure {σ | allOccupied O k σ}) + ≤ ∑ k ∈ Finset.range m, ((O.card : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k := + Finset.sum_le_sum fun k hk => + toOuterMeasure_allOccupied_le_pow O (le_of_lt (Finset.mem_range.mp hk)) + _ ≤ (m : ℝ≥0∞) / ((m - O.card : ℕ) : ℝ≥0∞) := sum_div_pow_le hO m + +/-- **Half-load corollary of Theorem 11.6 ([CLRS2009], Section 11.4)**: +when `0 < m` and `2 * O.card ≤ m` (load factor `α = O.card / m ≤ 1 / 2`), the expected number of +probes in an unsuccessful search is at most `2`. -/ +theorem expect_unsuccessfulProbes_le_two {m : ℕ} {O : Finset (Fin m)} + (hm : 0 < m) (h2n : 2 * O.card ≤ m) : + (∑' σ : Equiv.Perm (Fin m), + PMF.uniformOfFintype (Equiv.Perm (Fin m)) σ * (unsuccessfulProbes O σ : ℝ≥0∞)) ≤ 2 := + (expect_unsuccessfulProbes_le (by omega)).trans (div_sub_le_two hm h2n) + + +/-! ## Section 3: Open-Addressed Table Model & Theorem 11.8 ([CLRS2009], Section 11.4) -/ + +/-- Pure open-addressed hash table of `m` slots over key universe `Fin U`, where each slot holds +`none` (empty) or `some (u, v)` (occupied by key `u` with value `v`). -/ +abbrev Table (U m : ℕ) := Fin m → Option (Fin U × ℕ) + +/-- The initially empty table of `m` slots. -/ +def emptyTable (U m : ℕ) : Table U m := fun _ => none + +/-- The finite set of currently occupied slot indices in table `T`. -/ +def occupiedSlots {U m : ℕ} (T : Table U m) : Finset (Fin m) := + Finset.univ.filter fun j => (T j).isSome + +/-- The finite set of slot indices in `T` occupied by keys distinct from `u`. -/ +def otherOccupiedSlots {U m : ℕ} (T : Table U m) (u : Fin U) : Finset (Fin m) := + Finset.univ.filter fun j => (T j).isSome ∧ (T j).map Prod.fst ≠ some u + +/-- Total number of slot probes `1 + missedProbes (otherOccupiedSlots T u) σ` made when probing +table `T` for key `u` along permutation `σ`. -/ +def probeCount {U m : ℕ} (T : Table U m) (u : Fin U) (σ : Equiv.Perm (Fin m)) : ℕ := + unsuccessfulProbes (otherOccupiedSlots T u) σ + +@[simp] +theorem occupiedSlots_emptyTable (U m : ℕ) : + occupiedSlots (emptyTable U m) = ∅ := by + ext j + simp [occupiedSlots, emptyTable] + +@[simp] +theorem otherOccupiedSlots_emptyTable (U m : ℕ) (u : Fin U) : + otherOccupiedSlots (emptyTable U m) u = ∅ := by + ext j + simp [otherOccupiedSlots, emptyTable] + +theorem mem_otherOccupiedSlots_iff {U m : ℕ} {T : Table U m} {u : Fin U} {j : Fin m} : + j ∈ otherOccupiedSlots T u ↔ ∃ u' v', T j = some (u', v') ∧ u' ≠ u := by + simp only [otherOccupiedSlots, Finset.mem_filter, Finset.mem_univ, true_and] + cases h : T j with + | none => simp + | some p => + rcases p with ⟨u', v'⟩ + simp + +theorem otherOccupiedSlots_subset_occupiedSlots {U m : ℕ} (T : Table U m) (u : Fin U) : + otherOccupiedSlots T u ⊆ occupiedSlots T := by + intro j hj + rcases mem_otherOccupiedSlots_iff.mp hj with ⟨u', v', hTj, -⟩ + simp [occupiedSlots, hTj] + +/-- Open-addressed table representation invariant linking concrete slot array `T : Table U m` to +abstract dictionary state `s : Dictionary.State U` under permutation family +`π : Fin U → Equiv.Perm (Fin m)`. -/ +structure TableRep {U m : ℕ} (π : Fin U → Equiv.Perm (Fin m)) (s : Dictionary.State U) + (T : Table U m) : Prop where + /-- Every occupied slot `j` holds `(u, v)` with `s u = some v` at key `u`'s stopping probe index. + -/ + slot_spec : ∀ j u v, T j = some (u, v) → + s u = some v ∧ ∃ h : missedProbes (otherOccupiedSlots T u) (π u) < m, + j = π u ⟨missedProbes (otherOccupiedSlots T u) (π u), h⟩ + /-- Every present key `u` with `s u = some v` is stored at its stopping probe index. -/ + key_spec : ∀ u v, s u = some v → + ∃ h : missedProbes (otherOccupiedSlots T u) (π u) < m, + T (π u ⟨missedProbes (otherOccupiedSlots T u) (π u), h⟩) = some (u, v) + +/-- The empty table represents `Dictionary.empty U` under any permutation family `π`. -/ +theorem tableRep_empty {U m : ℕ} (π : Fin U → Equiv.Perm (Fin m)) : + TableRep π (Dictionary.empty U) (emptyTable U m) where + slot_spec j _ _ h := by simp [emptyTable] at h + key_spec u _ h := by simp [Dictionary.empty] at h + +/-- Under `TableRep π s T`, each key `u` occupies at most one slot in `T`. -/ +theorem TableRep.slot_eq_of_same_key {U m : ℕ} {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) + {j₁ j₂ : Fin m} {u : Fin U} {v₁ v₂ : ℕ} + (h₁ : T j₁ = some (u, v₁)) (h₂ : T j₂ = some (u, v₂)) : j₁ = j₂ := by + obtain ⟨-, hlt₁, rfl⟩ := hrep.slot_spec j₁ u v₁ h₁ + obtain ⟨-, _, rfl⟩ := hrep.slot_spec j₂ u v₂ h₂ + rfl + +/-- Under `TableRep π s T`, the number of occupied slots in `T` equals `(Dictionary.dom s).card`. -/ +theorem TableRep.card_occupiedSlots {U m : ℕ} {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) : + (occupiedSlots T).card = (Dictionary.dom s).card := by + refine Finset.card_bij + (fun j hj => ((T j).get (by simpa [occupiedSlots] using hj)).1) ?_ ?_ ?_ + · intro j hj + have his : (T j).isSome := by simpa [occupiedSlots] using hj + obtain ⟨⟨u, v⟩, hTj⟩ := Option.isSome_iff_exists.mp his + have hsu := (hrep.slot_spec j u v hTj).1 + simp [Dictionary.dom, hTj, hsu] + · intro j₁ hj₁ j₂ hj₂ heq + obtain ⟨⟨u₁, v₁⟩, h₁⟩ := Option.isSome_iff_exists.mp (by simpa [occupiedSlots] using hj₁) + obtain ⟨⟨u₂, v₂⟩, h₂⟩ := Option.isSome_iff_exists.mp (by simpa [occupiedSlots] using hj₂) + have hu : u₁ = u₂ := by simpa [h₁, h₂] using heq + subst hu + exact hrep.slot_eq_of_same_key h₁ h₂ + · intro u hu + obtain ⟨v, hsu⟩ := Dictionary.mem_dom_iff.mp hu + obtain ⟨hlt, hTj⟩ := hrep.key_spec u v hsu + refine ⟨π u ⟨missedProbes (otherOccupiedSlots T u) (π u), hlt⟩, ?_, ?_⟩ + · simp [occupiedSlots, hTj] + · simp [hTj] + +/-- Under `TableRep π s T`, `(otherOccupiedSlots T u).card = ((Dictionary.dom s).erase u).card`. -/ +theorem TableRep.card_otherOccupiedSlots {U m : ℕ} {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) (u : Fin U) : + (otherOccupiedSlots T u).card = ((Dictionary.dom s).erase u).card := by + refine Finset.card_bij + (fun j hj => ((T j).get (by + exact (Finset.mem_filter.mp hj).2.1)).1) ?_ ?_ ?_ + · intro j hj + rcases mem_otherOccupiedSlots_iff.mp hj with ⟨u', v', hTj, hne⟩ + have hsu' := (hrep.slot_spec j u' v' hTj).1 + simp [Dictionary.dom, hTj, hsu', hne] + · intro j₁ hj₁ j₂ hj₂ heq + rcases mem_otherOccupiedSlots_iff.mp hj₁ with ⟨u₁, v₁, h₁, -⟩ + rcases mem_otherOccupiedSlots_iff.mp hj₂ with ⟨u₂, v₂, h₂, -⟩ + have hu : u₁ = u₂ := by simpa [h₁, h₂] using heq + subst hu + exact hrep.slot_eq_of_same_key h₁ h₂ + · intro u' hu' + rcases Finset.mem_erase.mp hu' with ⟨hne, hu'_dom⟩ + obtain ⟨v', hsu'⟩ := Dictionary.mem_dom_iff.mp hu'_dom + obtain ⟨hlt, hTj⟩ := hrep.key_spec u' v' hsu' + refine ⟨π u' ⟨missedProbes (otherOccupiedSlots T u') (π u'), hlt⟩, ?_, ?_⟩ + · exact mem_otherOccupiedSlots_iff.mpr ⟨u', v', hTj, hne⟩ + · simp [hTj] + +/-- Under `TableRep π s T`, `(otherOccupiedSlots T u).card ≤ (Dictionary.dom s).card`. -/ +theorem TableRep.card_otherOccupiedSlots_le {U m : ℕ} {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) (u : Fin U) : + (otherOccupiedSlots T u).card ≤ (Dictionary.dom s).card := by + rw [hrep.card_otherOccupiedSlots u] + exact Finset.card_erase_le + +/-- When `s u = none`, no slot in `T` holds key `u`, so `otherOccupiedSlots T u = occupiedSlots T`. +-/ +theorem TableRep.otherOccupiedSlots_eq_occupiedSlots_of_eq_none {U m : ℕ} + {π : Fin U → Equiv.Perm (Fin m)} {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) {u : Fin U} (hu : s u = none) : + otherOccupiedSlots T u = occupiedSlots T := by + ext j + refine ⟨fun hj => otherOccupiedSlots_subset_occupiedSlots T u hj, fun hj => ?_⟩ + simp only [occupiedSlots, Finset.mem_filter, Finset.mem_univ, true_and] at hj + obtain ⟨⟨u', v'⟩, hTj⟩ := Option.isSome_iff_exists.mp hj + have hsu' := (hrep.slot_spec j u' v' hTj).1 + have hne : u' ≠ u := by + rintro rfl + rw [hu] at hsu' + cases hsu' + exact mem_otherOccupiedSlots_iff.mpr ⟨u', v', hTj, hne⟩ + +/-- When `TableRep π s T` and `(Dictionary.dom s).card < m`, the stopping index for any key `u` +satisfies `missedProbes (otherOccupiedSlots T u) (π u) < m`. -/ +theorem TableRep.missedProbes_lt {U m : ℕ} {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) + (hcard : (Dictionary.dom s).card < m) (u : Fin U) : + missedProbes (otherOccupiedSlots T u) (π u) < m := + missedProbes_lt_of_card_lt (lt_of_le_of_lt (hrep.card_otherOccupiedSlots_le u) hcard) (π u) + +/-- Target slot `π u ⟨missedProbes (otherOccupiedSlots T u) (π u), _⟩` where key `u` is found or +inserted when `(otherOccupiedSlots T u).card < m`. -/ +def targetSlot {U m : ℕ} (π : Fin U → Equiv.Perm (Fin m)) (T : Table U m) (u : Fin U) + (hO : (otherOccupiedSlots T u).card < m) : Fin m := + π u (probeSlotIndex (otherOccupiedSlots T u) (π u) hO) + +/-- Under `TableRep π s T` and `hO : (otherOccupiedSlots T u).card < m`, `T (targetSlot π T u hO)` +equals `(s u).map (fun v => (u, v))`. -/ +theorem TableRep.apply_targetSlot {U m : ℕ} {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) (u : Fin U) + (hO : (otherOccupiedSlots T u).card < m) : + T (targetSlot π T u hO) = (s u).map (fun v => (u, v)) := by + cases hsu : s u with + | none => + simp only [Option.map_none] + have hnot : targetSlot π T u hO ∉ otherOccupiedSlots T u := + notMem_apply_probeSlotIndex hO + cases hTj : T (targetSlot π T u hO) with + | none => rfl + | some p => + rcases p with ⟨u', v'⟩ + by_cases hne : u' = u + · subst hne + have hsu' := (hrep.slot_spec _ u' v' hTj).1 + rw [hsu] at hsu' + cases hsu' + · exact False.elim (hnot (mem_otherOccupiedSlots_iff.mpr ⟨u', v', hTj, hne⟩)) + | some v => + simp only [Option.map_some] + obtain ⟨_, hTj⟩ := hrep.key_spec u v hsu + exact hTj + +/-- Pure open-addressed lookup `tableGet hm π T u` when `0 < m`: inspects slot +`π u ⟨min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1), _⟩` and returns `some v` if it +contains `(u, v)`, else `none`. -/ +def tableGet {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) (T : Table U m) + (u : Fin U) : Option ℕ := + let idx := min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1) + match T (π u ⟨idx, by omega⟩) with + | some (u', v) => if u' = u then some v else none + | none => none + +/-- Pure open-addressed insertion `tableInsert hm π T u v`: writes `some (u, v)` at slot +`π u ⟨min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1), _⟩`. -/ +def tableInsert {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) (T : Table U m) + (u : Fin U) (v : ℕ) : Table U m := + let idx := min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1) + Function.update T (π u ⟨idx, by omega⟩) (some (u, v)) + +theorem tableInsert_eq_update_targetSlot {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) + (T : Table U m) (u : Fin U) (v : ℕ) (hO : (otherOccupiedSlots T u).card < m) : + tableInsert hm π T u v = Function.update T (targetSlot π T u hO) (some (u, v)) := by + have hlt : missedProbes (otherOccupiedSlots T u) (π u) < m := + missedProbes_lt_of_card_lt hO (π u) + have hmin : min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1) = + missedProbes (otherOccupiedSlots T u) (π u) := Nat.min_eq_left (by omega) + have hslot : (⟨min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1), by omega⟩ : Fin m) = + probeSlotIndex (otherOccupiedSlots T u) (π u) hO := Fin.ext hmin + dsimp only [tableInsert, targetSlot] + rw [hslot] + +/-- Under `TableRep π s T` and `(otherOccupiedSlots T u).card < m`, `tableGet hm π T u = s u`. -/ +theorem TableRep.tableGet_eq {U m : ℕ} (hm : 0 < m) {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) (u : Fin U) + (hO : (otherOccupiedSlots T u).card < m) : + tableGet hm π T u = s u := by + have hlt : missedProbes (otherOccupiedSlots T u) (π u) < m := + missedProbes_lt_of_card_lt hO (π u) + have hmin : min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1) = + missedProbes (otherOccupiedSlots T u) (π u) := Nat.min_eq_left (by omega) + dsimp only [tableGet] + have hslot : (⟨min (missedProbes (otherOccupiedSlots T u) (π u)) (m - 1), by omega⟩ : Fin m) = + probeSlotIndex (otherOccupiedSlots T u) (π u) hO := Fin.ext hmin + rw [hslot] + have htarget := hrep.apply_targetSlot u hO + dsimp only [targetSlot] at htarget + rw [htarget] + cases s u <;> simp + +/-- Inserting `(u, v)` leaves `otherOccupiedSlots` for `u` itself unchanged. -/ +theorem otherOccupiedSlots_tableInsert_self {U m : ℕ} (hm : 0 < m) + {π : Fin U → Equiv.Perm (Fin m)} {s : Dictionary.State U} {T : Table U m} + (_hrep : TableRep π s T) (u : Fin U) (v : ℕ) + (hO : (otherOccupiedSlots T u).card < m) : + otherOccupiedSlots (tableInsert hm π T u v) u = otherOccupiedSlots T u := by + rw [tableInsert_eq_update_targetSlot hm π T u v hO] + ext j + by_cases hj : j = targetSlot π T u hO + · subst hj + have hnot : targetSlot π T u hO ∉ otherOccupiedSlots T u := + notMem_apply_probeSlotIndex hO + simp [mem_otherOccupiedSlots_iff, hnot] + · simp [mem_otherOccupiedSlots_iff, Function.update_of_ne hj] + +/-- Inserting `(u, v)` preserves the stopping index `missedProbes (otherOccupiedSlots · w) (π w)` +for every key `w` already present in `s` (`(s w).isSome`). -/ +theorem TableRep.missedProbes_tableInsert_of_isSome {U m : ℕ} (hm : 0 < m) + {π : Fin U → Equiv.Perm (Fin m)} {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) (u : Fin U) (v : ℕ) + (hO : (otherOccupiedSlots T u).card < m) + {w : Fin U} (hw : (s w).isSome) : + missedProbes (otherOccupiedSlots (tableInsert hm π T u v) w) (π w) = + missedProbes (otherOccupiedSlots T w) (π w) := by + by_cases hwu : w = u + · subst hwu + rw [otherOccupiedSlots_tableInsert_self hm hrep w v hO] + · obtain ⟨vw, hsw⟩ := Option.isSome_iff_exists.mp hw + obtain ⟨hlt_w, hTw⟩ := hrep.key_spec w vw hsw + set iw : Fin m := ⟨missedProbes (otherOccupiedSlots T w) (π w), hlt_w⟩ + have hiw_spec := (missedProbes_eq_iff (O := otherOccupiedSlots T w) (σ := π w) iw).mp rfl + have htarget_u := hrep.apply_targetSlot u hO + have hne_slot : π w iw ≠ targetSlot π T u hO := by + intro heq + rw [← heq, hTw] at htarget_u + cases hsu : s u with + | none => simp [hsu] at htarget_u + | some vu => + simp only [hsu, Option.map_some, Option.some.injEq, Prod.mk.injEq] at htarget_u + exact hwu htarget_u.1 + rw [tableInsert_eq_update_targetSlot hm π T u v hO] + apply (missedProbes_eq_iff iw).mpr + refine ⟨fun i hi => ?_, ?_⟩ + · have hi_mem : π w i ∈ otherOccupiedSlots T w := hiw_spec.1 i hi + rcases mem_otherOccupiedSlots_iff.mp hi_mem with ⟨u', v', hTi, hne'⟩ + rw [mem_otherOccupiedSlots_iff] + by_cases h_eq : π w i = targetSlot π T u hO + · refine ⟨u, v, ?_, Ne.symm hwu⟩ + simp [h_eq] + · refine ⟨u', v', ?_, hne'⟩ + simp [Function.update_of_ne h_eq, hTi] + · rw [mem_otherOccupiedSlots_iff] + rintro ⟨u', v', hTiw, hne'⟩ + simp only [Function.update_of_ne hne_slot] at hTiw + rw [hTw] at hTiw + injection hTiw with hpair + injection hpair with hu' _ + exact hne' hu'.symm + +/-- **Preservation of `TableRep` under `tableInsert`**: if `TableRep π s T` and +`(otherOccupiedSlots T u).card < m`, then `tableInsert hm π T u v` satisfies +`TableRep π (Function.update s u (some v))`. -/ +theorem TableRep.tableInsert {U m : ℕ} (hm : 0 < m) {π : Fin U → Equiv.Perm (Fin m)} + {s : Dictionary.State U} {T : Table U m} (hrep : TableRep π s T) (u : Fin U) (v : ℕ) + (hO : (otherOccupiedSlots T u).card < m) : + TableRep π (Function.update s u (some v)) (tableInsert hm π T u v) := by + have hlt_u : missedProbes (otherOccupiedSlots T u) (π u) < m := + missedProbes_lt_of_card_lt hO (π u) + have hself_mp : + missedProbes (otherOccupiedSlots (UniformProbing.tableInsert hm π T u v) u) (π u) = + missedProbes (otherOccupiedSlots T u) (π u) := by + rw [otherOccupiedSlots_tableInsert_self hm hrep u v hO] + have hupd := tableInsert_eq_update_targetSlot hm π T u v hO + refine ⟨?_, ?_⟩ + · intro j w vw hj + by_cases hju : j = targetSlot π T u hO + · subst hju + rw [hupd] at hj + simp only [Function.update_self, Option.some.injEq, Prod.mk.injEq] at hj + rcases hj with ⟨rfl, rfl⟩ + refine ⟨by simp, ⟨by omega, ?_⟩⟩ + ext + simp [targetSlot, probeSlotIndex, hself_mp] + · rw [hupd, Function.update_of_ne hju] at hj + obtain ⟨hsw, hlt_w, hj_eq⟩ := hrep.slot_spec j w vw hj + have hwu : w ≠ u := by + rintro rfl + have ⟨_, hTu⟩ := hrep.key_spec w vw hsw + have h_same := hrep.slot_eq_of_same_key hj hTu + exact hju h_same + have hw_mp := hrep.missedProbes_tableInsert_of_isSome hm u v hO (by rw [hsw]; rfl) + refine ⟨by simp [Function.update_of_ne hwu, hsw], ⟨by omega, ?_⟩⟩ + ext + simpa [hw_mp] using congrArg Fin.val hj_eq + · intro w vw hsw' + by_cases hwu : w = u + · subst hwu + simp only [Function.update_self, Option.some.injEq] at hsw' + subst hsw' + refine ⟨by omega, ?_⟩ + have hidx_eq : + (⟨missedProbes (otherOccupiedSlots (UniformProbing.tableInsert hm π T w v) w) (π w), + by omega⟩ : Fin m) = probeSlotIndex (otherOccupiedSlots T w) (π w) hO := by + ext + simp [probeSlotIndex, hself_mp] + rw [hidx_eq, hupd] + simp [targetSlot] + · simp only [Function.update_of_ne hwu] at hsw' + obtain ⟨hlt_w, hTw⟩ := hrep.key_spec w vw hsw' + have hw_mp := hrep.missedProbes_tableInsert_of_isSome hm u v hO (by rw [hsw']; rfl) + refine ⟨by omega, ?_⟩ + have hidx_eq : + (⟨missedProbes (otherOccupiedSlots (UniformProbing.tableInsert hm π T u v) w) (π w), + by omega⟩ : Fin m) = ⟨missedProbes (otherOccupiedSlots T w) (π w), hlt_w⟩ := by + ext + exact hw_mp + rw [hidx_eq, hupd] + have hne_slot : + π w ⟨missedProbes (otherOccupiedSlots T w) (π w), hlt_w⟩ ≠ targetSlot π T u hO := by + intro heq + have htarget_u := hrep.apply_targetSlot u hO + rw [← heq, hTw] at htarget_u + cases hsu : s u with + | none => simp [hsu] at htarget_u + | some vu => + simp only [hsu, Option.map_some, Option.some.injEq, Prod.mk.injEq] at htarget_u + exact hwu htarget_u.1 + simp [Function.update_of_ne hne_slot, hTw] + +/-- Inserting `(u, v)` leaves `probeCount · u (π u)` unchanged. -/ +theorem TableRep.probeCount_tableInsert_self {U m : ℕ} (hm : 0 < m) + {π : Fin U → Equiv.Perm (Fin m)} {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) (u : Fin U) (v : ℕ) + (hO : (otherOccupiedSlots T u).card < m) : + probeCount (UniformProbing.tableInsert hm π T u v) u (π u) = probeCount T u (π u) := by + simp [probeCount, unsuccessfulProbes, otherOccupiedSlots_tableInsert_self hm hrep u v hO] + +/-- **Probe invariance for present keys ([CLRS2009], Section 11.4, p. 275)**: +once a key `w` is present in `s` (`(s w).isSome`), inserting or updating any key `(u, v)` leaves +the number of probes `probeCount · w (π w)` made when searching for `w` unchanged. -/ +theorem TableRep.probeCount_tableInsert_of_isSome {U m : ℕ} (hm : 0 < m) + {π : Fin U → Equiv.Perm (Fin m)} {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) (u : Fin U) (v : ℕ) + (hO : (otherOccupiedSlots T u).card < m) + {w : Fin U} (hw : (s w).isSome) : + probeCount (UniformProbing.tableInsert hm π T u v) w (π w) = probeCount T w (π w) := by + simp [probeCount, unsuccessfulProbes, hrep.missedProbes_tableInsert_of_isSome hm u v hO hw] + +/-- Execute a single `Dictionary.Op U` on open-addressed table `T`. -/ +def tableStep {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) (T : Table U m) + (o : Dictionary.Op U) : Table U m × (Dictionary.spec U 0 0).Result o := + match o with + | .get u => (T, tableGet hm π T u) + | .insert u v => (tableInsert hm π T u v, ()) + +/-- Execute a sequence of `Dictionary.Op U` operations on `T` under permutation family `π`. -/ +def runTableFrom {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) : + Table U m → List (Dictionary.Op U) → Table U m + | T, [] => T + | T, o :: os => runTableFrom hm π (tableStep hm π T o).1 os + +/-- Table produced by running `ops` starting from `emptyTable U m`. -/ +def runTable {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) + (ops : List (Dictionary.Op U)) : Table U m := + runTableFrom hm π (emptyTable U m) ops + +/-- Under any admissible operation `(Dictionary.spec U V n_max).valid s o` with `n_max ≤ m` and +`TableRep π s T`, `(otherOccupiedSlots T u).card < m` holds for the operated key `u`. -/ +theorem TableRep.card_otherOccupiedSlots_lt_of_valid {U V n_max m : ℕ} (_hm : 0 < m) + (hcap : n_max ≤ m) {π : Fin U → Equiv.Perm (Fin m)} {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) (hs : (Dictionary.dom s).card ≤ n_max) + {u : Fin U} {v : ℕ} (hval : (Dictionary.spec U V n_max).valid s (.insert u v)) : + (otherOccupiedSlots T u).card < m := by + rcases hval.1 with hlt | hisSome + · have hle := hrep.card_otherOccupiedSlots_le u + omega + · rw [hrep.card_otherOccupiedSlots u] + have hmem : u ∈ Dictionary.dom s := Dictionary.mem_dom_iff_isSome.mpr hisSome + have herase := Finset.card_erase_lt_of_mem hmem + omega + +/-- Any valid operation history preserves `TableRep`. -/ +theorem tableRep_runTableFrom {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) + (π : Fin U → Equiv.Perm (Fin m)) {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) (hs : (Dictionary.dom s).card ≤ n_max) + {ops : List (Dictionary.Op U)} (hvh : (Dictionary.spec U V n_max).ValidHistory s ops) : + TableRep π ((Dictionary.spec U V n_max).run s ops).1 (runTableFrom hm π T ops) := by + induction ops generalizing s T with + | nil => exact hrep + | cons o os ih => + have hs' : (Dictionary.dom ((Dictionary.spec U V n_max).step s o).1).card ≤ n_max := + Dictionary.card_dom_step_le_of_valid hs hvh.1 + cases o with + | get u => + exact ih hrep hs' hvh.2 + | insert u v => + have hO : (otherOccupiedSlots T u).card < m := + hrep.card_otherOccupiedSlots_lt_of_valid hm hcap hs hvh.1 + exact ih (hrep.tableInsert hm u v hO) hs' hvh.2 + +/-- Running any valid operation history from `emptyTable U m` produces a table satisfying +`TableRep`. -/ +theorem tableRep_runTable {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) + (π : Fin U → Equiv.Perm (Fin m)) {ops : List (Dictionary.Op U)} + (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops) : + TableRep π ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1 + (runTable hm π ops) := + tableRep_runTableFrom hm hcap π (tableRep_empty π) (by simp) hvh + +/-- If key `w` is already present in `s`, then running any valid continuation `ops` from `(s, T)` +preserves `probeCount · w (π w)`. -/ +theorem probeCount_runTableFrom_of_isSome {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) + (π : Fin U → Equiv.Perm (Fin m)) {s : Dictionary.State U} {T : Table U m} + (hrep : TableRep π s T) (hs : (Dictionary.dom s).card ≤ n_max) + {w : Fin U} (hw : (s w).isSome) + {ops : List (Dictionary.Op U)} (hvh : (Dictionary.spec U V n_max).ValidHistory s ops) : + probeCount (runTableFrom hm π T ops) w (π w) = probeCount T w (π w) := by + induction ops generalizing s T with + | nil => rfl + | cons o os ih => + have hs' : (Dictionary.dom ((Dictionary.spec U V n_max).step s o).1).card ≤ n_max := + Dictionary.card_dom_step_le_of_valid hs hvh.1 + cases o with + | get u => + exact ih hrep hs' hw hvh.2 + | insert u v => + have hO : (otherOccupiedSlots T u).card < m := + hrep.card_otherOccupiedSlots_lt_of_valid hm hcap hs hvh.1 + have hw' : ((Function.update s u (some v)) w).isSome := by + by_cases hwu : w = u + · subst hwu; simp + · simpa [Function.update_of_ne hwu] using hw + rw [runTableFrom, tableStep, ih (hrep.tableInsert hm u v hO) hs' hw' hvh.2] + exact hrep.probeCount_tableInsert_of_isSome hm u v hO hw + +/-- Prefix + suffix decomposition for `ValidHistory` and `runTableFrom`. -/ +theorem validHistory_and_runTableFrom_append {U V n_max m : ℕ} (hm : 0 < m) + (π : Fin U → Equiv.Perm (Fin m)) : + ∀ {l₁ l₂ : List (Dictionary.Op U)} {s : Dictionary.State U} (T : Table U m), + (Dictionary.spec U V n_max).ValidHistory s (l₁ ++ l₂) → + (Dictionary.spec U V n_max).ValidHistory s l₁ ∧ + (Dictionary.spec U V n_max).ValidHistory ((Dictionary.spec U V n_max).run s l₁).1 l₂ ∧ + runTableFrom hm π T (l₁ ++ l₂) = runTableFrom hm π (runTableFrom hm π T l₁) l₂ + | [], _, _, _, h => ⟨trivial, h, rfl⟩ + | o :: l₁, l₂, s, T, h => by + obtain ⟨h₁, h₂, hrun⟩ := + validHistory_and_runTableFrom_append hm π + (s := ((Dictionary.spec U V n_max).step s o).1) (tableStep hm π T o).1 h.2 + exact ⟨⟨h.1, h₁⟩, h₂, hrun⟩ + +/-- **Pathwise Theorem 11.8 of [CLRS2009] (`probeCount_runTable_eq_firstInsert`)**: +if `ops = ops₁ ++ (.insert u v₀ :: ops₂)` is a valid operation history where `ops₁` is the prefix +before `u` is inserted (`((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1 u = none`), +then the number of probes made when searching for `u` after the entire history `ops` equals the +number of probes made when `u` was first inserted after `ops₁`. -/ +theorem probeCount_runTable_eq_firstInsert {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) + (π : Fin U → Equiv.Perm (Fin m)) + {ops₁ ops₂ : List (Dictionary.Op U)} {u : Fin U} {v₀ : ℕ} + (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) + (ops₁ ++ (.insert u v₀ :: ops₂))) : + probeCount (runTable hm π (ops₁ ++ (.insert u v₀ :: ops₂))) u (π u) = + probeCount (runTable hm π ops₁) u (π u) := by + obtain ⟨hvh₁, hvh_rest, hrun_eq⟩ := + validHistory_and_runTableFrom_append hm π (emptyTable U m) hvh + set s₁ : Dictionary.State U := ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1 + set T₁ := runTable hm π ops₁ + have hrep₁ : TableRep π s₁ T₁ := tableRep_runTable hm hcap π hvh₁ + have hs₁ : (Dictionary.dom s₁).card ≤ n_max := Dictionary.card_dom_run_empty_le hvh₁ + have hO₁ : (otherOccupiedSlots T₁ u).card < m := + hrep₁.card_otherOccupiedSlots_lt_of_valid hm hcap hs₁ hvh_rest.1 + have hrep_ins : TableRep π (Function.update s₁ u (some v₀)) (tableInsert hm π T₁ u v₀) := + hrep₁.tableInsert hm u v₀ hO₁ + have hs_ins : (Dictionary.dom (Function.update s₁ u (some v₀))).card ≤ n_max := + Dictionary.card_dom_step_le_of_valid hs₁ hvh_rest.1 + change probeCount (runTableFrom hm π (emptyTable U m) (ops₁ ++ (.insert u v₀ :: ops₂))) u (π u) = + probeCount T₁ u (π u) + rw [hrun_eq] + change probeCount (runTableFrom hm π (tableInsert hm π T₁ u v₀) ops₂) u (π u) = + probeCount T₁ u (π u) + rw [probeCount_runTableFrom_of_isSome hm hcap π hrep_ins hs_ins + (by rw [Function.update_self]; rfl) hvh_rest.2] + exact hrep₁.probeCount_tableInsert_self hm u v₀ hO₁ + +/-- If key `u` is never inserted during `ops` (`∀ v, .insert u v ∉ ops`), then `runTableFrom` is +independent of `π u`: updating `π` at `u` leaves the resulting table unchanged. -/ +theorem runTableFrom_update_eq {U m : ℕ} (hm : 0 < m) (π : Fin U → Equiv.Perm (Fin m)) + (u : Fin U) (σ : Equiv.Perm (Fin m)) (T : Table U m) : + ∀ {ops : List (Dictionary.Op U)}, + (∀ v, Dictionary.Op.insert u v ∉ ops) → + runTableFrom hm (Function.update π u σ) T ops = runTableFrom hm π T ops + | [], _ => rfl + | .get _ :: _, hno => by + dsimp only [runTableFrom, tableStep] + exact runTableFrom_update_eq hm π u σ T fun v hv => hno v (List.mem_cons_of_mem _ hv) + | .insert w v :: _, hno => by + have hwu : w ≠ u := by + rintro rfl + exact hno v List.mem_cons_self + have hins : tableInsert hm (Function.update π u σ) T w v = tableInsert hm π T w v := by + simp [tableInsert, Function.update_of_ne hwu] + dsimp only [runTableFrom, tableStep] + rw [hins] + exact runTableFrom_update_eq hm π u σ (tableInsert hm π T w v) + fun v' hv' => hno v' (List.mem_cons_of_mem _ hv') + +/-- Once `(s₀ u).isSome`, running any list of operations `l` preserves `isSome` at `u`. -/ +private theorem isSome_run_of_isSome {U V n_max : ℕ} {u : Fin U} : + ∀ (l : List (Dictionary.Op U)) {s₀ : Dictionary.State U}, + (s₀ u).isSome → (((Dictionary.spec U V n_max).run s₀ l).1 u).isSome + | [], _, h => h + | .get _ :: l, _, h => isSome_run_of_isSome l h + | .insert w _ :: l, s₀, h => by + apply isSome_run_of_isSome l (s₀ := ((Dictionary.spec U V n_max).step s₀ (.insert w _)).1) + dsimp only [Dictionary.spec] + by_cases hwu : u = w + · subst hwu; simp + · simpa [Function.update_of_ne hwu] using h + +/-- If `((Dictionary.spec U V n_max).run s ops).1 u = none`, then no `.insert u v` appears in `ops`. +-/ +theorem not_mem_insert_of_run_eq_none {U V n_max : ℕ} {u : Fin U} : + ∀ {ops : List (Dictionary.Op U)} {s : Dictionary.State U}, + ((Dictionary.spec U V n_max).run s ops).1 u = none → + ∀ v, Dictionary.Op.insert u v ∉ ops + | [], _, _, _ => by simp + | o :: os, s, hnone, v => by + intro hmem + rcases List.mem_cons.mp hmem with rfl | htail + · have his : + (((Dictionary.spec U V n_max).run + (((Dictionary.spec U V n_max).step s (.insert u v)).1) os).1 u).isSome := + isSome_run_of_isSome (V := V) (n_max := n_max) os + (by dsimp [Dictionary.spec]; rw [Function.update_self]; rfl) + have hnone' : + (((Dictionary.spec U V n_max).run + (((Dictionary.spec U V n_max).step s (.insert u v)).1) os).1 u) = none := hnone + rw [hnone'] at his + cases his + · exact not_mem_insert_of_run_eq_none + (s := ((Dictionary.spec U V n_max).step s o).1) hnone v htail + +/-- **Theorem 11.6 & Corollary 11.7 of [CLRS2009] over `PMF.uniformOfFintype`**: +for any valid operation history `ops` reaching state `s` with `(Dictionary.dom s).card < m` and any +absent key `u` (`s u = none`), the expected number of probes made by an unsuccessful lookup `get u` +or fresh-key insertion `insert u v` on `runTable hm π ops` over +`π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))` is at most +`m / (m - (Dictionary.dom s).card)` (`≤ 2` when `2 * (Dictionary.dom s).card ≤ m`). -/ +theorem expect_probeCount_of_eq_none_le {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) + {ops : List (Dictionary.Op U)} + (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops) + {u : Fin U} (hu : ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1 u = none) + (hcard : + (Dictionary.dom ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1).card < m) : + let s := ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1 + (∑' π : Fin U → Equiv.Perm (Fin m), + PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m)) π * + (probeCount (runTable hm π ops) u (π u) : ℝ≥0∞)) ≤ + (m : ℝ≥0∞) / ((m - (Dictionary.dom s).card : ℕ) : ℝ≥0∞) := by + intro s + have hcard_s : (Dictionary.dom s).card < m := hcard + have : Nonempty (Equiv.Perm (Fin m)) := ⟨1⟩ + let μ_all := PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m)) + let μ_one := PMF.uniformOfFintype (Equiv.Perm (Fin m)) + have hno : ∀ v, Dictionary.Op.insert u v ∉ ops := not_mem_insert_of_run_eq_none hu + have hupd : μ_all = μ_one.bind (fun σ => μ_all.map (fun π => Function.update π u σ)) := + (PMF.uniformOfFintype_update (ι := Fin U) (γ := Equiv.Perm (Fin m)) u).symm + calc + (∑' π, μ_all π * (probeCount (runTable hm π ops) u (π u) : ℝ≥0∞)) + = ∑' π, (μ_one.bind (fun σ => μ_all.map (fun π₀ => Function.update π₀ u σ))) π * + (probeCount (runTable hm π ops) u (π u) : ℝ≥0∞) := by + rw [← hupd] + _ = ∑' σ, μ_one σ * ∑' π₀, μ_all π₀ * + (probeCount (runTable hm (Function.update π₀ u σ) ops) u + ((Function.update π₀ u σ) u) : ℝ≥0∞) := by + rw [PMF.tsum_bind_mul] + refine tsum_congr fun σ => congrArg (μ_one σ * ·) ?_ + exact PMF.tsum_map_mul μ_all (fun π₀ => Function.update π₀ u σ) + (fun π => (probeCount (runTable hm π ops) u (π u) : ℝ≥0∞)) + _ = ∑' π₀, μ_all π₀ * ∑' σ, μ_one σ * + (unsuccessfulProbes (occupiedSlots (runTable hm π₀ ops)) σ : ℝ≥0∞) := by + simp_rw [← ENNReal.tsum_mul_left] + rw [tsum_comm] + refine tsum_congr fun π₀ => tsum_congr fun σ => ?_ + have hrep : TableRep π₀ s (runTable hm π₀ ops) := tableRep_runTable hm hcap π₀ hvh + have htbl : runTable hm (Function.update π₀ u σ) ops = runTable hm π₀ ops := + runTableFrom_update_eq hm π₀ u σ (emptyTable U m) hno + rw [htbl, Function.update_self, probeCount, + hrep.otherOccupiedSlots_eq_occupiedSlots_of_eq_none hu] + ring + _ ≤ ∑' π₀, μ_all π₀ * ((m : ℝ≥0∞) / ((m - (Dictionary.dom s).card : ℕ) : ℝ≥0∞)) := by + refine tsum_le_tsum fun π₀ => ?_ + gcongr + have hrep : TableRep π₀ s (runTable hm π₀ ops) := tableRep_runTable hm hcap π₀ hvh + have hcard_occ : (occupiedSlots (runTable hm π₀ ops)).card = (Dictionary.dom s).card := + hrep.card_occupiedSlots + rw [← hcard_occ] + exact expect_unsuccessfulProbes_le (by omega) + _ = (m : ℝ≥0∞) / ((m - (Dictionary.dom s).card : ℕ) : ℝ≥0∞) := by + rw [ENNReal.tsum_mul_right, μ_all.tsum_coe, one_mul] + +/-- **Half-load corollary of Theorem 11.6 & Corollary 11.7 ([CLRS2009], Section 11.4)**: +when `2 * n_max ≤ m`, any unsuccessful lookup or fresh-key insertion after a valid history `ops` +makes at most `2` probes in expectation over +`π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`. -/ +theorem expect_probeCount_of_eq_none_le_two {U V n_max m : ℕ} (hm : 0 < m) (h2cap : 2 * n_max ≤ m) + {ops : List (Dictionary.Op U)} + (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops) + {u : Fin U} (hu : ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1 u = none) : + (∑' π : Fin U → Equiv.Perm (Fin m), + PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m)) π * + (probeCount (runTable hm π ops) u (π u) : ℝ≥0∞)) ≤ 2 := by + have hdom_le : + (Dictionary.dom ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1).card ≤ n_max := + Dictionary.card_dom_run_empty_le hvh + refine (expect_probeCount_of_eq_none_le hm (by omega) hvh hu (by omega)).trans ?_ + exact div_sub_le_two hm (by omega) + +/-- **Theorem 11.8 of [CLRS2009] over `PMF.uniformOfFintype` (Expected probes in a successful +search)**: +if `ops = ops₁ ++ (.insert u v₀ :: ops₂)` is a valid operation history where `ops₁` is the prefix +before `u` is first inserted (`s₁ u = none`), then the expected number of probes made when +searching for `u` after `ops` equals the expected number of probes when `u` was first inserted +after `ops₁`, and is at most `m / (m - (Dictionary.dom s₁).card) ≤ 2` when `2 * n_max ≤ m`. -/ +theorem expect_probeCount_of_mem_dom_le {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) + {ops₁ ops₂ : List (Dictionary.Op U)} {u : Fin U} {v₀ : ℕ} + (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) + (ops₁ ++ (.insert u v₀ :: ops₂))) + (hu₁ : ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1 u = none) + (hcard₁ : + (Dictionary.dom ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1).card < m) : + let s₁ := ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1 + (∑' π : Fin U → Equiv.Perm (Fin m), + PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m)) π * + (probeCount (runTable hm π (ops₁ ++ (.insert u v₀ :: ops₂))) u (π u) : ℝ≥0∞)) ≤ + (m : ℝ≥0∞) / ((m - (Dictionary.dom s₁).card : ℕ) : ℝ≥0∞) := by + intro s₁ + have hvh₁ : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops₁ := + (validHistory_and_runTableFrom_append hm (fun _ => 1) (emptyTable U m) hvh).1 + have hcongr : ∀ π : Fin U → Equiv.Perm (Fin m), + probeCount (runTable hm π (ops₁ ++ (.insert u v₀ :: ops₂))) u (π u) = + probeCount (runTable hm π ops₁) u (π u) := + fun π => probeCount_runTable_eq_firstInsert hm hcap π hvh + simp_rw [hcongr] + exact expect_probeCount_of_eq_none_le hm hcap hvh₁ hu₁ hcard₁ + +/-- **Half-load corollary of Theorem 11.8 ([CLRS2009], Section 11.4)**: +when `2 * n_max ≤ m`, any successful lookup of a present key `u` (first inserted after prefix +`ops₁`) makes at most `2` probes in expectation over +`π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`. -/ +theorem expect_probeCount_of_mem_dom_le_two {U V n_max m : ℕ} (hm : 0 < m) (h2cap : 2 * n_max ≤ m) + {ops₁ ops₂ : List (Dictionary.Op U)} {u : Fin U} {v₀ : ℕ} + (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) + (ops₁ ++ (.insert u v₀ :: ops₂))) + (hu₁ : ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1 u = none) : + (∑' π : Fin U → Equiv.Perm (Fin m), + PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m)) π * + (probeCount (runTable hm π (ops₁ ++ (.insert u v₀ :: ops₂))) u (π u) : ℝ≥0∞)) ≤ 2 := by + have hvh₁ : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops₁ := + (validHistory_and_runTableFrom_append hm (fun _ => 1) (emptyTable U m) hvh).1 + have hdom₁_le : + (Dictionary.dom ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1).card ≤ n_max := + Dictionary.card_dom_run_empty_le hvh₁ + refine (expect_probeCount_of_mem_dom_le hm (by omega) hvh hu₁ (by omega)).trans ?_ + exact div_sub_le_two hm (by omega) + +/-! ## Section 4: Lazy Random-Oracle Equivalence (`RandomOracle.uniform_interpret_eq_lazy`) -/ + +/-- Any adaptive oracle program querying per-key permutations `u ↦ π u : Equiv.Perm (Fin m)` has the +exact same output distribution whether the permutation table +`π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))` is sampled eagerly upfront or lazily on +first query via `Cslib.RandomOracle.answer`. -/ +theorem uniform_interpret_eq_lazy_perm {U m : ℕ} {α : Type} + (program : OracleComp (fun _ : Fin U => Equiv.Perm (Fin m)) α) : + (do + let π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m)) + OracleComp.interpret (fun u => PMF.pure (π u)) program) = + ((OracleComp.interpret + (Cslib.RandomOracle.answer (fun _ : Fin U => PMF.uniformOfFintype (Equiv.Perm (Fin m)))) + program).run (fun _ => none)).map Prod.fst := + Cslib.RandomOracle.uniform_interpret_eq_lazy program + +end Cslib.Algorithms.Lean.UniformProbing diff --git a/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean b/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean new file mode 100644 index 000000000..c0659b66d --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean @@ -0,0 +1,2206 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Algorithms.Lean.Dictionary +public import Complexity.Algorithms.Lean.UniformProbing +public import Complexity.Algorithms.WordRAM.Hashing.MultiplyShift +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation +public import Complexity.Computability.WordRAM.ExactNotation +public import Complexity.Computability.WordRAM.Randomized.Subroutine +import Complexity.Probability.PMF.Expect + +/-! +# Self-Contained Open-Addressed Permutation-Probing Dictionary on the Word-RAM + +This file implements a self-contained open-addressed permutation-probing dictionary realizing +`Cslib.Algorithms.Lean.Dictionary.spec U V n_max` (`get` and `insert` up to `n_max` distinct keys) +on the Word-RAM (`Cslib.WordRAM.Basis.exact` and `Cslib.WordRAM.Basis.standard`), following the +double-hashing construction of +[Cormen, Leiserson, Rivest, and Stein, Section 11.4, "Double hashing"][CLRS2009] with simple +tabulation due to Zobrist (1970) and Pătraşcu and Thorup (2012). + +## Design and reuse of `Cslib.Algorithms.Lean.UniformProbing` + +In the ideal uniform-probing model (`Complexity/Algorithms/Lean/UniformProbing.lean`), each key +`u : Fin U` is assigned an independent random permutation `π u : Equiv.Perm (Fin m)`. Storing an +explicit table of `U` permutations requires `Ω(U)` words when `U = poly(n_max) ≫ n_max`. + +To achieve a self-contained `O(n_max)`-word data structure (when `U = poly(n_max)`), we construct +for each key `u : Fin U` a full-cycle affine permutation of `m = 2 ^ M` slots +(`M = Nat.clog 2 (max (4 * n_max) 4)`): +`probePerm θ u i = (h₁(u) + i * h₂(u)) mod 2 ^ M`, where: +* `h₁(u) ∈ Fin (2 ^ M)` is a 2-universal multiply-shift hash + (`Cslib.Algorithms.WordRAM.Hashing.MultiplyShift`); +* `h₂(u) ∈ Fin (2 ^ M)` is an odd stride computed by `d`-character simple tabulation + (`Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride`, with character bit-width + `b = max 1 (⌊log₂ (max n_max 2)⌋ / 2)` and `d = numChars U n_max`), following Zobrist (1970) + and Pătraşcu and Thorup (2012). + +Because `h₂(u)` is always odd and therefore coprime to `2 ^ M`, `probePerm θ u` is a bijection on +`Fin (2 ^ M)` for **every** seed `θ` and key `u` (`TabulatedOddStride.bijective_probeSeq`). We +therefore instantiate `Cslib.Algorithms.Lean.UniformProbing.TableRep (probePerm θ)` directly on the +concrete Word-RAM slot arrays. Note that `TabulatedOddStride` proves full-cycle permutation +bijectivity (`bijective_probeSeq`) rather than full uniform-permutation independence: accordingly, +`subroutineDS` certifies the concrete deterministic full-cycle body bound +`charge = opSteps U n_max = 13 * numBuckets n_max + 8 * numChars U n_max + 49` +(`O(n_max + log U / log n_max)`), while `expect_opPathSteps_le_of_probeBound` and +`expect_idealProbeSteps_le` connect the exact probe-path body cost +`13 * probeCount + 8 * numChars U n_max + 49` to `≤ 8 * numChars U n_max + 75` expected +instructions under an ideal uniform permutation law +(`Complexity/Algorithms/Lean/UniformProbing.lean`, Theorems 11.6 and 11.8). + +## Model, specialization, and cost scope + +* For fixed universe `U` and capacity bound `n_max` and an admissible external machine word width + `w` (`2 * hashBits U n_max ≤ w`, `Fits w (base + space U n_max)`, `Fits w V`), + `subroutineCode U n_max` uses precomputed configuration literals (`bucketBits`, `hashBits`, + `charBits`, `numChars`, masks, and layout offsets baked as instruction literals, with + `initParams = Fin.elim0`). +* Expected and amortized bounds in `seq_expected_amortized` charge initialization (`initFresh`) and + subroutine operation bodies (`C.ExecSteps`) across an operation history `ops` fixed independently + of the sampled seed (oblivious adversary), excluding caller argument delivery, caller + control/dispatch loops, and code/configuration construction. +* `Dictionary.spec U V n_max` is insert-only (`get` and `insert` up to `n_max` distinct keys), so + open-addressed slots require no tombstones. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.bucketBits`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.numBuckets`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.hashBits`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.charBits`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.numChars`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.space`: table dimension parameters and memory + footprint. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.Seed`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.seedLaw`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.primaryHash`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.strideHash`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.probePerm`: seed space, uniform seed law, + multiply-shift primary hash, tabulated odd stride, and full-cycle probe permutation. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.extractSlots`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.Rep`: slot-array extractor and Word-RAM + representation invariant. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.allocSpan`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.initSetup`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.opCode`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.subroutineCode`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.subroutineDS`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.subroutineDSStandard`: structured Word-RAM + programs and `Randomized.SubroutineDS` certificates over `Basis.exact` and `Basis.standard`. + +## Main results + +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.setup_exec_spec`: randomized seed and table + initialization in `setupSteps U n_max = 5 * (numChars U n_max * 2 ^ charBits n_max) + 25` + expected instructions. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.opCode_execWithin`: every `get` or `insert` + operation on key `u` executes in at most + `opPathSteps U n_max θ T u = 13 * probeCount T u (probePerm θ u) + 8 * numChars U n_max + 49 ≤ + opSteps U n_max` instructions for every seed `θ`. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.seq_expected_amortized`: end-to-end expected + instruction bound across any admissible sequence `ops : List (Dictionary.Op U)`. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.isUniversalHash_primaryHash`: + `(2 / numBuckets n_max)`-universality of `primaryHash` under `seedLaw U n_max`. +* `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.expect_opPathSteps_le_of_probeBound`, + `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing.expect_idealProbeSteps_le`: parametric + expected-step transfer (`13 * B + 8 * numChars U n_max + 49`) from any expected-probe bound `B`, + yielding `≤ 8 * numChars U n_max + 75` under the ideal uniform permutation law (`B = 2`). + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, + *Introduction to Algorithms*, Section 11.4][CLRS2009] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] +-/ + +@[expose] public section + +namespace Cslib.Algorithms.WordRAM.Dictionary.UniformProbing + +open Cslib.Algorithms.Lean +open Cslib.Algorithms.Lean.UniformProbing +open Cslib.Algorithms.WordRAM.Hashing +open Cslib.Algorithms.WordRAM.Hashing.Tabulation +open Cslib.Probability +open Cslib.WordRAM +open Cslib.WordRAM.Randomized +open scoped BigOperators ENNReal + +/-! ## Section 1: Table Parameters, Seed Space, and Full-Cycle Probe Permutations -/ + +/-- Log₂ of the power-of-2 table size: `M = ⌈log₂ (max (4 * n_max) 4)⌉ ≥ 2`. -/ +def bucketBits (n_max : ℕ) : ℕ := + Nat.clog 2 (max (4 * n_max) 4) + +/-- Power-of-2 number of open-addressed table slots `m = 2 ^ bucketBits n_max ≥ max (4 * n_max) 4`. +-/ +def numBuckets (n_max : ℕ) : ℕ := + 2 ^ bucketBits n_max + +/-- Bit-width `w₀ = max (bucketBits n_max) ⌈log₂ (max U 2)⌉` for the multiply-shift primary hash. +-/ +def hashBits (U n_max : ℕ) : ℕ := + max (bucketBits n_max) (Nat.clog 2 (max U 2)) + +/-- Character bit-width `b = max 1 (⌊log₂ (max n_max 2)⌋ / 2)` for the tabulated odd stride. -/ +def charBits (n_max : ℕ) : ℕ := + TabulatedOddStride.charBits n_max + +/-- Number of base-`2 ^ b` characters `d = max 1 ⌈⌈log₂ (max U 2)⌉ / b⌉` used by the tabulated odd +stride. When `U = poly(n_max)`, `numChars U n_max = O(1)`. -/ +def numChars (U n_max : ℕ) : ℕ := + max 1 ((Nat.clog 2 (max U 2) + charBits n_max - 1) / charBits n_max) + +/-- Memory footprint `4 + d * 2 ^ b` words of the `TabulatedOddStride` table at `[base, ...)`. -/ +def tabSpace (U n_max : ℕ) : ℕ := + TabulatedOddStride.space n_max (numChars U n_max) + +/-- Offset `tabSpace U n_max + 1` of the open-addressed slot arrays relative to `base` (after the +`TabulatedOddStride` table at `[base, base + tabSpace U n_max)` and the 1-word multiply-shift seed +at `base + tabSpace U n_max`). -/ +def slotOffset (U n_max : ℕ) : ℕ := + tabSpace U n_max + 1 + +/-- Total memory footprint `slotOffset U n_max + 3 * numBuckets n_max` words in +`[base, base + space U n_max)`. -/ +def space (U n_max : ℕ) : ℕ := + slotOffset U n_max + 3 * numBuckets n_max + +theorem two_le_bucketBits (n_max : ℕ) : 2 ≤ bucketBits n_max := by + dsimp only [bucketBits] + calc + 2 = Nat.clog 2 (2 ^ 2) := (Nat.clog_pow 2 2 (by decide)).symm + _ ≤ Nat.clog 2 (max (4 * n_max) 4) := Nat.clog_mono_right 2 (by omega) + +theorem bucketBits_pos (n_max : ℕ) : 0 < bucketBits n_max := + lt_of_lt_of_le (by decide : 0 < 2) (two_le_bucketBits n_max) + +theorem max_four_mul_le_numBuckets (n_max : ℕ) : + max (4 * n_max) 4 ≤ numBuckets n_max := + Nat.le_pow_clog (by decide) (max (4 * n_max) 4) + +theorem four_le_numBuckets (n_max : ℕ) : 4 ≤ numBuckets n_max := + le_trans (Nat.le_max_right (4 * n_max) 4) (max_four_mul_le_numBuckets n_max) + +theorem four_mul_le_numBuckets (n_max : ℕ) : 4 * n_max ≤ numBuckets n_max := + le_trans (Nat.le_max_left (4 * n_max) 4) (max_four_mul_le_numBuckets n_max) + +theorem two_mul_le_numBuckets (n_max : ℕ) : 2 * n_max ≤ numBuckets n_max := by + have := four_mul_le_numBuckets n_max + omega + +theorem lt_numBuckets (n_max : ℕ) : n_max < numBuckets n_max := by + have h4 := four_le_numBuckets n_max + have h4n := four_mul_le_numBuckets n_max + omega + +theorem numBuckets_pos (n_max : ℕ) : 0 < numBuckets n_max := + lt_of_lt_of_le (by decide : 0 < 4) (four_le_numBuckets n_max) + +instance instNonemptyFinNumBuckets (n_max : ℕ) : Nonempty (Fin (numBuckets n_max)) := + ⟨⟨0, numBuckets_pos n_max⟩⟩ + +theorem bucketBits_le_hashBits (U n_max : ℕ) : bucketBits n_max ≤ hashBits U n_max := + Nat.le_max_left _ _ + +theorem hashBits_pos (U n_max : ℕ) : 0 < hashBits U n_max := + lt_of_lt_of_le (bucketBits_pos n_max) (bucketBits_le_hashBits U n_max) + +theorem one_le_numChars (U n_max : ℕ) : 1 ≤ numChars U n_max := + Nat.le_max_left 1 _ + +theorem lt_two_pow_hashBits {U : ℕ} (n_max : ℕ) (u : Fin U) : + u.val < 2 ^ hashBits U n_max := by + have h1 : u.val < max U 2 := lt_of_lt_of_le u.isLt (Nat.le_max_left U 2) + have h2 : max U 2 ≤ 2 ^ Nat.clog 2 (max U 2) := Nat.le_pow_clog (by decide) (max U 2) + have h3 : 2 ^ Nat.clog 2 (max U 2) ≤ 2 ^ hashBits U n_max := + Nat.pow_le_pow_right (by decide) (Nat.le_max_right _ _) + exact lt_of_lt_of_le h1 (le_trans h2 h3) + +/-- Embed key `u : Fin U` into `Fin (2 ^ hashBits U n_max)`. -/ +def encodeKey {U : ℕ} (n_max : ℕ) (u : Fin U) : Fin (2 ^ hashBits U n_max) := + ⟨u.val, lt_two_pow_hashBits n_max u⟩ + +@[simp] +theorem val_encodeKey {U : ℕ} (n_max : ℕ) (u : Fin U) : + (encodeKey n_max u).val = u.val := rfl + +theorem encodeKey_injective {U : ℕ} (n_max : ℕ) : + Function.Injective (encodeKey (U := U) n_max) := by + intro u₁ u₂ h + exact Fin.ext (congrArg (fun x : Fin (2 ^ hashBits U n_max) => x.val) h) + +/-- Combined random seed `(b_ms, T)` consisting of the multiply-shift seed +`b_ms : Fin (2 ^ (hashBits U n_max - 1))` and the character table +`T : Fin (numChars U n_max * 2 ^ charBits n_max) → Fin (numBuckets n_max)`. -/ +abbrev Seed (U n_max : ℕ) : Type := + Fin (2 ^ (hashBits U n_max - 1)) × + (Fin (numChars U n_max * 2 ^ charBits n_max) → Fin (numBuckets n_max)) + +instance instNonemptySeed (U n_max : ℕ) : Nonempty (Seed U n_max) := + ⟨(⟨0, Nat.two_pow_pos _⟩, fun _ => ⟨0, numBuckets_pos n_max⟩)⟩ + +/-- Uniform reference distribution over `Seed U n_max`. -/ +noncomputable def seedLaw (U n_max : ℕ) : PMF (Seed U n_max) := + PMF.uniformOfFintype (Seed U n_max) + +theorem multiplyShift_lt_numBuckets {U n_max : ℕ} + (b : Fin (2 ^ (hashBits U n_max - 1))) (x : Fin (2 ^ hashBits U n_max)) : + multiplyShift (hashBits U n_max) (bucketBits n_max) b x < numBuckets n_max := by + dsimp only [multiplyShift, numBuckets] + refine Nat.div_lt_of_lt_mul ?_ + rw [← pow_add, Nat.sub_add_cancel (bucketBits_le_hashBits U n_max)] + exact Nat.mod_lt _ (Nat.two_pow_pos _) + +/-- Multiply-shift primary slot `h₁(u) ∈ Fin (numBuckets n_max)`. -/ +def primaryHash {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) : Fin (numBuckets n_max) := + ⟨multiplyShift (hashBits U n_max) (bucketBits n_max) θ.1 (encodeKey n_max u), + multiplyShift_lt_numBuckets θ.1 (encodeKey n_max u)⟩ + +@[simp] +theorem val_primaryHash {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) : + (primaryHash θ u).val = + multiplyShift (hashBits U n_max) (bucketBits n_max) θ.1 (encodeKey n_max u) := + rfl + +/-- Raw simple-tabulation character XOR sum `simpleTab b d M θ.2 u.val : Fin (numBuckets n_max)`. -/ +def rawTabHash {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) : Fin (numBuckets n_max) := + TabulatedOddStride.simpleTab (charBits n_max) (numChars U n_max) (bucketBits n_max) θ.2 u.val + +/-- Tabulated odd stride `h₂(u) = (2 * rawTabHash θ u + 1) mod 2 ^ M ∈ Fin (numBuckets n_max)`. -/ +def strideHash {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) : Fin (numBuckets n_max) := + TabulatedOddStride.tabulatedOddStride (charBits n_max) (numChars U n_max) (bucketBits n_max) + θ.2 u.val + +/-- Full-cycle double-hashing probe permutation +`probePerm θ u : Equiv.Perm (Fin (numBuckets n_max))` sending probe index `i` to +`(h₁(u) + i * h₂(u)) mod 2 ^ M`, certified bijective by `TabulatedOddStride.bijective_probeSeq`. -/ +noncomputable def probePerm {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) : + Equiv.Perm (Fin (numBuckets n_max)) := + Equiv.ofBijective + (TabulatedOddStride.probeSeq (bucketBits n_max) (primaryHash θ u).val (rawTabHash θ u).val) + (TabulatedOddStride.bijective_probeSeq (bucketBits n_max) + (primaryHash θ u).val (rawTabHash θ u).val) + +theorem probePerm_apply {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) + (i : Fin (numBuckets n_max)) : + probePerm θ u i = + TabulatedOddStride.probeSeq (bucketBits n_max) (primaryHash θ u).val (rawTabHash θ u).val i := + rfl + +@[simp] +theorem val_probePerm {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) + (i : Fin (numBuckets n_max)) : + (probePerm θ u i).val = + ((primaryHash θ u).val + i.val * (strideHash θ u).val) % numBuckets n_max := + rfl + +@[simp] +theorem probePerm_zero {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) : + probePerm θ u ⟨0, numBuckets_pos n_max⟩ = primaryHash θ u := by + apply Fin.ext + rw [val_probePerm] + simpa using Nat.mod_eq_of_lt (primaryHash θ u).isLt + +/-- One step of the Word-RAM probe loop `(slot + stride) &&& (2 ^ M - 1)` advances from +`probePerm θ u ⟨i, _⟩` to `probePerm θ u ⟨i + 1, _⟩`. -/ +theorem val_probePerm_succ {U n_max : ℕ} (θ : Seed U n_max) (u : Fin U) + {i : ℕ} (hi : i + 1 < numBuckets n_max) : + (probePerm θ u ⟨i + 1, hi⟩).val = + ((probePerm θ u ⟨i, by omega⟩).val + (strideHash θ u).val) &&& + (numBuckets n_max - 1) := by + change (probePerm θ u ⟨i + 1, hi⟩).val = + ((probePerm θ u ⟨i, by omega⟩).val + (strideHash θ u).val) &&& + (2 ^ bucketBits n_max - 1) + rw [Nat.and_two_pow_sub_one_eq_mod, val_probePerm, val_probePerm] + simp [numBuckets, Nat.add_mul, one_mul, Nat.add_assoc] + +/-! ## Section 2: Concrete Slot Extraction and Representation Invariant -/ + +/-- Well-formedness of the flat open-addressed slot arrays at +`[base + slotOffset U n_max, base + slotOffset U n_max + 3 * numBuckets n_max)`: +each `used[j]` word is `0` or `1`, and whenever `used[j] = 1`, `key[j] < U`. -/ +def SlotArrayWF (base U n_max : ℕ) (mem : ℕ → ℕ) : Prop := + let sBase := base + slotOffset U n_max + let m := numBuckets n_max + ∀ j : Fin m, + (mem (sBase + j.val) = 0 ∨ mem (sBase + j.val) = 1) ∧ + (mem (sBase + j.val) = 1 → mem (sBase + m + j.val) < U) + +/-- Extract the abstract open-addressed table `UniformProbing.Table U (numBuckets n_max)` from +Word-RAM memory `mem`. -/ +def extractSlots (base U n_max : ℕ) (mem : ℕ → ℕ) : + UniformProbing.Table U (numBuckets n_max) := + let sBase := base + slotOffset U n_max + let m := numBuckets n_max + fun j => + if mem (sBase + j.val) = 0 then none + else if h : mem (sBase + m + j.val) < U then + some (⟨mem (sBase + m + j.val), h⟩, mem (sBase + 2 * m + j.val)) + else none + +theorem extractSlots_eq_none_iff {base U n_max : ℕ} {mem : ℕ → ℕ} + (hwf : SlotArrayWF base U n_max mem) (j : Fin (numBuckets n_max)) : + extractSlots base U n_max mem j = none ↔ + mem (base + slotOffset U n_max + j.val) = 0 := by + rcases hwf j with ⟨h0 | h1, hkey⟩ + · simp [extractSlots, h0] + · have hlt : mem (base + slotOffset U n_max + numBuckets n_max + j.val) < U := hkey h1 + simp [extractSlots, h1, hlt] + +theorem extractSlots_eq_some_iff {base U n_max : ℕ} {mem : ℕ → ℕ} + (hwf : SlotArrayWF base U n_max mem) (j : Fin (numBuckets n_max)) + (u : Fin U) (v : ℕ) : + extractSlots base U n_max mem j = some (u, v) ↔ + mem (base + slotOffset U n_max + j.val) = 1 ∧ + mem (base + slotOffset U n_max + numBuckets n_max + j.val) = u.val ∧ + mem (base + slotOffset U n_max + 2 * numBuckets n_max + j.val) = v := by + rcases hwf j with ⟨h0 | h1, hkey⟩ + · simp [extractSlots, h0] + · have hlt : mem (base + slotOffset U n_max + numBuckets n_max + j.val) < U := hkey h1 + simp [extractSlots, h1, hlt, Fin.ext_iff] + +theorem mem_otherOccupiedSlots_extractSlots_iff {base U n_max : ℕ} {mem : ℕ → ℕ} + (hwf : SlotArrayWF base U n_max mem) (u : Fin U) (j : Fin (numBuckets n_max)) : + j ∈ otherOccupiedSlots (extractSlots base U n_max mem) u ↔ + mem (base + slotOffset U n_max + j.val) = 1 ∧ + mem (base + slotOffset U n_max + numBuckets n_max + j.val) ≠ u.val := by + rw [mem_otherOccupiedSlots_iff] + constructor + · rintro ⟨u', v', hsome, hne⟩ + rw [extractSlots_eq_some_iff hwf] at hsome + refine ⟨hsome.1, ?_⟩ + rw [hsome.2.1] + exact fun h => hne (Fin.ext h) + · rintro ⟨h1, hne⟩ + have hlt : mem (base + slotOffset U n_max + numBuckets n_max + j.val) < U := + (hwf j).2 h1 + refine ⟨⟨mem (base + slotOffset U n_max + numBuckets n_max + j.val), hlt⟩, + mem (base + slotOffset U n_max + 2 * numBuckets n_max + j.val), ?_, ?_⟩ + · rw [extractSlots_eq_some_iff hwf] + exact ⟨h1, rfl, rfl⟩ + · intro heq + exact hne (congrArg Fin.val heq) + +/-- Coupled representation invariant `Rep w base U V n_max θ s x` on `Data 12`: +* `x.Bounded w`; +* `x.regs 0 = base`; +* `TabulatedOddStride.Rep` holds for `θ.2` at `[base, base + tabSpace U n_max)`; +* `x.mem (base + tabSpace U n_max) = θ.1.val` holds the multiply-shift seed; +* `SlotArrayWF base U n_max x.mem` and `UniformProbing.TableRep (probePerm θ) s (extractSlots ...)` + hold on the slot arrays; +* `(Dictionary.dom s).card ≤ n_max` and `Dictionary.ValuesBounded V s`. -/ +structure Rep (w base U V n_max : ℕ) (θ : Seed U n_max) (s : Dictionary.State U) + (x : Data 12) : Prop where + bounded : x.Bounded w + regs_base : x.regs 0 = base + tab_rep : TabulatedOddStride.Rep base (charBits n_max) (numChars U n_max) + (bucketBits n_max) θ.2 x.mem + ms_seed : x.mem (base + tabSpace U n_max) = θ.1.val + slot_wf : SlotArrayWF base U n_max x.mem + table_rep : UniformProbing.TableRep (probePerm θ) s (extractSlots base U n_max x.mem) + dom_le : (Dictionary.dom s).card ≤ n_max + vals_bounded : Dictionary.ValuesBounded V s + +theorem Rep.unique {w base U V n_max : ℕ} {θ : Seed U n_max} + {s₁ s₂ : Dictionary.State U} {x : Data 12} + (h₁ : Rep w base U V n_max θ s₁ x) (h₂ : Rep w base U V n_max θ s₂ x) : + s₁ = s₂ := by + funext u + cases hs₁ : s₁ u with + | none => + cases hs₂ : s₂ u with + | none => rfl + | some v₂ => + obtain ⟨hlt, hT⟩ := h₂.table_rep.key_spec u v₂ hs₂ + have hs₁' := (h₁.table_rep.slot_spec _ u v₂ hT).1 + rw [hs₁] at hs₁' + cases hs₁' + | some v₁ => + obtain ⟨hlt, hT⟩ := h₁.table_rep.key_spec u v₁ hs₁ + exact (h₂.table_rep.slot_spec _ u v₁ hT).1.symm + +theorem mem_footprint_of_lt_space {base U n_max a : ℕ} + (hlo : base ≤ a) (hhi : a < base + space U n_max) : + a ∈ (Block.mk base (space U n_max)).footprint := by + rw [Block.mem_footprint] + exact ⟨hlo, hhi⟩ + +theorem SlotArrayWF.congr_mem {base U n_max : ℕ} {mem mem' : ℕ → ℕ} + (hmem : ∀ a ∈ (Block.mk base (space U n_max)).footprint, mem' a = mem a) + (hwf : SlotArrayWF base U n_max mem) : + SlotArrayWF base U n_max mem' := by + intro j + have hj := j.isLt + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have h0 : mem' (base + slotOffset U n_max + j.val) = + mem (base + slotOffset U n_max + j.val) := + hmem _ (mem_footprint_of_lt_space (by omega) (by omega)) + have h1 : mem' (base + slotOffset U n_max + numBuckets n_max + j.val) = + mem (base + slotOffset U n_max + numBuckets n_max + j.val) := + hmem _ (mem_footprint_of_lt_space (by omega) (by omega)) + dsimp only [SlotArrayWF] at hwf ⊢ + rw [h0, h1] + exact hwf j + +theorem extractSlots_congr_mem {base U n_max : ℕ} {mem mem' : ℕ → ℕ} + (hmem : ∀ a ∈ (Block.mk base (space U n_max)).footprint, mem' a = mem a) : + extractSlots base U n_max mem' = extractSlots base U n_max mem := by + funext j + have hj := j.isLt + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have h0 : mem' (base + slotOffset U n_max + j.val) = + mem (base + slotOffset U n_max + j.val) := + hmem _ (mem_footprint_of_lt_space (by omega) (by omega)) + have h1 : mem' (base + slotOffset U n_max + numBuckets n_max + j.val) = + mem (base + slotOffset U n_max + numBuckets n_max + j.val) := + hmem _ (mem_footprint_of_lt_space (by omega) (by omega)) + have h2 : mem' (base + slotOffset U n_max + 2 * numBuckets n_max + j.val) = + mem (base + slotOffset U n_max + 2 * numBuckets n_max + j.val) := + hmem _ (mem_footprint_of_lt_space (by omega) (by omega)) + dsimp only [extractSlots] + rw [h0, h1, h2] + +theorem Rep.congr {w base U V n_max : ℕ} {θ : Seed U n_max} {s : Dictionary.State U} + {x x' : Data 12} + (hbdd : x'.Bounded w) (hbase : x'.regs 0 = x.regs 0) + (hmem : ∀ a ∈ (Block.mk base (space U n_max)).footprint, x'.mem a = x.mem a) + (hrep : Rep w base U V n_max θ s x) : + Rep w base U V n_max θ s x' := by + have hsp : space U n_max = + (4 + numChars U n_max * 2 ^ charBits n_max) + 1 + 3 * numBuckets n_max := rfl + have htab_eq : tabSpace U n_max = 4 + numChars U n_max * 2 ^ charBits n_max := rfl + have htab : TabulatedOddStride.Rep base (charBits n_max) (numChars U n_max) + (bucketBits n_max) θ.2 x'.mem := by + rcases hrep.tab_rep with ⟨hm0, hm1, hm2, hm3, hmt⟩ + refine ⟨?_, ?_, ?_, ?_, fun i => ?_⟩ + · rw [hmem base (mem_footprint_of_lt_space (by omega) (by omega)), hm0] + · rw [hmem (base + 1) (mem_footprint_of_lt_space (by omega) (by omega)), hm1] + · rw [hmem (base + 2) (mem_footprint_of_lt_space (by omega) (by omega)), hm2] + · rw [hmem (base + 3) (mem_footprint_of_lt_space (by omega) (by omega)), hm3] + · have hi := i.isLt + rw [hmem (base + 4 + i.val) (mem_footprint_of_lt_space (by omega) (by omega)), hmt i] + have hms : x'.mem (base + tabSpace U n_max) = θ.1.val := by + rw [hmem (base + tabSpace U n_max) (mem_footprint_of_lt_space (by omega) (by omega)), + hrep.ms_seed] + refine ⟨hbdd, hbase.trans hrep.regs_base, htab, hms, + hrep.slot_wf.congr_mem hmem, ?_, hrep.dom_le, hrep.vals_bounded⟩ + rw [extractSlots_congr_mem hmem] + exact hrep.table_rep + +theorem slotArrayWF_of_zero {base U n_max : ℕ} {mem : ℕ → ℕ} + (hz : ∀ j : Fin (numBuckets n_max), mem (base + slotOffset U n_max + j.val) = 0) : + SlotArrayWF base U n_max mem := by + intro j + refine ⟨Or.inl (hz j), ?_⟩ + rw [hz j] + intro h + omega + +theorem extractSlots_eq_emptyTable_of_zero {base U n_max : ℕ} {mem : ℕ → ℕ} + (hz : ∀ j : Fin (numBuckets n_max), mem (base + slotOffset U n_max + j.val) = 0) : + extractSlots base U n_max mem = emptyTable U (numBuckets n_max) := by + funext j + simp [extractSlots, emptyTable, hz j] + +/-! ## Section 3: Register Layout, Allocation, and Randomized Initialization -/ + +/-- Register `0` (`baseReg`, persistent): holds the base memory address `base`. -/ +abbrev regBase : Fin 12 := 0 + +/-- Register `1` (`curReg`): loop condition / multiply-shift seed / tabulation key scratch. -/ +abbrev regCond : Fin 12 := 1 + +/-- Register `2` (`zeroReg`): memory address / multiply-shift key / tabulation output scratch. -/ +abbrev regAddr : Fin 12 := 2 + +/-- Register `3` (`oneReg`): current probe slot `slot ∈ Fin (numBuckets n_max)`. -/ +abbrev regSlot : Fin 12 := 3 + +/-- Register `4`: slot array base address `base + slotOffset U n_max`. -/ +abbrev regSlotBase : Fin 12 := 4 + +/-- Register `5`: bucket mask `numBuckets n_max - 1`. -/ +abbrev regMaskM : Fin 12 := 5 + +/-- Register `6`: odd probe stride `strideHash θ u`. -/ +abbrev regStride : Fin 12 := 6 + +/-- Register `7`: table size `numBuckets n_max`. -/ +abbrev regM : Fin 12 := 7 + +/-- Register `8`: constant `0` during probing / general scratch. -/ +abbrev regZero : Fin 12 := 8 + +/-- Register `9` (`output`): operation tag (`0` for `get`, `1` for `insert`) on input; decoded +return value (`0` for `none`/`()`, `v + 1` for `some v`) on output. -/ +abbrev regOp : Fin 12 := 9 + +/-- Register `10`: input key `u.val < U`. -/ +abbrev regU : Fin 12 := 10 + +/-- Register `11` (`frontierReg`): input value `v < V` for `insert` / allocation frontier. -/ +abbrev regV : Fin 12 := 11 + +/-- Register injection `Fin 5 → Fin 12` embedding `MultiplyShift` into registers `1..5` +(preserving `regBase = 0` and `6..11`). -/ +def msMap : Fin 5 → Fin 12 := + fun i => ⟨i.val + 1, by omega⟩ + +theorem injective_msMap : Function.Injective msMap := by + intro i j h + ext + simpa [msMap] using congrArg Fin.val h + +theorem notMem_range_msMap {j : Fin 12} (hj : j.val = 0 ∨ 6 ≤ j.val) : + j ∉ Set.range msMap := by + rintro ⟨i, rfl⟩ + have := i.isLt + simp only [msMap] at hj + omega + +/-- Register injection `Fin 9 → Fin 12` embedding `TabulatedOddStride` into registers `0..8` +(with `tabMap 0 = regBase` and preserving `regOp = 9`, `regU = 10`, `regV = 11`). -/ +def tabMap : Fin 9 → Fin 12 := + fun i => ⟨i.val, by omega⟩ + +theorem injective_tabMap : Function.Injective tabMap := by + intro i j h + ext + simpa [tabMap] using congrArg Fin.val h + +theorem notMem_range_tabMap {j : Fin 12} (hj : 9 ≤ j.val) : + j ∉ Set.range tabMap := by + rintro ⟨i, rfl⟩ + have := i.isLt + simp only [tabMap] at hj + omega + +/-- Allocation program `allocSpan U n_max` (3 instructions): copies `frontierReg` (`regV`) to +`baseReg` (`regBase`) and advances `frontierReg` by `space U n_max`. -/ +def allocSpan (U n_max : ℕ) : Code .exact 12 := + .seq (.op (.copy regBase regV)) + (.seq (.op (.const regCond (space U n_max))) + (.op (.exactBin .add regV regBase regCond))) + +theorem allocSpan_execWithin {w base U n_max : ℕ} {x : Data 12} + (hS : Fits w (base + space U n_max)) (hfr : x.regs regV = base) : + ∃ x₁ : Data 12, + ExecWithin w (allocSpan U n_max) x x₁ 3 ∧ + x₁.mem = x.mem ∧ + x₁.regs regBase = base ∧ + x₁.regs regV = base + space U n_max := by + have hfit_sp : Fits w (space U n_max) := hS.of_le (by omega) + set s1 := x.set regBase base + have e1 : Exec w (.op (.copy regBase regV) : Code .exact 12) x s1 1 := by + simpa [s1, hfr] using (Exec.copy (B := .exact) (w := w) (d := regBase) (s := regV) (x := x)) + set s2 := s1.set regCond (space U n_max) + have e2 : Exec w (.op (.const regCond (space U n_max)) : Code .exact 12) s1 s2 1 := + Exec.const hfit_sp + set s3 := s2.set regV (base + space U n_max) + have e3 : Exec w (.op (.exactBin .add regV regBase regCond) : Code .exact 12) s2 s3 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s2, s1]) hS + exact ⟨s3, (e1.seq (e2.seq e3)).within, rfl, + by simp +decide [s3, s2, s1], by simp +decide [s3]⟩ + +/-- Deterministic 5-instruction glue between `MultiplyShift.initCode` and +`TabulatedOddStride.initFromBCode`: stores the multiply-shift seed `regCond` at +`base + tabSpace U n_max` and loads `charBits n_max` into `regMaskM` and `bucketBits n_max` into +`regAddr`. -/ +def initMidCode (U n_max : ℕ) : Code .exact 12 := + .seq (.op (.const regAddr (tabSpace U n_max))) + (.seq (.op (.exactBin .add regAddr regBase regAddr)) + (.seq (.op (.store regAddr regCond)) + (.seq (.op (.const regMaskM (charBits n_max))) + (.op (.const regAddr (bucketBits n_max)))))) + +/-- Closed-form machine state after `initMidCode U n_max`. -/ +def initMidData (base U n_max : ℕ) (d : Data 12) : Data 12 := + ((((d.set regAddr (tabSpace U n_max)).set regAddr (base + tabSpace U n_max)).write + (base + tabSpace U n_max) (d.regs regCond)).set regMaskM (charBits n_max)).set + regAddr (bucketBits n_max) + +theorem fits_of_space {w base U n_max : ℕ} (hS : Fits w (base + space U n_max)) : + Fits w (base + tabSpace U n_max) ∧ + Fits w (2 ^ charBits n_max) ∧ + Fits w (2 ^ bucketBits n_max) ∧ + Fits w (numChars U n_max) ∧ + Fits w (charBits n_max) ∧ + Fits w (bucketBits n_max) := by + have hd : 1 ≤ numChars U n_max := one_le_numChars U n_max + have hpowB_pos : 1 ≤ 2 ^ charBits n_max := Nat.one_le_two_pow + have hpowB_le : 2 ^ charBits n_max ≤ numChars U n_max * 2 ^ charBits n_max := + Nat.le_mul_of_pos_left _ hd + have hd_le : numChars U n_max ≤ numChars U n_max * 2 ^ charBits n_max := + Nat.le_mul_of_pos_right _ hpowB_pos + have htab : tabSpace U n_max = 4 + numChars U n_max * 2 ^ charBits n_max := rfl + have hsp : space U n_max = tabSpace U n_max + 1 + 3 * numBuckets n_max := rfl + have hcb_lt : charBits n_max < 2 ^ charBits n_max := Nat.lt_two_pow_self + have hbb_lt : bucketBits n_max < 2 ^ bucketBits n_max := Nat.lt_two_pow_self + have hM : 2 ^ bucketBits n_max = numBuckets n_max := rfl + refine ⟨hS.of_le (by omega), hS.of_le (by omega), hS.of_le (by omega), + hS.of_le (by omega), hS.of_le (by omega), hS.of_le (by omega)⟩ + +theorem initMidCode_exec {w base U n_max : ℕ} {d : Data 12} + (hbase : d.regs regBase = base) (hS : Fits w (base + space U n_max)) : + Exec w (initMidCode U n_max) d (initMidData base U n_max d) 5 := by + obtain ⟨hfit_btab, _, _, _, hfit_cb, hfit_bb⟩ := fits_of_space hS + have hfit_tab : Fits w (tabSpace U n_max) := hfit_btab.of_le (by omega) + set s1 := d.set regAddr (tabSpace U n_max) + have e1 : Exec w (.op (.const regAddr (tabSpace U n_max)) : Code .exact 12) d s1 1 := + Exec.const hfit_tab + set s2 := s1.set regAddr (base + tabSpace U n_max) + have e2 : Exec w (.op (.exactBin .add regAddr regBase regAddr) : Code .exact 12) s1 s2 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s1, hbase]) hfit_btab + set s3 := s2.write (base + tabSpace U n_max) (d.regs regCond) + have e3 : Exec w (.op (.store regAddr regCond) : Code .exact 12) s2 s3 1 := by + simpa +decide [s3, s2, s1] using + (Exec.store (B := .exact) (w := w) (a := regAddr) (s := regCond) (x := s2)) + set s4 := s3.set regMaskM (charBits n_max) + have e4 : Exec w (.op (.const regMaskM (charBits n_max)) : Code .exact 12) s3 s4 1 := + Exec.const hfit_cb + set s5 := s4.set regAddr (bucketBits n_max) + have e5 : Exec w (.op (.const regAddr (bucketBits n_max)) : Code .exact 12) s4 s5 1 := + Exec.const hfit_bb + exact e1.seq (e2.seq (e3.seq (e4.seq e5))) + +/-- Complete randomized setup program `initSetup U n_max`: samples the multiply-shift seed, +stores it at `base + tabSpace U n_max`, and initializes the `TabulatedOddStride` table at +`[base, base + tabSpace U n_max)`. -/ +def initSetup (U n_max : ℕ) : Code .exact 12 := + .seq ((MultiplyShift.initCode (hashBits U n_max)).rename msMap) + (.seq (initMidCode U n_max) + ((TabulatedOddStride.initFromBCode (numChars U n_max)).rename tabMap)) + +/-- Total instruction count `5 * (numChars U n_max * 2 ^ charBits n_max) + 25` of `initSetup`. -/ +def setupSteps (U n_max : ℕ) : ℕ := + 5 * (numChars U n_max * 2 ^ charBits n_max) + 25 + +/-- Extract the multiply-shift seed from `mem (base + tabSpace U n_max)`. -/ +def extractMSSeedFromMem (base U n_max : ℕ) (mem : ℕ → ℕ) : Fin (2 ^ (hashBits U n_max - 1)) := + if h : mem (base + tabSpace U n_max) < 2 ^ (hashBits U n_max - 1) then + ⟨mem (base + tabSpace U n_max), h⟩ + else + ⟨0, Nat.two_pow_pos _⟩ + +/-- Extract the combined seed `Seed U n_max` from machine state `y`. -/ +def extractSeedPair (base U n_max : ℕ) (y : Data 12) : Seed U n_max := + (extractMSSeedFromMem base U n_max y.mem, + TabulatedOddStride.extractTable base (charBits n_max) (numChars U n_max) (bucketBits n_max) + y.mem) + +/-- Stage-1 state distribution produced by +`(MultiplyShift.initCode (hashBits U n_max)).rename msMap`. -/ +noncomputable def msInitPMF (U n_max : ℕ) (x₁ : Data 12) : PMF (Data 12) := + (Prod.fst <$> + ((fun z => (z.1.extend msMap x₁, z.2)) <$> + MultiplyShift.initComp (hashBits U n_max) (x₁.comap msMap))).eval + +/-- Stage-3 state distribution produced by +`(TabulatedOddStride.initFromBCode (numChars U n_max)).rename tabMap`. -/ +noncomputable def tabInitPMF (base U n_max : ℕ) (y₂ : Data 12) : PMF (Data 12) := + (Prod.fst <$> + ((fun z => (z.1.extend tabMap y₂, z.2)) <$> + TabulatedOddStride.initFromBComp base (charBits n_max) (numChars U n_max) (bucketBits n_max) + (y₂.comap tabMap))).eval + +/-- Output state distribution of `initSetup U n_max`. -/ +noncomputable def initSetupPMF (base U n_max : ℕ) (x₁ : Data 12) : PMF (Data 12) := + (msInitPMF U n_max x₁).bind (fun y₁ => tabInitPMF base U n_max (initMidData base U n_max y₁)) + +/-- Coupled `(Seed × Data 12)` distribution produced by `initSetup U n_max`. -/ +noncomputable def initSeedStatePMF (base U n_max : ℕ) (x₁ : Data 12) : + PMF (Seed U n_max × Data 12) := + (initSetupPMF base U n_max x₁).map (fun y => (extractSeedPair base U n_max y, y)) + +theorem mem_support_msInitPMF {U n_max : ℕ} {x₁ y₁ : Data 12} + (hy₁ : y₁ ∈ (msInitPMF U n_max x₁).support) : + y₁.regs regBase = x₁.regs regBase ∧ + y₁.regs regV = x₁.regs regV ∧ + y₁.regs regCond < 2 ^ (hashBits U n_max - 1) ∧ + (y₁.comap msMap).regs MultiplyShift.regB = y₁.regs regCond ∧ + y₁.mem = x₁.mem := by + dsimp only [msInitPMF] at hy₁ + rw [Functor.map_map, ProbComp.eval_map, PMF.mem_support_map_iff] at hy₁ + obtain ⟨z, hz, rfl⟩ := hy₁ + rw [ProbComp.support_eval] at hz + dsimp only [MultiplyShift.initComp] at hz + obtain ⟨k, -, hk⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + cases OracleComp.eq_of_mem_supportWith_pure hk + have hlt : k.val < 2 ^ (hashBits U n_max - 1) := by + have hpos : 1 ≤ 2 ^ (hashBits U n_max - 1) := Nat.one_le_two_pow + have hk_lt := k.isLt + omega + refine ⟨Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide)), + Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide)), ?_, rfl, rfl⟩ + change ((MultiplyShift.initData (hashBits U n_max) (x₁.comap msMap) k.val).extend msMap + x₁ |>.comap msMap).regs MultiplyShift.regB < 2 ^ (hashBits U n_max - 1) + rw [Data.comap_extend injective_msMap] + simpa [MultiplyShift.initData] using hlt + +theorem map_extractSeed_msInitPMF (U n_max : ℕ) (x₁ : Data 12) : + (msInitPMF U n_max x₁).map + (fun y₁ => MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap)) = + PMF.uniformOfFintype (Fin (2 ^ (hashBits U n_max - 1))) := by + dsimp only [msInitPMF] + rw [Functor.map_map, ProbComp.eval_map, PMF.map_comp] + have hcomp : + ((fun y₁ : Data 12 => MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap)) ∘ + fun x : Data 5 × ℕ => x.1.extend msMap x₁) = + fun x : Data 5 × ℕ => MultiplyShift.extractSeed (hashBits U n_max) x.1 := by + funext z + simp [Data.comap_extend injective_msMap] + rw [hcomp] + exact MultiplyShift.map_extractSeed_eval_initComp (hashBits U n_max) (x₁.comap msMap) + +theorem mem_support_tabInitPMF {base U n_max : ℕ} {y₂ y : Data 12} + (hbase : y₂.regs regBase = base) + (hy : y ∈ (tabInitPMF base U n_max y₂).support) : + TabulatedOddStride.Rep base (charBits n_max) (numChars U n_max) (bucketBits n_max) + (TabulatedOddStride.extractTable base (charBits n_max) (numChars U n_max) (bucketBits n_max) + y.mem) y.mem ∧ + y.regs regBase = base ∧ + y.regs regV = y₂.regs regV ∧ + (∀ a, (a < base ∨ base + tabSpace U n_max ≤ a) → y.mem a = y₂.mem a) := by + dsimp only [tabInitPMF] at hy + rw [Functor.map_map, ProbComp.eval_map, PMF.mem_support_map_iff] at hy + obtain ⟨z, hz, rfl⟩ := hy + rw [ProbComp.support_eval] at hz + have hbase_comap : (y₂.comap tabMap).regs TabulatedOddStride.regBase = base := hbase + obtain ⟨-, hrep, hbase', hframe⟩ := + TabulatedOddStride.mem_support_initFromBComp hbase_comap hz + have hbase_ext : (z.1.extend tabMap y₂).regs regBase = base := by + change ((z.1.extend tabMap y₂).comap tabMap).regs TabulatedOddStride.regBase = base + rw [Data.comap_extend injective_tabMap] + exact hbase' + have hv_ext : (z.1.extend tabMap y₂).regs regV = y₂.regs regV := + Data.extend_regs_of_notMem _ _ (notMem_range_tabMap (by decide)) + exact ⟨hrep, hbase_ext, hv_ext, hframe⟩ + +theorem map_extractSeedPair_tabInitPMF {base U n_max : ℕ} {y₁ : Data 12} + (hbase : y₁.regs regBase = base) + (hcond : y₁.regs regCond < 2 ^ (hashBits U n_max - 1)) : + (tabInitPMF base U n_max (initMidData base U n_max y₁)).map (extractSeedPair base U n_max) = + (PMF.uniformOfFintype + (Fin (numChars U n_max * 2 ^ charBits n_max) → Fin (numBuckets n_max))).map + (fun T => (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), T)) := by + set y₂ := initMidData base U n_max y₁ + have hy₂_base : y₂.regs regBase = base := by + simpa [y₂, initMidData, regBase, regAddr, regMaskM] using hbase + have hcongr : + (tabInitPMF base U n_max y₂).map (extractSeedPair base U n_max) = + (tabInitPMF base U n_max y₂).map + (fun y => + (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), + TabulatedOddStride.extractTable base (charBits n_max) (numChars U n_max) + (bucketBits n_max) y.mem)) := by + apply PMF.map_congr_on_support + intro y hy + obtain ⟨-, -, -, hframe⟩ := mem_support_tabInitPMF hy₂_base hy + have hms_mem : y.mem (base + tabSpace U n_max) = y₁.regs regCond := by + rw [hframe (base + tabSpace U n_max) (Or.inr le_rfl)] + simp [y₂, initMidData, Data.write] + have hms_eq : extractMSSeedFromMem base U n_max y.mem = + MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap) := by + have hregB : y₁.regs (msMap MultiplyShift.regB) = y₁.regs regCond := rfl + apply Fin.ext + dsimp only [extractMSSeedFromMem, MultiplyShift.extractSeed, Data.comap] + simp [hms_mem, hregB, hcond] + exact Prod.ext hms_eq rfl + rw [hcongr] + dsimp only [tabInitPMF] + rw [Functor.map_map, ProbComp.eval_map, PMF.map_comp] + have hcomp : + ((fun y : Data 12 => + (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), + TabulatedOddStride.extractTable base (charBits n_max) (numChars U n_max) + (bucketBits n_max) y.mem)) ∘ + fun z : Data 9 × ℕ => z.1.extend tabMap y₂) = + ((fun T => (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), T)) ∘ + fun z : Data 9 × ℕ => + TabulatedOddStride.extractTable base (charBits n_max) (numChars U n_max) + (bucketBits n_max) z.1.mem) := rfl + rw [hcomp, ← PMF.map_comp] + change (((TabulatedOddStride.initFromBComp base (charBits n_max) (numChars U n_max) + (bucketBits n_max) (y₂.comap tabMap)).eval).map + (fun z => TabulatedOddStride.extractTable base (charBits n_max) (numChars U n_max) + (bucketBits n_max) z.1.mem)).map + (fun T => (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), T)) = + (PMF.uniformOfFintype + (Fin (numChars U n_max * 2 ^ charBits n_max) → Fin (2 ^ bucketBits n_max))).map + (fun T => (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), T)) + rw [TabulatedOddStride.map_extractTable_eval_initFromBComp] + +/-- **Exact uniformity of the sampled seed pair**: the seed marginal of `initSeedStatePMF` is the +exact product uniform distribution `seedLaw U n_max`. -/ +theorem map_fst_initSeedStatePMF {base U n_max : ℕ} {x₁ : Data 12} + (hbase : x₁.regs regBase = base) : + (initSeedStatePMF base U n_max x₁).map Prod.fst = seedLaw U n_max := by + dsimp only [initSeedStatePMF, initSetupPMF] + rw [PMF.map_comp] + have hfst : Prod.fst ∘ (fun y : Data 12 => (extractSeedPair base U n_max y, y)) = + extractSeedPair base U n_max := rfl + rw [hfst, PMF.map_bind] + let μ_tab : PMF (Fin (numChars U n_max * 2 ^ charBits n_max) → Fin (numBuckets n_max)) := + PMF.uniformOfFintype + (Fin (numChars U n_max * 2 ^ charBits n_max) → Fin (numBuckets n_max)) + have hbind : + (msInitPMF U n_max x₁).bind + (fun y₁ => (tabInitPMF base U n_max (initMidData base U n_max y₁)).map + (extractSeedPair base U n_max)) = + (msInitPMF U n_max x₁).bind + (fun y₁ => μ_tab.map + (fun T => (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), T))) := by + ext θ + rw [PMF.bind_apply, PMF.bind_apply] + refine tsum_congr fun y₁ => ?_ + by_cases hy₁ : y₁ ∈ (msInitPMF U n_max x₁).support + · obtain ⟨hb₁, -, hcond₁, -, -⟩ := mem_support_msInitPMF hy₁ + rw [map_extractSeedPair_tabInitPMF (hb₁.trans hbase) hcond₁] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy₁, zero_mul, zero_mul] + rw [hbind] + have hmap_bind : + (msInitPMF U n_max x₁).bind + (fun y₁ => μ_tab.map + (fun T => (MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap), T))) = + ((msInitPMF U n_max x₁).map + (fun y₁ => MultiplyShift.extractSeed (hashBits U n_max) (y₁.comap msMap))).bind + (fun b => μ_tab.map (fun T => (b, T))) := by + rw [PMF.bind_map] + rfl + rw [hmap_bind, map_extractSeed_msInitPMF] + exact Cslib.Probability.PMF.uniformOfFintype_prod _ _ + +theorem initSetup_execCodeDist {w base U n_max : ℕ} {x₁ : Data 12} + (hbase : x₁.regs regBase = base) (hS : Fits w (base + space U n_max)) + (hw : 2 * hashBits U n_max ≤ w) : + ExecCodeDist w (initSetup U n_max) x₁ (initSetupPMF base U n_max x₁) (setupSteps U n_max) := by + have hw₀ : hashBits U n_max ≤ w := by omega + obtain ⟨hfit_btab, hfit_powB, hfit_powM, hfit_d, -, -⟩ := fits_of_space hS + have hms_rand := + (MultiplyShift.initCode_execRand (w := w) hw₀ (x₁.comap msMap)).rename injective_msMap + have hms_dist : ExecCodeDist w + ((MultiplyShift.initCode (hashBits U n_max)).rename msMap) x₁ + (msInitPMF U n_max x₁) 2 := by + refine (ExecCodeDist.of_execRand hms_rand).congr_dist rfl ?_ + rw [ProbComp.expect_map] + refine (ProbComp.expect_mono_on_support fun z hz => ?_).trans + (le_of_eq (ProbComp.expect_const _ 2)) + dsimp only [MultiplyShift.initComp] at hz + obtain ⟨k, -, hk⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + cases OracleComp.eq_of_mem_supportWith_pure hk + exact le_rfl + have hrest : ∀ y₁ ∈ (msInitPMF U n_max x₁).support, + ExecCodeDist w + (.seq (initMidCode U n_max) + ((TabulatedOddStride.initFromBCode (numChars U n_max)).rename tabMap)) + y₁ (tabInitPMF base U n_max (initMidData base U n_max y₁)) + (5 + (5 * (numChars U n_max * 2 ^ charBits n_max) + 18 : ℕ)) := by + intro y₁ hy₁ + obtain ⟨hb₁, -, -, -, -⟩ := mem_support_msInitPMF hy₁ + have hmid := ExecCodeDist.of_exec (initMidCode_exec (hb₁.trans hbase) hS) + set y₂ := initMidData base U n_max y₁ + have hy₂_base : (y₂.comap tabMap).regs TabulatedOddStride.regBase = base := by + simpa +decide [y₂, initMidData, tabMap, Data.comap] using hb₁.trans hbase + have hy₂_b : (y₂.comap tabMap).regs TabulatedOddStride.regB = charBits n_max := by + simp +decide [y₂, initMidData, tabMap, Data.comap] + have hy₂_M : (y₂.comap tabMap).regs TabulatedOddStride.regOut = bucketBits n_max := by + simp +decide [y₂, initMidData, tabMap, Data.comap] + have htab_rand := + (TabulatedOddStride.initFromBCode_execRand (w := w) + hy₂_base hy₂_b hy₂_M hfit_btab hfit_powB hfit_powM hfit_d).rename injective_tabMap + have htab_dist : ExecCodeDist w + ((TabulatedOddStride.initFromBCode (numChars U n_max)).rename tabMap) y₂ + (tabInitPMF base U n_max y₂) + (5 * (numChars U n_max * 2 ^ charBits n_max) + 18 : ℕ) := by + refine (ExecCodeDist.of_execRand htab_rand).congr_dist rfl ?_ + rw [ProbComp.expect_map] + refine (ProbComp.expect_mono_on_support fun z hz => ?_).trans + (le_of_eq (ProbComp.expect_const _ _)) + obtain ⟨hsteps, -, -, -⟩ := TabulatedOddStride.mem_support_initFromBComp hy₂_base hz + exact_mod_cast hsteps.le + exact hmid.seq_pure htab_dist + have hseq := hms_dist.seq_const hrest (fun _ _ => le_rfl) + refine hseq.congr_dist rfl ?_ + dsimp only [setupSteps] + norm_cast + omega + +/-- Full `SubroutineDS.setup_exec` certificate for `initSetup U n_max`. -/ +theorem setup_exec_spec {w base U V n_max : ℕ} {x₁ : Data 12} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) + (hx₁ : x₁.Bounded w) (hbase : x₁.regs regBase = base) + (hfr : x₁.regs regV = base + space U n_max) + (hz : ∀ a ∈ (Block.mk base (space U n_max)).footprint, x₁.mem a = 0) : + ∃ ν₀ : PMF (Seed U n_max × Data 12), + ExecCodeDist w (initSetup U n_max) x₁ (ν₀.map Prod.snd) (setupSteps U n_max) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ 1 * seedLaw U n_max θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs regV = base + space U n_max ∧ + Rep w base U V n_max p.1 (Dictionary.empty U) p.2 ∧ + (0 : ℕ) ≤ 0 ∧ + Modifies (Block.mk base (space U n_max)).footprint x₁.mem p.2.mem := by + refine ⟨initSeedStatePMF base U n_max x₁, ?_, ?_, ?_⟩ + · have hsnd : (initSeedStatePMF base U n_max x₁).map Prod.snd = + initSetupPMF base U n_max x₁ := by + dsimp only [initSeedStatePMF] + rw [PMF.map_comp] + exact PMF.map_id _ + rw [hsnd] + exact initSetup_execCodeDist hbase hS hw + · intro θ + rw [map_fst_initSeedStatePMF hbase, one_mul] + · intro p hp + dsimp only [initSeedStatePMF] at hp + rw [PMF.mem_support_map_iff] at hp + obtain ⟨y, hy, rfl⟩ := hp + have hbdd_y : y.Bounded w := + (initSetup_execCodeDist hbase hS hw).bounded hx₁ y hy + dsimp only [initSetupPMF] at hy + rw [PMF.mem_support_bind_iff] at hy + obtain ⟨y₁, hy₁, hy₃⟩ := hy + obtain ⟨hb₁, hv₁, hcond₁, -, hmem₁⟩ := mem_support_msInitPMF hy₁ + set y₂ := initMidData base U n_max y₁ + have hy₂_base : y₂.regs regBase = base := by + simpa +decide [y₂, initMidData] using hb₁.trans hbase + obtain ⟨htab_rep, hy_base, hy_v, hframe₃⟩ := mem_support_tabInitPMF hy₂_base hy₃ + have hy_v_eq : y.regs regV = base + space U n_max := by + rw [hy_v] + simpa +decide [y₂, initMidData, hv₁] using hfr + have hms_mem : y.mem (base + tabSpace U n_max) = y₁.regs regCond := by + rw [hframe₃ (base + tabSpace U n_max) (Or.inr le_rfl)] + simp [y₂, initMidData, Data.write] + have hms_seed : y.mem (base + tabSpace U n_max) = (extractSeedPair base U n_max y).1.val := by + dsimp only [extractSeedPair, extractMSSeedFromMem] + have hlt_mem : y.mem (base + tabSpace U n_max) < 2 ^ (hashBits U n_max - 1) := by + rwa [hms_mem] + simp [hlt_mem] + have hz_slots : ∀ j : Fin (numBuckets n_max), + y.mem (base + slotOffset U n_max + j.val) = 0 := by + intro j + have hj := j.isLt + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have hso : slotOffset U n_max = tabSpace U n_max + 1 := rfl + rw [hframe₃ (base + slotOffset U n_max + j.val) (Or.inr (by omega))] + have hne : base + slotOffset U n_max + j.val ≠ base + tabSpace U n_max := by omega + simpa [y₂, initMidData, Data.write, Function.update_of_ne hne, hmem₁] using + hz _ (mem_footprint_of_lt_space (by omega) (by omega)) + have hwf : SlotArrayWF base U n_max y.mem := slotArrayWF_of_zero hz_slots + have hempty : extractSlots base U n_max y.mem = emptyTable U (numBuckets n_max) := + extractSlots_eq_emptyTable_of_zero hz_slots + have htrep : TableRep (probePerm (extractSeedPair base U n_max y)) + (Dictionary.empty U) (extractSlots base U n_max y.mem) := by + rw [hempty] + exact tableRep_empty _ + have hmod : Modifies (Block.mk base (space U n_max)).footprint x₁.mem y.mem := by + intro a ha + have ha_out : a < base ∨ base + space U n_max ≤ a := by + by_contra h + rw [not_or, not_lt, not_le] at h + exact ha (mem_footprint_of_lt_space h.1 h.2) + have hsp : space U n_max = tabSpace U n_max + 1 + 3 * numBuckets n_max := rfl + have ha₃ : a < base ∨ base + tabSpace U n_max ≤ a := by omega + have hne : a ≠ base + tabSpace U n_max := by omega + rw [hframe₃ a ha₃] + simp [y₂, initMidData, Data.write, Function.update_of_ne hne, hmem₁] + refine ⟨hy_v_eq, ⟨hbdd_y, hy_base, htab_rep, hms_seed, hwf, htrep, ?_, ?_⟩, le_rfl, hmod⟩ + · simp + · exact Dictionary.valuesBounded_empty U V + +/-! ## Section 4: Lookup and Insertion Operations (`opCode`) -/ + +/-- Probe inspection snippet `probeCheckCode` (`≤ 9` instructions): checks whether the slot in +`regSlot` belongs to `otherOccupiedSlots (extractSlots base U n_max d.mem) u` (i.e., +`used[slot] = 1` and `key[slot] ≠ u.val`), storing `1` in `regCond` if so and `0` otherwise. -/ +def probeCheckCode : Code .exact 12 := + .seq (.op (.exactBin .add regAddr regSlotBase regSlot)) + (.seq (.op (.load regCond regAddr)) + (.branch (.eq regCond regZero) + (.op (.const regCond 0)) + (.seq (.op (.exactBin .add regAddr regAddr regM)) + (.seq (.op (.load regCond regAddr)) + (.branch (.eq regCond regU) + (.op (.const regCond 0)) + (.op (.const regCond 1))))))) + +theorem probeCheckCode_execWithin {w base U n_max : ℕ} {d : Data 12} + (hS : Fits w (base + space U n_max)) + (hwf : SlotArrayWF base U n_max d.mem) + (u : Fin U) (slot : Fin (numBuckets n_max)) + (hsbase : d.regs regSlotBase = base + slotOffset U n_max) + (hslot : d.regs regSlot = slot.val) + (hm : d.regs regM = numBuckets n_max) + (hzero : d.regs regZero = 0) + (hu : d.regs regU = u.val) : + ∃ d' : Data 12, + ExecWithin w probeCheckCode d d' 9 ∧ + d'.mem = d.mem ∧ + d'.regs regCond = + (if slot ∈ otherOccupiedSlots (extractSlots base U n_max d.mem) u then 1 else 0) ∧ + ∀ r : Fin 12, r ≠ regAddr → r ≠ regCond → d'.regs r = d.regs r := by + have hslot_lt := slot.isLt + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have hfit_0 : Fits w 0 := hS.of_le (by omega) + have hfit_1 : Fits w 1 := hS.of_le (by omega) + let sBase := base + slotOffset U n_max + let m := numBuckets n_max + have hfit_a0 : Fits w (sBase + slot.val) := hS.of_le (by omega) + have hfit_a1 : Fits w (sBase + slot.val + m) := hS.of_le (by omega) + set s1 := d.set regAddr (sBase + slot.val) + have e1 : Exec w (.op (.exactBin .add regAddr regSlotBase regSlot) : Code .exact 12) d s1 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, hsbase, hslot, sBase]) hfit_a0 + let usedVal := d.mem (sBase + slot.val) + set s2 := s1.set regCond usedVal + have e2 : Exec w (.op (.load regCond regAddr) : Code .exact 12) s1 s2 1 := by + simpa +decide [s2, s1, usedVal] using + (Exec.load (B := .exact) (w := w) (d := regCond) (a := regAddr) (x := s1)) + rcases (hwf slot).1 with hused0 | hused1 + · have hc : (Cond.eq regCond regZero).Holds s2 := by + simp +decide [Cond.Holds, s2, s1, usedVal, sBase, hused0, hzero] + set s3 := s2.set regCond 0 + have e3 : Exec w (.op (.const regCond 0) : Code .exact 12) s2 s3 1 := Exec.const hfit_0 + have hnot_mem : slot ∉ otherOccupiedSlots (extractSlots base U n_max d.mem) u := by + rw [mem_otherOccupiedSlots_extractSlots_iff hwf] + omega + refine ⟨s3, + (e1.within.seq (e2.within.seq (ExecWithin.branch_pos hc e3.within))).mono (by omega), + rfl, ?_, ?_⟩ + · simp +decide [s3, hnot_mem] + · intro r hr1 hr2 + simp [s3, s2, s1, Data.set, Function.update_of_ne hr2, Function.update_of_ne hr1] + · have hc : ¬(Cond.eq regCond regZero).Holds s2 := by + simp +decide [Cond.Holds, s2, s1, usedVal, sBase, hused1, hzero] + set s3 := s2.set regAddr (sBase + slot.val + m) + have e3 : Exec w (.op (.exactBin .add regAddr regAddr regM) : Code .exact 12) s2 s3 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s2, s1, hm, m]) hfit_a1 + have haddr_eq : sBase + slot.val + m = sBase + m + slot.val := by omega + let keyVal := d.mem (sBase + m + slot.val) + set s4 := s3.set regCond keyVal + have e4 : Exec w (.op (.load regCond regAddr) : Code .exact 12) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, haddr_eq, keyVal] using + (Exec.load (B := .exact) (w := w) (d := regCond) (a := regAddr) (x := s3)) + by_cases hkey : keyVal = u.val + · have hc2 : (Cond.eq regCond regU).Holds s4 := by + simp +decide [Cond.Holds, s4, s3, s2, s1, hkey, hu] + set s5 := s4.set regCond 0 + have e5 : Exec w (.op (.const regCond 0) : Code .exact 12) s4 s5 1 := Exec.const hfit_0 + have hnot_mem : slot ∉ otherOccupiedSlots (extractSlots base U n_max d.mem) u := by + rw [mem_otherOccupiedSlots_extractSlots_iff hwf] + exact fun h => h.2 hkey + refine ⟨s5, + (e1.within.seq (e2.within.seq + (ExecWithin.branch_neg hc + (e3.within.seq (e4.within.seq (ExecWithin.branch_pos hc2 e5.within)))))).mono + (by omega), + rfl, ?_, ?_⟩ + · simp +decide [s5, hnot_mem] + · intro r hr1 hr2 + simp [s5, s4, s3, s2, s1, Data.set, Function.update_of_ne hr2, Function.update_of_ne hr1] + · have hc2 : ¬(Cond.eq regCond regU).Holds s4 := by + simp +decide [Cond.Holds, s4, s3, s2, s1, hkey, hu] + set s5 := s4.set regCond 1 + have e5 : Exec w (.op (.const regCond 1) : Code .exact 12) s4 s5 1 := Exec.const hfit_1 + have hmem : slot ∈ otherOccupiedSlots (extractSlots base U n_max d.mem) u := by + rw [mem_otherOccupiedSlots_extractSlots_iff hwf] + exact ⟨hused1, hkey⟩ + refine ⟨s5, + (e1.within.seq (e2.within.seq + (ExecWithin.branch_neg hc + (e3.within.seq (e4.within.seq (ExecWithin.branch_neg hc2 e5.within)))))).mono + (by omega), + rfl, ?_, ?_⟩ + · simp +decide [s5, hmem] + · intro r hr1 hr2 + simp [s5, s4, s3, s2, s1, Data.set, Function.update_of_ne hr2, Function.update_of_ne hr1] + +/-- One iteration of the probe loop (`≤ 11` instructions): advances `regSlot` by `regStride` modulo +`numBuckets n_max` (`regMaskM`) and runs `probeCheckCode`. -/ +def probeStepCode : Code .exact 12 := + .seq (.op (.exactBin .add regSlot regSlot regStride)) + (.seq (.op (.exactBin .and regSlot regSlot regMaskM)) + probeCheckCode) + +/-- Probe loop: continues advancing `regSlot` while `0 < regCond` (i.e., while `regSlot` is occupied +by a key other than `u`). -/ +def probeLoop : Code .exact 12 := + .loop (.lt regZero regCond) probeStepCode + +theorem probeLoop_execWithin_aux {w base U n_max : ℕ} (θ : Seed U n_max) + (hS : Fits w (base + space U n_max)) + {mem : ℕ → ℕ} (hwf : SlotArrayWF base U n_max mem) + (u : Fin U) + (hO : (otherOccupiedSlots (extractSlots base U n_max mem) u).card < numBuckets n_max) : + let T := extractSlots base U n_max mem + let O := otherOccupiedSlots T u + let k := missedProbes O (probePerm θ u) + ∀ (rem i : ℕ) (hi_lt : i < numBuckets n_max) (d : Data 12), + i + rem = k → + d.mem = mem → + d.regs regSlotBase = base + slotOffset U n_max → + d.regs regMaskM = numBuckets n_max - 1 → + d.regs regStride = (strideHash θ u).val → + d.regs regM = numBuckets n_max → + d.regs regZero = 0 → + d.regs regU = u.val → + d.regs regSlot = (probePerm θ u ⟨i, hi_lt⟩).val → + d.regs regCond = (if probePerm θ u ⟨i, hi_lt⟩ ∈ O then 1 else 0) → + ∃ d' : Data 12, + ExecWithin w probeLoop d d' (13 * rem + 2) ∧ + d'.mem = mem ∧ + d'.regs regSlot = (targetSlot (probePerm θ) T u hO).val ∧ + ∀ r : Fin 12, r ≠ regSlot → r ≠ regAddr → r ≠ regCond → d'.regs r = d.regs r := by + intro T O k rem + have hk_lt : k < numBuckets n_max := missedProbes_lt_of_card_lt hO (probePerm θ u) + induction rem with + | zero => + intro i hi_lt d hik hmem hsbase hmask hstride hm hzero hu hslot hcond + have hi_eq : i = k := by omega + have hnot_mem : probePerm θ u ⟨i, hi_lt⟩ ∉ O := by + have heq_idx : (⟨i, hi_lt⟩ : Fin (numBuckets n_max)) = + probeSlotIndex O (probePerm θ u) hO := Fin.ext hi_eq + rw [heq_idx] + exact notMem_apply_probeSlotIndex hO + have hcond0 : d.regs regCond = 0 := by + rw [hcond] + exact ite_eq_right_iff.mpr (fun h => False.elim (hnot_mem h)) + have hc : ¬(Cond.lt regZero regCond).Holds d := by + simp [Cond.Holds, hzero, hcond0] + refine ⟨d, (Exec.loop_exit hc).within, hmem, ?_, fun _ _ _ _ => rfl⟩ + rw [hslot] + dsimp only [targetSlot, probeSlotIndex] + congr 2 + ext + exact hi_eq + | succ rem ih => + intro i hi_lt d hik hmem hsbase hmask hstride hm hzero hu hslot hcond + have hi_lt_k : i < k := by omega + have hi1_lt : i + 1 < numBuckets n_max := by omega + have hmem_i : probePerm θ u ⟨i, hi_lt⟩ ∈ O := + mem_of_lt_probeSlotIndex hO (i := ⟨i, hi_lt⟩) hi_lt_k + have hcond1 : d.regs regCond = 1 := by + rw [hcond] + exact ite_eq_left_iff.mpr (fun h => False.elim (h hmem_i)) + have hc : (Cond.lt regZero regCond).Holds d := by + simp [Cond.Holds, hzero, hcond1] + let slot_i := probePerm θ u ⟨i, hi_lt⟩ + let slot_next := probePerm θ u ⟨i + 1, hi1_lt⟩ + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have hfit_add : Fits w (slot_i.val + (strideHash θ u).val) := by + have := slot_i.isLt + have := (strideHash θ u).isLt + exact hS.of_le (by omega) + have hfit_next : Fits w slot_next.val := by + have := slot_next.isLt + exact hS.of_le (by omega) + set s1 := d.set regSlot (slot_i.val + (strideHash θ u).val) + have e1 : Exec w (.op (.exactBin .add regSlot regSlot regStride) : Code .exact 12) d s1 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, hslot, hstride, slot_i]) hfit_add + set s2 := s1.set regSlot slot_next.val + have e2 : Exec w (.op (.exactBin .and regSlot regSlot regMaskM) : Code .exact 12) s1 s2 1 := by + refine Exec.alu ?_ hfit_next + have hs1_slot : s1.regs regSlot = slot_i.val + (strideHash θ u).val := by + simp +decide [s1] + have hs1_mask : s1.regs regMaskM = numBuckets n_max - 1 := by + simp +decide [s1, hmask] + change some (s1.regs regSlot &&& s1.regs regMaskM) = some slot_next.val + rw [hs1_slot, hs1_mask] + exact congrArg some (val_probePerm_succ θ u hi1_lt).symm + have hwf_s2 : SlotArrayWF base U n_max s2.mem := by simpa [s2, s1, hmem] using hwf + obtain ⟨s3, e3, hs3_mem, hs3_cond, hs3_regs⟩ := + probeCheckCode_execWithin hS hwf_s2 u slot_next + (by simp +decide [s2, s1, hsbase]) + (by simp +decide [s2]) + (by simp +decide [s2, s1, hm]) + (by simp +decide [s2, s1, hzero]) + (by simp +decide [s2, s1, hu]) + have eStep : ExecWithin w probeStepCode d s3 11 := + (e1.within.seq (e2.within.seq e3)).mono (by omega) + have hs2_mem_eq : s2.mem = mem := hmem + have hs3_mem_eq : s3.mem = mem := hs3_mem.trans hs2_mem_eq + have hs3_slot : s3.regs regSlot = slot_next.val := by + rw [hs3_regs regSlot (by decide) (by decide)] + simp +decide [s2] + have hs3_cond' : s3.regs regCond = (if slot_next ∈ O then 1 else 0) := by + rw [hs2_mem_eq] at hs3_cond + exact hs3_cond + obtain ⟨d', eRec, hd'_mem, hd'_slot, hd'_regs⟩ := + ih (i + 1) hi1_lt s3 (by omega) hs3_mem_eq + (by rw [hs3_regs regSlotBase (by decide) (by decide)]; simp +decide [s2, s1, hsbase]) + (by rw [hs3_regs regMaskM (by decide) (by decide)]; simp +decide [s2, s1, hmask]) + (by rw [hs3_regs regStride (by decide) (by decide)]; simp +decide [s2, s1, hstride]) + (by rw [hs3_regs regM (by decide) (by decide)]; simp +decide [s2, s1, hm]) + (by rw [hs3_regs regZero (by decide) (by decide)]; simp +decide [s2, s1, hzero]) + (by rw [hs3_regs regU (by decide) (by decide)]; simp +decide [s2, s1, hu]) + hs3_slot + hs3_cond' + refine ⟨d', (ExecWithin.loop_iter hc eStep eRec).mono (by omega), hd'_mem, hd'_slot, ?_⟩ + intro r hr_slot hr_addr hr_cond + rw [hd'_regs r hr_slot hr_addr hr_cond, hs3_regs r hr_addr hr_cond] + simp [s2, s1, Data.set, Function.update_of_ne hr_slot] + +/-- Deterministic 5-instruction glue before `MultiplyShift.evalCode`: saves the stride from +`regAddr` into `regStride`, loads the multiply-shift seed from `base + tabSpace U n_max` into +`regCond` (`msMap regB`), and copies `regU` into `regAddr` (`msMap regX`). -/ +def opPreMSCode (U n_max : ℕ) : Code .exact 12 := + .seq (.op (.copy regStride regAddr)) + (.seq (.op (.const regAddr (tabSpace U n_max))) + (.seq (.op (.exactBin .add regAddr regBase regAddr)) + (.seq (.op (.load regCond regAddr)) + (.op (.copy regAddr regU))))) + +/-- Deterministic 5-instruction glue after `MultiplyShift.evalCode`: initializes `regSlotBase`, +`regMaskM`, `regM`, and `regZero`. -/ +def opPostMSCode (U n_max : ℕ) : Code .exact 12 := + .seq (.op (.const regSlotBase (slotOffset U n_max))) + (.seq (.op (.exactBin .add regSlotBase regBase regSlotBase)) + (.seq (.op (.const regMaskM (numBuckets n_max - 1))) + (.seq (.op (.const regM (numBuckets n_max))) + (.op (.const regZero 0))))) + +/-- Hash evaluation and probe-register setup `opEvalHashesCode U n_max` +(`8 * numChars U n_max + 41` instructions): evaluates `strideHash θ u` into `regStride`, +`primaryHash θ u` into `regSlot`, and initializes `regSlotBase`, `regMaskM`, `regM`, +and `regZero`. -/ +def opEvalHashesCode (U n_max : ℕ) : Code .exact 12 := + .seq (.op (.copy regCond regU)) + (.seq ((TabulatedOddStride.evalCode (numChars U n_max)).rename tabMap) + (.seq (opPreMSCode U n_max) + (.seq ((MultiplyShift.evalCode (hashBits U n_max) (bucketBits n_max)).rename msMap) + (opPostMSCode U n_max)))) + +theorem opEvalHashesCode_execWithin {w base U V n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x : Data 12} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) + (hrep : Rep w base U V n_max θ s x) (u : Fin U) (hu : x.regs regU = u.val) : + ∃ d : Data 12, + ExecWithin w (opEvalHashesCode U n_max) x d (8 * numChars U n_max + 41) ∧ + d.mem = x.mem ∧ + d.regs regBase = base ∧ + d.regs regSlot = (probePerm θ u ⟨0, numBuckets_pos n_max⟩).val ∧ + d.regs regSlotBase = base + slotOffset U n_max ∧ + d.regs regMaskM = numBuckets n_max - 1 ∧ + d.regs regStride = (strideHash θ u).val ∧ + d.regs regM = numBuckets n_max ∧ + d.regs regZero = 0 ∧ + d.regs regOp = x.regs regOp ∧ + d.regs regU = u.val ∧ + d.regs regV = x.regs regV := by + obtain ⟨hfit_btab, _, hfit_powM, hfit_d, _, _⟩ := fits_of_space hS + have hsp : space U n_max = tabSpace U n_max + 1 + 3 * numBuckets n_max := rfl + have hso : slotOffset U n_max = tabSpace U n_max + 1 := rfl + have hM : 2 ^ bucketBits n_max = numBuckets n_max := rfl + have hfit_2powM : Fits w (2 * 2 ^ bucketBits n_max) := hS.of_le (by omega) + have hfit_u : Fits w u.val := by + rw [fits_iff] + have hu_lt : u.val < 2 ^ hashBits U n_max := (encodeKey n_max u).isLt + have hpow : 2 ^ hashBits U n_max ≤ 2 ^ w := Nat.pow_le_pow_right two_pos (by omega) + exact hu_lt.trans_le hpow + set s1 := x.set regCond u.val + have e1 : Exec w (.op (.copy regCond regU) : Code .exact 12) x s1 1 := by + simpa [s1, hu] using (Exec.copy (B := .exact) (w := w) (d := regCond) (s := regU) (x := x)) + have hs1_base : (s1.comap tabMap).regs TabulatedOddStride.regBase = base := by + simpa +decide [s1, tabMap, Data.comap] using hrep.regs_base + have hs1_key : (s1.comap tabMap).regs TabulatedOddStride.regKey = u.val := by + simp +decide [s1, tabMap, Data.comap] + obtain ⟨yTab, htab_exec, hyTab_out, hyTab_base, hyTab_mem⟩ := + TabulatedOddStride.evalCode_execWithin hrep.tab_rep hs1_base hs1_key + hfit_btab hfit_2powM hfit_d hfit_u + set s2 := yTab.extend tabMap s1 + have e2 : ExecWithin w ((TabulatedOddStride.evalCode (numChars U n_max)).rename tabMap) + s1 s2 (8 * numChars U n_max + 21) := + htab_exec.rename injective_tabMap + have hs2_mem : s2.mem = x.mem := by + change yTab.mem = x.mem + exact hyTab_mem + have hs2_base : s2.regs regBase = base := by + change (s2.comap tabMap).regs TabulatedOddStride.regBase = base + rw [Data.comap_extend injective_tabMap] + exact hyTab_base + have hs2_addr : s2.regs regAddr = (strideHash θ u).val := by + change (s2.comap tabMap).regs TabulatedOddStride.regOut = (strideHash θ u).val + rw [Data.comap_extend injective_tabMap] + exact hyTab_out + have hs2_op : s2.regs regOp = x.regs regOp := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_tabMap (by decide))] + simp +decide [s1] + have hs2_u : s2.regs regU = u.val := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_tabMap (by decide))] + simp +decide [s1, hu] + have hs2_v : s2.regs regV = x.regs regV := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_tabMap (by decide))] + simp +decide [s1] + -- Execute opPreMSCode + have hfit_tab : Fits w (tabSpace U n_max) := hfit_btab.of_le (by omega) + set s3 := s2.set regStride (strideHash θ u).val + have e3 : Exec w (.op (.copy regStride regAddr) : Code .exact 12) s2 s3 1 := by + simpa [s3, hs2_addr] using + (Exec.copy (B := .exact) (w := w) (d := regStride) (s := regAddr) (x := s2)) + set s4 := s3.set regAddr (tabSpace U n_max) + have e4 : Exec w (.op (.const regAddr (tabSpace U n_max)) : Code .exact 12) s3 s4 1 := + Exec.const hfit_tab + set s5 := s4.set regAddr (base + tabSpace U n_max) + have e5 : Exec w (.op (.exactBin .add regAddr regBase regAddr) : Code .exact 12) s4 s5 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s4, s3, hs2_base]) hfit_btab + set s6 := s5.set regCond θ.1.val + have e6 : Exec w (.op (.load regCond regAddr) : Code .exact 12) s5 s6 1 := by + simpa +decide [s6, s5, s4, s3, hs2_mem, hrep.ms_seed] using + (Exec.load (B := .exact) (w := w) (d := regCond) (a := regAddr) (x := s5)) + set s7 := s6.set regAddr u.val + have e7 : Exec w (.op (.copy regAddr regU) : Code .exact 12) s6 s7 1 := by + simpa +decide [s7, s6, s5, s4, s3, hs2_u] using + (Exec.copy (B := .exact) (w := w) (d := regAddr) (s := regU) (x := s6)) + have ePre : Exec w (opPreMSCode U n_max) s2 s7 5 := + e3.seq (e4.seq (e5.seq (e6.seq e7))) + -- Execute MultiplyShift.evalCode + have hs7_b : (s7.comap msMap).regs MultiplyShift.regB = θ.1.val := by + simp +decide [s7, s6, msMap, Data.comap] + have hs7_x : (s7.comap msMap).regs MultiplyShift.regX = (encodeKey n_max u).val := by + simp +decide [s7, msMap, Data.comap, encodeKey] + have hms_eval := + MultiplyShift.evalCode_exec (w₀ := hashBits U n_max) (M := bucketBits n_max) (w := w) + (hashBits_pos U n_max) hw + (by rw [hs7_b]; exact θ.1.isLt) + (by rw [hs7_x]; exact (encodeKey n_max u).isLt) + set yMS := MultiplyShift.evalData (hashBits U n_max) (bucketBits n_max) (s7.comap msMap) + set s8 := yMS.extend msMap s7 + have e8 : Exec w ((MultiplyShift.evalCode (hashBits U n_max) (bucketBits n_max)).rename msMap) + s7 s8 9 := + hms_eval.rename injective_msMap + have hs8_mem : s8.mem = x.mem := by + change (s7.comap msMap).mem = x.mem + exact hs2_mem + have hs8_slot : s8.regs regSlot = (primaryHash θ u).val := by + change (s8.comap msMap).regs MultiplyShift.regOut = (primaryHash θ u).val + rw [Data.comap_extend injective_msMap] + exact MultiplyShift.evalData_regs_out_eq_multiplyShift + (hashBits U n_max) (bucketBits n_max) (s7.comap msMap) θ.1 (encodeKey n_max u) hs7_b hs7_x + have hs8_base : s8.regs regBase = base := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide))] + simp +decide [s7, s6, s5, s4, s3, hs2_base] + have hs8_stride : s8.regs regStride = (strideHash θ u).val := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide))] + simp +decide [s7, s6, s5, s4, s3] + have hs8_op : s8.regs regOp = x.regs regOp := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide))] + simp +decide [s7, s6, s5, s4, s3, hs2_op] + have hs8_u : s8.regs regU = u.val := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide))] + simp +decide [s7, s6, s5, s4, s3, hs2_u] + have hs8_v : s8.regs regV = x.regs regV := by + rw [Data.extend_regs_of_notMem _ _ (notMem_range_msMap (by decide))] + simp +decide [s7, s6, s5, s4, s3, hs2_v] + -- Execute opPostMSCode + have hfit_so : Fits w (slotOffset U n_max) := hS.of_le (by omega) + have hfit_bso : Fits w (base + slotOffset U n_max) := hS.of_le (by omega) + have hfit_mask : Fits w (numBuckets n_max - 1) := hS.of_le (by omega) + have hfit_m : Fits w (numBuckets n_max) := hS.of_le (by omega) + have hfit_0 : Fits w 0 := hS.of_le (by omega) + set s9 := s8.set regSlotBase (slotOffset U n_max) + have e9 : Exec w (.op (.const regSlotBase (slotOffset U n_max)) : Code .exact 12) s8 s9 1 := + Exec.const hfit_so + set s10 := s9.set regSlotBase (base + slotOffset U n_max) + have e10 : Exec w (.op (.exactBin .add regSlotBase regBase regSlotBase) : Code .exact 12) + s9 s10 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s9, hs8_base]) hfit_bso + set s11 := s10.set regMaskM (numBuckets n_max - 1) + have e11 : Exec w (.op (.const regMaskM (numBuckets n_max - 1)) : Code .exact 12) s10 s11 1 := + Exec.const hfit_mask + set s12 := s11.set regM (numBuckets n_max) + have e12 : Exec w (.op (.const regM (numBuckets n_max)) : Code .exact 12) s11 s12 1 := + Exec.const hfit_m + set s13 := s12.set regZero 0 + have e13 : Exec w (.op (.const regZero 0) : Code .exact 12) s12 s13 1 := + Exec.const hfit_0 + have ePost : Exec w (opPostMSCode U n_max) s8 s13 5 := + e9.seq (e10.seq (e11.seq (e12.seq e13))) + have eAll : ExecWithin w (opEvalHashesCode U n_max) x s13 (8 * numChars U n_max + 41) := + (e1.within.seq (e2.seq (ePre.within.seq (e8.within.seq ePost.within)))).mono (by omega) + refine ⟨s13, eAll, hs8_mem, + by simp +decide [s13, s12, s11, s10, s9, hs8_base], + by rw [probePerm_zero]; simp +decide [s13, s12, s11, s10, s9, hs8_slot], + by simp +decide [s13, s12, s11, s10], + by simp +decide [s13, s12, s11], + by simp +decide [s13, s12, s11, s10, s9, hs8_stride], + by simp +decide [s13, s12], + by simp +decide [s13], + by simp +decide [s13, s12, s11, s10, s9, hs8_op], + by simp +decide [s13, s12, s11, s10, s9, hs8_u], + by simp +decide [s13, s12, s11, s10, s9, hs8_v]⟩ + +/-- Lookup completion program `finishGetCode` (`≤ 9` instructions): inspects `used[regSlot]`; +if `0`, writes `0` (`none`) into `regOp`, and if `1`, loads `val[regSlot]` and writes +`val[regSlot] + 1` (`some v`) into `regOp`. -/ +def finishGetCode : Code .exact 12 := + .seq (.op (.exactBin .add regAddr regSlotBase regSlot)) + (.seq (.op (.load regCond regAddr)) + (.branch (.eq regCond regZero) + (.op (.const regOp 0)) + (.seq (.op (.exactBin .add regAddr regAddr regM)) + (.seq (.op (.exactBin .add regAddr regAddr regM)) + (.seq (.op (.load regOp regAddr)) + (.seq (.op (.const regCond 1)) + (.op (.exactBin .add regOp regOp regCond)))))))) + +theorem finishGetCode_execWithin {w base U V n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x d : Data 12} + (hS : Fits w (base + space U n_max)) (hV : Fits w V) + (hrep : Rep w base U V n_max θ s x) + (u : Fin U) + (hO : (otherOccupiedSlots (extractSlots base U n_max x.mem) u).card < numBuckets n_max) + (hmem : d.mem = x.mem) + (hbase : d.regs regBase = base) + (hsbase : d.regs regSlotBase = base + slotOffset U n_max) + (hslot : d.regs regSlot = (targetSlot (probePerm θ) (extractSlots base U n_max x.mem) u hO).val) + (hm : d.regs regM = numBuckets n_max) + (hzero : d.regs regZero = 0) : + ∃ y : Data 12, + ExecWithin w finishGetCode d y 9 ∧ + y.mem = x.mem ∧ + y.regs regBase = base ∧ + y.regs regOp = (match s u with | none => 0 | some v => v + 1) := by + let T := extractSlots base U n_max x.mem + let slot := targetSlot (probePerm θ) T u hO + have hslot_lt := slot.isLt + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have hfit_0 : Fits w 0 := hS.of_le (by omega) + have hfit_1 : Fits w 1 := hS.of_le (by omega) + let sBase := base + slotOffset U n_max + let m := numBuckets n_max + have hfit_a0 : Fits w (sBase + slot.val) := hS.of_le (by omega) + have hfit_a1 : Fits w (sBase + slot.val + m) := hS.of_le (by omega) + have hfit_a2 : Fits w (sBase + slot.val + m + m) := hS.of_le (by omega) + set s1 := d.set regAddr (sBase + slot.val) + have e1 : Exec w (.op (.exactBin .add regAddr regSlotBase regSlot) : Code .exact 12) d s1 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, hsbase, hslot, sBase, slot, T]) hfit_a0 + let usedVal := x.mem (sBase + slot.val) + set s2 := s1.set regCond usedVal + have e2 : Exec w (.op (.load regCond regAddr) : Code .exact 12) s1 s2 1 := by + simpa +decide [s2, s1, hmem, usedVal] using + (Exec.load (B := .exact) (w := w) (d := regCond) (a := regAddr) (x := s1)) + have htarget := hrep.table_rep.apply_targetSlot u hO + cases hsu : s u with + | none => + have hT_none : extractSlots base U n_max x.mem slot = none := by + simpa [slot, T, hsu] using htarget + have hused0 : usedVal = 0 := (extractSlots_eq_none_iff hrep.slot_wf slot).mp hT_none + have hc : (Cond.eq regCond regZero).Holds s2 := by + simp +decide [Cond.Holds, s2, s1, hused0, hzero] + set s3 := s2.set regOp 0 + have e3 : Exec w (.op (.const regOp 0) : Code .exact 12) s2 s3 1 := Exec.const hfit_0 + refine ⟨s3, + (e1.within.seq (e2.within.seq (ExecWithin.branch_pos hc e3.within))).mono (by omega), + hmem, by simp +decide [s3, s2, s1, hbase], by simp +decide [s3]⟩ + | some v => + have hT_some : extractSlots base U n_max x.mem slot = some (u, v) := by + simpa [slot, T, hsu] using htarget + obtain ⟨hused1, _, hval_eq⟩ := (extractSlots_eq_some_iff hrep.slot_wf slot u v).mp hT_some + have hv_lt : v < V := hrep.vals_bounded u v hsu + have hfit_v1 : Fits w (v + 1) := hV.of_le (by omega) + have hc : ¬(Cond.eq regCond regZero).Holds s2 := by + simp +decide [Cond.Holds, s2, s1, usedVal, sBase, hused1, hzero] + set s3 := s2.set regAddr (sBase + slot.val + m) + have e3 : Exec w (.op (.exactBin .add regAddr regAddr regM) : Code .exact 12) s2 s3 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s2, s1, hm, m]) hfit_a1 + set s4 := s3.set regAddr (sBase + slot.val + m + m) + have e4 : Exec w (.op (.exactBin .add regAddr regAddr regM) : Code .exact 12) s3 s4 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s3, s2, s1, hm, m]) hfit_a2 + have haddr2_eq : + sBase + slot.val + m + m = base + slotOffset U n_max + 2 * numBuckets n_max + slot.val := by + dsimp only [sBase, m] + omega + set s5 := s4.set regOp v + have e5 : Exec w (.op (.load regOp regAddr) : Code .exact 12) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, hmem, haddr2_eq, hval_eq] using + (Exec.load (B := .exact) (w := w) (d := regOp) (a := regAddr) (x := s4)) + set s6 := s5.set regCond 1 + have e6 : Exec w (.op (.const regCond 1) : Code .exact 12) s5 s6 1 := Exec.const hfit_1 + set s7 := s6.set regOp (v + 1) + have e7 : Exec w (.op (.exactBin .add regOp regOp regCond) : Code .exact 12) s6 s7 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s6, s5]) hfit_v1 + refine ⟨s7, + (e1.within.seq (e2.within.seq + (ExecWithin.branch_neg hc + (e3.within.seq (e4.within.seq (e5.within.seq (e6.within.seq e7.within))))))).mono + (by omega), + hmem, by simp +decide [s7, s6, s5, s4, s3, s2, s1, hbase], by simp +decide [s7]⟩ + +/-- Insertion completion program `finishInsertCode` (`8` instructions): writes `used[regSlot] := 1`, +`key[regSlot] := regU`, `val[regSlot] := regV`, and sets `regOp := 0`. -/ +def finishInsertCode : Code .exact 12 := + .seq (.op (.exactBin .add regAddr regSlotBase regSlot)) + (.seq (.op (.const regCond 1)) + (.seq (.op (.store regAddr regCond)) + (.seq (.op (.exactBin .add regAddr regAddr regM)) + (.seq (.op (.store regAddr regU)) + (.seq (.op (.exactBin .add regAddr regAddr regM)) + (.seq (.op (.store regAddr regV)) + (.op (.const regOp 0)))))))) + +theorem finishInsertCode_execWithin {w base U V n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x d : Data 12} + (hS : Fits w (base + space U n_max)) + (hrep : Rep w base U V n_max θ s x) + (u : Fin U) (v : ℕ) + (hvalid : ((Dictionary.spec U V n_max).valid s (.insert u v))) + (hO : (otherOccupiedSlots (extractSlots base U n_max x.mem) u).card < numBuckets n_max) + (hmem : d.mem = x.mem) + (hbase : d.regs regBase = base) + (hsbase : d.regs regSlotBase = base + slotOffset U n_max) + (hslot : d.regs regSlot = (targetSlot (probePerm θ) (extractSlots base U n_max x.mem) u hO).val) + (hm : d.regs regM = numBuckets n_max) + (hu : d.regs regU = u.val) + (hv : d.regs regV = v) : + ∃ y : Data 12, + ExecWithin w finishInsertCode d y 8 ∧ + y.regs regBase = base ∧ + y.regs regOp = 0 ∧ + TabulatedOddStride.Rep base (charBits n_max) (numChars U n_max) + (bucketBits n_max) θ.2 y.mem ∧ + y.mem (base + tabSpace U n_max) = θ.1.val ∧ + SlotArrayWF base U n_max y.mem ∧ + TableRep (probePerm θ) (Function.update s u (some v)) (extractSlots base U n_max y.mem) ∧ + (Dictionary.dom (Function.update s u (some v))).card ≤ n_max ∧ + Dictionary.ValuesBounded V (Function.update s u (some v)) ∧ + Modifies (Block.mk base (space U n_max)).footprint x.mem y.mem := by + let T := extractSlots base U n_max x.mem + let slot := targetSlot (probePerm θ) T u hO + have hslot_lt := slot.isLt + have hsp : space U n_max = slotOffset U n_max + 3 * numBuckets n_max := rfl + have hso : slotOffset U n_max = tabSpace U n_max + 1 := rfl + have htab_eq : tabSpace U n_max = 4 + numChars U n_max * 2 ^ charBits n_max := rfl + have hfit_0 : Fits w 0 := hS.of_le (by omega) + have hfit_1 : Fits w 1 := hS.of_le (by omega) + let sBase := base + slotOffset U n_max + let m := numBuckets n_max + have hfit_a0 : Fits w (sBase + slot.val) := hS.of_le (by omega) + have hfit_a1 : Fits w (sBase + slot.val + m) := hS.of_le (by omega) + have hfit_a2 : Fits w (sBase + slot.val + m + m) := hS.of_le (by omega) + have ha1_eq : sBase + slot.val + m = sBase + m + slot.val := by omega + have ha2_eq : sBase + slot.val + m + m = sBase + 2 * m + slot.val := by omega + set s1 := d.set regAddr (sBase + slot.val) + have e1 : Exec w (.op (.exactBin .add regAddr regSlotBase regSlot) : Code .exact 12) d s1 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, hsbase, hslot, sBase, slot, T]) hfit_a0 + set s2 := s1.set regCond 1 + have e2 : Exec w (.op (.const regCond 1) : Code .exact 12) s1 s2 1 := Exec.const hfit_1 + set s3 := s2.write (sBase + slot.val) 1 + have e3 : Exec w (.op (.store regAddr regCond) : Code .exact 12) s2 s3 1 := by + simpa +decide [s3, s2, s1] using + (Exec.store (B := .exact) (w := w) (a := regAddr) (s := regCond) (x := s2)) + set s4 := s3.set regAddr (sBase + slot.val + m) + have e4 : Exec w (.op (.exactBin .add regAddr regAddr regM) : Code .exact 12) s3 s4 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s3, s2, s1, hm, m]) hfit_a1 + set s5 := s4.write (sBase + m + slot.val) u.val + have e5 : Exec w (.op (.store regAddr regU) : Code .exact 12) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, ha1_eq, hu] using + (Exec.store (B := .exact) (w := w) (a := regAddr) (s := regU) (x := s4)) + set s6 := s5.set regAddr (sBase + slot.val + m + m) + have e6 : Exec w (.op (.exactBin .add regAddr regAddr regM) : Code .exact 12) s5 s6 1 := + Exec.alu (by simp +decide [StandardOp.evalExact, s5, s4, s3, s2, s1, hm, m]) hfit_a2 + set s7 := s6.write (sBase + 2 * m + slot.val) v + have e7 : Exec w (.op (.store regAddr regV) : Code .exact 12) s6 s7 1 := by + simpa +decide [s7, s6, s5, s4, s3, s2, s1, ha2_eq, hv] using + (Exec.store (B := .exact) (w := w) (a := regAddr) (s := regV) (x := s6)) + set s8 := s7.set regOp 0 + have e8 : Exec w (.op (.const regOp 0) : Code .exact 12) s7 s8 1 := Exec.const hfit_0 + have eAll := (e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq e8))))))).within + have hs8_mem_below : ∀ a < sBase, s8.mem a = x.mem a := by + intro a ha + have hne0 : a ≠ sBase + slot.val := by omega + have hne1 : a ≠ sBase + m + slot.val := by omega + have hne2 : a ≠ sBase + 2 * m + slot.val := by omega + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne hne0, Function.update_of_ne hne1, Function.update_of_ne hne2, hmem] + have htab : TabulatedOddStride.Rep base (charBits n_max) (numChars U n_max) + (bucketBits n_max) θ.2 s8.mem := by + rcases hrep.tab_rep with ⟨hm0, hm1, hm2, hm3, hmt⟩ + refine ⟨?_, ?_, ?_, ?_, fun i => ?_⟩ + · rw [hs8_mem_below base (by omega), hm0] + · rw [hs8_mem_below (base + 1) (by omega), hm1] + · rw [hs8_mem_below (base + 2) (by omega), hm2] + · rw [hs8_mem_below (base + 3) (by omega), hm3] + · have hi := i.isLt + rw [hs8_mem_below (base + 4 + i.val) (by omega), hmt i] + have hms : s8.mem (base + tabSpace U n_max) = θ.1.val := by + rw [hs8_mem_below (base + tabSpace U n_max) (by omega), hrep.ms_seed] + have hs8_slot0 : s8.mem (sBase + slot.val) = 1 := by + have hne2 : sBase + slot.val ≠ sBase + 2 * m + slot.val := by omega + have hne1 : sBase + slot.val ≠ sBase + m + slot.val := by omega + simp [s8, s7, s6, s5, s4, s3, Data.write, + Function.update_of_ne hne2, Function.update_of_ne hne1] + have hs8_slot1 : s8.mem (sBase + m + slot.val) = u.val := by + have hne2 : sBase + m + slot.val ≠ sBase + 2 * m + slot.val := by omega + simp [s8, s7, s6, s5, Data.write, Function.update_of_ne hne2] + have hs8_slot2 : s8.mem (sBase + 2 * m + slot.val) = v := by + simp [s8, s7, Data.write] + have hwf : SlotArrayWF base U n_max s8.mem := by + intro j + have hj := j.isLt + by_cases hjs : j = slot + · subst hjs + exact ⟨Or.inr hs8_slot0, fun _ => hs8_slot1 ▸ u.isLt⟩ + · have hne_val : j.val ≠ slot.val := fun h => hjs (Fin.ext h) + have h0 : s8.mem (sBase + j.val) = x.mem (sBase + j.val) := by + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne (show sBase + j.val ≠ sBase + 2 * m + slot.val by omega), + Function.update_of_ne (show sBase + j.val ≠ sBase + m + slot.val by omega), + Function.update_of_ne (show sBase + j.val ≠ sBase + slot.val by omega), hmem] + have h1 : s8.mem (sBase + m + j.val) = x.mem (sBase + m + j.val) := by + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne (show sBase + m + j.val ≠ sBase + 2 * m + slot.val by omega), + Function.update_of_ne (show sBase + m + j.val ≠ sBase + m + slot.val by omega), + Function.update_of_ne (show sBase + m + j.val ≠ sBase + slot.val by omega), hmem] + have hwf_j := hrep.slot_wf j + change (s8.mem (sBase + j.val) = 0 ∨ s8.mem (sBase + j.val) = 1) ∧ + (s8.mem (sBase + j.val) = 1 → s8.mem (sBase + m + j.val) < U) + rw [h0, h1] + exact hwf_j + have hext : extractSlots base U n_max s8.mem = + tableInsert (numBuckets_pos n_max) (probePerm θ) T u v := by + rw [tableInsert_eq_update_targetSlot (numBuckets_pos n_max) (probePerm θ) T u v hO] + funext j + have hj := j.isLt + by_cases hjs : j = slot + · subst hjs + simp only [slot, Function.update_self] + rw [extractSlots_eq_some_iff hwf] + exact ⟨hs8_slot0, hs8_slot1, hs8_slot2⟩ + · have hne_val : j.val ≠ slot.val := fun h => hjs (Fin.ext h) + simp only [slot, Function.update_of_ne hjs] + have h0 : s8.mem (sBase + j.val) = x.mem (sBase + j.val) := by + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne (show sBase + j.val ≠ sBase + 2 * m + slot.val by omega), + Function.update_of_ne (show sBase + j.val ≠ sBase + m + slot.val by omega), + Function.update_of_ne (show sBase + j.val ≠ sBase + slot.val by omega), hmem] + have h1 : s8.mem (sBase + m + j.val) = x.mem (sBase + m + j.val) := by + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne (show sBase + m + j.val ≠ sBase + 2 * m + slot.val by omega), + Function.update_of_ne (show sBase + m + j.val ≠ sBase + m + slot.val by omega), + Function.update_of_ne (show sBase + m + j.val ≠ sBase + slot.val by omega), hmem] + have h2 : s8.mem (sBase + 2 * m + j.val) = x.mem (sBase + 2 * m + j.val) := by + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne (show sBase + 2 * m + j.val ≠ sBase + 2 * m + slot.val by omega), + Function.update_of_ne (show sBase + 2 * m + j.val ≠ sBase + m + slot.val by omega), + Function.update_of_ne (show sBase + 2 * m + j.val ≠ sBase + slot.val by omega), hmem] + change extractSlots base U n_max s8.mem j = extractSlots base U n_max x.mem j + dsimp only [extractSlots] + rw [show base + slotOffset U n_max + j.val = sBase + j.val from rfl, + show base + slotOffset U n_max + numBuckets n_max + j.val = sBase + m + j.val from rfl, + show base + slotOffset U n_max + 2 * numBuckets n_max + j.val = + sBase + 2 * m + j.val from rfl, + h0, h1, h2] + have htrep : TableRep (probePerm θ) (Function.update s u (some v)) + (extractSlots base U n_max s8.mem) := by + rw [hext] + exact hrep.table_rep.tableInsert (numBuckets_pos n_max) u v hO + have hdom_le : (Dictionary.dom (Function.update s u (some v))).card ≤ n_max := + Dictionary.card_dom_step_le_of_valid (V := V) hrep.dom_le (o := .insert u v) hvalid + have hvals : Dictionary.ValuesBounded V (Function.update s u (some v)) := + Dictionary.valuesBounded_step_of_valid (n_max := n_max) hrep.vals_bounded + (o := .insert u v) hvalid + have hmod : Modifies (Block.mk base (space U n_max)).footprint x.mem s8.mem := by + intro a ha + have ha_out : a < base ∨ base + space U n_max ≤ a := by + by_contra h + rw [not_or, not_lt, not_le] at h + exact ha (mem_footprint_of_lt_space h.1 h.2) + have hne0 : a ≠ sBase + slot.val := by omega + have hne1 : a ≠ sBase + m + slot.val := by omega + have hne2 : a ≠ sBase + 2 * m + slot.val := by omega + simp [s8, s7, s6, s5, s4, s3, s2, s1, Data.write, + Function.update_of_ne hne0, Function.update_of_ne hne1, Function.update_of_ne hne2, hmem] + exact ⟨s8, eAll, + by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, hbase], + by simp +decide [s8], + htab, hms, hwf, htrep, hdom_le, hvals, hmod⟩ + +/-- Complete operation body `opCode U n_max` for both `get` and `insert`. -/ +def opCode (U n_max : ℕ) : Code .exact 12 := + .seq (opEvalHashesCode U n_max) + (.seq probeCheckCode + (.seq probeLoop + (.branch (.eq regOp regZero) finishGetCode finishInsertCode))) + +/-- Exact pathwise instruction budget +`13 * probeCount T u (probePerm θ u) + 8 * numChars U n_max + 49` for `opCode U n_max` on table `T` +and key `u`. -/ +noncomputable def opPathSteps (U n_max : ℕ) (θ : Seed U n_max) + (T : UniformProbing.Table U (numBuckets n_max)) (u : Fin U) : ℕ := + 13 * probeCount T u (probePerm θ u) + 8 * numChars U n_max + 49 + +/-- Worst-case instruction bound `13 * numBuckets n_max + 8 * numChars U n_max + 49` for +`opCode`. -/ +def opSteps (U n_max : ℕ) : ℕ := + 13 * numBuckets n_max + 8 * numChars U n_max + 49 + +theorem opPathSteps_le_opSteps {U n_max : ℕ} (θ : Seed U n_max) + (T : UniformProbing.Table U (numBuckets n_max)) (u : Fin U) + (hO : (otherOccupiedSlots T u).card < numBuckets n_max) : + opPathSteps U n_max θ T u ≤ opSteps U n_max := by + dsimp only [opPathSteps, opSteps, probeCount, unsuccessfulProbes] + have hlt := missedProbes_lt_of_card_lt hO (probePerm θ u) + omega + +/-- Key extracted from a `Dictionary.Op U`. -/ +def opKey {U : ℕ} : Dictionary.Op U → Fin U + | .get u => u + | .insert u _ => u + +/-- Operation tag (`0` for `get`, `1` for `insert`). -/ +def opTag {U : ℕ} : Dictionary.Op U → ℕ + | .get _ => 0 + | .insert _ _ => 1 + +/-- Encoded output word in `regOp` after executing `op` from state `s`. -/ +def opOutWord {U : ℕ} (s : Dictionary.State U) : Dictionary.Op U → ℕ + | .get u => match s u with | none => 0 | some v => v + 1 + | .insert _ _ => 0 + +/-- **Pathwise correctness and complexity of `opCode U n_max`**: for any seed `θ`, dictionary +state `s`, machine state `x` satisfying `Rep w base U V n_max θ s x`, and admissible operation +`op`, `opCode U n_max` halts deterministically in at most +`opPathSteps U n_max θ (extractSlots base U n_max x.mem) (opKey op) ≤ opSteps U n_max` instructions, +preserving `Rep` for `((Dictionary.spec U V n_max).step s op).1`, writing `opOutWord s op` into +`regOp`, and modifying memory only within `[base, base + space U n_max)`. -/ +theorem opCode_execWithin {w base U V n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x : Data 12} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) (hV : Fits w V) + (hrep : Rep w base U V n_max θ s x) (op : Dictionary.Op U) + (hvalid : (Dictionary.spec U V n_max).valid s op) + (hop : x.regs regOp = opTag op) + (hu : x.regs regU = (opKey op).val) + (hv : ∀ u v, op = .insert u v → x.regs regV = v) : + ∃ y : Data 12, + ExecWithin w (opCode U n_max) x y + (opPathSteps U n_max θ (extractSlots base U n_max x.mem) (opKey op)) ∧ + Rep w base U V n_max θ ((Dictionary.spec U V n_max).step s op).1 y ∧ + y.regs regOp = opOutWord s op ∧ + Modifies (Block.mk base (space U n_max)).footprint x.mem y.mem := by + have hO : + (otherOccupiedSlots (extractSlots base U n_max x.mem) (opKey op)).card < numBuckets n_max := + lt_of_le_of_lt (hrep.table_rep.card_otherOccupiedSlots_le (opKey op)) + (lt_of_le_of_lt hrep.dom_le (lt_numBuckets n_max)) + obtain ⟨d1, e1, hd1_mem, hd1_base, hd1_slot, hd1_sbase, hd1_mask, hd1_stride, hd1_m, + hd1_zero, hd1_op, hd1_u, hd1_v⟩ := + opEvalHashesCode_execWithin hS hw hrep (opKey op) hu + have hwf_d1 : SlotArrayWF base U n_max d1.mem := by simpa [hd1_mem] using hrep.slot_wf + obtain ⟨d2, e2, hd2_mem, hd2_cond, hd2_regs⟩ := + probeCheckCode_execWithin hS hwf_d1 (opKey op) + (probePerm θ (opKey op) ⟨0, numBuckets_pos n_max⟩) + hd1_sbase hd1_slot hd1_m hd1_zero hd1_u + have hd2_mem_eq : d2.mem = x.mem := hd2_mem.trans hd1_mem + have hd2_cond' : d2.regs regCond = + (if probePerm θ (opKey op) ⟨0, numBuckets_pos n_max⟩ ∈ + otherOccupiedSlots (extractSlots base U n_max x.mem) (opKey op) then 1 else 0) := by + rw [hd1_mem] at hd2_cond + exact hd2_cond + obtain ⟨d3, e3, hd3_mem, hd3_slot, hd3_regs⟩ := + probeLoop_execWithin_aux θ hS hrep.slot_wf (opKey op) hO + (missedProbes (otherOccupiedSlots (extractSlots base U n_max x.mem) (opKey op)) + (probePerm θ (opKey op))) 0 (numBuckets_pos n_max) d2 + (zero_add _) hd2_mem_eq + (by rw [hd2_regs regSlotBase (by decide) (by decide), hd1_sbase]) + (by rw [hd2_regs regMaskM (by decide) (by decide), hd1_mask]) + (by rw [hd2_regs regStride (by decide) (by decide), hd1_stride]) + (by rw [hd2_regs regM (by decide) (by decide), hd1_m]) + (by rw [hd2_regs regZero (by decide) (by decide), hd1_zero]) + (by rw [hd2_regs regU (by decide) (by decide), hd1_u]) + (by rw [hd2_regs regSlot (by decide) (by decide), hd1_slot]) + hd2_cond' + have hd3_base : d3.regs regBase = base := by + rw [hd3_regs regBase (by decide) (by decide) (by decide), + hd2_regs regBase (by decide) (by decide), hd1_base] + have hd3_sbase : d3.regs regSlotBase = base + slotOffset U n_max := by + rw [hd3_regs regSlotBase (by decide) (by decide) (by decide), + hd2_regs regSlotBase (by decide) (by decide), hd1_sbase] + have hd3_m : d3.regs regM = numBuckets n_max := by + rw [hd3_regs regM (by decide) (by decide) (by decide), + hd2_regs regM (by decide) (by decide), hd1_m] + have hd3_zero : d3.regs regZero = 0 := by + rw [hd3_regs regZero (by decide) (by decide) (by decide), + hd2_regs regZero (by decide) (by decide), hd1_zero] + have hd3_op : d3.regs regOp = opTag op := by + rw [hd3_regs regOp (by decide) (by decide) (by decide), + hd2_regs regOp (by decide) (by decide), hd1_op, hop] + have hd3_u : d3.regs regU = (opKey op).val := by + rw [hd3_regs regU (by decide) (by decide) (by decide), + hd2_regs regU (by decide) (by decide), hd1_u] + have hd3_v : d3.regs regV = x.regs regV := by + rw [hd3_regs regV (by decide) (by decide) (by decide), + hd2_regs regV (by decide) (by decide), hd1_v] + cases op with + | get u₀ => + have hc : (Cond.eq regOp regZero).Holds d3 := by + simp [Cond.Holds, hd3_op, hd3_zero, opTag] + obtain ⟨y, eFin, hy_mem, hy_base, hy_op⟩ := + finishGetCode_execWithin hS hV hrep u₀ hO hd3_mem hd3_base hd3_sbase hd3_slot hd3_m hd3_zero + have eBr : ExecWithin w (.branch (.eq regOp regZero) finishGetCode finishInsertCode) d3 y 10 := + ExecWithin.branch_pos hc eFin + have eTot : ExecWithin w (opCode U n_max) x y + (opPathSteps U n_max θ (extractSlots base U n_max x.mem) u₀) := by + refine (e1.seq (e2.seq (e3.seq eBr))).mono ?_ + dsimp only [opPathSteps, probeCount, unsuccessfulProbes, opKey] at * + omega + have hbdd_y : y.Bounded w := eTot.bounded hrep.bounded + have hrep_y : Rep w base U V n_max θ s y := + hrep.congr hbdd_y (hy_base.trans hrep.regs_base.symm) (fun a _ => by rw [hy_mem]) + have hmod : Modifies (Block.mk base (space U n_max)).footprint x.mem y.mem := by + intro a _ + rw [hy_mem] + exact ⟨y, eTot, hrep_y, hy_op, hmod⟩ + | insert u₀ v₀ => + have hc : ¬(Cond.eq regOp regZero).Holds d3 := by + simp [Cond.Holds, hd3_op, hd3_zero, opTag] + have hd3_v₀ : d3.regs regV = v₀ := hd3_v.trans (hv u₀ v₀ rfl) + obtain ⟨y, eFin, hy_base, hy_op, hy_tab, hy_ms, hy_wf, hy_trep, hy_dom, hy_vals, hy_mod⟩ := + finishInsertCode_execWithin hS hrep u₀ v₀ hvalid hO + hd3_mem hd3_base hd3_sbase hd3_slot hd3_m hd3_u hd3_v₀ + have eBr : ExecWithin w (.branch (.eq regOp regZero) finishGetCode finishInsertCode) d3 y 10 := + ExecWithin.branch_neg hc eFin + have eTot : ExecWithin w (opCode U n_max) x y + (opPathSteps U n_max θ (extractSlots base U n_max x.mem) u₀) := by + refine (e1.seq (e2.seq (e3.seq eBr))).mono ?_ + dsimp only [opPathSteps, probeCount, unsuccessfulProbes, opKey] at * + omega + have hbdd_y : y.Bounded w := eTot.bounded hrep.bounded + exact ⟨y, eTot, + ⟨hbdd_y, hy_base, hy_tab, hy_ms, hy_wf, hy_trep, hy_dom, hy_vals⟩, + hy_op, hy_mod⟩ + +/-! ## Section 5: `SubroutineCode`, `SubroutineDS`, and Expected Probe / Collision Bounds -/ + +/-- Relocatable `SubroutineCode` over `Basis.exact` (`12` registers, `0` runtime init parameters) +for the open-addressed permutation-probing dictionary. -/ +def subroutineCode (U n_max : ℕ) : SubroutineCode .exact 12 0 where + allocSpan := allocSpan U n_max + initSetup := initSetup U n_max + code := opCode U n_max + frontierReg := regV + baseReg := regBase + initParams := Fin.elim0 + curReg := regCond + zeroReg := regAddr + oneReg := regSlot + incCur := .exactBin .add regCond regCond regSlot + input := fun i => decide (i ≠ regBase) + output := regOp + +/-- Standalone argument encoder for `Dictionary.Op U` on `12` registers. -/ +def encodeOp {U : ℕ} : Dictionary.Op U → Fin 12 → ℕ + | .get u, i => + if i = regOp then 0 + else if i = regU then u.val + else 0 + | .insert u v, i => + if i = regOp then 1 + else if i = regU then u.val + else if i = regV then v + else 0 + +/-- Partial argument-matching predicate for inline and standalone callers. -/ +def argMatch {U : ℕ} (op : Dictionary.Op U) (regs : Fin 12 → ℕ) : Prop := + regs regOp = opTag op ∧ + regs regU = (opKey op).val ∧ + ∀ u v, op = .insert u v → regs regV = v + +/-- Output decoder reading typed operation observations from final registers. -/ +def decodeOp {U V n_max : ℕ} : (op : Dictionary.Op U) → (Fin 12 → ℕ) → + Option ((Dictionary.spec U V n_max).Result op) + | .get _, regs => some (if regs regOp = 0 then none else some (regs regOp - 1)) + | .insert _ _, _ => some () + +theorem encode_matches {U n_max : ℕ} (op : Dictionary.Op U) (regs : Fin 12 → ℕ) : + argMatch op (fun i => if (subroutineCode U n_max).input i then encodeOp op i else regs i) := by + cases op with + | get u => + refine ⟨rfl, ?_, fun _ _ h => by nomatch h⟩ + simp +decide [subroutineCode, encodeOp, opKey] + | insert u v => + refine ⟨rfl, ?_, fun u' v' h => ?_⟩ + · simp +decide [subroutineCode, encodeOp, opKey] + · cases h + simp +decide [subroutineCode, encodeOp] + +theorem arguments_fit {w U V n_max : ℕ} + (hw : 2 * hashBits U n_max ≤ w) (hV : Fits w V) + {s : Dictionary.State U} (op : Dictionary.Op U) + (hv : (Dictionary.spec U V n_max).valid s op) + (i : Fin 12) : + Fits w (encodeOp op i) := by + have hpos_hb := hashBits_pos U n_max + have hfit0 : Fits w 0 := fits_zero w + have hfit1 : Fits w 1 := fits_one (by omega) + have hfitU : ∀ u : Fin U, Fits w u.val := by + intro u + rw [fits_iff] + have hu_lt : u.val < 2 ^ hashBits U n_max := lt_two_pow_hashBits n_max u + have hpow : 2 ^ hashBits U n_max ≤ 2 ^ w := Nat.pow_le_pow_right two_pos (by omega) + exact hu_lt.trans_le hpow + cases op with + | get u => + dsimp only [encodeOp] + split_ifs + · exact hfit0 + · exact hfitU u + · exact hfit0 + | insert u v => + dsimp only [encodeOp] + split_ifs + · exact hfit1 + · exact hfitU u + · exact hV.of_le (by have := hv.2; omega) + · exact hfit0 + +theorem decodeOp_eq_step {U V n_max : ℕ} {s : Dictionary.State U} + (op : Dictionary.Op U) (regs : Fin 12 → ℕ) + (h : regs regOp = opOutWord s op) : + decodeOp (V := V) (n_max := n_max) op regs = + some ((Dictionary.spec U V n_max).step s op).2 := by + cases op with + | get u => + dsimp only [decodeOp, Dictionary.spec, opOutWord] at h ⊢ + rw [h] + cases s u with + | none => simp + | some v => simp + | insert u v => + rfl + +theorem step_exec_spec {w base U V n_max : ℕ} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) (hV : Fits w V) + {s : Dictionary.State U} (o : Dictionary.Op U) + (ν : PMF (Seed U n_max × Data 12)) + (hv : (Dictionary.spec U V n_max).valid s o) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ 1 * seedLaw U n_max θ) + (hsupp : ∀ p ∈ ν.support, Rep w base U V n_max p.1 s p.2 ∧ argMatch o p.2.regs) : + ∃ (μ_step : Seed U n_max × Data 12 → PMF (Seed U n_max × Data 12)) + (T_step : Seed U n_max × Data 12 → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (opCode U n_max) p.2 ((μ_step p).map Prod.snd) (T_step p) ∧ + ∀ q ∈ (μ_step p).support, + decodeOp (V := V) (n_max := n_max) o q.2.regs = + some ((Dictionary.spec U V n_max).step s o).2 ∧ + Rep w base U V n_max q.1 ((Dictionary.spec U V n_max).step s o).1 q.2 ∧ + Modifies (Block.mk base (space U n_max)).footprint p.2.mem q.2.mem) ∧ + (∀ θ', ((ν.bind μ_step).map Prod.fst) θ' ≤ 1 * seedLaw U n_max θ') ∧ + (∑' p, ν p * (T_step p + ∑' q, μ_step p q * ((0 : ℕ) : ℝ≥0∞))) ≤ + (opSteps U n_max : ℝ≥0∞) + ∑' p, ν p * ((0 : ℕ) : ℝ≥0∞) := by + classical + have hEx : ∀ p : Seed U n_max × Data 12, ∃ y : Data 12, + p ∈ ν.support → + ExecWithin w (opCode U n_max) p.2 y (opSteps U n_max) ∧ + Rep w base U V n_max p.1 ((Dictionary.spec U V n_max).step s o).1 y ∧ + y.regs regOp = opOutWord s o ∧ + Modifies (Block.mk base (space U n_max)).footprint p.2.mem y.mem := by + intro p + by_cases hp : p ∈ ν.support + · obtain ⟨hrep, harg1, harg2, harg3⟩ := hsupp p hp + have hO : + (otherOccupiedSlots (extractSlots base U n_max p.2.mem) (opKey o)).card < + numBuckets n_max := + lt_of_le_of_lt (hrep.table_rep.card_otherOccupiedSlots_le (opKey o)) + (lt_of_le_of_lt hrep.dom_le (lt_numBuckets n_max)) + obtain ⟨y, e, hrep_y, hout, hmod⟩ := + opCode_execWithin hS hw hV hrep o hv harg1 harg2 harg3 + exact ⟨y, fun _ => ⟨e.mono (opPathSteps_le_opSteps p.1 _ _ hO), hrep_y, hout, hmod⟩⟩ + · exact ⟨p.2, fun h => False.elim (hp h)⟩ + let yOf : Seed U n_max × Data 12 → Data 12 := fun p => Classical.choose (hEx p) + refine ⟨fun p => PMF.pure (p.1, yOf p), fun _ => (opSteps U n_max : ℝ≥0∞), ?_, ?_, ?_⟩ + · intro p hp + obtain ⟨e, hrep_y, hout, hmod⟩ := Classical.choose_spec (hEx p) hp + refine ⟨?_, ?_⟩ + · simpa [yOf] using ExecCodeDist.of_execWithin e + · intro q hq + rw [PMF.support_pure, Set.mem_singleton_iff] at hq + subst hq + exact ⟨decodeOp_eq_step o (yOf p).regs hout, hrep_y, hmod⟩ + · intro θ' + have hmap : ((ν.bind fun p => PMF.pure (p.1, yOf p)).map Prod.fst) = ν.map Prod.fst := by + rw [PMF.map_bind] + simp only [PMF.map, PMF.pure_bind] + rfl + rw [hmap] + exact hseed θ' + · simp [ENNReal.tsum_mul_right, ν.tsum_coe] + +/-- **Composable randomized Word-RAM `SubroutineDS` certificate over `Basis.exact`** for the +open-addressed permutation-probing dictionary realizing `Dictionary.spec U V n_max`. -/ +noncomputable def subroutineDS {w base U V n_max : ℕ} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) (hV : Fits w V) : + Randomized.SubroutineDS (subroutineCode U n_max) + (Dictionary.spec U V n_max) w base Fin.elim0 (Dictionary.empty U) where + Seed := Seed U n_max + seedLaw := seedLaw U n_max + seedSlack := 1 + space := space U n_max + spanTime := 3 + setupTime := (setupSteps U n_max : ℝ≥0∞) + initPotential := 0 + charge _ _ := opSteps U n_max + rep := Rep w base U V n_max + potential _ _ := 0 + encode := encodeOp + argMatch := argMatch + decode := decodeOp + bounded hrep := hrep.bounded + rep_unique h1 h2 := h1.unique h2 + arguments_fit o hv i _ := arguments_fit hw hV o hv i + encode_matches := encode_matches + rep_supported {θ s x x'} hregs hmem hbdd hrep := by + have hbase : x'.regs regBase = x.regs regBase := hregs regBase rfl + exact hrep.congr hbdd hbase hmem + potential_supported _ _ _ := rfl + clear_nodup := by + dsimp only [subroutineCode] + decide + params_not_clear j := Fin.elim0 j + incCur_eval {x} h1 hfit := by + dsimp only [subroutineCode] at h1 hfit ⊢ + simp [Op.exactBin, Op.eval, StandardOp.evalExact, Data.put, h1, hfit] + span_exec {x} _ hfr _ := by + obtain ⟨x₁, he, hm, hb, hfr'⟩ := allocSpan_execWithin hS hfr + exact ⟨x₁, he, hm, hb, hfr', fun j => Fin.elim0 j⟩ + setup_exec {x₁} hx₁ hb hfr _ hz := + setup_exec_spec hS hw hx₁ hb hfr hz + step_exec {s} o ν hv hseed hsupp := + step_exec_spec hS hw hV o ν hv hseed hsupp + +/-- Standard-basis (`Basis.standard`) randomized `SubroutineDS` certificate for the open-addressed +permutation-probing dictionary. -/ +noncomputable abbrev subroutineDSStandard {w base U V n_max : ℕ} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) (hV : Fits w V) : + Randomized.SubroutineDS (subroutineCode U n_max).toStandard + (Dictionary.spec U V n_max) w base Fin.elim0 (Dictionary.empty U) := + (subroutineDS hS hw hV).toStandard + +/-- **End-to-end multi-operation expected-time theorem (`seq_expected_amortized`)**: +from fresh zeroed memory `[base, base + space U n_max)`, initializing the permutation-probing +dictionary (`initFresh`) and executing the subroutine operation bodies (`ExecSteps`) across any +admissible operation history `ops : List (Dictionary.Op U)` fixed independently of the sampled seed +(oblivious adversary) takes at most `3 + setupSteps U n_max + ops.length * opSteps U n_max` expected +Word-RAM instructions (charging the deterministic full-cycle body bound `opSteps U n_max` per +operation and excluding caller argument delivery, caller control/dispatch loops, and +code/configuration construction). -/ +theorem seq_expected_amortized {w base U V n_max : ℕ} + (hS : Fits w (base + space U n_max)) (hw : 2 * hashBits U n_max ≤ w) (hV : Fits w V) + {x : Data 12} (ops : List (Dictionary.Op U)) + (hx : x.Bounded w) (hfr : x.regs regV = base) + (hz : ∀ a ∈ (Block.mk base (space U n_max)).footprint, x.mem a = 0) + (hv : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops) : + ∃ (ν₀ ν_final : PMF (Seed U n_max × Data 12)) (T_ops : ℝ≥0∞), + ExecCodeDist w (subroutineCode U n_max).initFresh x (ν₀.map Prod.snd) + (subroutineDS (V := V) hS hw hV).initTime ∧ + (subroutineDS (V := V) hS hw hV).ExecSteps (Dictionary.empty U) ops ν₀ ν_final T_ops ∧ + (∀ θ, (ν_final.map Prod.fst) θ ≤ seedLaw U n_max θ) ∧ + (∀ p ∈ ν_final.support, + Rep w base U V n_max p.1 + ((Dictionary.spec U V n_max).run (Dictionary.empty U) ops).1 p.2) ∧ + (subroutineDS (V := V) hS hw hV).initTime + T_ops ≤ + (3 + setupSteps U n_max : ℝ≥0∞) + + ((Dictionary.spec U V n_max).totalCost (fun _ _ => opSteps U n_max) + (Dictionary.empty U) ops : ℝ≥0∞) := by + obtain ⟨ν₀, ν_final, T_ops, he_init, hex_steps, hseed_final, hrep_final, hbound⟩ := + (subroutineDS (V := V) hS hw hV).seq_expected_amortized ops hx hfr (fun j => Fin.elim0 j) hz hv + refine ⟨ν₀, ν_final, T_ops, he_init, hex_steps, ?_, hrep_final, ?_⟩ + · intro θ + have h1 : (ν_final.map Prod.fst) θ ≤ 1 * seedLaw U n_max θ := hseed_final θ + rwa [one_mul] at h1 + · have hinit : (subroutineDS (V := V) hS hw hV).initTime = (3 + setupSteps U n_max : ℝ≥0∞) := rfl + rw [hinit] + have h2 : (3 + (setupSteps U n_max : ℝ≥0∞)) + T_ops + + ∑' q : Seed U n_max × Data 12, ν_final q * ((0 : ℕ) : ℝ≥0∞) ≤ + (3 + (setupSteps U n_max : ℝ≥0∞)) + ((0 : ℕ) : ℝ≥0∞) + + ((Dictionary.spec U V n_max).totalCost (fun _ _ => opSteps U n_max) + (Dictionary.empty U) ops : ℝ≥0∞) := hbound + simpa only [Nat.cast_zero, mul_zero, tsum_zero, add_zero, Nat.cast_add, Nat.cast_ofNat] using h2 + +/-- The first projection of `seedLaw U n_max` is the uniform distribution on multiply-shift +seeds `Fin (2 ^ (hashBits U n_max - 1))`. -/ +theorem map_fst_seedLaw (U n_max : ℕ) : + (seedLaw U n_max).map Prod.fst = PMF.uniformOfFintype (Fin (2 ^ (hashBits U n_max - 1))) := + Cslib.Probability.PMF.uniformOfFintype_prod_map_fst + +/-- **Primary-slot `2 / numBuckets n_max`-universality (`isUniversalHash_primaryHash`)**: +the primary hash `primaryHash θ : Fin U → Fin (numBuckets n_max)` under `seedLaw U n_max` is +`(2 / numBuckets n_max)`-universal. -/ +theorem isUniversalHash_primaryHash (U n_max : ℕ) : + Cslib.Probability.IsUniversalHash (seedLaw U n_max) (primaryHash (U := U) (n_max := n_max)) + (2 / (numBuckets n_max : ℝ≥0∞)) := by + intro u₁ u₂ hne + have h1 := Cslib.Probability.isUniversalHash_multiplyShift + (hashBits_pos U n_max) (bucketBits_le_hashBits U n_max) + (encodeKey n_max u₁) (encodeKey n_max u₂) + (fun h => hne (encodeKey_injective n_max h)) + rw [← map_fst_seedLaw U n_max, PMF.toOuterMeasure_map_apply] at h1 + have heq : (Prod.fst ⁻¹' {σ : Fin (2 ^ (hashBits U n_max - 1)) | + Cslib.Probability.multiplyShift (hashBits U n_max) (bucketBits n_max) σ + (encodeKey n_max u₁) = + Cslib.Probability.multiplyShift (hashBits U n_max) (bucketBits n_max) σ + (encodeKey n_max u₂)}) = + {θ : Seed U n_max | primaryHash θ u₁ = primaryHash θ u₂} := by + ext θ + simp [primaryHash, Fin.ext_iff] + rw [heq] at h1 + have hnum : (2 / (numBuckets n_max : ℝ≥0∞)) = 2 / 2 ^ bucketBits n_max := by + simp [numBuckets] + rw [hnum] + exact h1 + +/-- **Parametric expected-step bound from any expected-probe bound**: +for any permutation distribution `μ : PMF α` and family +`π : α → Equiv.Perm (Fin (numBuckets n_max))` whose expected probe count on table `T` and key `u` +is at most `B`, the expected Word-RAM instruction count of the probe path is at most +`13 * B + (8 * numChars U n_max + 49)`. -/ +theorem expect_opPathSteps_le_of_probeBound {α : Type*} {U n_max : ℕ} + (μ : PMF α) (π : α → Equiv.Perm (Fin (numBuckets n_max))) + (T : Fin (numBuckets n_max) → Option (Fin U × ℕ)) (u : Fin U) (B : ℝ≥0∞) + (hB : (∑' a, μ a * (UniformProbing.probeCount T u (π a) : ℝ≥0∞)) ≤ B) : + (∑' a, μ a * + ((13 * UniformProbing.probeCount T u (π a) + 8 * numChars U n_max + 49 : ℕ) : ℝ≥0∞)) ≤ + 13 * B + (8 * numChars U n_max + 49 : ℝ≥0∞) := by + have hterm : ∀ a : α, + μ a * ((13 * UniformProbing.probeCount T u (π a) + 8 * numChars U n_max + 49 : ℕ) : ℝ≥0∞) = + 13 * (μ a * (UniformProbing.probeCount T u (π a) : ℝ≥0∞)) + + μ a * (8 * numChars U n_max + 49 : ℝ≥0∞) := by + intro a + push_cast + ring + simp_rw [hterm] + rw [tsum_add, ENNReal.tsum_mul_left, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] + gcongr + +/-- **Ideal uniform-probing expected-step corollary (`≤ 8 * numChars U n_max + 75`)**: +when the probe permutation `π` is drawn uniformly from `Equiv.Perm (Fin (numBuckets n_max))` on a +table `T` with at most `n_max` other occupied slots (`(otherOccupiedSlots T u).card ≤ n_max`), the +expected Word-RAM step count `13 * probeCount T u π + 8 * numChars U n_max + 49` is at most +`8 * numChars U n_max + 75` (combining `expect_opPathSteps_le_of_probeBound` with `B = 2` from +`UniformProbing.expect_unsuccessfulProbes_le_two`). -/ +theorem expect_idealProbeSteps_le {U n_max : ℕ} + (T : Fin (numBuckets n_max) → Option (Fin U × ℕ)) (u : Fin U) + (hcard : (UniformProbing.otherOccupiedSlots T u).card ≤ n_max) : + (∑' π : Equiv.Perm (Fin (numBuckets n_max)), + (PMF.uniformOfFintype (Equiv.Perm (Fin (numBuckets n_max)))) π * + ((13 * UniformProbing.probeCount T u π + 8 * numChars U n_max + 49 : ℕ) : ℝ≥0∞)) ≤ + (8 * numChars U n_max + 75 : ℝ≥0∞) := by + have h2m : 2 * (UniformProbing.otherOccupiedSlots T u).card ≤ numBuckets n_max := by + have h4 := four_mul_le_numBuckets n_max + omega + have hB : (∑' π : Equiv.Perm (Fin (numBuckets n_max)), + (PMF.uniformOfFintype (Equiv.Perm (Fin (numBuckets n_max)))) π * + (UniformProbing.probeCount T u π : ℝ≥0∞)) ≤ 2 := + UniformProbing.expect_unsuccessfulProbes_le_two (numBuckets_pos n_max) h2m + have hstep := expect_opPathSteps_le_of_probeBound (U := U) (n_max := n_max) + (PMF.uniformOfFintype (Equiv.Perm (Fin (numBuckets n_max)))) id T u 2 hB + refine hstep.trans ?_ + have heq : (13 : ℝ≥0∞) * 2 + (8 * numChars U n_max + 49 : ℝ≥0∞) = + (8 * numChars U n_max + 75 : ℝ≥0∞) := by + ring + exact le_of_eq heq + +end Cslib.Algorithms.WordRAM.Dictionary.UniformProbing diff --git a/ComplexityTests/WordRAMUniformProbing.lean b/ComplexityTests/WordRAMUniformProbing.lean new file mode 100644 index 000000000..fa57dfdb8 --- /dev/null +++ b/ComplexityTests/WordRAMUniformProbing.lean @@ -0,0 +1,219 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Algorithms.Lean.UniformProbing +public import Complexity.Algorithms.WordRAM.Dictionary.UniformProbing + +/-! +# Tests for Uniform Probing (`Lean.UniformProbing` and `WordRAM.Dictionary.UniformProbing`) + +This test file verifies: +1. Concrete open-addressed probing behavior (`tableGet`, `tableInsert`, `runTable`, `probeCount`) + under primary-slot collisions and in-place updates, together with the textbook expected-probe + bounds (`≤ m / (m - n) ≤ 2` for both unsuccessful and successful lookups) and lazy random-oracle + equivalence in `Cslib.Algorithms.Lean.UniformProbing`. +2. Concrete parameter values, full-cycle stride permutations, pathwise Word-RAM execution + (`opCode_execWithin`), composable `SubroutineDS` certificates over `Basis.exact` and + `Basis.standard`, end-to-end multi-operation expected complexity (`seq_expected_amortized`), + primary-slot universality (`isUniversalHash_primaryHash`), and expected-step transfer + (`expect_idealProbeSteps_le`) in `Cslib.Algorithms.WordRAM.Dictionary.UniformProbing`. +-/ + +public section + +namespace ComplexityTests.WordRAMUniformProbing + +open Cslib +open Cslib.Algorithms.Lean +open Cslib.Algorithms.Lean.UniformProbing +open Cslib.Algorithms.WordRAM.Dictionary.UniformProbing +open Cslib.Probability +open Cslib.WordRAM +open Cslib.WordRAM.Randomized +open scoped ENNReal + +/-! ## Part 1: Pure Textbook Uniform Probing (`Cslib.Algorithms.Lean.UniformProbing`) -/ + +/-- A concrete permutation family on `U = 4` keys and `m = 4` slots where keys `0` and `1` both +start at slot `0`, and key `1` swaps slots `1` and `3`. -/ +def testPermFamily : Fin 4 → Equiv.Perm (Fin 4) + | ⟨0, _⟩ => Equiv.refl (Fin 4) + | ⟨1, _⟩ => Equiv.swap (1 : Fin 4) (3 : Fin 4) + | _ => Equiv.refl (Fin 4) + +/-- Concrete trace of `tableInsert`, `tableGet`, and `probeCount` under a primary-slot collision +and an in-place overwrite at full capacity `n_max = 2`. -/ +theorem test_pure_collision_and_overwrite : + let hm : 0 < 4 := by decide + let T₀ := emptyTable 4 4 + let u₀ : Fin 4 := ⟨0, by decide⟩ + let u₁ : Fin 4 := ⟨1, by decide⟩ + let u₂ : Fin 4 := ⟨2, by decide⟩ + let T₁ := tableInsert hm testPermFamily T₀ u₀ 10 + let T₂ := tableInsert hm testPermFamily T₁ u₁ 20 + let T₃ := tableInsert hm testPermFamily T₂ u₁ 99 + -- Initial empty table lookup misses in 1 probe: + tableGet hm testPermFamily T₀ u₀ = none ∧ + probeCount T₀ u₀ (testPermFamily u₀) = 1 ∧ + -- First insertion lands at primary slot 0 in 1 probe: + T₁ ⟨0, by decide⟩ = some (u₀, 10) ∧ + probeCount T₁ u₀ (testPermFamily u₀) = 1 ∧ + -- Colliding key u₁ probes slot 0 (occupied by u₀) and lands at testPermFamily u₁ 1 = 3: + probeCount T₁ u₁ (testPermFamily u₁) = 2 ∧ + T₂ ⟨3, by decide⟩ = some (u₁, 20) ∧ + tableGet hm testPermFamily T₂ u₀ = some 10 ∧ + tableGet hm testPermFamily T₂ u₁ = some 20 ∧ + tableGet hm testPermFamily T₂ u₂ = none ∧ + -- In-place overwrite of u₁ updates slot 3 to 99 and preserves probeCount = 2: + T₃ ⟨3, by decide⟩ = some (u₁, 99) ∧ + tableGet hm testPermFamily T₃ u₁ = some 99 ∧ + probeCount T₃ u₁ (testPermFamily u₁) = 2 := by + decide + +/-- Concrete instantiation of the textbook expected-probe theorems (`≤ 2`) for both unsuccessful +lookups (`[CLRS2009]`, Theorem 11.6 / Corollary 11.7) and successful lookups (`[CLRS2009]`, +Theorem 11.8) on a table of `m = 8` slots with `U = 16` keys and capacity `n_max = 4`. -/ +theorem test_expected_probes_le_two + (ops₁ ops₂ : List (Dictionary.Op 16)) + (u u_abs : Fin 16) (v₀ : ℕ) + (hvh : (Dictionary.spec 16 100 4).ValidHistory (Dictionary.empty 16) + (ops₁ ++ (.insert u v₀ :: ops₂))) + (hu₁ : ((Dictionary.spec 16 100 4).run (Dictionary.empty 16) ops₁).1 u = none) + (habs : + ((Dictionary.spec 16 100 4).run (Dictionary.empty 16) + (ops₁ ++ (.insert u v₀ :: ops₂))).1 u_abs = none) : + let hm : 0 < 8 := by decide + let ops := ops₁ ++ (.insert u v₀ :: ops₂) + let μ := PMF.uniformOfFintype (Fin 16 → Equiv.Perm (Fin 8)) + (∑' π : Fin 16 → Equiv.Perm (Fin 8), + μ π * (probeCount (runTable hm π ops) u (π u) : ℝ≥0∞)) ≤ 2 ∧ + (∑' π : Fin 16 → Equiv.Perm (Fin 8), + μ π * (probeCount (runTable hm π ops) u_abs (π u_abs) : ℝ≥0∞)) ≤ 2 := by + have hm : 0 < 8 := by decide + have h2m : 2 * 4 ≤ 8 := by decide + exact ⟨expect_probeCount_of_mem_dom_le_two hm h2m hvh hu₁, + expect_probeCount_of_eq_none_le_two hm h2m hvh habs⟩ + +/-- Concrete instantiation of the lazy random-oracle equivalence for uniform probing on `U = 16` +keys and `m = 8` slots. -/ +theorem test_lazy_random_oracle_equiv + (prog : OracleComp (fun _ : Fin 16 => Equiv.Perm (Fin 8)) Bool) : + (do + let π ← PMF.uniformOfFintype (Fin 16 → Equiv.Perm (Fin 8)) + OracleComp.interpret (fun u => PMF.pure (π u)) prog) = + ((OracleComp.interpret + (Cslib.RandomOracle.answer (fun _ : Fin 16 => PMF.uniformOfFintype (Equiv.Perm (Fin 8)))) + prog).run (fun _ => none)).map Prod.fst := + uniform_interpret_eq_lazy_perm prog + +/-! ## Part 2: Word-RAM Realization (`Cslib.Algorithms.WordRAM.Dictionary.UniformProbing`) -/ + +/-- Verify concrete table dimension parameters at `U = 16`, `n_max = 2`. -/ +theorem test_wordram_params : + bucketBits 2 = 3 ∧ + numBuckets 2 = 8 ∧ + hashBits 16 2 = 4 ∧ + charBits 2 = 1 ∧ + numChars 16 2 = 4 ∧ + tabSpace 16 2 = 12 ∧ + slotOffset 16 2 = 13 ∧ + space 16 2 = 37 ∧ + setupSteps 16 2 = 65 ∧ + opSteps 16 2 = 185 := by + decide + +/-- For every seed `θ : Seed 16 2` and key `u : Fin 16`, `probePerm θ u` is a bijection on the `8` +table slots, and its `0`-th probe slot is `primaryHash θ u`. -/ +theorem test_wordram_probePerm_bijective (θ : Seed 16 2) (u : Fin 16) : + Function.Bijective (probePerm θ u) ∧ + probePerm θ u ⟨0, by decide⟩ = primaryHash θ u := + ⟨(probePerm θ u).bijective, probePerm_zero θ u⟩ + +/-- Concrete admissible operation sequence on `U = 16`, `V = 100`, `n_max = 2` exercising fresh +insertions up to full capacity `n_max = 2`, an in-place overwrite at full capacity, successful +lookups, and an unsuccessful lookup of an absent key. -/ +def testOps : List (Dictionary.Op 16) := + [ .insert ⟨3, by decide⟩ 42, + .insert ⟨11, by decide⟩ 77, + .insert ⟨3, by decide⟩ 88, + .get ⟨3, by decide⟩, + .get ⟨11, by decide⟩, + .get ⟨5, by decide⟩ ] + +theorem testOps_validHistory : + (Dictionary.spec 16 100 2).ValidHistory (Dictionary.empty 16) testOps := by + dsimp only [testOps, Cslib.Stateful.Spec.ValidHistory, Dictionary.spec] + decide + +/-- Concrete instantiation of the `Basis.exact` and `Basis.standard` `SubroutineDS` certificates +and end-to-end `seq_expected_amortized` bound at `w = 64`, `base = 16`, `U = 16`, `V = 100`, +`n_max = 2`. -/ +theorem test_wordram_seq_expected_amortized : + have hS : Fits 64 (16 + space 16 2) := by decide + have hw : 2 * hashBits 16 2 ≤ 64 := by decide + have hV : Fits 64 100 := by decide + let x₀ : Data 12 := (⟨fun _ => 0, fun _ => 0⟩ : Data 12).set regV 16 + ∃ (ν₀ ν_final : PMF (Seed 16 2 × Data 12)) (T_ops : ℝ≥0∞), + ExecCodeDist 64 (subroutineCode 16 2).initFresh x₀ (ν₀.map Prod.snd) + (subroutineDS (V := 100) hS hw hV).initTime ∧ + (subroutineDS (V := 100) hS hw hV).ExecSteps (Dictionary.empty 16) testOps ν₀ ν_final + T_ops ∧ + (∀ θ, (ν_final.map Prod.fst) θ ≤ seedLaw 16 2 θ) ∧ + (∀ p ∈ ν_final.support, + Rep 64 16 16 100 2 p.1 + ((Dictionary.spec 16 100 2).run (Dictionary.empty 16) testOps).1 p.2) ∧ + (subroutineDS (V := 100) hS hw hV).initTime + T_ops ≤ (1178 : ℝ≥0∞) := by + have hS : Fits 64 (16 + space 16 2) := by decide + have hw : 2 * hashBits 16 2 ≤ 64 := by decide + have hV : Fits 64 100 := by decide + let x₀ : Data 12 := (⟨fun _ => 0, fun _ => 0⟩ : Data 12).set regV 16 + have hx₀ : x₀.Bounded 64 := by + refine ⟨fun i => ?_, fun a => ?_⟩ + · fin_cases i <;> decide + · change Fits 64 0 + exact fits_zero 64 + have hfr : x₀.regs regV = 16 := rfl + have hz : ∀ a ∈ (Block.mk 16 (space 16 2)).footprint, x₀.mem a = 0 := fun _ _ => rfl + obtain ⟨ν₀, ν_final, T_ops, he_init, hex_steps, hseed, hrep, hbound⟩ := + seq_expected_amortized (V := 100) hS hw hV testOps hx₀ hfr hz testOps_validHistory + refine ⟨ν₀, ν_final, T_ops, he_init, hex_steps, hseed, hrep, hbound.trans ?_⟩ + have htot : (3 + setupSteps 16 2 : ℝ≥0∞) + + ((Dictionary.spec 16 100 2).totalCost (fun _ _ => opSteps 16 2) + (Dictionary.empty 16) testOps : ℝ≥0∞) = 1178 := by + have h1 : setupSteps 16 2 = 65 := by decide + have h2 : (Dictionary.spec 16 100 2).totalCost (fun _ _ => opSteps 16 2) + (Dictionary.empty 16) testOps = 1110 := by decide + rw [h1, h2] + norm_num + exact le_of_eq htot + +/-- Concrete instantiation of the standard-basis (`Basis.standard`) certificate, primary-slot +`1/4`-universality (`2 / 8`), and ideal uniform-probing expected-step bound (`≤ 107` steps) at +`U = 16`, `n_max = 2`. -/ +theorem test_wordram_standard_and_collision_bounds : + have hS : Fits 64 (16 + space 16 2) := by decide + have hw : 2 * hashBits 16 2 ≤ 64 := by decide + have hV : Fits 64 100 := by decide + (subroutineDSStandard (V := 100) hS hw hV).space = 37 ∧ + Cslib.Probability.IsUniversalHash (seedLaw 16 2) (primaryHash (U := 16) (n_max := 2)) + (2 / (8 : ℝ≥0∞)) ∧ + ∀ (T : Fin (numBuckets 2) → Option (Fin 16 × ℕ)) (u : Fin 16), + (otherOccupiedSlots T u).card ≤ 2 → + (∑' π : Equiv.Perm (Fin (numBuckets 2)), + (PMF.uniformOfFintype (Equiv.Perm (Fin (numBuckets 2)))) π * + ((13 * probeCount T u π + 8 * numChars 16 2 + 49 : ℕ) : ℝ≥0∞)) ≤ 107 := by + refine ⟨by decide, isUniversalHash_primaryHash 16 2, fun T u hcard => ?_⟩ + have h := expect_idealProbeSteps_le T u hcard + have heq : (8 * numChars 16 2 + 75 : ℝ≥0∞) = 107 := by + have hc : numChars 16 2 = 4 := by decide + rw [hc] + norm_num + rwa [heq] at h + +end ComplexityTests.WordRAMUniformProbing From 9f9a7b123a9d1859c7a3d127a45ee59d1fa976ba Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Fri, 30 Oct 2026 02:54:56 +0000 Subject: [PATCH 18/18] docs(WordRAM/Dictionary): clarify CLRS 3rd-ed locators and Theorem 11.8 per-key bound --- .../Algorithms/Lean/UniformProbing.lean | 103 +++++++++++++----- .../WordRAM/Dictionary/UniformProbing.lean | 11 +- ComplexityTests/WordRAMUniformProbing.lean | 11 +- 3 files changed, 87 insertions(+), 38 deletions(-) diff --git a/Complexity/Algorithms/Lean/UniformProbing.lean b/Complexity/Algorithms/Lean/UniformProbing.lean index 3f7c56a26..123f675b7 100644 --- a/Complexity/Algorithms/Lean/UniformProbing.lean +++ b/Complexity/Algorithms/Lean/UniformProbing.lean @@ -18,17 +18,45 @@ import Mathlib.Topology.Instances.ENNReal.Lemmas /-! # Uniform Probing in Open-Addressed Hash Tables -This file formalizes the classical uniform-probing model of open-addressed hashing -([Cormen, Leiserson, Rivest, and Stein, Section 11.4, Theorems 11.6 and 11.8][CLRS2009]), -realizing the bounded-capacity dictionary specification `Cslib.Algorithms.Lean.Dictionary.spec` -over `m` table slots (`2 * n_max ≤ m`). - -In the textbook uniform-probing model, each key `u : Fin U` is assigned an independent uniform -random permutation `π u : Equiv.Perm (Fin m)` (`PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`), -which may equivalently be sampled lazily on demand via -`Cslib.RandomOracle.uniform_interpret_eq_lazy` +This file formalizes the classical uniform-probing model of open-addressed hashing from +[Cormen, Leiserson, Rivest, and Stein, *Introduction to Algorithms*, third edition (MIT Press, +2009), Section 11.4 ("Open addressing", pp. 270–277)][CLRS2009], realizing the bounded-capacity +insert-only dictionary specification `Cslib.Algorithms.Lean.Dictionary.spec` over `m` table slots +(`2 * n_max ≤ m`). + +In the textbook uniform-probing ("uniform hashing") model, each key `u : Fin U` is assigned an +independent uniform random permutation `π u : Equiv.Perm (Fin m)` +(`PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`), which may equivalently be sampled lazily on +demand via `Cslib.RandomOracle.uniform_interpret_eq_lazy` (`Complexity/Languages/Probabilistic/RandomOracle.lean`). +## Source statements and notation mapping + +In [CLRS2009], third edition, Section 11.4: +* An open-address table has `m` slots (`Fin m`) and holds `n` stored elements (`O.card` or + `(Dictionary.dom s).card`, bounded by `n_max` with `2 * n_max ≤ m`), giving load factor + `α = n / m < 1` (`α ≤ 1 / 2` when `2 * n ≤ m`). +* Under uniform hashing (p. 271), each key's probe sequence `h(k, 0), h(k, 1), …, h(k, m - 1)` is + equally likely to be any of the `m!` permutations of `{0, 1, …, m - 1}` (`Equiv.Perm (Fin m)`). + Every search or insertion counts all inspected slots along the probe sequence up to and including + the stopping slot (`unsuccessfulProbes O σ = missedProbes O σ + 1` and + `probeCount T u σ = unsuccessfulProbes (otherOccupiedSlots T u) σ`). +* **Theorem 11.6 (pp. 274–275)** states that in an open-address hash table with load factor + `α = n / m < 1`, the expected number of probes in an unsuccessful search is at most + `1 / (1 - α) = m / (m - n)` under uniform hashing; its proof establishes the exact occupancy tail + product `Pr{X ≥ i} = ∏_{j = 0}^{i - 2} ((n - j) / (m - j)) ≤ α ^ (i - 1)`. +* **Corollary 11.7 (p. 276)** states that inserting an element into an open-address hash table with + load factor `α < 1` requires at most `1 / (1 - α) = m / (m - n)` probes on average. +* **Theorem 11.8 (p. 276)** states that when each key in the table is equally likely to be searched + for, the expected number of probes in a successful search is at most `(1 / α) ln(1 / (1 - α))`. + Its proof first establishes for a fixed stored key `u` that searching for `u` follows the exact + same probe sequence as when `u` was inserted (`probeCount_runTable_eq_firstInsert`), so if `u` was + first inserted after a prefix `ops₁` reaching state `s₁` with `i = (Dictionary.dom s₁).card` keys, + the expected number of probes for `u` is at most `1 / (1 - i / m) = m / (m - i)` + (`expect_probeCount_of_mem_dom_le`, and `≤ 2` when `2 * n_max ≤ m` via + `expect_probeCount_of_mem_dom_le_two`). The uniform-key-averaged logarithmic endpoint + `(1 / α) ln(1 / (1 - α))` is not formalized here. + ## Main definitions * `Cslib.Algorithms.Lean.UniformProbing.allOccupied`, @@ -57,18 +85,25 @@ which may equivalently be sampled lazily on demand via `n = O.card ≤ m` occupied slots, the probability over `σ ← PMF.uniformOfFintype (Equiv.Perm (Fin m))` that the first `k ≤ m` probes all hit `O` (`k < unsuccessfulProbes O σ`) is - `∏ i ∈ Finset.range k, ((n - i : ℝ≥0∞) / (m - i : ℝ≥0∞)) ≤ ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k`. + `∏ i ∈ Finset.range k, ((n - i : ℝ≥0∞) / (m - i : ℝ≥0∞)) ≤ ((n : ℝ≥0∞) / (m : ℝ≥0∞)) ^ k` + ([CLRS2009], third edition, proof of Theorem 11.6, pp. 274–275). * `Cslib.Algorithms.Lean.UniformProbing.expect_unsuccessfulProbes_le`, `Cslib.Algorithms.Lean.UniformProbing.expect_unsuccessfulProbes_le_two`, - `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_eq_none_le`: Theorem 11.6 and - Corollary 11.7 of [CLRS2009] — an unsuccessful lookup or fresh-key insertion into a table holding - `n < m` keys makes at most `m / (m - n)` probes in expectation (`≤ 2` when `2 * n ≤ m`). + `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_eq_none_le`, + `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_eq_none_le_two`: Theorem 11.6 + (pp. 274–275) and Corollary 11.7 (p. 276) of [CLRS2009], third edition — an unsuccessful lookup or + fresh-key insertion into a table holding `n < m` keys makes at most `m / (m - n)` probes in + expectation (`≤ 2` when `2 * n ≤ m`). * `Cslib.Algorithms.Lean.UniformProbing.TableRep.probeCount_tableInsert_of_isSome`, `Cslib.Algorithms.Lean.UniformProbing.probeCount_runTable_eq_firstInsert`, - `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_mem_dom_le`: Theorem 11.8 of - [CLRS2009] — a successful lookup of a present key `u ∈ dom s` makes the exact same number of - probes as the operation that first inserted `u`, which is at most `m / (m - n) ≤ 2` in - expectation. + `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_mem_dom_le`, + `Cslib.Algorithms.Lean.UniformProbing.expect_probeCount_of_mem_dom_le_two`: pathwise + insertion-cost identity and per-key intermediate bound from the proof of Theorem 11.8 (p. 276) of + [CLRS2009], third edition — a successful lookup of a fixed present key `u ∈ dom s` makes the + exact same number of probes as the operation that first inserted `u` after prefix `ops₁` + (reaching state `s₁`), which is at most `m / (m - (Dictionary.dom s₁).card) ≤ 2` in expectation + when `2 * n_max ≤ m` (the uniform-key-averaged `(1 / α) ln(1 / (1 - α))` endpoint of Theorem 11.8 + is not formalized here). * `Cslib.Algorithms.Lean.UniformProbing.uniform_interpret_eq_lazy_perm`: eager sampling of the independent permutation family `Fin U → Equiv.Perm (Fin m)` agrees in distribution with lazy on-demand oracle evaluation (`RandomOracle.uniform_interpret_eq_lazy`). @@ -80,13 +115,14 @@ Storing an explicit table of independent permutations in `(Equiv.Perm (Fin m)) ^ `O(n_max)`-word self-contained Word-RAM dictionary, `Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean` pairs a 2-universal multiply-shift primary hash `h₁(u) ∈ Fin (2 ^ M)` with a tabulated odd stride `h₂(u) ∈ Fin (2 ^ M)` -([CLRS2009], Section 11.4, "Double hashing"), reusing the open-addressed table invariants proved -here. +([CLRS2009], third edition, Section 11.4, "Double hashing", pp. 272–274), reusing the +open-addressed table invariants proved here. ## References * [T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, - *Introduction to Algorithms*, Section 11.4][CLRS2009] + *Introduction to Algorithms*, third edition (MIT Press, 2009), Section 11.4, + pp. 270–277][CLRS2009] * [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] -/ @@ -670,7 +706,7 @@ theorem expect_unsuccessfulProbes_le_two {m : ℕ} {O : Finset (Fin m)} (expect_unsuccessfulProbes_le (by omega)).trans (div_sub_le_two hm h2n) -/-! ## Section 3: Open-Addressed Table Model & Theorem 11.8 ([CLRS2009], Section 11.4) -/ +/-! ## Section 3: Open-Addressed Table Model ([CLRS2009], Section 11.4, pp. 270–276) -/ /-- Pure open-addressed hash table of `m` slots over key universe `Fin U`, where each slot holds `none` (empty) or `some (u, v)` (occupied by key `u` with value `v`). -/ @@ -1159,7 +1195,8 @@ theorem validHistory_and_runTableFrom_append {U V n_max m : ℕ} (hm : 0 < m) (s := ((Dictionary.spec U V n_max).step s o).1) (tableStep hm π T o).1 h.2 exact ⟨⟨h.1, h₁⟩, h₂, hrun⟩ -/-- **Pathwise Theorem 11.8 of [CLRS2009] (`probeCount_runTable_eq_firstInsert`)**: +/-- **Pathwise insertion-cost identity from the proof of Theorem 11.8 of [CLRS2009], third edition, +p. 276 (`probeCount_runTable_eq_firstInsert`)**: if `ops = ops₁ ++ (.insert u v₀ :: ops₂)` is a valid operation history where `ops₁` is the prefix before `u` is inserted (`((Dictionary.spec U V n_max).run (Dictionary.empty U) ops₁).1 u = none`), then the number of probes made when searching for `u` after the entire history `ops` equals the @@ -1250,7 +1287,8 @@ theorem not_mem_insert_of_run_eq_none {U V n_max : ℕ} {u : Fin U} : · exact not_mem_insert_of_run_eq_none (s := ((Dictionary.spec U V n_max).step s o).1) hnone v htail -/-- **Theorem 11.6 & Corollary 11.7 of [CLRS2009] over `PMF.uniformOfFintype`**: +/-- **Theorem 11.6 (pp. 274–275) & Corollary 11.7 (p. 276) of [CLRS2009], third edition, over +`PMF.uniformOfFintype`**: for any valid operation history `ops` reaching state `s` with `(Dictionary.dom s).card < m` and any absent key `u` (`s u = none`), the expected number of probes made by an unsuccessful lookup `get u` or fresh-key insertion `insert u v` on `runTable hm π ops` over @@ -1309,7 +1347,8 @@ theorem expect_probeCount_of_eq_none_le {U V n_max m : ℕ} (hm : 0 < m) (hcap : _ = (m : ℝ≥0∞) / ((m - (Dictionary.dom s).card : ℕ) : ℝ≥0∞) := by rw [ENNReal.tsum_mul_right, μ_all.tsum_coe, one_mul] -/-- **Half-load corollary of Theorem 11.6 & Corollary 11.7 ([CLRS2009], Section 11.4)**: +/-- **Half-load corollary of Theorem 11.6 & Corollary 11.7 ([CLRS2009], third edition, Section 11.4, +pp. 274–276)**: when `2 * n_max ≤ m`, any unsuccessful lookup or fresh-key insertion after a valid history `ops` makes at most `2` probes in expectation over `π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`. -/ @@ -1326,12 +1365,15 @@ theorem expect_probeCount_of_eq_none_le_two {U V n_max m : ℕ} (hm : 0 < m) (h2 refine (expect_probeCount_of_eq_none_le hm (by omega) hvh hu (by omega)).trans ?_ exact div_sub_le_two hm (by omega) -/-- **Theorem 11.8 of [CLRS2009] over `PMF.uniformOfFintype` (Expected probes in a successful -search)**: +/-- **Per-key successful-search bound from the proof of Theorem 11.8 of [CLRS2009], third edition, +p. 276**: if `ops = ops₁ ++ (.insert u v₀ :: ops₂)` is a valid operation history where `ops₁` is the prefix before `u` is first inserted (`s₁ u = none`), then the expected number of probes made when -searching for `u` after `ops` equals the expected number of probes when `u` was first inserted -after `ops₁`, and is at most `m / (m - (Dictionary.dom s₁).card) ≤ 2` when `2 * n_max ≤ m`. -/ +searching for the fixed key `u` after `ops` equals the expected number of probes when `u` was first +inserted after `ops₁`, and is at most `m / (m - (Dictionary.dom s₁).card)` (this formalizes the +per-key intermediate bound `1 / (1 - i / m) = m / (m - i)` for `i = (Dictionary.dom s₁).card` from +the proof of Theorem 11.8; the uniform-key-averaged `(1 / α) ln(1 / (1 - α))` endpoint of Theorem +11.8 is not formalized here). -/ theorem expect_probeCount_of_mem_dom_le {U V n_max m : ℕ} (hm : 0 < m) (hcap : n_max ≤ m) {ops₁ ops₂ : List (Dictionary.Op U)} {u : Fin U} {v₀ : ℕ} (hvh : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) @@ -1354,8 +1396,9 @@ theorem expect_probeCount_of_mem_dom_le {U V n_max m : ℕ} (hm : 0 < m) (hcap : simp_rw [hcongr] exact expect_probeCount_of_eq_none_le hm hcap hvh₁ hu₁ hcard₁ -/-- **Half-load corollary of Theorem 11.8 ([CLRS2009], Section 11.4)**: -when `2 * n_max ≤ m`, any successful lookup of a present key `u` (first inserted after prefix +/-- **Half-load per-key successful-search bound from the proof of Theorem 11.8 ([CLRS2009], third +edition, Section 11.4, p. 276)**: +when `2 * n_max ≤ m`, any successful lookup of a fixed present key `u` (first inserted after prefix `ops₁`) makes at most `2` probes in expectation over `π ← PMF.uniformOfFintype (Fin U → Equiv.Perm (Fin m))`. -/ theorem expect_probeCount_of_mem_dom_le_two {U V n_max m : ℕ} (hm : 0 < m) (h2cap : 2 * n_max ≤ m) diff --git a/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean b/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean index c0659b66d..7a7942fd2 100644 --- a/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean +++ b/Complexity/Algorithms/WordRAM/Dictionary/UniformProbing.lean @@ -22,8 +22,9 @@ This file implements a self-contained open-addressed permutation-probing diction `Cslib.Algorithms.Lean.Dictionary.spec U V n_max` (`get` and `insert` up to `n_max` distinct keys) on the Word-RAM (`Cslib.WordRAM.Basis.exact` and `Cslib.WordRAM.Basis.standard`), following the double-hashing construction of -[Cormen, Leiserson, Rivest, and Stein, Section 11.4, "Double hashing"][CLRS2009] with simple -tabulation due to Zobrist (1970) and Pătraşcu and Thorup (2012). +[Cormen, Leiserson, Rivest, and Stein, *Introduction to Algorithms*, third edition (MIT Press, +2009), Section 11.4, "Double hashing", pp. 272–274][CLRS2009] with simple tabulation due to +Zobrist (1970) and Pătraşcu and Thorup (2012). ## Design and reuse of `Cslib.Algorithms.Lean.UniformProbing` @@ -53,7 +54,8 @@ bijectivity (`bijective_probeSeq`) rather than full uniform-permutation independ `expect_idealProbeSteps_le` connect the exact probe-path body cost `13 * probeCount + 8 * numChars U n_max + 49` to `≤ 8 * numChars U n_max + 75` expected instructions under an ideal uniform permutation law -(`Complexity/Algorithms/Lean/UniformProbing.lean`, Theorems 11.6 and 11.8). +(`Complexity/Algorithms/Lean/UniformProbing.lean`, Theorem 11.6, Corollary 11.7, and the per-key +bound from the proof of Theorem 11.8 of [CLRS2009], third edition, pp. 274–276). ## Model, specialization, and cost scope @@ -116,7 +118,8 @@ instructions under an ideal uniform permutation law ## References * [T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, - *Introduction to Algorithms*, Section 11.4][CLRS2009] + *Introduction to Algorithms*, third edition (MIT Press, 2009), Section 11.4, + pp. 270–277][CLRS2009] * [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] -/ diff --git a/ComplexityTests/WordRAMUniformProbing.lean b/ComplexityTests/WordRAMUniformProbing.lean index fa57dfdb8..bea269a9d 100644 --- a/ComplexityTests/WordRAMUniformProbing.lean +++ b/ComplexityTests/WordRAMUniformProbing.lean @@ -16,7 +16,9 @@ public import Complexity.Algorithms.WordRAM.Dictionary.UniformProbing This test file verifies: 1. Concrete open-addressed probing behavior (`tableGet`, `tableInsert`, `runTable`, `probeCount`) under primary-slot collisions and in-place updates, together with the textbook expected-probe - bounds (`≤ m / (m - n) ≤ 2` for both unsuccessful and successful lookups) and lazy random-oracle + bounds (`≤ m / (m - n) ≤ 2` for unsuccessful lookups by [CLRS2009], third edition, Theorem 11.6 + and Corollary 11.7, pp. 274–276, and for fixed-key successful lookups via the insertion-cost + identity and intermediate bound from the proof of Theorem 11.8, p. 276) and lazy random-oracle equivalence in `Cslib.Algorithms.Lean.UniformProbing`. 2. Concrete parameter values, full-cycle stride permutations, pathwise Word-RAM execution (`opCode_execWithin`), composable `SubroutineDS` certificates over `Basis.exact` and @@ -76,9 +78,10 @@ theorem test_pure_collision_and_overwrite : probeCount T₃ u₁ (testPermFamily u₁) = 2 := by decide -/-- Concrete instantiation of the textbook expected-probe theorems (`≤ 2`) for both unsuccessful -lookups (`[CLRS2009]`, Theorem 11.6 / Corollary 11.7) and successful lookups (`[CLRS2009]`, -Theorem 11.8) on a table of `m = 8` slots with `U = 16` keys and capacity `n_max = 4`. -/ +/-- Concrete instantiation of the textbook expected-probe bounds (`≤ 2`) for both unsuccessful +lookups (`[CLRS2009]`, third edition, Theorem 11.6 / Corollary 11.7, pp. 274–276) and fixed-key +successful lookups (via the insertion-cost identity and intermediate bound from the proof of +Theorem 11.8, p. 276) on a table of `m = 8` slots with `U = 16` keys and capacity `n_max = 4`. -/ theorem test_expected_probes_le_two (ops₁ ops₂ : List (Dictionary.Op 16)) (u u_abs : Fin 16) (v₀ : ℕ)