From 874872f77d1d5c4cb709661a259dedb0637d338b Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 15:59:05 +0000 Subject: [PATCH 01/18] feat(stateful,word-ram): add relational specifications and subroutine certificates --- Complexity.lean | 2 + .../Computability/WordRAM/RelSubroutine.lean | 1008 ++++++++++++++ .../Semantics/Stateful/Relational.lean | 1181 +++++++++++++++++ ComplexityTests/StatefulRelational.lean | 241 ++++ ComplexityTests/WordRAMRelSubroutine.lean | 351 +++++ 5 files changed, 2783 insertions(+) create mode 100644 Complexity/Computability/WordRAM/RelSubroutine.lean create mode 100644 Complexity/Foundations/Semantics/Stateful/Relational.lean create mode 100644 ComplexityTests/StatefulRelational.lean create mode 100644 ComplexityTests/WordRAMRelSubroutine.lean diff --git a/Complexity.lean b/Complexity.lean index a53d95e8e..5c5dbb976 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -2014,6 +2014,7 @@ public import Complexity.Computability.WordRAM.Randomized.Expected public import Complexity.Computability.WordRAM.Randomized.LoopSegment public import Complexity.Computability.WordRAM.Randomized.Structured public import Complexity.Computability.WordRAM.RegisterFrame +public import Complexity.Computability.WordRAM.RelSubroutine public import Complexity.Computability.WordRAM.Sequence public import Complexity.Computability.WordRAM.Signed.Arithmetic public import Complexity.Computability.WordRAM.Signed.Comparison @@ -2634,6 +2635,7 @@ public import Complexity.Foundations.Semantics.Stateful.CostHoare public import Complexity.Foundations.Semantics.Stateful.Hoare public import Complexity.Foundations.Semantics.Stateful.Module public import Complexity.Foundations.Semantics.Stateful.Realization +public import Complexity.Foundations.Semantics.Stateful.Relational public import Complexity.Foundations.Semantics.Stateful.Resource public import Complexity.Init public import Complexity.Languages.Probabilistic.Basic diff --git a/Complexity/Computability/WordRAM/RelSubroutine.lean b/Complexity/Computability/WordRAM/RelSubroutine.lean new file mode 100644 index 000000000..9b92e4dee --- /dev/null +++ b/Complexity/Computability/WordRAM/RelSubroutine.lean @@ -0,0 +1,1008 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Computability.WordRAM.Subroutine +public import Complexity.Foundations.Semantics.Stateful.Relational + +/-! +# Relational Word-RAM dynamic programs and composable data-structure subroutines + +This file extends `Complexity.Computability.WordRAM.Dynamic` and +`Complexity.Computability.WordRAM.Subroutine` from deterministic specifications (`Stateful.Spec`) +to relational specifications (`Stateful.RelSpec`): + +1. **Standalone relational fixed-program implementations (`RelDynamicProgram`, + `RelDynamicCertificate`):** Generalizes `DynamicProgram` and `DynamicCertificate` to + `D : Stateful.RelSpec`, where `execute` witnesses an angelic outcome `(s', a)` satisfying + `D.step s o s' a`. Because the Word-RAM structured semantics `Exec` is deterministic, + `RelDynamicCertificate.realization` establishes `Stateful.RelRealization` directly via + `Exec.unique`. +2. **Unified standalone and inline relational subroutine certificates (`RelSubroutineDS`):** + Mirrors `SubroutineDS` over `D : Stateful.RelSpec`, omitting `rep_unique` (since concrete + potential `potential : Data r_ds → ℕ` depends only on machine data) and existentially + quantifying over `(s', a)` with `D.step s o s' a` in `step_exec`. +3. **Zero-cost embedding of deterministic subroutines (`SubroutineDS.toRelSubroutineDS`):** + Every `SubroutineDS P D w base params s₀` embeds into + `RelSubroutineDS P D.toRelSpec w base params s₀`. +4. **Basis lifting, standalone `Stateful` projections, inline subroutine calls, and flat-program + operation sequences:** Provides `mapSimulates`, `toStandard`, `toRelDynamicProgram`, + `toRelCertificate`, `toRelAmortized`, `toRelBound`, `run_client_amortized`, `run_client_bound`, + `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, + `initClear_rename`, `code_rename`, `call_rename`, `frame_caller`, and flat-program sequence + execution (`RunsRelOps`, `seq_runsRelOps`, `run_ops_exact_fresh`, `run_ops_exact_reusable`). + +## Main definitions + +* `Cslib.WordRAM.RelDynamicProgram`, `Cslib.WordRAM.RelDynamicCertificate`: standalone fixed-program + realization and certificate for a relational specification `D : Stateful.RelSpec`. +* `Cslib.WordRAM.RelSubroutineDS`: unified standalone and inline subroutine specification and + amortized resource certificate for `P : SubroutineCode B r_ds k_init` realizing + `D : Stateful.RelSpec` (without `rep_unique`). +* `Cslib.WordRAM.SubroutineDS.toRelSubroutineDS`: embedding of a deterministic `SubroutineDS` into + `RelSubroutineDS`. +* `Cslib.WordRAM.RelSubroutineDS.RunsRelOps`: sequential flat-program execution of an operation list + recording the resulting abstract transcript `D.Transcript`. + +## Main results + +* `Cslib.WordRAM.RelDynamicCertificate.realization`, + `Cslib.WordRAM.RelDynamicCertificate.instruction_bound`: standalone `Stateful.RelRealization` + and `Stateful.RelBound` certificates. +* `Cslib.WordRAM.RelSubroutineDS.toRelAmortized`, + `Cslib.WordRAM.RelSubroutineDS.run_client_amortized`: standalone amortized potential certificate + and adaptive-client bound over `Basis.standard`. +* `Cslib.WordRAM.RelSubroutineDS.initFresh_rename`, + `Cslib.WordRAM.RelSubroutineDS.initClear_rename`, + `Cslib.WordRAM.RelSubroutineDS.call_rename`, + `Cslib.WordRAM.RelSubroutineDS.frame_caller`: relocated inline initialization, inline operation + calls, and bidirectional framing. +* `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_fresh`, + `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_reusable`: end-to-end flat-program execution of any + demonically admissible operation sequence `D.ValidOpSeq s₀ ops`. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Chapter 17][CLRS2009] +* [J. C. Reynolds, *Separation Logic: A Logic for Shared Mutable Data Structures*][Reynolds2002] +-/ + +@[expose] public section + +namespace Cslib.WordRAM + +universe u v z + +/-! ## Standalone relational dynamic programs (`RelDynamicProgram`, `RelDynamicCertificate`) -/ + +variable {r : ℕ} {D : Stateful.RelSpec.{u, v, z}} + +/-- A single `.standard`-basis Word-RAM program serving all operations of a relational +specification `D : Stateful.RelSpec` through a fixed register calling convention. -/ +structure RelDynamicProgram (D : Stateful.RelSpec.{u, v, z}) (r : ℕ) where + /-- The same structured program is executed for every operation. -/ + code : Code .standard r + /-- Register returned by the final `halt` instruction. -/ + output : Fin r + /-- Registers overwritten by externally supplied operation arguments. -/ + input : Fin r → Bool + /-- Encoded arguments, independent of the maintained machine state. -/ + encode : D.Op → Fin r → ℕ + /-- Interpretation of the final register tuple as the operation's typed observation. -/ + decode : (o : D.Op) → (Fin r → ℕ) → Option (D.Result o) + +/-- Embed a deterministic `DynamicProgram D r` as a `RelDynamicProgram D.toRelSpec r`. -/ +abbrev DynamicProgram.toRelDynamicProgram {D : Stateful.Spec.{u, v, z}} {r : ℕ} + (P : DynamicProgram D r) : RelDynamicProgram D.toRelSpec r where + code := P.code + output := P.output + input := P.input + encode := P.encode + decode := P.decode + +namespace RelDynamicProgram + +variable (P : RelDynamicProgram D r) + +/-- Deliver operation arguments without changing maintained memory or non-input registers. -/ +def prepare (o : D.Op) (x : Data r) : Data r where + regs i := if P.input i then P.encode o i else x.regs i + mem := x.mem + +/-- Delivering operation arguments preserves the machine memory unchanged. -/ +@[simp] +theorem prepare_mem (o : D.Op) (x : Data r) : (P.prepare o x).mem = x.mem := rfl + +/-- Registers outside the calling convention retain their contents. -/ +theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : D.Op) (x : Data r) : + (P.prepare o x).regs i = x.regs i := by + simp [prepare, hi] + +/-- Argument delivery preserves bounded words when the arguments fit. -/ +theorem prepare_bounded {w : ℕ} {o : D.Op} {x : Data r} (hx : x.Bounded w) + (ho : ∀ i, P.input i = true → Fits w (P.encode o i)) : (P.prepare o x).Bounded w := by + refine ⟨fun i => ?_, hx.2⟩ + change Fits w (if P.input i then P.encode o i else x.regs i) + split + · exact ho i (by assumption) + · exact hx.1 i + +/-- Evaluate one typed call, delivering its arguments and decoding its observation. The returned +cost includes the final `halt`. -/ +def eval (w fuel : ℕ) (o : D.Op) (x : Data r) : Option (Data r × D.Result o × ℕ) := + match P.code.eval w fuel (P.prepare o x) with + | none => none + | some (y, n) => (P.decode o y.regs).map fun a => (y, a, n + 1) + +/-- A bounded machine call with native execution evidence (`DynamicProgram.OperationTrace`). -/ +def ExecOp (w : ℕ) : Stateful.RelExecution D (Data r) DynamicProgram.OperationTrace := + fun x o a y trace => + (P.prepare o x).Bounded w ∧ ∃ n, TraceExec w P.code (P.prepare o x) y n trace.probes ∧ + P.decode o y.regs = some a ∧ trace.instructions = n + 1 + +/-- Successful typed evaluation supplies an exact operation trace when the delivered state is +bounded. -/ +theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result o} + (hb : (P.prepare o x).Bounded w) (h : P.eval w fuel o x = some (y, a, k)) : + ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := by + cases he : P.code.eval w fuel (P.prepare o x) with + | none => simp [eval, he] at h + | some yn => + obtain ⟨z, n⟩ := yn + cases hd : P.decode o z.regs with + | none => simp [eval, he, hd] at h + | some b => + simp only [eval, he, hd, Option.map_some, Option.some.injEq, Prod.mk.injEq] at h + obtain ⟨rfl, rfl, rfl⟩ := h + obtain ⟨probes, hp⟩ := (Exec.of_eval he).exists_trace + exact ⟨probes, hb, n, hp, hd, rfl⟩ + +/-- Every call is an execution of one fixed flat machine program. -/ +theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : + Executes w (P.code.program P.output) ⟨0, P.prepare o x⟩ (y.regs P.output) + trace.instructions := by + obtain ⟨_, n, he, _, ht⟩ := h + rw [ht] + exact he.exec.executes P.output + +/-- A call preserves bounded words. -/ +theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : y.Bounded w := by + obtain ⟨hb, _, he, _, _⟩ := h + exact he.exec.bounded hb + +/-- Calls are deterministic in their final representation, observation, and instruction count. -/ +theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} + {trace trace' : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) + (h' : P.ExecOp w x o a' y' trace') : + y = y' ∧ a = a' ∧ trace.instructions = trace'.instructions := by + obtain ⟨_, _, he, ha, ht⟩ := h + obtain ⟨_, _, he', ha', ht'⟩ := h' + obtain ⟨rfl, rfl⟩ := he.exec.unique he'.exec + exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ + +/-- A call's final state and instruction count agree with any structured execution of the code. -/ +theorem ExecOp.eq_of_exec {P : RelDynamicProgram D r} {w : ℕ} {x y z : Data r} {o : D.Op} + {a : D.Result o} {trace : DynamicProgram.OperationTrace} {m : ℕ} + (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : + y = z ∧ trace.instructions = m + 1 := by + obtain ⟨_, _, ht, _, hn⟩ := he + obtain ⟨rfl, rfl⟩ := ht.exec.unique h + exact ⟨rfl, hn⟩ + +/-- The write transcript supplies a frame rule for a call, including register-only delivery. -/ +theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) {Q : HeapAssertion} + (hQ : Q.holds x.mem) (hd : Disjoint Q.footprint (Probe.writes trace.probes)) : + Q.holds y.mem := by + obtain ⟨_, _, he, _, _⟩ := h + exact he.frame hQ hd + +end RelDynamicProgram + +/-- Per-operation angelic machine proof obligations for a `RelDynamicProgram`. -/ +structure RelDynamicCertificate (P : RelDynamicProgram D r) (w : ℕ) where + /-- Abstract state represented by the persistent machine data. -/ + rep : D.State → Data r → Prop + /-- Represented configurations contain only machine words. -/ + bounded : ∀ {s x}, rep s x → x.Bounded w + /-- Admissible arguments fit the machine's word size. -/ + arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (P.encode o i) + /-- A machine execution implements an admissible operation by realizing some permitted + `(s', a)` with `D.step s o s' a`. -/ + execute : ∀ {s x} (o), rep s x → D.valid s o → + ∃ y s' a n, Exec w P.code (P.prepare o x) y n ∧ + D.step s o s' a ∧ P.decode o y.regs = some a ∧ rep s' y + +/-- Embed a deterministic `DynamicCertificate` as a `RelDynamicCertificate`. -/ +def DynamicCertificate.toRelDynamicCertificate {D : Stateful.Spec.{u, v, z}} {r w : ℕ} + {P : DynamicProgram D r} (C : DynamicCertificate P w) : + RelDynamicCertificate P.toRelDynamicProgram w where + rep := C.rep + bounded := C.bounded + arguments_fit := C.arguments_fit + execute o hr hv := by + obtain ⟨y, n, he, hdec, hrep⟩ := C.execute o hr hv + exact ⟨y, (D.step _ o).1, (D.step _ o).2, n, he, rfl, hdec, hrep⟩ + +namespace RelDynamicCertificate + +variable {P : RelDynamicProgram D r} {w : ℕ} (C : RelDynamicCertificate P w) + +/-- Turn a successful typed evaluation on a represented state and admissible operation into +certified `ExecOp` evidence. -/ +theorem execOp_of_eval {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} {fuel k : ℕ} + (hr : C.rep s x) (hv : D.valid s o) (h : P.eval w fuel o x = some (y, a, k)) : + ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := + RelDynamicProgram.ExecOp.of_eval P + (P.prepare_bounded (C.bounded hr) (C.arguments_fit o hv)) h + +/-- Fixed-program relational certificates realize `D` via `Stateful.RelRealization`. -/ +def realization : Stateful.RelRealization D (P.ExecOp w) where + rep := C.rep + total o hr hv := by + obtain ⟨y, _, a, n, he, _, ha, _⟩ := C.execute o hr hv + obtain ⟨ts, hts⟩ := he.exists_trace + exact ⟨a, y, ⟨n + 1, ts⟩, + P.prepare_bounded (C.bounded hr) (C.arguments_fit o hv), n, hts, ha, rfl⟩ + correct hr hv he := by + obtain ⟨y, s', a, n, he', hstep, ha, hr'⟩ := C.execute _ hr hv + obtain ⟨_, _, ht, ha', _⟩ := he + obtain ⟨rfl, rfl⟩ := ht.exec.unique he' + have h_eq : _ = a := Option.some.inj (ha'.symm.trans ha) + subst h_eq + exact ⟨s', hstep, hr'⟩ + +/-- A separate per-operation instruction estimate supplies a reusable `Stateful.RelBound`. -/ +theorem instruction_bound (charge : D.State → D.Op → ℕ) + (hbound : ∀ {s x} (o) {y n}, C.rep s x → D.valid s o → + Exec w P.code (P.prepare o x) y n → n + 1 ≤ charge s o) : + Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions charge := by + intro s x o a y trace hr hv he + obtain ⟨_, n, ht, _, hn⟩ := he + rw [hn] + exact hbound o hr hv ht.exec + +/-- A uniform constant instruction bound follows from an existential per-operation execution. -/ +theorem instruction_bound_const (K : ℕ) + (hbound : ∀ {s x} (o), C.rep s x → D.valid s o → + ∃ y n, Exec w P.code (P.prepare o x) y n ∧ n + 1 ≤ K) : + Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions + (fun _ _ => K) := by + apply C.instruction_bound + intro s x o y n hr hv he + obtain ⟨_, _, he', hn⟩ := hbound o hr hv + obtain ⟨_, rfl⟩ := he.unique he' + exact hn + +/-- Adaptive client execution, final representation, and linear total instruction bound for a +uniform constant per-operation instruction certificate. -/ +theorem run_client_const {K : ℕ} + (B : Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions + (fun _ _ => K)) + {α : Type*} (p : D.Client α) {s : D.State} {x : Data r} + (hr : C.rep s x) (hv : D.DemonicValid s p) : + ∃ a y traces s' tr, + Stateful.RelExecutes (P.ExecOp w) p x a y traces ∧ + D.ClientRun s p a s' tr ∧ + C.rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ tr.length * K := by + obtain ⟨a, y, traces, s', tr, he, hrun, hr', hlen, hcost⟩ := B.exists_executes p hr hv + rw [D.totalCost_const] at hcost + exact ⟨a, y, traces, s', tr, he, hrun, hr', hlen, hcost⟩ + +/-- Lift a per-operation pointwise memory frame from `Exec` to `ExecOp`. -/ +theorem frame_of_execute {j : ℕ} + (hframe : ∀ {s x} (o), C.rep s x → D.valid s o → + ∃ z n, Exec w P.code (P.prepare o x) z n ∧ z.mem j = x.mem j) + {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.valid s o) (he : P.ExecOp w x o a y trace) : + y.mem j = x.mem j := by + obtain ⟨z, n, hz, hf⟩ := hframe o hr hv + exact (he.eq_of_exec hz).1 ▸ hf + +/-- Lift a per-operation `Modifies` bound from `Exec` to `ExecOp`. -/ +theorem modifies_of_execute {F : Finset ℕ} + (hmod : ∀ {s x} (o), C.rep s x → D.valid s o → + ∃ z n, Exec w P.code (P.prepare o x) z n ∧ Modifies F x.mem z.mem) + {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} + {trace : DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.valid s o) (he : P.ExecOp w x o a y trace) : + Modifies F x.mem y.mem := + fun j hj => C.frame_of_execute + (fun o hr' hv' => let ⟨z, n, hz, hf⟩ := hmod o hr' hv'; ⟨z, n, hz, hf j hj⟩) + hr hv he + +/-- Lift a per-operation pointwise memory frame across a demonically admissible adaptive client +execution. -/ +theorem executes_client_frame {j : ℕ} + (hframe : ∀ {s x y o a trace}, C.rep s x → D.valid s o → + P.ExecOp w x o a y trace → y.mem j = x.mem j) + {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} + {traces : List DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.DemonicValid s p) + (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + y.mem j = x.mem j := by + induction he generalizing s with + | pure _ _ => rfl + | @call o k c a c' trace _ _ _ hstep _ ih => + obtain ⟨hv_o, hv_k⟩ := (D.demonicValid_liftBind_iff s o k).mp hv + obtain ⟨s', hstep_d, hr'⟩ := C.realization.correct hr hv_o hstep + exact (ih hr' (hv_k s' a hstep_d)).trans (hframe hr hv_o hstep) + +/-- Lift a per-operation `Modifies` bound across a demonically admissible adaptive client +execution. -/ +theorem executes_client_modifies {F : Finset ℕ} + (hmod : ∀ {s x y o a trace}, C.rep s x → D.valid s o → + P.ExecOp w x o a y trace → Modifies F x.mem y.mem) + {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} + {traces : List DynamicProgram.OperationTrace} + (hr : C.rep s x) (hv : D.DemonicValid s p) + (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + Modifies F x.mem y.mem := + fun j hj => C.executes_client_frame + (fun hr' hv' he' => hmod hr' hv' he' j hj) hr hv he + +/-- Memory probes are bounded by the instruction charge. -/ +theorem probe_bound (charge : D.State → D.Op → ℕ) + (B : Stateful.RelBound C.realization DynamicProgram.OperationTrace.instructions charge) : + Stateful.RelBound C.realization (fun t => t.probes.length) charge := by + intro s x o a y trace hr hv he + have hb := B hr hv he + obtain ⟨_, n, ht, _, hn⟩ := he + exact ht.length_le.trans ((Nat.le_add_right n 1).trans (hn ▸ hb)) + +end RelDynamicCertificate + +/-! ## `RelSubroutineDS` and `SubroutineDS.toRelSubroutineDS` -/ + +namespace SubroutineCode + +variable {B : Basis} {r_ds k_init : ℕ} + +/-- Compiled flat `Program B r_ds` for `O(1)` fresh zeroed-memory initialization. -/ +def initFreshProg (P : SubroutineCode B r_ds k_init) : Program B r_ds := + P.initFresh.program P.output + +/-- Compiled flat `Program B r_ds` for `O(space)` dirty-memory reusable initialization. -/ +def initClearProg (P : SubroutineCode B r_ds k_init) : Program B r_ds := + P.initClear.program P.output + +/-- Compiled flat `Program B r_ds` serving all operations of the data structure. -/ +def opProg (P : SubroutineCode B r_ds k_init) : Program B r_ds := + P.code.program P.output + +end SubroutineCode + +/-- Unified standalone and subroutine specification/resource certificate for +`P : SubroutineCode B r_ds k_init` realizing a relational specification `D : Stateful.RelSpec` +at word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. + +In contrast to `SubroutineDS`, `rep_unique` is omitted (because concrete potential +`potential : Data r_ds → ℕ` depends only on machine data) and `step_exec` existentially +quantifies over a permitted outcome `(s', a)` satisfying `D.step s o s' a`. -/ +structure RelSubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) + (D : Stateful.RelSpec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) where + /-- Contiguous memory footprint length `[base, base + space)`. -/ + space : ℕ + /-- Instruction bound for `P.allocSpan`. -/ + spanTime : ℕ + /-- Instruction bound for `P.initSetup`. -/ + setupTime : ℕ + /-- Upper bound on initial concrete potential after initialization. -/ + initPotential : ℕ + /-- Amortized instruction charge per operation (excluding trailing standalone `halt`). -/ + charge : D.State → D.Op → ℕ + /-- Representation invariant linking mathematical state `s` to concrete `Data r_ds`. -/ + rep : D.State → Data r_ds → Prop + /-- Concrete integer potential depending only on `Data r_ds`. -/ + potential : Data r_ds → ℕ + /-- Standalone register encoding for `RelDynamicProgram.prepare`. -/ + encode : D.Op → Fin r_ds → ℕ + /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ + argMatch : D.Op → (Fin r_ds → ℕ) → Prop + /-- Output decoder reading typed operation observations from final registers. -/ + decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) + /-- Represented configurations are word-bounded. -/ + bounded : ∀ {s x}, rep s x → x.Bounded w + /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ + arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) + /-- Full standalone `prepare` delivery satisfies `argMatch`. -/ + encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), + argMatch o (fun i => if P.input i then encode o i else regs i) + /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. + -/ + rep_supported : ∀ {s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + x'.Bounded w → rep s x → rep s x' + /-- `potential` depends only on persistent registers and `[base, base + space)` under `rep`. -/ + potential_supported : ∀ {s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + rep s x → potential x' = potential x + /-- Distinctness of registers used by `zeroFillCode` relative to `baseReg` and `frontierReg`. -/ + clear_nodup : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup + /-- Initialization parameter registers are not overwritten by `zeroFillCode`. -/ + params_not_clear : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg] + /-- `P.incCur` increments `P.curReg` by `1` when `P.oneReg = 1` and the sum fits in `w` bits. -/ + incCur_eval : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → + P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1)) + /-- Execution of `P.allocSpan`: records `baseReg = base` and advances `frontierReg`. -/ + span_exec : ∀ {x : Data r_ds}, + x.Bounded w → x.regs P.frontierReg = base → (∀ j, x.regs (P.initParams j) = params j) → + ∃ x₁, ExecWithin w P.allocSpan x x₁ spanTime ∧ x₁.mem = x.mem ∧ + x₁.regs P.baseReg = base ∧ x₁.regs P.frontierReg = base + space ∧ + (∀ j, x₁.regs (P.initParams j) = params j) + /-- Execution of `P.initSetup` once `[base, base + space)` holds zeros. -/ + setup_exec : ∀ {x₁ : Data r_ds}, + x₁.Bounded w → x₁.regs P.baseReg = base → x₁.regs P.frontierReg = base + space → + (∀ j, x₁.regs (P.initParams j) = params j) → + (∀ a ∈ (Block.mk base space).footprint, x₁.mem a = 0) → + ∃ y, ExecWithin w P.initSetup x₁ y setupTime ∧ y.regs P.frontierReg = base + space ∧ + rep s₀ y ∧ potential y ≤ initPotential ∧ + Modifies (Block.mk base space).footprint x₁.mem y.mem + /-- Angelic execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`, + witnessing some permitted outcome `(s', a)` with `D.step s o s' a`. -/ + step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), + rep s x → D.valid s o → argMatch o x.regs → + ∃ y s' a n, Exec w P.code x y n ∧ + D.step s o s' a ∧ + decode o y.regs = some a ∧ + rep s' y ∧ + Modifies (Block.mk base space).footprint x.mem y.mem ∧ + n + potential y ≤ charge s o + potential x + +/-- Embed any deterministic `SubroutineDS P D w base params s₀` as a relational +`RelSubroutineDS P D.toRelSpec w base params s₀`. -/ +def SubroutineDS.toRelSubroutineDS {B : Basis} {r_ds k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + (C : SubroutineDS P D w base params s₀) : + RelSubroutineDS P D.toRelSpec w base params s₀ where + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep := C.rep + potential := C.potential + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec := C.setup_exec + step_exec o hr hv harg := by + obtain ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + exact ⟨y, (D.step _ o).1, (D.step _ o).2, n, he, rfl, hdec, hrep, hmod, hpot⟩ + +namespace RelSubroutineDS + +variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.RelSpec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-- Total instruction bound for `P.initFresh`. -/ +def initTime (C : RelSubroutineDS P D w base params s₀) : ℕ := + C.spanTime + C.setupTime + +/-! ### Projection 1: Basis lifting and standalone `RelDynamicProgram` / `Stateful` theorems -/ + +/-- Lift a `RelSubroutineDS` certificate along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : RelSubroutineDS P D w base params s₀) : + RelSubroutineDS (P.mapSimulates φ) D w base params s₀ where + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep := C.rep + potential := C.potential + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval h1 hfit := Op.eval_mapSimulates_of_some φ (C.incCur_eval h1 hfit) + span_exec hx hf hp := + let ⟨x₁, he, hm, hb, hfr, hpar⟩ := C.span_exec hx hf hp + ⟨x₁, he.mapSimulates φ, hm, hb, hfr, hpar⟩ + setup_exec hx₁ hb hfr hpar hz := + let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz + ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ + step_exec o hr hv harg := + let ⟨y, s', a, n, he, hstep, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + ⟨y, s', a, n, he.mapSimulates φ, hstep, hdec, hrep, hmod, hpot⟩ + +/-- Lift an exact-basis `RelSubroutineDS` certificate to `Basis.standard`. -/ +abbrev toStandard {P : SubroutineCode .exact r_ds k_init} + (C : RelSubroutineDS P D w base params s₀) : + RelSubroutineDS P.toStandard D w base params s₀ := + C.mapSimulates .exactStandard + +section Standard + +variable {P_std : SubroutineCode .standard r_ds k_init} + (C : RelSubroutineDS P_std D w base params s₀) + +/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicProgram`. -/ +def toRelDynamicProgram : RelDynamicProgram D r_ds where + code := P_std.code + output := P_std.output + input := P_std.input + encode := C.encode + decode := C.decode + +/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicCertificate`. -/ +def toRelCertificate : RelDynamicCertificate C.toRelDynamicProgram w where + rep := C.rep + bounded := C.bounded + arguments_fit := C.arguments_fit + execute {s x} o hr hv := by + have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := + C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨y, s', a, n, he, hstep, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg + exact ⟨y, s', a, n, he, hstep, hdec, hrep⟩ + +/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `Stateful.RelAmortized` +certificate, charging `+ 1` per operation for `RelDynamicProgram`'s trailing `halt` instruction. -/ +theorem toRelAmortized : + Stateful.RelAmortized C.toRelCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun _ x => C.potential x) + (fun s o => C.charge s o + 1) where + step {s x o a y trace} hr hv he := by + have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := + C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have hp_eq : C.potential (C.toRelDynamicProgram.prepare o x) = C.potential x := + C.potential_supported + (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hr + have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨z, s', a', m, hz, hstep, hdec, hrep, _, hpot⟩ := C.step_exec o hr' hv harg + obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz + obtain ⟨_, _, _, ha_trace, _⟩ := he + have ha_eq : a = a' := Option.some.inj (ha_trace.symm.trans hdec) + subst ha_eq + refine ⟨s', hstep, hrep, ?_⟩ + rw [hinst] + rw [hp_eq] at hpot + omega + +/-- Alias for `toRelAmortized` matching `SubroutineDS.toAmortized`. -/ +abbrev toAmortized := @toRelAmortized + +/-- When `C.potential` is identically zero, `RelSubroutineDS` projects directly to a worst-case +`Stateful.RelBound` certificate. -/ +theorem toRelBound (hpot : ∀ x, C.potential x = 0) : + Stateful.RelBound C.toRelCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun s o => C.charge s o + 1) := by + intro s x o a y trace hr hv he + obtain ⟨_, _, _, h⟩ := C.toRelAmortized.step hr hv he + simp only [hpot] at h + omega + +/-- Alias for `toRelBound` matching `SubroutineDS.toBound`. -/ +abbrev toBound := @toRelBound + +/-- Standalone adaptive-client amortized bound for any demonically admissible client `p`. -/ +theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} + (hr : C.rep s x) (hv : D.DemonicValid s p) : + ∃ a y traces s' tr, + Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + D.ClientRun s p a s' tr ∧ + C.rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces + C.potential y ≤ + D.totalCost (fun s o => C.charge s o + 1) s tr + C.potential x := + C.toRelAmortized.run_client_amortized p hr hv + +/-- Standalone adaptive-client worst-case bound when `C.potential` is identically zero. -/ +theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) + {s : D.State} {x : Data r_ds} (hr : C.rep s x) (hv : D.DemonicValid s p) : + ∃ a y traces s' tr, + Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + D.ClientRun s p a s' tr ∧ + C.rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ + D.totalCost (fun s o => C.charge s o + 1) s tr := + Stateful.RelBound.exists_executes (C.toRelBound hpot) p hr hv + +end Standard + +/-! ### Projection 2: Fresh `O(1)` initialization and register relocation -/ + +/-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ +theorem initFresh_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ y, ExecWithin w P.initFresh x y C.initTime ∧ + y.regs P.frontierReg = base + C.space ∧ + C.rep s₀ y ∧ + C.potential y ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem y.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + have hx₁ : x₁.Bounded w := he₁.bounded hx + have hz₁ : ∀ a ∈ (Block.mk base C.space).footprint, x₁.mem a = 0 := by + intro a ha + rw [hm₁] + exact hz a ha + obtain ⟨y, he₂, hfr₂, hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb₁ hfr₁ hpar₁ hz₁ + rw [hm₁] at hmod + exact ⟨y, he₁.seq he₂, hfr₂, hrep, hpot, hmod⟩ + +/-- Relocated `O(1)` fresh initialization along any injective register embedding `f`. -/ +theorem initFresh_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ z, ExecWithin w (P.initFresh.rename f) x z C.initTime ∧ + z.regs (f P.frontierReg) = base + C.space ∧ + C.rep s₀ (z.comap f) ∧ + C.potential (z.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x.regs j) := by + obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initFresh_exec hx hfr hpar hz + refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, + fun j hj => Data.extend_regs_of_notMem y x hj⟩ + · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`initClear`) -/ + +/-- Auxiliary deterministic `SubroutineDS` sharing `C`'s initialization fields so +`zeroFillCode_exec` is reused without duplicating its loop proof. -/ +private def toInitSubroutineDS (C : RelSubroutineDS P D w base params s₀) : + SubroutineDS P + { State := PUnit.{1}, Op := PUnit.{1}, Result := fun _ => PUnit.{1}, + step := fun _ _ => (⟨⟩, ⟨⟩), valid := fun _ _ => False } + w base params ⟨⟩ where + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge _ _ := 0 + rep _ x := C.rep s₀ x + potential := C.potential + encode _ _ := 0 + argMatch _ _ := True + decode _ _ := some ⟨⟩ + bounded := C.bounded + rep_unique _ _ := rfl + arguments_fit _ hv := False.elim hv + encode_matches _ _ := trivial + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec := C.setup_exec + step_exec _ _ hv := False.elim hv + +/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` +instructions over any basis `B`. -/ +theorem zeroFillCode_exec (C : RelSubroutineDS P D w base params s₀) {x₁ : Data r_ds} + (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) + (hfr₁ : x₁.regs P.frontierReg = base + C.space) + (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : + ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ + x₂.Bounded w ∧ + x₂.regs P.baseReg = base ∧ + x₂.regs P.frontierReg = base + C.space ∧ + (∀ j, x₂.regs (P.initParams j) = params j) ∧ + (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ + Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := + C.toInitSubroutineDS.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ + +/-- Execute `P.initClear` on arbitrary dirty memory in `C.initTime + 4 * C.space + 6` +instructions. -/ +theorem initClear_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) : + ∃ y, ExecWithin w P.initClear x y (C.initTime + 4 * C.space + 6) ∧ + y.regs P.frontierReg = base + C.space ∧ + C.rep s₀ y ∧ + C.potential y ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem y.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := + C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + obtain ⟨y, he₃, hfr₃, hrep, hpot, hmod₃⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ + rw [hm₁] at hmod₂ + have hmod : Modifies (Block.mk base C.space).footprint x.mem y.mem := by + simpa only [Finset.union_idempotent] using hmod₂.trans hmod₃ + exact ⟨y, (he₁.seq (he₂.seq he₃)).mono (by unfold initTime; omega), hfr₃, hrep, hpot, hmod⟩ + +/-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ +theorem initClear_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : + ∃ z, ExecWithin w (P.initClear.rename f) x z (C.initTime + 4 * C.space + 6) ∧ + z.regs (f P.frontierReg) = base + C.space ∧ + C.rep s₀ (z.comap f) ∧ + C.potential (z.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x.regs j) := by + obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initClear_exec hx hfr hpar + refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, + fun j hj => Data.extend_regs_of_notMem y x hj⟩ + · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Projection 4: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ + +/-- Execute `P.code.rename f` inline when arguments already match `(x.comap f).regs`, +witnessing a permitted outcome `(s', a)` with `D.step s o s' a`. -/ +theorem code_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} {x : Data r_caller} (o : D.Op) + (hr : C.rep s (x.comap f)) (hv : D.valid s o) (harg : C.argMatch o (x.comap f).regs) : + ∃ z s' a t, ExecWithin w (P.code.rename f) x z t ∧ + D.step s o s' a ∧ + C.decode o (z.comap f).regs = some a ∧ + C.rep s' (z.comap f) ∧ + Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x.regs j) ∧ + t + C.potential (z.comap f) ≤ C.charge s o + C.potential (x.comap f) := by + obtain ⟨y, s', a, n, he, hstep, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + refine ⟨y.extend f x, s', a, n, (he.rename hf).within, hstep, ?_, ?_, hmod, + fun j hj => Data.extend_regs_of_notMem y x hj, ?_⟩ + · rw [Data.comap_extend hf] + exact hdec + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-- Execute `.seq prep (P.code.rename f)` inline after a caller argument-preparation block `prep` +that preserves persistent DS registers and memory in `[base, base + C.space)`. -/ +theorem call_rename (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} {x x' : Data r_caller} {prep : Code B r_caller} {t_prep : ℕ} (o : D.Op) + (hr : C.rep s (x.comap f)) (hv : D.valid s o) + (hprep : ExecWithin w prep x x' t_prep) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hmem : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a) + (harg : C.argMatch o (x'.comap f).regs) : + ∃ z s' a t, ExecWithin w (.seq prep (P.code.rename f)) x z t ∧ + D.step s o s' a ∧ + C.decode o (z.comap f).regs = some a ∧ + C.rep s' (z.comap f) ∧ + Modifies (Block.mk base C.space).footprint x'.mem z.mem ∧ + (∀ j ∉ Set.range f, z.regs j = x'.regs j) ∧ + t + C.potential (z.comap f) ≤ t_prep + C.charge s o + C.potential (x.comap f) := by + have hr' : C.rep s (x'.comap f) := + C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hmem hb' hr + have hp_eq : C.potential (x'.comap f) = C.potential (x.comap f) := + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hmem hr + obtain ⟨z, s', a, t_op, he_op, hstep, hdec, hrep_z, hmod_z, hregs_z, hpot_z⟩ := + C.code_rename hf o hr' hv harg + exact ⟨z, s', a, t_prep + t_op, hprep.seq he_op, hstep, hdec, hrep_z, hmod_z, hregs_z, + by rw [hp_eq] at hpot_z; omega⟩ + +/-! ### Projection 5: Caller-to-DS bidirectional framing (`frame_caller`) -/ + +/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS +registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from +`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ +theorem frame_caller (C : RelSubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} {s : D.State} {x x' : Data r_caller} {F : Finset ℕ} + (hr : C.rep s (x.comap f)) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hmem : Modifies F x.mem x'.mem) : + C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by + have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => + hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) + exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ + +/-! ### Projection 6: Flat-program operation-sequence execution (`RunsRelOps`, `run_ops_exact_*`) -/ + +/-- Default register-preparation function overwriting `P.input` registers with `C.encode o`. -/ +def prepareRegs (C : RelSubroutineDS P D w base params s₀) (o : D.Op) + (regs : Fin r_ds → ℕ) : Fin r_ds → ℕ := + fun i => if P.input i then C.encode o i else regs i + +/-- Sequential flat-program execution of `ops : List D.Op` by repeated calls to `P.opProg`: +each call prepares input registers via `prepRegs`, runs `P.opProg` from `pc = 0` to `halt`, +decodes a permitted outcome `(s₁, a)` with `D.step s o s₁ a`, modifies memory only within +`[base, base + C.space)`, records the resulting transcript `tr : D.Transcript`, and accumulates +total instructions executed (including each trailing `halt`). -/ +inductive RunsRelOps (C : RelSubroutineDS P D w base params s₀) + (prepRegs : D.Op → (Fin r_ds → ℕ) → (Fin r_ds → ℕ) := C.prepareRegs) : + D.State → Data r_ds → List D.Op → D.Transcript → D.State → Data r_ds → ℕ → Prop + | nil {s x} : RunsRelOps C prepRegs s x [] [] s x 0 + | cons {s s₁ s₂ x o a ops tr pc y z steps rest} : + Runs w P.opProg ⟨0, ⟨prepRegs o x.regs, x.mem⟩⟩ ⟨pc, y⟩ steps → + (∃ out, step w P.opProg ⟨pc, y⟩ = .halt out) → + D.step s o s₁ a → + C.decode o y.regs = some a → + Modifies (Block.mk base C.space).footprint x.mem y.mem → + RunsRelOps C prepRegs s₁ y ops tr s₂ z rest → + RunsRelOps C prepRegs s x (o :: ops) (⟨o, a, s₁⟩ :: tr) s₂ z ((steps + 1) + rest) + +private theorem step_program_halt (c : Code B r_ds) (out : Fin r_ds) (y : Data r_ds) : + step w (c.program out) ⟨0 + c.size, y⟩ = .halt (y.regs out) := + step_halt (by + simp only [Code.program, Nat.zero_add, + List.getElem?_append_right (Code.length_assemble c 0).le, + Code.length_assemble, Nat.sub_self, List.getElem?_cons_zero]) + +/-- Flat-program sequence execution under a custom register-preparation function `prepRegs`. -/ +theorem seq_runsRelOps_of_prep (C : RelSubroutineDS P D w base params s₀) + (prepRegs : D.Op → (Fin r_ds → ℕ) → (Fin r_ds → ℕ)) + (hfit : ∀ {s} (o) (regs : Fin r_ds → ℕ), D.valid s o → (∀ i, Fits w (regs i)) → + ∀ i, Fits w (prepRegs o regs i)) + (hnot_in : ∀ (o : D.Op) (regs : Fin r_ds → ℕ) (i : Fin r_ds), + P.input i = false → prepRegs o regs i = regs i) + (harg : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), C.argMatch o (prepRegs o regs)) : + ∀ (ops : List D.Op) {s : D.State} {x : Data r_ds}, + C.rep s x → D.ValidOpSeq s ops → + ∃ tr s_fin z t_ops, + C.RunsRelOps prepRegs s x ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s tr s_fin ∧ + C.rep s_fin z ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s tr + C.potential x + | [], s, x, hr, _ => + ⟨[], s, x, 0, .nil, rfl, .nil s, hr, Nat.zero_add (C.potential x) ▸ le_rfl⟩ + | o :: ops, s, x, hr, ⟨hv_o, hv_rest⟩ => by + let x' : Data r_ds := ⟨prepRegs o x.regs, x.mem⟩ + have hbx := C.bounded hr + have hb' : x'.Bounded w := ⟨hfit o x.regs hv_o hbx.1, hbx.2⟩ + have hreg' : ∀ i, P.input i = false → x'.regs i = x.regs i := + fun i hi => hnot_in o x.regs i hi + have hr' : C.rep s x' := C.rep_supported hreg' (fun _ _ => rfl) hb' hr + have hpot' : C.potential x' = C.potential x := + C.potential_supported hreg' (fun _ _ => rfl) hr + obtain ⟨y, s₁, a, m, he_op, hstep, hdec, hr_y, hmod, hpot_m⟩ := + C.step_exec o hr' hv_o (harg o x.regs) + rw [hpot'] at hpot_m + have hruns : Runs w P.opProg ⟨0, x'⟩ ⟨0 + P.code.size, y⟩ m := + he_op.runs (pc := 0) (p := P.opProg) (Program.Contains.self _).append_left + have hhalt : step w P.opProg ⟨0 + P.code.size, y⟩ = .halt (y.regs P.output) := + step_program_halt P.code P.output y + obtain ⟨tr, s_fin, z, rest, hrest, hops, htr, hr_z, hpot_rest⟩ := + seq_runsRelOps_of_prep C prepRegs hfit hnot_in harg ops hr_y (hv_rest s₁ a hstep) + refine ⟨⟨o, a, s₁⟩ :: tr, s_fin, z, (m + 1) + rest, + .cons hruns ⟨_, hhalt⟩ hstep hdec hmod hrest, + congrArg (o :: ·) hops, .cons _ hv_o hstep htr, hr_z, ?_⟩ + simp only [Stateful.RelSpec.totalCost_cons] + omega + +/-- Flat-program sequence execution using `C.prepareRegs`. -/ +theorem seq_runsRelOps (C : RelSubroutineDS P D w base params s₀) + (ops : List D.Op) {s : D.State} {x : Data r_ds} + (hr : C.rep s x) (hv : D.ValidOpSeq s ops) : + ∃ tr s_fin z t_ops, + C.RunsRelOps C.prepareRegs s x ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s tr s_fin ∧ + C.rep s_fin z ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s tr + C.potential x := by + refine C.seq_runsRelOps_of_prep C.prepareRegs ?_ ?_ C.encode_matches ops hr hv + · intro s' o regs hv_o hregs i + simp only [prepareRegs] + split_ifs with hi + · exact C.arguments_fit o hv_o i hi + · exact hregs i + · intro o regs i hi + simp [prepareRegs, hi] + +/-- Standalone flat-program operation-sequence theorem with `O(1)` fresh initialization on zeroed +`[base, base + C.space)`. -/ +theorem run_ops_exact_fresh (C : RelSubroutineDS P D w base params s₀) + {d₀ : Data r_ds} (hd₀ : d₀.Bounded w) + (hfr : d₀.regs P.frontierReg = base) + (hpar : ∀ j, d₀.regs (P.initParams j) = params j) + (hzero : ∀ a ∈ (Block.mk base C.space).footprint, d₀.mem a = 0) + (ops : List D.Op) (hv : D.ValidOpSeq s₀ ops) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w P.initFreshProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w P.initFreshProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base C.space).footprint d₀.mem x₀.mem ∧ + C.RunsRelOps C.prepareRegs s₀ x₀ ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s₀ tr s_fin ∧ + C.rep s_fin z ∧ + t_init ≤ C.initTime ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s₀ tr + C.initPotential := by + obtain ⟨x₀, ⟨t_init, ht_init, he_init⟩, _, hr₀, hpot₀, hmod₀⟩ := + C.initFresh_exec hd₀ hfr hpar hzero + have hruns₀ : Runs w P.initFreshProg ⟨0, d₀⟩ ⟨0 + P.initFresh.size, x₀⟩ t_init := + he_init.runs (pc := 0) (p := P.initFreshProg) (Program.Contains.self _).append_left + have hhalt₀ : step w P.initFreshProg ⟨0 + P.initFresh.size, x₀⟩ = .halt (x₀.regs P.output) := + step_program_halt P.initFresh P.output x₀ + obtain ⟨tr, s_fin, z, t_ops, hruns_ops, hops, htr, hr_z, hbound⟩ := + C.seq_runsRelOps ops hr₀ hv + exact ⟨_, x₀, t_init, tr, s_fin, z, t_ops, hruns₀, ⟨_, hhalt₀⟩, hmod₀, hruns_ops, + hops, htr, hr_z, ht_init, by omega⟩ + +/-- Standalone flat-program operation-sequence theorem with `O(space)` reusable initialization on +arbitrary `w`-bounded memory. -/ +theorem run_ops_exact_reusable (C : RelSubroutineDS P D w base params s₀) + {d₀ : Data r_ds} (hd₀ : d₀.Bounded w) + (hfr : d₀.regs P.frontierReg = base) + (hpar : ∀ j, d₀.regs (P.initParams j) = params j) + (ops : List D.Op) (hv : D.ValidOpSeq s₀ ops) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w P.initClearProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w P.initClearProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base C.space).footprint d₀.mem x₀.mem ∧ + C.RunsRelOps C.prepareRegs s₀ x₀ ops tr s_fin z t_ops ∧ + tr.ops = ops ∧ + D.ValidTranscript s₀ tr s_fin ∧ + C.rep s_fin z ∧ + t_init ≤ C.initTime + 4 * C.space + 6 ∧ + t_ops + C.potential z ≤ + D.totalCost (fun st o => C.charge st o + 1) s₀ tr + C.initPotential := by + obtain ⟨x₀, ⟨t_init, ht_init, he_init⟩, _, hr₀, hpot₀, hmod₀⟩ := + C.initClear_exec hd₀ hfr hpar + have hruns₀ : Runs w P.initClearProg ⟨0, d₀⟩ ⟨0 + P.initClear.size, x₀⟩ t_init := + he_init.runs (pc := 0) (p := P.initClearProg) (Program.Contains.self _).append_left + have hhalt₀ : step w P.initClearProg ⟨0 + P.initClear.size, x₀⟩ = .halt (x₀.regs P.output) := + step_program_halt P.initClear P.output x₀ + obtain ⟨tr, s_fin, z, t_ops, hruns_ops, hops, htr, hr_z, hbound⟩ := + C.seq_runsRelOps ops hr₀ hv + exact ⟨_, x₀, t_init, tr, s_fin, z, t_ops, hruns₀, ⟨_, hhalt₀⟩, hmod₀, hruns_ops, + hops, htr, hr_z, ht_init, by omega⟩ + +end RelSubroutineDS + +end Cslib.WordRAM diff --git a/Complexity/Foundations/Semantics/Stateful/Relational.lean b/Complexity/Foundations/Semantics/Stateful/Relational.lean new file mode 100644 index 000000000..40d7761e7 --- /dev/null +++ b/Complexity/Foundations/Semantics/Stateful/Relational.lean @@ -0,0 +1,1181 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Foundations.Semantics.Stateful.Amortized +public import Complexity.Foundations.Semantics.Stateful.CostHoare +public import Cslib.Foundations.Semantics.LTS.Basic + +/-! +# Relational stateful specifications, demonic adaptive clients, and angelic realizations + +A relational stateful specification (`Cslib.Stateful.RelSpec`) generalizes `Cslib.Stateful.Spec` +from a deterministic transition function `step : State → (o : Op) → State × Result o` to a +transition relation `step : State → (o : Op) → State → Result o → Prop`. This allows data-structure +specifications to leave implementation choices (such as which free slot or handle is allocated, or +how equal-key ties are broken) underspecified in the abstract interface while keeping internal +allocator state hidden inside concrete representation invariants. + +The semantics is **demonic** for callers and **angelic** for implementations: +* An adaptive client `p : D.Client α` is demonically admissible (`D.DemonicValid s p`) when each + issued operation is admissible (`D.valid`) and the continuation remains admissible under *every* + outcome `(s', a)` permitted by `D.step`. Caller postconditions (`D.Hoare`) and abstract cost + budgets (`D.CostHoare`) hold across *all* permitted client runs (`D.ClientRun`). +* A concrete backend realization (`RelRealization`, `RelAmortized`) is angelic in the abstract + post-state: each concrete operation execution witnesses *some* permitted abstract transition + `D.step s o s' a` maintaining the representation invariant and potential inequality. + +Every deterministic `D : Spec` embeds into `RelSpec` with zero overhead via `Spec.toRelSpec` +(preserving `State`, `Op`, `Result`, `valid`, `signature`, and `Client α` definitionally), and +`DemonicValid`, `ClientRun`, `Hoare`, and `CostHoare` on `D.toRelSpec` are equivalent to their +deterministic counterparts on `D`. + +## Main definitions + +* `Cslib.Stateful.RelSpec`: relational stateful specification with `State`, `Op`, `Result`, + transition relation `step`, and input admissibility `valid`. +* `Cslib.Stateful.RelSpec.Total`: typeclass asserting that every admissible `(s, o)` has at least + one permitted outcome `(s', a)`. +* `Cslib.Stateful.Spec.toRelSpec`: zero-cost embedding of a deterministic `Spec` into `RelSpec`. +* `Cslib.Stateful.RelSpec.toLTS`: projection of a `RelSpec` to a `Cslib.LTS` labelled by + input-output pairs `Σ o : D.Op, D.Result o`. +* `Cslib.Stateful.RelSpec.Step`, `Cslib.Stateful.RelSpec.Transcript`, + `Cslib.Stateful.RelSpec.ValidTranscript`, `Cslib.Stateful.RelSpec.ValidOpSeq`, + `Cslib.Stateful.RelSpec.totalCost`, `Cslib.Stateful.RelSpec.totalCostOutcome`: execution + transcripts, non-adaptive operation-sequence admissibility, and abstract cost accounting. +* `Cslib.Stateful.RelSpec.ClientRun`, `Cslib.Stateful.RelSpec.DemonicValid`, + `Cslib.Stateful.RelSpec.Hoare`, `Cslib.Stateful.RelSpec.CostHoare`: adaptive client runs, + demonic admissibility, and compositional Hoare / CostHoare triples. +* `Cslib.Stateful.RelExecution`, `Cslib.Stateful.RelExecutes`, + `Cslib.Stateful.RelRealization`, `Cslib.Stateful.RelBound`, `Cslib.Stateful.RelAmortized`: + native execution traces, angelic representation simulation, and potential-method certificates. + +## Main results + +* `Cslib.Stateful.Spec.toRelSpec_demonicValid_iff`, + `Cslib.Stateful.Spec.toRelSpec_clientRun_iff`, + `Cslib.Stateful.Spec.toRelSpec_hoare_iff`, + `Cslib.Stateful.Spec.toRelSpec_costHoare_iff`: exact collapse of relational client semantics to + deterministic `Spec` semantics on `D.toRelSpec`. +* `Cslib.Stateful.RelRealization.correct_client`, + `Cslib.Stateful.RelRealization.exists_executes`: soundness and existence of native client + executions under `DemonicValid`. +* `Cslib.Stateful.RelAmortized.executes_le`, `Cslib.Stateful.RelAmortized.run_client_amortized`: + amortized potential telescoping along adaptive client executions. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Chapter 17][CLRS2009] +* [D. Sangiorgi, *Introduction to Bisimulation and Coinduction*][Sangiorgi2011] +-/ + +@[expose] public section + +namespace Cslib.Stateful + +universe u v w x y + +/-! ## Core relational specification and totality -/ + +/-- A relational stateful specification with state type `State`, operation type `Op`, +operation-dependent observation family `Result`, transition relation `step`, and input +admissibility predicate `valid`. -/ +structure RelSpec where + /-- Mathematical states, independent of a concrete storage representation. -/ + State : Type u + /-- Operations available to clients. -/ + Op : Type v + /-- The observation type returned by each operation. -/ + Result : Op → Type w + /-- Relational step specification: `step s o s' a` holds when transitioning from `s` to `s'` + and returning `a : Result o` is a permitted outcome of `o`. -/ + step : State → (o : Op) → State → Result o → Prop + /-- Preconditions under which an operation's implementation must satisfy its contract. -/ + valid : State → Op → Prop := fun _ _ => True + +namespace RelSpec + +variable (D : RelSpec.{u, v, w}) + +/-- The set of permitted `(nextState, result)` outcomes of operation `o` from state `s`. -/ +def outcomes (s : D.State) (o : D.Op) : Set (D.State × D.Result o) := + {p | D.step s o p.1 p.2} + +/-- Membership in `D.outcomes s o` unfolds to `D.step s o`. -/ +@[simp] +theorem mem_outcomes_iff (s : D.State) (o : D.Op) (p : D.State × D.Result o) : + p ∈ D.outcomes s o ↔ D.step s o p.1 p.2 := Iff.rfl + +/-- A relational specification is total on admissible inputs if every valid `(s, o)` admits at +least one permitted outcome `(s', a)`. -/ +class Total : Prop where + /-- Every admissible operation call has at least one permitted post-state and result. -/ + step_total : ∀ s o, D.valid s o → ∃ s' a, D.step s o s' a + +/-- Obtain a permitted outcome `(s', a)` for an admissible operation under `[D.Total]`. -/ +theorem exists_step [D.Total] {s : D.State} {o : D.Op} (hv : D.valid s o) : + ∃ s' a, D.step s o s' a := + Total.step_total s o hv + +/-- The polynomial operation signature `⟨D.Op, D.Result⟩`. -/ +abbrev signature : PFunctor.{v, w} := ⟨D.Op, D.Result⟩ + +/-- An adaptive client over `D`: each continuation may branch on the observed operation result. -/ +abbrev Client (α : Type y) : Type (max v w y) := D.signature.FreeM α + +/-- Invoke a single operation `o : D.Op` in an adaptive `D.Client`. -/ +abbrev call (o : D.Op) : D.Client (D.Result o) := .liftBind o .pure + +/-- An input-output transition label pairing an operation `o : D.Op` with its result `a`. -/ +abbrev Label : Type (max v w) := Σ o : D.Op, D.Result o + +/-- View a relational stateful specification as a labelled transition system (`Cslib.LTS`) +labelled by input-output pairs `⟨o, a⟩ : Σ o : D.Op, D.Result o`. -/ +def toLTS : LTS D.State D.Label where + Tr s lbl s' := D.step s lbl.1 s' lbl.2 + +/-- A transition in `D.toLTS` is a step of `D`. -/ +@[simp] +theorem toLTS_tr (s s' : D.State) (lbl : D.Label) : + D.toLTS.Tr s lbl s' ↔ D.step s lbl.1 s' lbl.2 := Iff.rfl + +end RelSpec + +/-! ## Zero-cost embedding of deterministic specifications (`Spec.toRelSpec`) -/ + +/-- Embed a deterministic `Stateful.Spec` as a relational `RelSpec` whose transition relation +holds if and only if `(s', a) = D.step s o`. -/ +abbrev Spec.toRelSpec (D : Spec.{u, v, w}) : RelSpec.{u, v, w} where + State := D.State + Op := D.Op + Result := D.Result + step s o s' a := (s', a) = D.step s o + valid := D.valid + +instance : Coe Spec.{u, v, w} RelSpec.{u, v, w} := ⟨Spec.toRelSpec⟩ + +namespace Spec + +variable (D : Spec.{u, v, w}) + +/-- `toRelSpec` preserves the state type definitionally. -/ +@[simp] +theorem toRelSpec_state : D.toRelSpec.State = D.State := rfl + +/-- `toRelSpec` preserves the operation type definitionally. -/ +@[simp] +theorem toRelSpec_op : D.toRelSpec.Op = D.Op := rfl + +/-- `toRelSpec` preserves the result family definitionally. -/ +@[simp] +theorem toRelSpec_result : D.toRelSpec.Result = D.Result := rfl + +/-- A step in `D.toRelSpec` holds iff `(s', a) = D.step s o`. -/ +@[simp] +theorem toRelSpec_step (s : D.State) (o : D.Op) (s' : D.State) (a : D.Result o) : + D.toRelSpec.step s o s' a ↔ (s', a) = D.step s o := Iff.rfl + +/-- `toRelSpec` preserves the admissibility predicate definitionally. -/ +@[simp] +theorem toRelSpec_valid (s : D.State) (o : D.Op) : + D.toRelSpec.valid s o ↔ D.valid s o := Iff.rfl + +/-- `toRelSpec` preserves the polynomial signature definitionally. -/ +@[simp] +theorem toRelSpec_signature : D.toRelSpec.signature = D.signature := rfl + +/-- `toRelSpec` preserves the adaptive client type definitionally. -/ +@[simp] +theorem toRelSpec_client (α : Type y) : D.toRelSpec.Client α = D.Client α := rfl + +/-- Every embedded deterministic specification is automatically total. -/ +instance : D.toRelSpec.Total where + step_total s o _ := ⟨(D.step s o).1, (D.step s o).2, Prod.mk.eta⟩ + +/-- Transitions in `D.toRelSpec.toLTS` agree with `D.toFLTS` on states and `(D.step s o).2` on +observations. -/ +theorem toRelSpec_toLTS_tr (s s' : D.State) (o : D.Op) (a : D.Result o) : + D.toRelSpec.toLTS.Tr s ⟨o, a⟩ s' ↔ s' = D.toFLTS.tr s o ∧ a = (D.step s o).2 := by + simp [RelSpec.toLTS, Spec.toRelSpec, Spec.toFLTS, Prod.ext_iff] + +/-- The labelled transition system of an embedded deterministic specification is deterministic. -/ +instance : D.toRelSpec.toLTS.Deterministic where + deterministic s lbl s₁ s₂ h₁ h₂ := by + have h₁' := congrArg Prod.fst h₁ + have h₂' := congrArg Prod.fst h₂ + exact h₁'.trans h₂'.symm + +/-- Multi-step execution `D.run` produces a valid multi-step trace in `D.toRelSpec.toLTS` ending +at `D.toFLTS.mtr s os`. -/ +theorem toRelSpec_toLTS_mtr_run (s : D.State) (os : List D.Op) : + D.toRelSpec.toLTS.MTr s (D.run s os).2 (D.toFLTS.mtr s os) := by + induction os generalizing s with + | nil => exact .refl + | cons o os ih => + exact .stepL Prod.mk.eta (ih (D.step s o).1) + +end Spec + +/-! ## Transcripts, non-adaptive operation sequences, and cost accounting -/ + +namespace RelSpec + +variable (D : RelSpec.{u, v, w}) + +/-- One recorded step of a relational specification: the issued operation `op`, its returned +observation `result`, and the resulting post-state `nextState`. -/ +structure Step : Type (max u v w) where + /-- The operation invoked at this step. -/ + op : D.Op + /-- The observation returned by `op`. -/ + result : D.Result op + /-- The abstract post-state reached after `op`. -/ + nextState : D.State + +/-- A finite execution transcript is a list of recorded steps. -/ +abbrev Transcript : Type (max u v w) := List D.Step + +/-- Extract the underlying sequence of operations from a transcript. -/ +def Transcript.ops {D : RelSpec.{u, v, w}} (tr : D.Transcript) : List D.Op := + tr.map Step.op + +/-- Extract the `Cslib.LTS` input-output labels from a transcript. -/ +def Transcript.labels {D : RelSpec.{u, v, w}} (tr : D.Transcript) : List D.Label := + tr.map fun st => ⟨st.op, st.result⟩ + +@[simp] +theorem Transcript.ops_nil {D : RelSpec.{u, v, w}} : + Transcript.ops ([] : D.Transcript) = [] := rfl + +@[simp] +theorem Transcript.ops_cons {D : RelSpec.{u, v, w}} (st : D.Step) (tr : D.Transcript) : + Transcript.ops (st :: tr) = st.op :: Transcript.ops tr := rfl + +@[simp] +theorem Transcript.ops_append {D : RelSpec.{u, v, w}} (tr₁ tr₂ : D.Transcript) : + Transcript.ops (tr₁ ++ tr₂) = Transcript.ops tr₁ ++ Transcript.ops tr₂ := + List.map_append + +@[simp] +theorem Transcript.length_ops {D : RelSpec.{u, v, w}} (tr : D.Transcript) : + (Transcript.ops tr).length = tr.length := + List.length_map _ + +@[simp] +theorem Transcript.labels_nil {D : RelSpec.{u, v, w}} : + Transcript.labels ([] : D.Transcript) = [] := rfl + +@[simp] +theorem Transcript.labels_cons {D : RelSpec.{u, v, w}} (st : D.Step) (tr : D.Transcript) : + Transcript.labels (st :: tr) = ⟨st.op, st.result⟩ :: Transcript.labels tr := rfl + +@[simp] +theorem Transcript.labels_append {D : RelSpec.{u, v, w}} (tr₁ tr₂ : D.Transcript) : + Transcript.labels (tr₁ ++ tr₂) = Transcript.labels tr₁ ++ Transcript.labels tr₂ := + List.map_append + +@[simp] +theorem Transcript.length_labels {D : RelSpec.{u, v, w}} (tr : D.Transcript) : + (Transcript.labels tr).length = tr.length := + List.length_map _ + +/-- `D.ValidTranscript s tr s'` holds when `tr` is a valid step-by-step execution starting at `s` +and ending at `s'`, where each step is admissible (`D.valid`) and permitted by `D.step`. -/ +inductive ValidTranscript : D.State → D.Transcript → D.State → Prop + | nil (s : D.State) : ValidTranscript s [] s + | cons {s : D.State} (st : D.Step) {tr : D.Transcript} {s_fin : D.State} : + D.valid s st.op → + D.step s st.op st.nextState st.result → + ValidTranscript st.nextState tr s_fin → + ValidTranscript s (st :: tr) s_fin + +/-- The empty transcript is valid iff the initial and final states coincide. -/ +@[simp] +theorem validTranscript_nil_iff {D : RelSpec.{u, v, w}} (s s' : D.State) : + D.ValidTranscript s [] s' ↔ s = s' := by + constructor + · intro h; cases h; rfl + · rintro rfl; exact .nil s + +/-- A nonempty transcript is valid iff its head step is admissible and permitted and its tail is +valid from `st.nextState`. -/ +@[simp] +theorem validTranscript_cons_iff {D : RelSpec.{u, v, w}} (s s' : D.State) (st : D.Step) + (tr : D.Transcript) : + D.ValidTranscript s (st :: tr) s' ↔ + D.valid s st.op ∧ D.step s st.op st.nextState st.result ∧ + D.ValidTranscript st.nextState tr s' := by + constructor + · intro h + cases h with + | cons _ hv hstep htr => exact ⟨hv, hstep, htr⟩ + · rintro ⟨hv, hstep, htr⟩ + exact .cons st hv hstep htr + +/-- Valid transcripts compose across concatenation. -/ +theorem ValidTranscript.append {D : RelSpec.{u, v, w}} {s s₁ s₂ : D.State} + {tr₁ tr₂ : D.Transcript} (h₁ : D.ValidTranscript s tr₁ s₁) + (h₂ : D.ValidTranscript s₁ tr₂ s₂) : + D.ValidTranscript s (tr₁ ++ tr₂) s₂ := by + induction h₁ with + | nil _ => exact h₂ + | cons st hv hstep _ ih => exact .cons st hv hstep (ih h₂) + +/-- Every valid transcript projects to a multi-step transition in `D.toLTS`. -/ +theorem ValidTranscript.toLTS_mtr {D : RelSpec.{u, v, w}} {s s' : D.State} + {tr : D.Transcript} (h : D.ValidTranscript s tr s') : + D.toLTS.MTr s tr.labels s' := by + induction h with + | nil _ => exact .refl + | cons _ _ hstep _ ih => exact .stepL hstep ih + +/-- Demonic admissibility of a non-adaptive operation sequence `ops : List D.Op` from `s`: +the head operation is admissible at `s`, and the tail sequence remains admissible from every +permitted post-state `s'` of the head operation. -/ +def ValidOpSeq (s : D.State) : List D.Op → Prop + | [] => True + | o :: ops => D.valid s o ∧ ∀ s' a, D.step s o s' a → ValidOpSeq s' ops + +@[simp] +theorem validOpSeq_nil (s : D.State) : D.ValidOpSeq s [] ↔ True := Iff.rfl + +@[simp] +theorem validOpSeq_cons (s : D.State) (o : D.Op) (ops : List D.Op) : + D.ValidOpSeq s (o :: ops) ↔ D.valid s o ∧ ∀ s' a, D.step s o s' a → D.ValidOpSeq s' ops := + Iff.rfl + +/-- Total interfaces have no operation-sequence admissibility obligations. -/ +theorem validOpSeq_of_forall_valid (h : ∀ s o, D.valid s o) (s : D.State) (ops : List D.Op) : + D.ValidOpSeq s ops := by + induction ops generalizing s with + | nil => trivial + | cons o ops ih => exact ⟨h s o, fun s' _ _ => ih s'⟩ + +/-- Under `[D.Total]`, every demonically admissible operation sequence admits at least one valid +transcript. -/ +theorem exists_validTranscript_of_validOpSeq [D.Total] {s : D.State} {ops : List D.Op} + (hv : D.ValidOpSeq s ops) : + ∃ tr s', tr.ops = ops ∧ D.ValidTranscript s tr s' := by + induction ops generalizing s with + | nil => exact ⟨[], s, rfl, .nil s⟩ + | cons o ops ih => + obtain ⟨hv_o, hv_rest⟩ := hv + obtain ⟨s₁, a, hstep⟩ := D.exists_step hv_o + obtain ⟨tr, s₂, hops, htr⟩ := ih (hv_rest s₁ a hstep) + exact ⟨⟨o, a, s₁⟩ :: tr, s₂, congrArg (o :: ·) hops, .cons _ hv_o hstep htr⟩ + +/-- State-dependent charges summed along a transcript. -/ +def totalCost (charge : D.State → D.Op → ℕ) : D.State → D.Transcript → ℕ + | _, [] => 0 + | s, st :: tr => charge s st.op + totalCost charge st.nextState tr + +/-- Outcome-dependent charges summed along a transcript, where each step charge may also inspect +the post-state `st.nextState` and result `st.result`. -/ +def totalCostOutcome (charge : D.State → (o : D.Op) → D.State → D.Result o → ℕ) : + D.State → D.Transcript → ℕ + | _, [] => 0 + | s, st :: tr => + charge s st.op st.nextState st.result + totalCostOutcome charge st.nextState tr + +@[simp] +theorem totalCost_nil (charge : D.State → D.Op → ℕ) (s : D.State) : + D.totalCost charge s [] = 0 := rfl + +@[simp] +theorem totalCost_cons (charge : D.State → D.Op → ℕ) (s : D.State) (st : D.Step) + (tr : D.Transcript) : + D.totalCost charge s (st :: tr) = charge s st.op + D.totalCost charge st.nextState tr := rfl + +@[simp] +theorem totalCostOutcome_nil (charge : D.State → (o : D.Op) → D.State → D.Result o → ℕ) + (s : D.State) : + D.totalCostOutcome charge s [] = 0 := rfl + +@[simp] +theorem totalCostOutcome_cons (charge : D.State → (o : D.Op) → D.State → D.Result o → ℕ) + (s : D.State) (st : D.Step) (tr : D.Transcript) : + D.totalCostOutcome charge s (st :: tr) = + charge s st.op st.nextState st.result + D.totalCostOutcome charge st.nextState tr := rfl + +/-- `totalCost` is the special case of `totalCostOutcome` that ignores the post-state and +result. -/ +theorem totalCost_eq_totalCostOutcome (charge : D.State → D.Op → ℕ) (s : D.State) + (tr : D.Transcript) : + D.totalCost charge s tr = D.totalCostOutcome (fun s o _ _ => charge s o) s tr := by + induction tr generalizing s with + | nil => rfl + | cons st tr ih => simp [ih] + +/-- Along a valid transcript prefix `D.ValidTranscript s tr₁ s₁`, `totalCost` splits additively at +the intermediate state `s₁`. -/ +theorem totalCost_append {D : RelSpec.{u, v, w}} (charge : D.State → D.Op → ℕ) + {s s₁ : D.State} {tr₁ : D.Transcript} (h₁ : D.ValidTranscript s tr₁ s₁) + (tr₂ : D.Transcript) : + D.totalCost charge s (tr₁ ++ tr₂) = + D.totalCost charge s tr₁ + D.totalCost charge s₁ tr₂ := by + induction h₁ with + | nil _ => simp + | cons st _ _ _ ih => simp [ih, Nat.add_assoc] + +/-- Uniform per-operation bounds scale by the transcript length on any valid transcript. -/ +theorem totalCost_le_length_mul {D : RelSpec.{u, v, w}} (charge : D.State → D.Op → ℕ) (T : ℕ) + (h : ∀ s o, D.valid s o → charge s o ≤ T) {s s' : D.State} {tr : D.Transcript} + (hv : D.ValidTranscript s tr s') : + D.totalCost charge s tr ≤ tr.length * T := by + induction hv with + | nil _ => simp + | cons st hv_o _ _ ih => + have hstep := h _ st.op hv_o + simp only [totalCost_cons, List.length_cons, Nat.add_mul, Nat.one_mul] + omega + +/-- A constant charge `K` per operation totals `tr.length * K`. -/ +theorem totalCost_const (K : ℕ) (s : D.State) (tr : D.Transcript) : + D.totalCost (fun _ _ => K) s tr = tr.length * K := by + induction tr generalizing s with + | nil => simp + | cons st tr ih => simp [ih, Nat.add_mul, Nat.add_comm] + +/-- A unit charge per operation totals the transcript length. -/ +theorem totalCost_one (s : D.State) (tr : D.Transcript) : + D.totalCost (fun _ _ => 1) s tr = tr.length := by + simpa using D.totalCost_const 1 s tr + +end RelSpec + +/-! ## Adaptive clients (`ClientRun`, `DemonicValid`) and demonic `Hoare` / `CostHoare` -/ + +namespace RelSpec + +variable (D : RelSpec.{u, v, w}) {α β : Type y} + +/-- `D.ClientRun s p v s' tr` holds when running the adaptive client `p : D.Client α` from initial +state `s` can produce return value `v : α`, final state `s' : D.State`, and transcript `tr` by +making admissible and permitted transitions at each operation call. -/ +inductive ClientRun : D.State → D.Client α → α → D.State → D.Transcript → Prop + | pure (s : D.State) (a : α) : + ClientRun s (.pure a) a s [] + | liftBind {s s₁ s₂ : D.State} (o : D.Op) (k : D.Result o → D.Client α) + (a : D.Result o) {v : α} {tr : D.Transcript} : + D.valid s o → + D.step s o s₁ a → + ClientRun s₁ (k a) v s₂ tr → + ClientRun s (.liftBind o k) v s₂ (⟨o, a, s₁⟩ :: tr) + +/-- Every `ClientRun` produces a valid transcript. -/ +theorem ClientRun.validTranscript {D : RelSpec.{u, v, w}} {s s' : D.State} {p : D.Client α} + {v : α} {tr : D.Transcript} (h : D.ClientRun s p v s' tr) : + D.ValidTranscript s tr s' := by + induction h with + | pure s _ => exact .nil s + | liftBind _ _ _ hv hstep _ ih => exact .cons _ hv hstep ih + +theorem clientRun_pure_iff {D : RelSpec.{u, v, w}} (s : D.State) (a v : α) (s' : D.State) + (tr : D.Transcript) : + D.ClientRun s (.pure a) v s' tr ↔ v = a ∧ s' = s ∧ tr = [] := by + constructor + · intro h + cases h + exact ⟨rfl, rfl, rfl⟩ + · rintro ⟨rfl, rfl, rfl⟩ + exact .pure s' v + +theorem clientRun_liftBind_iff {D : RelSpec.{u, v, w}} (s : D.State) (o : D.Op) + (k : D.Result o → D.Client α) (v : α) (s₂ : D.State) (tr : D.Transcript) : + D.ClientRun s (.liftBind o k) v s₂ tr ↔ + D.valid s o ∧ ∃ a s₁ tr', D.step s o s₁ a ∧ + D.ClientRun s₁ (k a) v s₂ tr' ∧ tr = ⟨o, a, s₁⟩ :: tr' := by + constructor + · intro h + cases h with + | liftBind _ _ a hv hstep hrest => + exact ⟨hv, a, _, _, hstep, hrest, rfl⟩ + · rintro ⟨hv, a, s₁, tr', hstep, hrest, rfl⟩ + exact .liftBind o k a hv hstep hrest + +/-- Sequential client runs compose by concatenating their transcripts. -/ +theorem ClientRun.bind {D : RelSpec.{u, v, w}} {s s₁ s₂ : D.State} {p : D.Client α} + {k : α → D.Client β} {a : α} {b : β} {tr₁ tr₂ : D.Transcript} + (h₁ : D.ClientRun s p a s₁ tr₁) (h₂ : D.ClientRun s₁ (k a) b s₂ tr₂) : + D.ClientRun s (p.bind k) b s₂ (tr₁ ++ tr₂) := by + induction h₁ with + | pure _ _ => exact h₂ + | liftBind o cont res hv hstep _ ih => + exact .liftBind o (fun r => (cont r).bind k) res hv hstep (ih h₂) + +/-- A client run of `p.bind k` splits into a run of `p` and a run of `k` from the intermediate +result and state. -/ +theorem clientRun_bind_iff {D : RelSpec.{u, v, w}} {s s₂ : D.State} {p : D.Client α} + {k : α → D.Client β} {b : β} {tr : D.Transcript} : + D.ClientRun s (p.bind k) b s₂ tr ↔ + ∃ a s₁ tr₁ tr₂, D.ClientRun s p a s₁ tr₁ ∧ + D.ClientRun s₁ (k a) b s₂ tr₂ ∧ tr = tr₁ ++ tr₂ := by + constructor + · intro h + induction p generalizing s tr with + | pure a => + exact ⟨a, s, [], tr, .pure s a, h, rfl⟩ + | liftBind o cont ih => + change D.ClientRun s (.liftBind o fun r => (cont r).bind k) b s₂ tr at h + cases h with + | liftBind _ _ res hv hstep hrest => + obtain ⟨a, s₁, tr₁, tr₂, hp, hk, rfl⟩ := ih res hrest + exact ⟨a, s₁, ⟨o, res, _⟩ :: tr₁, tr₂, .liftBind o cont res hv hstep hp, hk, rfl⟩ + · rintro ⟨a, s₁, tr₁, tr₂, hp, hk, rfl⟩ + exact hp.bind hk + +/-- Demonic admissibility of an adaptive client `p` from state `s`: +* `.pure a` is always admissible; +* `.liftBind o k` requires `D.valid s o` now, and for *every* permitted outcome `(s', a)` of `o` + from `s`, the continuation `k a` must be demonically admissible from `s'`. -/ +inductive DemonicValid : D.State → D.Client α → Prop + | pure (s : D.State) (a : α) : + DemonicValid s (.pure a) + | liftBind {s : D.State} (o : D.Op) (k : D.Result o → D.Client α) : + D.valid s o → + (∀ s' a, D.step s o s' a → DemonicValid s' (k a)) → + DemonicValid s (.liftBind o k) + +theorem demonicValid_pure_iff {D : RelSpec.{u, v, w}} (s : D.State) (a : α) : + D.DemonicValid s (.pure a) ↔ True := + ⟨fun _ => trivial, fun _ => .pure s a⟩ + +theorem demonicValid_liftBind_iff {D : RelSpec.{u, v, w}} (s : D.State) (o : D.Op) + (k : D.Result o → D.Client α) : + D.DemonicValid s (.liftBind o k) ↔ + D.valid s o ∧ ∀ s' a, D.step s o s' a → D.DemonicValid s' (k a) := by + constructor + · intro h + cases h with + | liftBind _ _ hv hk => exact ⟨hv, hk⟩ + · rintro ⟨hv, hk⟩; exact .liftBind o k hv hk + +/-- On a specification where every operation is valid in every state, every client is demonically +admissible. -/ +theorem demonicValid_of_forall_valid (h : ∀ s o, D.valid s o) (s : D.State) (p : D.Client α) : + D.DemonicValid s p := by + induction p generalizing s with + | pure a => exact .pure s a + | liftBind o k ih => exact .liftBind o k (h s o) fun s' a _ => ih a s' + +/-- Under `[D.Total]`, every demonically admissible client admits at least one valid `ClientRun`. -/ +theorem DemonicValid.exists_clientRun {D : RelSpec.{u, v, w}} [D.Total] {s : D.State} + {p : D.Client α} (hv : D.DemonicValid s p) : + ∃ v s' tr, D.ClientRun s p v s' tr := by + induction hv with + | pure s a => exact ⟨a, s, [], .pure s a⟩ + | liftBind o k hv_o _ ih => + obtain ⟨s₁, a, hstep⟩ := D.exists_step hv_o + obtain ⟨v, s₂, tr, hrun⟩ := ih s₁ a hstep + exact ⟨v, s₂, ⟨o, a, s₁⟩ :: tr, .liftBind o k a hv_o hstep hrun⟩ + +/-- Demonic admissibility composes across `bind` when the continuation is demonically admissible +after every valid run of the first client. -/ +theorem DemonicValid.bind {D : RelSpec.{u, v, w}} {s : D.State} {p : D.Client α} + {k : α → D.Client β} (hp : D.DemonicValid s p) + (hk : ∀ a s' tr, D.ClientRun s p a s' tr → D.DemonicValid s' (k a)) : + D.DemonicValid s (p.bind k) := by + induction hp with + | pure s a => exact hk a s [] (.pure s a) + | liftBind o cont hv_o _ ih => + refine .liftBind o (fun r => (cont r).bind k) hv_o fun s₁ r hstep => ?_ + exact ih s₁ r hstep fun a s₂ tr hrun => hk a s₂ (⟨o, r, s₁⟩ :: tr) + (.liftBind o cont r hv_o hstep hrun) + +/-- Demonic Hoare triple for a relational specification: from any initial state satisfying `pre`, +the client `p` is demonically admissible and *every* valid `ClientRun` satisfies `post`. -/ +def Hoare (pre : D.State → Prop) (p : D.Client α) (post : α → D.State → Prop) : Prop := + ∀ s, pre s → D.DemonicValid s p ∧ ∀ v s' tr, D.ClientRun s p v s' tr → post v s' + +namespace Hoare + +variable {D : RelSpec.{u, v, w}} + +/-- Returning a pure value leaves the state unchanged. -/ +theorem pure (pre : D.State → Prop) (a : α) (post : α → D.State → Prop) + (h : ∀ s, pre s → post a s) : D.Hoare pre (.pure a) post := by + intro s hs + refine ⟨.pure s a, ?_⟩ + intro v s' tr hrun + cases hrun + exact h s hs + +/-- Calling an operation requires `D.valid s o` and establishes `post a s'` for every permitted +outcome `D.step s o s' a`. -/ +theorem call (pre : D.State → Prop) (o : D.Op) (post : D.Result o → D.State → Prop) + (h : ∀ s, pre s → D.valid s o ∧ ∀ s' a, D.step s o s' a → post a s') : + D.Hoare pre (.liftBind o fun a => .pure a) post := by + intro s hs + obtain ⟨hv, hpost⟩ := h s hs + refine ⟨.liftBind o (fun a => .pure a) hv fun s' a _ => .pure s' a, ?_⟩ + intro v s' tr hrun + cases hrun with + | liftBind _ _ a _ hstep hrest => + cases hrest + exact hpost s' v hstep + +/-- Sequential composition of demonic Hoare triples. -/ +theorem bind {pre : D.State → Prop} {mid : α → D.State → Prop} + {post : β → D.State → Prop} {p : D.Client α} {k : α → D.Client β} + (hp : D.Hoare pre p mid) (hk : ∀ a, D.Hoare (mid a) (k a) post) : + D.Hoare pre (p.bind k) post := by + intro s hs + obtain ⟨hvp, hpost_p⟩ := hp s hs + refine ⟨hvp.bind fun a s₁ tr₁ hrun₁ => (hk a s₁ (hpost_p a s₁ tr₁ hrun₁)).1, ?_⟩ + intro b s₂ tr hrun + obtain ⟨a, s₁, tr₁, tr₂, hrun₁, hrun₂, _⟩ := clientRun_bind_iff.mp hrun + exact (hk a s₁ (hpost_p a s₁ tr₁ hrun₁)).2 b s₂ tr₂ hrun₂ + +/-- Consequence rule for demonic Hoare triples. -/ +theorem consequence {pre pre' : D.State → Prop} {post post' : α → D.State → Prop} + {p : D.Client α} (h : D.Hoare pre p post) (hp : ∀ s, pre' s → pre s) + (hq : ∀ a s, post a s → post' a s) : D.Hoare pre' p post' := by + intro s hs + obtain ⟨hv, hpost⟩ := h s (hp s hs) + exact ⟨hv, fun v s' tr hrun => hq v s' (hpost v s' tr hrun)⟩ + +end Hoare + +/-- Demonic `CostHoare` triple: strengthens `D.Hoare` with a worst-case abstract charge bound +`D.totalCost charge s tr ≤ budget s` across *every* valid `ClientRun`. -/ +def CostHoare (charge : D.State → D.Op → ℕ) (pre : D.State → Prop) + (p : D.Client α) (post : α → D.State → Prop) (budget : D.State → ℕ) : Prop := + D.Hoare pre p post ∧ + ∀ s, pre s → ∀ v s' tr, D.ClientRun s p v s' tr → D.totalCost charge s tr ≤ budget s + +namespace CostHoare + +variable {D : RelSpec.{u, v, w}} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} + {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} + +/-- Project a `CostHoare` triple to its underlying `Hoare` triple. -/ +theorem hoare (h : D.CostHoare charge pre p post budget) : D.Hoare pre p post := h.1 + +/-- Returning a pure value incurs zero operation cost. -/ +theorem pure (a : α) (h : ∀ s, pre s → post a s) : + D.CostHoare charge pre (.pure a) post (fun _ => 0) := by + refine ⟨Hoare.pure pre a post h, ?_⟩ + intro s _ v s' tr hrun + cases hrun + exact Nat.le_refl 0 + +/-- Calling a single operation charges `charge s o` at the pre-state `s`. -/ +theorem call (o : D.Op) {post : D.Result o → D.State → Prop} + (h : ∀ s, pre s → D.valid s o ∧ ∀ s' a, D.step s o s' a → post a s') : + D.CostHoare charge pre (.liftBind o fun a => .pure a) post (fun s => charge s o) := by + refine ⟨Hoare.call pre o post h, ?_⟩ + intro s _ v s' tr hrun + cases hrun with + | liftBind _ _ a _ _ hrest => + cases hrest + simp + +/-- Compose `CostHoare` budgets at every intermediate state satisfying `mid`. -/ +theorem bind {mid : α → D.State → Prop} {post : β → D.State → Prop} + {k : α → D.Client β} {first : D.State → ℕ} {next : α → D.State → ℕ} + (hp : D.CostHoare charge pre p mid first) + (hk : ∀ a, D.CostHoare charge (mid a) (k a) post (next a)) + (hb : ∀ s, pre s → ∀ a s', mid a s' → first s + next a s' ≤ budget s) : + D.CostHoare charge pre (p.bind k) post budget := by + refine ⟨Hoare.bind hp.hoare (fun a => (hk a).hoare), ?_⟩ + intro s hs b s₂ tr hrun + obtain ⟨a, s₁, tr₁, tr₂, hrun₁, hrun₂, rfl⟩ := clientRun_bind_iff.mp hrun + have hmid : mid a s₁ := (hp.hoare s hs).2 a s₁ tr₁ hrun₁ + have hf := hp.2 s hs a s₁ tr₁ hrun₁ + have hn := (hk a).2 s₁ hmid b s₂ tr₂ hrun₂ + rw [D.totalCost_append charge hrun₁.validTranscript tr₂] + exact (Nat.add_le_add hf hn).trans (hb s hs a s₁ hmid) + +/-- Consequence rule for demonic `CostHoare` triples. -/ +theorem consequence {pre' : D.State → Prop} {post' : α → D.State → Prop} + {budget' : D.State → ℕ} (h : D.CostHoare charge pre p post budget) + (hp : ∀ s, pre' s → pre s) (hq : ∀ a s, post a s → post' a s) + (hb : ∀ s, pre' s → budget s ≤ budget' s) : + D.CostHoare charge pre' p post' budget' := + ⟨h.hoare.consequence hp hq, + fun s hs v s' tr hrun => (h.2 s (hp s hs) v s' tr hrun).trans (hb s hs)⟩ + +end CostHoare + +end RelSpec + +/-! ## Equivalence of relational and deterministic client semantics on `D.toRelSpec` -/ + +namespace Spec + +variable (D : Spec.{u, v, w}) {α : Type y} + +/-- On `D.toRelSpec`, non-adaptive operation-sequence admissibility `ValidOpSeq` coincides with +`D.ValidHistory`. -/ +theorem toRelSpec_validOpSeq_iff (s : D.State) (ops : List D.Op) : + D.toRelSpec.ValidOpSeq s ops ↔ D.ValidHistory s ops := by + induction ops generalizing s with + | nil => rfl + | cons o ops ih => + change (D.valid s o ∧ ∀ s' a, (s', a) = D.step s o → D.toRelSpec.ValidOpSeq s' ops) ↔ + D.valid s o ∧ D.ValidHistory (D.step s o).1 ops + refine and_congr_right fun _ => ?_ + constructor + · intro h + exact (ih (D.step s o).1).mp (h (D.step s o).1 (D.step s o).2 Prod.mk.eta) + · intro h s' a heq + have hs' : s' = (D.step s o).1 := congrArg Prod.fst heq + subst hs' + exact (ih (D.step s o).1).mpr h + +/-- The unique canonical transcript determined by a deterministic specification `D` on an +operation list `os` from state `s`. -/ +def canonicalTranscript (s : D.State) : List D.Op → D.toRelSpec.Transcript + | [] => [] + | o :: os => + let next := D.step s o + ⟨o, next.2, next.1⟩ :: canonicalTranscript next.1 os + +@[simp] +theorem canonicalTranscript_nil (s : D.State) : D.canonicalTranscript s [] = [] := rfl + +@[simp] +theorem canonicalTranscript_cons (s : D.State) (o : D.Op) (os : List D.Op) : + D.canonicalTranscript s (o :: os) = + ⟨o, (D.step s o).2, (D.step s o).1⟩ :: D.canonicalTranscript (D.step s o).1 os := rfl + +/-- The underlying operation list of `D.canonicalTranscript s os` is `os`. -/ +@[simp] +theorem canonicalTranscript_ops (s : D.State) (os : List D.Op) : + (D.canonicalTranscript s os).ops = os := by + induction os generalizing s with + | nil => rfl + | cons o os ih => simp [ih] + +/-- `totalCost` along `D.canonicalTranscript s os` equals `D.totalCost charge s os`. -/ +@[simp] +theorem toRelSpec_totalCost_canonicalTranscript (charge : D.State → D.Op → ℕ) (s : D.State) + (os : List D.Op) : + D.toRelSpec.totalCost charge s (D.canonicalTranscript s os) = D.totalCost charge s os := by + induction os generalizing s with + | nil => rfl + | cons o os ih => + change charge s o + + D.toRelSpec.totalCost charge (D.step s o).1 (D.canonicalTranscript (D.step s o).1 os) = + charge s o + D.totalCost charge (D.step s o).1 os + rw [ih] + +/-- On `D.toRelSpec`, demonic client admissibility `DemonicValid s p` is equivalent to +`D.ValidHistory s (D.runClient s p).2.2`. -/ +theorem toRelSpec_demonicValid_iff (s : D.State) (p : D.Client α) : + D.toRelSpec.DemonicValid s p ↔ D.ValidHistory s (D.runClient s p).2.2 := by + induction p generalizing s with + | pure a => + exact ⟨fun _ => trivial, fun _ => RelSpec.DemonicValid.pure (D := D.toRelSpec) s a⟩ + | liftBind o k ih => + rw [RelSpec.demonicValid_liftBind_iff (D := D.toRelSpec), runClient_liftBind, validHistory_cons] + refine and_congr_right fun _ => ?_ + constructor + · intro h + exact (ih (D.step s o).2 (D.step s o).1).mp + (h (D.step s o).1 (D.step s o).2 Prod.mk.eta) + · intro h s' a heq + have hs' : s' = (D.step s o).1 := congrArg Prod.fst heq + have ha : a = (D.step s o).2 := congrArg Prod.snd heq + subst hs'; subst ha + exact (ih (D.step s o).2 (D.step s o).1).mpr h + +/-- Every client with an admissible deterministic history executes canonically in `D.toRelSpec`. -/ +theorem toRelSpec_clientRun_canonical (s : D.State) (p : D.Client α) + (hv : D.ValidHistory s (D.runClient s p).2.2) : + D.toRelSpec.ClientRun s p (D.runClient s p).1 (D.runClient s p).2.1 + (D.canonicalTranscript s (D.runClient s p).2.2) := by + induction p generalizing s with + | pure a => + exact RelSpec.ClientRun.pure (D := D.toRelSpec) s a + | liftBind o k ih => + exact RelSpec.ClientRun.liftBind (D := D.toRelSpec) o k (D.step s o).2 hv.1 Prod.mk.eta + (ih (D.step s o).2 (D.step s o).1 hv.2) + +private theorem toRelSpec_clientRun_imp {s : D.toRelSpec.State} {p : D.toRelSpec.Client α} + {v : α} {s' : D.toRelSpec.State} {tr : D.toRelSpec.Transcript} + (h : D.toRelSpec.ClientRun s p v s' tr) : + D.ValidHistory s (D.runClient s p).2.2 ∧ + v = (D.runClient s p).1 ∧ + s' = (D.runClient s p).2.1 ∧ + tr = D.canonicalTranscript s (D.runClient s p).2.2 := by + induction h with + | pure s a => + exact ⟨trivial, rfl, rfl, rfl⟩ + | @liftBind s s₁ s₂ o k res v tr hv hstep _ ih => + have hs₁ : s₁ = (D.step s o).1 := congrArg Prod.fst hstep + have hres : res = (D.step s o).2 := congrArg Prod.snd hstep + subst hs₁; subst hres + obtain ⟨hv_rest, rfl, rfl, rfl⟩ := ih + exact ⟨⟨hv, hv_rest⟩, rfl, rfl, rfl⟩ + +/-- On `D.toRelSpec`, `ClientRun` collapses uniquely to `D.runClient` and `D.ValidHistory`. -/ +theorem toRelSpec_clientRun_iff (s : D.State) (p : D.Client α) (v : α) (s' : D.State) + (tr : D.toRelSpec.Transcript) : + D.toRelSpec.ClientRun s p v s' tr ↔ + D.ValidHistory s (D.runClient s p).2.2 ∧ + v = (D.runClient s p).1 ∧ + s' = (D.runClient s p).2.1 ∧ + tr = D.canonicalTranscript s (D.runClient s p).2.2 := + ⟨D.toRelSpec_clientRun_imp, fun ⟨hv, h₁, h₂, h₃⟩ => + h₁ ▸ h₂ ▸ h₃ ▸ D.toRelSpec_clientRun_canonical s p hv⟩ + +/-- On `D.toRelSpec`, `RelSpec.Hoare` is equivalent to `Spec.Hoare`. -/ +theorem toRelSpec_hoare_iff (pre : D.State → Prop) (p : D.Client α) (post : α → D.State → Prop) : + D.toRelSpec.Hoare pre p post ↔ D.Hoare pre p post := by + refine forall_congr' fun s => imp_congr_right fun _ => ?_ + rw [toRelSpec_demonicValid_iff] + constructor + · rintro ⟨hv, hpost⟩ + exact ⟨hv, hpost _ _ _ (D.toRelSpec_clientRun_canonical s p hv)⟩ + · rintro ⟨hv, hpost⟩ + refine ⟨hv, fun v s' tr hrun => ?_⟩ + obtain ⟨_, rfl, rfl, _⟩ := (D.toRelSpec_clientRun_iff s p v s' tr).mp hrun + exact hpost + +/-- On `D.toRelSpec`, `RelSpec.CostHoare` is equivalent to `Spec.CostHoare`. -/ +theorem toRelSpec_costHoare_iff (charge : D.State → D.Op → ℕ) (pre : D.State → Prop) + (p : D.Client α) (post : α → D.State → Prop) (budget : D.State → ℕ) : + D.toRelSpec.CostHoare charge pre p post budget ↔ D.CostHoare charge pre p post budget := by + unfold RelSpec.CostHoare Spec.CostHoare + rw [toRelSpec_hoare_iff] + refine and_congr_right fun hhoare => forall_congr' fun s => imp_congr_right fun hs => ?_ + have hv := (hhoare s hs).1 + constructor + · intro h + have hb := h _ _ _ (D.toRelSpec_clientRun_canonical s p hv) + rwa [D.toRelSpec_totalCost_canonicalTranscript] at hb + · intro h v s' tr hrun + obtain ⟨_, _, _, rfl⟩ := (D.toRelSpec_clientRun_iff s p v s' tr).mp hrun + rwa [D.toRelSpec_totalCost_canonicalTranscript] + +end Spec + +/-! ## Native backend executions (`RelExecution`, `RelExecutes`), `RelRealization`, `RelAmortized` +-/ + +/-- A concrete backend execution relation for a relational specification `D : RelSpec`. -/ +abbrev RelExecution (D : RelSpec.{u, v, w}) (C Trace : Type x) := + C → (o : D.Op) → D.Result o → C → Trace → Prop + +/-- Adaptive client execution collecting native execution evidence in call order over a +relational specification `D : RelSpec`. -/ +inductive RelExecutes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + (exec : RelExecution D C τ) : D.Client α → C → α → C → List τ → Prop + | pure (a c) : RelExecutes exec (.pure a) c a c [] + | call {o k c a c' trace b c'' traces} : + exec c o a c' trace → RelExecutes exec (k a) c' b c'' traces → + RelExecutes exec (.liftBind o k) c b c'' (trace :: traces) + +/-- Sequential client execution splits its native trace at the call boundary. -/ +theorem relExecutes_bind_iff {D : RelSpec.{u, v, w}} {C τ : Type x} {α β : Type y} + (exec : RelExecution D C τ) (p : D.Client α) (k : α → D.Client β) + {c c' : C} {b : β} {ts : List τ} : + RelExecutes exec (p.bind k) c b c' ts ↔ + ∃ a mid front back, RelExecutes exec p c a mid front ∧ + RelExecutes exec (k a) mid b c' back ∧ ts = front ++ back := by + induction p generalizing c ts with + | pure a => + constructor + · intro h; exact ⟨a, c, [], ts, .pure a c, h, rfl⟩ + · rintro ⟨a', mid, front, back, hp, hk, ht⟩ + cases hp + simpa only [ht, List.nil_append, PFunctor.FreeM.bind] using hk + | liftBind o cont ih => + change RelExecutes exec (.liftBind o fun a => (cont a).bind k) c b c' ts ↔ _ + constructor + · intro h + cases h with + | call ho rest => + obtain ⟨a, mid, front, back, hp, hk, ht⟩ := (ih _).mp rest + exact ⟨a, mid, _, back, .call ho hp, hk, by simp [ht]⟩ + · rintro ⟨a, mid, front, back, hp, hk, ht⟩ + cases hp with + | call ho rest => + have hrest := (ih _).mpr ⟨a, mid, _, back, rest, hk, rfl⟩ + have h := RelExecutes.call (k := fun a => (cont a).bind k) ho hrest + simpa only [ht, List.cons_append] using h + +private theorem Spec.toRelSpec_relExecutes_imp {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : Execution D C τ} {p : D.toRelSpec.Client α} {c : C} {a : α} {c' : C} + {traces : List τ} (h : RelExecutes (D := D.toRelSpec) exec p c a c' traces) : + Executes exec p c a c' traces := by + induction h with + | pure a c => exact .pure a c + | call ho _ ih => exact .call ho ih + +/-- On `D.toRelSpec`, `RelExecutes` coincides with `Executes`. -/ +theorem Spec.toRelSpec_relExecutes_iff {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} + (exec : Execution D C τ) (p : D.Client α) (c : C) (a : α) (c' : C) (traces : List τ) : + RelExecutes (D := D.toRelSpec) exec p c a c' traces ↔ Executes exec p c a c' traces := by + refine ⟨Spec.toRelSpec_relExecutes_imp, fun h => ?_⟩ + induction h with + | pure a c => exact RelExecutes.pure (D := D.toRelSpec) a c + | call ho _ ih => exact RelExecutes.call (D := D.toRelSpec) ho ih + +/-- Angelic semantic realization of a relational specification `D : RelSpec`: +* `total`: every represented state and admissible operation admits a native execution; +* `correct`: every native execution from a represented state on an admissible operation realizes + *some* permitted abstract post-state `s'` with `D.step s o s' a ∧ rep s' c'`. -/ +structure RelRealization (D : RelSpec.{u, v, w}) {C τ : Type x} + (exec : RelExecution D C τ) where + /-- Representation invariant linking abstract states to concrete backend states. -/ + rep : D.State → C → Prop + /-- Existence of a native execution for every represented state and admissible operation. -/ + total : ∀ {s c} o, rep s c → D.valid s o → ∃ a c' trace, exec c o a c' trace + /-- Angelic step correctness: every native execution realizes some permitted abstract step. -/ + correct : ∀ {s c o a c' trace}, rep s c → D.valid s o → exec c o a c' trace → + ∃ s', D.step s o s' a ∧ rep s' c' + +namespace RelRealization + +variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} (R : RelRealization D exec) + +/-- From related states, every native execution of a demonically admissible client realizes a +valid `ClientRun` ending in a related state. -/ +theorem correct_client {p : D.Client α} {s : D.State} {c c' : C} {a : α} + {traces : List τ} (hr : R.rep s c) (hv : D.DemonicValid s p) + (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by + induction he generalizing s with + | pure a c => + exact ⟨s, [], .pure s a, hr, rfl⟩ + | @call o k c res c₁ trace b c₂ traces ho hes ih => + obtain ⟨hv_o, hv_k⟩ := (D.demonicValid_liftBind_iff s o k).mp hv + obtain ⟨s₁, hstep, hr₁⟩ := R.correct hr hv_o ho + obtain ⟨s₂, tr, hrun, hr₂, hlen⟩ := ih hr₁ (hv_k s₁ res hstep) + exact ⟨s₂, ⟨o, res, s₁⟩ :: tr, .liftBind o k res hv_o hstep hrun, hr₂, + congrArg (· + 1) hlen⟩ + +/-- From related states, every demonically admissible client admits a native execution realizing a +valid `ClientRun` ending in a related state. -/ +theorem exists_executes (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) + (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by + induction hv generalizing c with + | pure s a => + exact ⟨a, c, [], s, [], .pure a c, .pure s a, hr, rfl⟩ + | liftBind o k hv_o _ ih => + obtain ⟨res, c₁, trace, ho⟩ := R.total o hr hv_o + obtain ⟨s₁, hstep, hr₁⟩ := R.correct hr hv_o ho + obtain ⟨b, c₂, traces, s₂, tr, hes, hrun, hr₂, hlen⟩ := ih s₁ res hstep hr₁ + exact ⟨b, c₂, trace :: traces, s₂, ⟨o, res, s₁⟩ :: tr, + .call ho hes, .liftBind o k res hv_o hstep hrun, hr₂, congrArg (· + 1) hlen⟩ + +end RelRealization + +/-- Lift a deterministic `Realization D exec` to a relational `RelRealization D.toRelSpec exec`. -/ +def Realization.toRelRealization {D : Spec.{u, v, w}} {C τ : Type x} + {exec : Execution D C τ} (R : Realization D exec) : + RelRealization D.toRelSpec exec where + rep := R.rep + total o hr hv := R.total o hr hv + correct {s c o a c' trace} hr hv he := by + obtain ⟨ha, hr'⟩ := R.correct hr hv he + exact ⟨(D.step s o).1, Prod.ext rfl ha, hr'⟩ + +/-- Every native execution of a verified demonic `Hoare` client satisfies its abstract +postcondition in a related final state. -/ +theorem RelSpec.Hoare.realize {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} + {exec : RelExecution D C Trace} (R : RelRealization D exec) + {pre : D.State → Prop} {post : α → D.State → Prop} {p : D.Client α} + (h : D.Hoare pre p post) {s : D.State} {c c' : C} {a : α} {traces : List Trace} + (hp : pre s) (hr : R.rep s c) (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' := by + obtain ⟨hv, hpost⟩ := h s hp + obtain ⟨s', tr, hrun, hr', _⟩ := R.correct_client hr hv he + exact ⟨s', tr, hrun, hr', hpost a s' tr hrun⟩ + +/-- Per-operation worst-case resource certificate over a relational realization `R`. -/ +def RelBound {D : RelSpec.{u, v, w}} {C τ : Type x} {exec : RelExecution D C τ} + (R : RelRealization D exec) (weight : τ → ℕ) (charge : D.State → D.Op → ℕ) : Prop := + ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → + weight trace ≤ charge s o + +/-- Lift a deterministic `Bound R weight charge` to `RelBound R.toRelRealization weight charge`. -/ +theorem Bound.toRelBound {D : Spec.{u, v, w}} {C τ : Type x} {exec : Execution D C τ} + {R : Realization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + (B : Bound R weight charge) : + RelBound R.toRelRealization weight charge := + fun hr hv he => B hr hv he + +/-- Potential-method amortized resource certificate over a relational realization `R`: +for every native operation execution from `R.rep s c` on an admissible `o`, there exists a +permitted abstract post-state `s'` satisfying `D.step s o s' a`, `R.rep s' c'`, and the +single-step potential inequality `weight trace + potential s' c' ≤ charge s o + potential s c`. -/ +structure RelAmortized {D : RelSpec.{u, v, w}} {C Trace : Type x} + {exec : RelExecution D C Trace} (R : RelRealization D exec) (weight : Trace → ℕ) + (potential : D.State → C → ℕ) (charge : D.State → D.Op → ℕ) : Prop where + /-- Angelic single-step potential inequality witnessing a permitted post-state `s'`. -/ + step : ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → + ∃ s', D.step s o s' a ∧ R.rep s' c' ∧ + weight trace + potential s' c' ≤ charge s o + potential s c + +namespace RelAmortized + +variable {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} + {exec : RelExecution D C Trace} {R : RelRealization D exec} + {weight : Trace → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} + (A : RelAmortized R weight potential charge) + +include A in +/-- **Amortized telescoping along adaptive clients (`executes_le`).** +For any demonically admissible client `p` and native execution `RelExecutes exec p c a c' traces`, +there exist a final abstract state `s'` and transcript `tr` such that `D.ClientRun s p a s' tr`, +`R.rep s' c'`, and +`total weight traces + potential s' c' ≤ D.totalCost charge s tr + potential s c`. -/ +theorem executes_le {p : D.Client α} {s : D.State} {c c' : C} {a : α} + {traces : List Trace} (hr : R.rep s c) (hv : D.DemonicValid s p) + (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces + potential s' c' ≤ + D.totalCost charge s tr + potential s c := by + induction he generalizing s with + | pure a c => + exact ⟨s, [], .pure s a, hr, rfl, Nat.le_refl _⟩ + | @call o k c res c₁ trace b c₂ traces ho hes ih => + obtain ⟨hv_o, hv_k⟩ := (D.demonicValid_liftBind_iff s o k).mp hv + obtain ⟨s₁, hstep, hr₁, hpot₁⟩ := A.step hr hv_o ho + obtain ⟨s₂, tr, hrun, hr₂, hlen, hpot₂⟩ := ih hr₁ (hv_k s₁ res hstep) + refine ⟨s₂, ⟨o, res, s₁⟩ :: tr, .liftBind o k res hv_o hstep hrun, hr₂, + congrArg (· + 1) hlen, ?_⟩ + simp only [total_cons, RelSpec.totalCost_cons] + omega + +/-- Alias for `RelAmortized.executes_le` matching `Amortized.client`. -/ +abbrev client := @executes_le + +include A in +/-- **Existence, correctness, and amortized bound (`run_client_amortized`).** +From related initial states, every demonically admissible client `p` admits a native execution +satisfying `D.ClientRun`, `R.rep`, and the telescoping potential bound. -/ +theorem run_client_amortized (p : D.Client α) {s : D.State} {c : C} + (hr : R.rep s c) (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces + potential s' c' ≤ + D.totalCost charge s tr + potential s c := by + obtain ⟨a, c', traces, _, _, he, _, _, _⟩ := R.exists_executes p hr hv + obtain ⟨s', tr, hrun, hr', hlen, hpot⟩ := A.executes_le hr hv he + exact ⟨a, c', traces, s', tr, he, hrun, hr', hlen, hpot⟩ + +include A in +/-- Alias for `RelAmortized.run_client_amortized` matching `Amortized.exists_executes`. -/ +theorem exists_executes (p : D.Client α) {s : D.State} {c : C} + (hr : R.rep s c) (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces + potential s' c' ≤ + D.totalCost charge s tr + potential s c := + A.run_client_amortized p hr hv + +/-- A worst-case `RelBound` is a zero-potential `RelAmortized` certificate. -/ +theorem of_relBound (B : RelBound R weight charge) : + RelAmortized R weight (fun _ _ => 0) charge where + step hr hv he := by + obtain ⟨s', hstep, hr'⟩ := R.correct hr hv he + exact ⟨s', hstep, hr', by simpa using B hr hv he⟩ + +end RelAmortized + +/-- Lift a deterministic `Amortized R weight potential charge` certificate to +`RelAmortized R.toRelRealization weight potential charge`. -/ +theorem Amortized.toRelAmortized {D : Spec.{u, v, w}} {C Trace : Type x} + {exec : Execution D C Trace} {R : Realization D exec} + {weight : Trace → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} + (A : Amortized R weight potential charge) : + RelAmortized R.toRelRealization weight potential charge where + step {s c o a c' trace} hr hv he := by + obtain ⟨ha, hr'⟩ := R.correct hr hv he + exact ⟨(D.step s o).1, Prod.ext rfl ha, hr', A.step hr hv he⟩ + +namespace RelBound + +variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + (B : RelBound R weight charge) + +include B in +/-- Under a worst-case `RelBound`, every native execution of a demonically admissible client has +total weight at most `D.totalCost charge s tr` for some valid `ClientRun` transcript `tr`. -/ +theorem client {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List τ} + (hr : R.rep s c) (hv : D.DemonicValid s p) (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces ≤ D.totalCost charge s tr := by + obtain ⟨s', tr, hrun, hr', hlen, hpot⟩ := (RelAmortized.of_relBound B).executes_le hr hv he + exact ⟨s', tr, hrun, hr', hlen, by simpa using hpot⟩ + +include B in +/-- Under a worst-case `RelBound`, every demonically admissible client admits a native execution +meeting the total charge bound. -/ +theorem exists_executes (p : D.Client α) {s : D.State} {c : C} + (hr : R.rep s c) (hv : D.DemonicValid s p) : + ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ + traces.length = tr.length ∧ + total weight traces ≤ D.totalCost charge s tr := by + obtain ⟨a, c', traces, s', tr, he, hrun, hr', hlen, hpot⟩ := + (RelAmortized.of_relBound B).run_client_amortized p hr hv + exact ⟨a, c', traces, s', tr, he, hrun, hr', hlen, by simpa using hpot⟩ + +end RelBound + +/-- Transport a demonic `CostHoare` triple through a `RelAmortized` certificate to bound native +execution weight plus final potential by `budget s + potential s c`. -/ +theorem RelSpec.CostHoare.realize_amortized {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} + {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} + {p : D.Client α} (A : RelAmortized R weight potential charge) + (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} + {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) + (he : RelExecutes exec p c a c' traces) : + ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' ∧ + total weight traces + potential s' c' ≤ budget s + potential s c := by + obtain ⟨hv, hpost⟩ := h.1 s hs + obtain ⟨s', tr, hrun, hr', _, hpot⟩ := A.executes_le hr hv he + have hbud := h.2 s hs a s' tr hrun + exact ⟨s', tr, hrun, hr', hpost a s' tr hrun, by omega⟩ + +/-- Transport a demonic `CostHoare` triple through a worst-case `RelBound` certificate. -/ +theorem RelSpec.CostHoare.realize {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} + {p : D.Client α} (B : RelBound R weight charge) + (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} + {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) + (he : RelExecutes exec p c a c' traces) : + (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by + obtain ⟨s', _, _, hr', hpost, hpot⟩ := + h.realize_amortized (RelAmortized.of_relBound B) hs hr he + exact ⟨⟨s', hr', hpost⟩, by simpa using hpot⟩ + +/-- Existence of a native execution satisfying a demonic `CostHoare` contract under `RelBound`. -/ +theorem RelSpec.CostHoare.exists_executes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} + {exec : RelExecution D C τ} {R : RelRealization D exec} + {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} + {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} + {p : D.Client α} (B : RelBound R weight charge) + (h : D.CostHoare charge pre p post budget) {s : D.State} {c : C} + (hs : pre s) (hr : R.rep s c) : + ∃ a c' traces, RelExecutes exec p c a c' traces ∧ + (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by + obtain ⟨a, c', traces, _, _, he, _, _, _⟩ := R.exists_executes p hr (h.1 s hs).1 + exact ⟨a, c', traces, he, h.realize B hs hr he⟩ + +end Cslib.Stateful diff --git a/ComplexityTests/StatefulRelational.lean b/ComplexityTests/StatefulRelational.lean new file mode 100644 index 000000000..8d3373331 --- /dev/null +++ b/ComplexityTests/StatefulRelational.lean @@ -0,0 +1,241 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: local-only (test driver) + +public import Complexity.Init +public import Complexity.Foundations.Semantics.Stateful.Relational + +/-! +# Tests for Relational Stateful Specifications + +Concrete non-vacuous tests for `Cslib.Stateful.RelSpec`, its totality class `RelSpec.Total`, +projection `RelSpec.toLTS`, zero-cost deterministic embedding `Spec.toRelSpec`, demonic adaptive +client semantics (`ClientRun`, `DemonicValid`, `RelSpec.Hoare`, `RelSpec.CostHoare`), and angelic +backend realization and amortized analysis (`RelRealization`, `RelBound`, `RelAmortized`). +-/ + +@[expose] public section + +namespace Cslib.Stateful.Tests + +open Cslib.Stateful + +/-! ## 1. A genuinely nondeterministic relational specification (two-slot allocator) -/ + +/-- Operations of a two-slot index allocator: `alloc` nondeterministically returns any free +slot in `Fin 2`; `free i` releases slot `i`. -/ +inductive TwoSlotOp : Type + | alloc : TwoSlotOp + | free : Fin 2 → TwoSlotOp + deriving DecidableEq + +/-- Relational specification of a two-slot allocator whose state `(b₀, b₁) : Bool × Bool` +records whether slots `0` and `1` are currently occupied. When both slots are free, `alloc` +nondeterministically returns either `0` or `1`. -/ +abbrev TwoSlotAlloc : RelSpec where + State := Bool × Bool + Op := TwoSlotOp + Result + | .alloc => Fin 2 + | .free _ => Unit + valid s + | .alloc => s.1 = false ∨ s.2 = false + | .free _ => True + step s o s' a := + match o, a with + | .alloc, i => + (i = 0 ∧ s.1 = false ∧ s' = (true, s.2)) ∨ + (i = 1 ∧ s.2 = false ∧ s' = (s.1, true)) + | .free i, () => + s' = if i = 0 then (false, s.2) else (s.1, false) + +instance : TwoSlotAlloc.Total where + step_total s o hv := by + cases o with + | alloc => + rcases hv with h0 | h1 + · exact ⟨(true, s.2), 0, Or.inl ⟨rfl, h0, rfl⟩⟩ + · exact ⟨(s.1, true), 1, Or.inr ⟨rfl, h1, rfl⟩⟩ + | free i => + exact ⟨if i = 0 then (false, s.2) else (s.1, false), (), rfl⟩ + +/-- Both slot `0` and slot `1` are valid outcomes of `alloc` from `(false, false)`. -/ +theorem twoSlotAlloc_nondeterministic : + ((true, false), (0 : Fin 2)) ∈ TwoSlotAlloc.outcomes (false, false) .alloc ∧ + ((false, true), (1 : Fin 2)) ∈ TwoSlotAlloc.outcomes (false, false) .alloc := + ⟨Or.inl ⟨rfl, rfl, rfl⟩, Or.inr ⟨rfl, rfl, rfl⟩⟩ + +/-- Sample LTS transition via `RelSpec.toLTS`. -/ +theorem twoSlotAlloc_toLTS_tr : + TwoSlotAlloc.toLTS.Tr (false, false) ⟨.alloc, (1 : Fin 2)⟩ (false, true) := + Or.inr ⟨rfl, rfl, rfl⟩ + +/-! ## 2. Adaptive client, demonic admissibility, and Hoare / CostHoare verification -/ + +/-- Adaptive client that allocates two slots sequentially, frees the first slot returned, and +returns the pair `(i, j)` of allocated slots. -/ +abbrev allocTwoAndFreeFirst : TwoSlotAlloc.Client (Fin 2 × Fin 2) := + .liftBind TwoSlotOp.alloc fun i => + .liftBind TwoSlotOp.alloc fun j => + .liftBind (TwoSlotOp.free i) fun _ => + .pure (i, j) + +/-- From `(false, false)`, `allocTwoAndFreeFirst` is demonically admissible across all +nondeterministic choices of the first and second allocations. -/ +theorem allocTwoAndFreeFirst_demonicValid : + TwoSlotAlloc.DemonicValid (false, false) allocTwoAndFreeFirst := by + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) TwoSlotOp.alloc _ (Or.inl rfl) + fun s₁ i h₁ => ?_ + have hv₂ : TwoSlotAlloc.valid s₁ TwoSlotOp.alloc := by + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · exact Or.inr rfl + · exact Or.inl rfl + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) TwoSlotOp.alloc _ hv₂ + fun s₂ j _ => ?_ + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) (TwoSlotOp.free i) _ trivial + fun s₃ _ _ => .pure s₃ (i, j) + +/-- Demonic `CostHoare` specification: under unit per-operation charge, every relational execution +of `allocTwoAndFreeFirst` from `(false, false)` returns distinct slots `i ≠ j`, leaves `i` free +and `j` occupied, and incurs abstract cost `3`. -/ +theorem allocTwoAndFreeFirst_costHoare : + TwoSlotAlloc.CostHoare (fun _ _ => 1) + (fun s => s = (false, false)) + allocTwoAndFreeFirst + (fun p s' => p.1 ≠ p.2 ∧ (if p.1 = 0 then s'.1 = false else s'.2 = false)) + (fun _ => 3) := by + refine ⟨fun s hs => ?_, fun s hs v s' tr hrun => ?_⟩ + · subst hs + refine ⟨allocTwoAndFreeFirst_demonicValid, fun v s' tr hrun => ?_⟩ + obtain ⟨_, i, s₁, tr₁, h₁, hrun₁, _⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun + obtain ⟨_, j, s₂, tr₂, h₂, hrun₂, _⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₁ + obtain ⟨_, _, s₃, tr₃, h₃, hrun₃, _⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₂ + obtain ⟨rfl, rfl, rfl⟩ := (RelSpec.clientRun_pure_iff ..).mp hrun₃ + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · rcases h₂ with ⟨_, hfalse, _⟩ | ⟨rfl, _, rfl⟩ + · cases hfalse + · subst h₃ + exact ⟨by decide, by decide⟩ + · rcases h₂ with ⟨rfl, _, rfl⟩ | ⟨_, hfalse, _⟩ + · subst h₃ + exact ⟨by decide, by decide⟩ + · cases hfalse + · subst hs + obtain ⟨_, i, s₁, tr₁, _, hrun₁, rfl⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun + obtain ⟨_, j, s₂, tr₂, _, hrun₂, rfl⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₁ + obtain ⟨_, _, s₃, tr₃, _, hrun₃, rfl⟩ := (RelSpec.clientRun_liftBind_iff ..).mp hrun₂ + obtain ⟨rfl, rfl, rfl⟩ := (RelSpec.clientRun_pure_iff ..).mp hrun₃ + simp + +/-- Consequence to plain demonic `Hoare`. -/ +theorem allocTwoAndFreeFirst_hoare : + TwoSlotAlloc.Hoare + (fun s => s = (false, false)) + allocTwoAndFreeFirst + (fun p s' => p.1 ≠ p.2 ∧ (if p.1 = 0 then s'.1 = false else s'.2 = false)) := + allocTwoAndFreeFirst_costHoare.hoare + +/-! ## 3. Concrete first-fit backend realization and amortized certificate -/ + +/-- Concrete first-fit implementation of `TwoSlotAlloc`: `alloc` chooses slot `0` when free and +slot `1` otherwise, recording `1` unit of execution trace cost per operation. -/ +def firstFitExec : RelExecution TwoSlotAlloc (Bool × Bool) ℕ + | c, .alloc, i, c', n => + n = 1 ∧ + if c.1 = false then i = 0 ∧ c' = (true, c.2) + else i = 1 ∧ c' = (c.1, true) + | c, .free i, (), c', n => + n = 1 ∧ c' = (if i = 0 then (false, c.2) else (c.1, false)) + +/-- Angelic realization of `TwoSlotAlloc` by `firstFitExec`. -/ +def firstFitRealization : RelRealization TwoSlotAlloc firstFitExec where + rep s c := s = c + total {s c} o hr _ := by + subst hr + cases o with + | alloc => + rcases Bool.dichotomy s.1 with h0 | h0 + · exact ⟨0, (true, s.2), 1, rfl, by simp [h0]⟩ + · exact ⟨1, (s.1, true), 1, rfl, by simp [h0]⟩ + | free i => + exact ⟨(), if i = 0 then (false, s.2) else (s.1, false), 1, rfl, rfl⟩ + correct {s c o a c' n} hr hv he := by + subst hr + cases o with + | alloc => + obtain ⟨_, hbranch⟩ := he + rcases Bool.dichotomy s.1 with h0 | h0 + · simp only [h0, ↓reduceIte] at hbranch + obtain ⟨rfl, rfl⟩ := hbranch + exact ⟨(true, s.2), Or.inl ⟨rfl, h0, rfl⟩, rfl⟩ + · simp only [h0, Bool.true_eq_false, ↓reduceIte] at hbranch + obtain ⟨rfl, rfl⟩ := hbranch + have h1 : s.2 = false := hv.resolve_left (by simp [h0]) + exact ⟨(s.1, true), Or.inr ⟨rfl, h1, rfl⟩, by simp [h0]⟩ + | free i => + obtain ⟨_, rfl⟩ := he + cases a + exact ⟨if i = 0 then (false, s.2) else (s.1, false), rfl, rfl⟩ + +/-- Per-operation worst-case bound for `firstFitRealization`. -/ +theorem firstFitBound : + RelBound firstFitRealization id (fun _ _ => 1) := by + intro s c o a c' n _ _ he + cases o with + | alloc => exact Nat.le_of_eq he.1 + | free _ => exact Nat.le_of_eq he.1 + +/-- Amortized certificate for `firstFitRealization` with zero potential. -/ +theorem firstFitAmortized : + RelAmortized firstFitRealization id (fun _ _ => 0) (fun _ _ => 1) := + RelAmortized.of_relBound firstFitBound + +/-- End-to-end execution of `allocTwoAndFreeFirst` on `firstFitRealization` via +`RelAmortized.run_client_amortized` and `RelSpec.CostHoare.realize_amortized`. -/ +theorem firstFit_allocTwoAndFreeFirst_executes : + ∃ v c' traces s' tr, + RelExecutes firstFitExec allocTwoAndFreeFirst (false, false) v c' traces ∧ + TwoSlotAlloc.ClientRun (false, false) allocTwoAndFreeFirst v s' tr ∧ + firstFitRealization.rep s' c' ∧ + (v.1 ≠ v.2 ∧ (if v.1 = 0 then s'.1 = false else s'.2 = false)) ∧ + total id traces ≤ 3 := by + obtain ⟨v, c', traces, _, _, he, _, _, _, _⟩ := + firstFitAmortized.run_client_amortized allocTwoAndFreeFirst + (s := (false, false)) (c := (false, false)) rfl allocTwoAndFreeFirst_demonicValid + obtain ⟨s', tr, hrun, hr, hpost, hcost⟩ := + allocTwoAndFreeFirst_costHoare.realize_amortized firstFitAmortized rfl rfl he + exact ⟨v, c', traces, s', tr, he, hrun, hr, hpost, by omega⟩ + +/-! ## 4. Layer 1 (`Spec.toRelSpec`) embedding and collapse equivalences -/ + +/-- Simple deterministic counter specification to exercise `Spec.toRelSpec`. -/ +abbrev CounterSpec : Spec where + State := ℕ + Op := Unit + Result _ := ℕ + step n _ := (n + 1, n) + +/-- Adaptive client on `CounterSpec` that increments twice and returns the sum of results. -/ +def counterTwice : CounterSpec.Client ℕ := + .liftBind () fun a => .liftBind () fun b => .pure (a + b) + +/-- `DemonicValid` on `CounterSpec.toRelSpec` collapses to `ValidHistory` on `CounterSpec`. -/ +theorem counterTwice_demonicValid (n : ℕ) : + CounterSpec.toRelSpec.DemonicValid n counterTwice := by + rw [CounterSpec.toRelSpec_demonicValid_iff] + exact ⟨trivial, trivial, trivial⟩ + +/-- `RelSpec.Hoare` on `CounterSpec.toRelSpec` is equivalent to `Spec.Hoare`. -/ +theorem counterTwice_relHoare : + CounterSpec.toRelSpec.Hoare (fun n => n = 5) counterTwice (fun v s' => v = 11 ∧ s' = 7) := by + rw [CounterSpec.toRelSpec_hoare_iff] + intro s hs + subst hs + exact ⟨⟨trivial, trivial, trivial⟩, rfl, rfl⟩ + +end Cslib.Stateful.Tests diff --git a/ComplexityTests/WordRAMRelSubroutine.lean b/ComplexityTests/WordRAMRelSubroutine.lean new file mode 100644 index 000000000..1eecc6baa --- /dev/null +++ b/ComplexityTests/WordRAMRelSubroutine.lean @@ -0,0 +1,351 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: local-only (test driver) + +public import Complexity.Init +public import Complexity.Computability.WordRAM.RelSubroutine + +/-! +# Tests for Relational Word-RAM Subroutine Realizations + +Concrete non-vacuous tests for `Cslib.WordRAM.RelSubroutineDS`, `RelDynamicProgram`, +`RelDynamicCertificate`, `SubroutineDS.toRelSubroutineDS`, `RelSubroutineDS.toStandard`, +`RelSubroutineDS.call_rename`, `RelSubroutineDS.run_client_amortized`, +`RelSubroutineDS.run_ops_exact_fresh`, and `RelSubroutineDS.run_ops_exact_reusable` on a +genuinely nondeterministic two-slot allocator specification. +-/ + +@[expose] public section + +namespace Cslib.WordRAM.Tests + +open Cslib Cslib.Stateful Cslib.WordRAM + +/-! ## 1. Nondeterministic two-slot allocator specification -/ + +/-- Relational specification of a two-slot allocator: from state `(b₀, b₁) : Bool × Bool`, an +allocation operation `()` nondeterministically returns any free slot `i : Fin 2` and marks slot +`i` occupied. When both slots are free (`(false, false)`), both `0` and `1` are permitted. -/ +abbrev TwoSlotAlloc : RelSpec where + State := Bool × Bool + Op := Unit + Result _ := Fin 2 + valid s _ := s.1 = false ∨ s.2 = false + step s _ s' i := + (i = 0 ∧ s.1 = false ∧ s' = (true, s.2)) ∨ + (i = 1 ∧ s.2 = false ∧ s' = (s.1, true)) + +instance : TwoSlotAlloc.Total where + step_total s _ hv := by + rcases hv with h0 | h1 + · exact ⟨(true, s.2), 0, Or.inl ⟨rfl, h0, rfl⟩⟩ + · exact ⟨(s.1, true), 1, Or.inr ⟨rfl, h1, rfl⟩⟩ + +/-! ## 2. Concrete `SubroutineCode .exact` implementation and `RelSubroutineDS` certificate -/ + +/-- Six-register `.exact` Word-RAM subroutine implementing `TwoSlotAlloc` by sequential +allocation of slot `0` followed by slot `1`: +* register `0`: `baseReg` (persistent) +* register `1`: `frontierReg` (persistent) +* register `2`: `curReg` (scratch) +* register `3`: `zeroReg` (scratch) +* register `4`: `oneReg` / `output` (scratch) +* register `5`: persistent next-free-slot counter (`0`, `1`, or `2`). -/ +abbrev twoSlotCode : SubroutineCode .exact 6 0 where + allocSpan := .op (.copy 0 1) + initSetup := .op (.const 5 0) + code := .seq (.op (.copy 4 5)) (.seq (.op (.const 3 1)) (.op (.exactBin .add 5 5 3))) + frontierReg := 1 + baseReg := 0 + initParams := Fin.elim0 + curReg := 2 + zeroReg := 3 + oneReg := 4 + incCur := .exactBin .add 2 2 4 + input i := i == 2 || i == 3 || i == 4 + output := 4 + +/-- Representation invariant for `twoSlotCode` at word width `w` and base address `base`. -/ +abbrev twoSlotRep (w base : ℕ) (s : Bool × Bool) (x : Data 6) : Prop := + x.Bounded w ∧ x.regs 0 = base ∧ + ((s = (false, false) ∧ x.regs 5 = 0) ∨ + (s = (true, false) ∧ x.regs 5 = 1) ∨ + (s = (true, true) ∧ x.regs 5 = 2)) + +/-- `RelSubroutineDS` certificate showing that `twoSlotCode` realizes the nondeterministic +`TwoSlotAlloc` specification whenever `Fits w 2`. -/ +abbrev twoSlotSubroutine (w base : ℕ) (h2 : Fits w 2) : + RelSubroutineDS twoSlotCode TwoSlotAlloc w base Fin.elim0 (false, false) where + space := 0 + spanTime := 1 + setupTime := 1 + initPotential := 0 + charge _ _ := 3 + rep := twoSlotRep w base + potential _ := 0 + encode _ _ := 0 + argMatch _ _ := True + decode _ regs := some ⟨regs 4 % 2, Nat.mod_lt _ (by decide)⟩ + bounded hr := hr.1 + arguments_fit _ _ _ _ := fits_zero w + encode_matches _ _ := trivial + rep_supported {s x x'} hregs _ hx' hr := by + refine ⟨hx', ?_, ?_⟩ + · rw [hregs 0 rfl]; exact hr.2.1 + · rw [hregs 5 rfl]; exact hr.2.2 + potential_supported _ _ _ := rfl + clear_nodup := by decide + params_not_clear j := j.elim0 + incCur_eval {x} hone hfit := by + simp only [Op.eval_exactBin, BinOp.evalExact, hone, Option.bind_some, Data.put_of_fits hfit] + span_exec {x} _ hfr _ := by + refine ⟨x.set 0 (x.regs 1), ⟨1, Nat.le_refl 1, Exec.copy⟩, rfl, ?_, ?_, fun j => j.elim0⟩ + · simp [Data.set, hfr] + · simp [Data.set, hfr] + setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by + refine ⟨x₁.set 5 0, ⟨1, Nat.le_refl 1, Exec.const (fits_zero w)⟩, ?_, ?_, Nat.le_refl 0, + fun _ _ => rfl⟩ + · simp [Data.set, hfr₁] + · refine ⟨hx₁.set (fits_zero w), ?_, Or.inl ⟨rfl, by simp [Data.set]⟩⟩ + simp [Data.set, hb₁] + step_exec {s x} _ hr hv _ := by + have h1 : Fits w 1 := h2.of_le (by decide) + rcases hr with ⟨hx, hb, ⟨rfl, h5⟩ | ⟨rfl, h5⟩ | ⟨rfl, _⟩⟩ + · let x₁ := x.set 4 (x.regs 5) + let x₂ := x₁.set 3 1 + let y := x₂.set 5 (x₂.regs 5 + x₂.regs 3) + have hfit : Fits w (x₂.regs 5 + x₂.regs 3) := by + simpa [x₂, x₁, Data.set, h5] using h1 + have heval : (Op.exactBin .add (5 : Fin 6) 5 3).eval w x₂ = some y := by + simp only [Op.eval_exactBin, BinOp.evalExact, Option.bind_some, Data.put_of_fits hfit, y] + have hex : Exec w twoSlotCode.code x y 3 := + Exec.copy.seq ((Exec.const h1).seq (Exec.op heval)) + have hy_b : y.Bounded w := hex.bounded hx + refine ⟨y, (true, false), 0, 3, hex, Or.inl ⟨rfl, rfl, rfl⟩, ?_, ?_, + fun _ _ => rfl, Nat.le_refl 3⟩ + · simp [y, x₂, x₁, Data.set, h5] + · refine ⟨hy_b, ?_, Or.inr (Or.inl ⟨rfl, ?_⟩)⟩ + · simp [y, x₂, x₁, Data.set, hb] + · simp [y, x₂, x₁, Data.set, h5] + · let x₁ := x.set 4 (x.regs 5) + let x₂ := x₁.set 3 1 + let y := x₂.set 5 (x₂.regs 5 + x₂.regs 3) + have hfit : Fits w (x₂.regs 5 + x₂.regs 3) := by + simpa [x₂, x₁, Data.set, h5] using h2 + have heval : (Op.exactBin .add (5 : Fin 6) 5 3).eval w x₂ = some y := by + simp only [Op.eval_exactBin, BinOp.evalExact, Option.bind_some, Data.put_of_fits hfit, y] + have hex : Exec w twoSlotCode.code x y 3 := + Exec.copy.seq ((Exec.const h1).seq (Exec.op heval)) + have hy_b : y.Bounded w := hex.bounded hx + refine ⟨y, (true, true), 1, 3, hex, Or.inr ⟨rfl, rfl, rfl⟩, ?_, ?_, + fun _ _ => rfl, Nat.le_refl 3⟩ + · simp [y, x₂, x₁, Data.set, h5] + · refine ⟨hy_b, ?_, Or.inr (Or.inr ⟨rfl, ?_⟩)⟩ + · simp [y, x₂, x₁, Data.set, hb] + · simp [y, x₂, x₁, Data.set, h5] + · cases hv with + | inl h => cases h + | inr h => cases h + +/-! ## 3. Exercising `call_rename`, `run_client_amortized`, and flat-program execution -/ + +/-- Adaptive client allocating two slots sequentially and returning both indices. -/ +abbrev allocPair : TwoSlotAlloc.Client (Fin 2 × Fin 2) := + .liftBind () fun i => + .liftBind () fun j => + .pure (i, j) + +/-- `allocPair` is demonically admissible from `(false, false)` across all permitted outcomes. -/ +theorem allocPair_demonicValid : + TwoSlotAlloc.DemonicValid (false, false) allocPair := by + refine RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) () _ (Or.inl rfl) fun s₁ i h₁ => ?_ + have hv₂ : TwoSlotAlloc.valid s₁ () := by + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · exact Or.inr rfl + · exact Or.inl rfl + exact RelSpec.DemonicValid.liftBind (D := TwoSlotAlloc) () _ hv₂ + fun s₂ j _ => .pure s₂ (i, j) + +/-- Standalone adaptive-client amortized theorem via `RelSubroutineDS.toStandard` and +`RelSubroutineDS.run_client_amortized`. -/ +theorem twoSlot_run_client_amortized (w base : ℕ) (h2 : Fits w 2) + {x : Data 6} (hr : (twoSlotSubroutine w base h2).rep (false, false) x) : + ∃ a y traces s' tr, + Stateful.RelExecutes + ((twoSlotSubroutine w base h2).toStandard.toRelDynamicProgram.ExecOp w) + allocPair x a y traces ∧ + TwoSlotAlloc.ClientRun (false, false) allocPair a s' tr ∧ + (twoSlotSubroutine w base h2).rep s' y ∧ + traces.length = tr.length ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ + TwoSlotAlloc.totalCost (fun _ _ => 4) (false, false) tr := + (twoSlotSubroutine w base h2).toStandard.run_client_bound (fun _ => rfl) + allocPair hr allocPair_demonicValid + +/-- Canonical injective register embedding `Fin 6 → Fin 8`. -/ +def embed6to8 : Fin 6 → Fin 8 := Fin.castLE (by decide) + +theorem embed6to8_injective : Function.Injective embed6to8 := + (Fin.castLEOrderEmb (by decide : 6 ≤ 8)).injective + +/-- Relocated inline call via `RelSubroutineDS.call_rename` inside an 8-register caller. -/ +theorem twoSlot_call_rename (w base : ℕ) (h2 : Fits w 2) + {x : Data 8} (hx : (x.comap embed6to8).Bounded w) + (hr : (twoSlotSubroutine w base h2).rep (false, false) (x.comap embed6to8)) : + ∃ z s' a t, + ExecWithin w (.seq (.op (.const 2 0)) (twoSlotCode.code.rename embed6to8)) x z t ∧ + TwoSlotAlloc.step (false, false) () s' a ∧ + (twoSlotSubroutine w base h2).decode () + (z.comap embed6to8).regs = some a ∧ + (twoSlotSubroutine w base h2).rep s' (z.comap embed6to8) ∧ + Modifies (Block.mk base 0).footprint (x.set 2 0).mem z.mem ∧ + (∀ j ∉ Set.range embed6to8, z.regs j = (x.set 2 0).regs j) ∧ + t ≤ 4 := by + have hprep : ExecWithin (B := .exact) w (.op (.const (2 : Fin 8) 0)) x (x.set 2 0) 1 := + ⟨1, Nat.le_refl 1, Exec.const (fits_zero w)⟩ + have hb' : ((x.set 2 0).comap embed6to8).Bounded w := by + refine ⟨fun i => ?_, hx.2⟩ + by_cases hi : embed6to8 i = 2 + · simp [Data.comap, Data.set, hi, fits_zero w] + · simp only [Fin.isValue, Data.comap_regs, Data.set_regs, Function.update_of_ne hi] + exact hx.1 i + have hreg : ∀ i : Fin 6, twoSlotCode.input i = false → + (x.set 2 0).regs (embed6to8 i) = x.regs (embed6to8 i) := by + intro i hi + have hne : embed6to8 i ≠ 2 := by + fin_cases i <;> revert hi <;> decide + simp [Data.set, Function.update_of_ne hne] + obtain ⟨z, s', a, t, he, hstep, hdec, hrep, hmod, hframe, ht⟩ := + (twoSlotSubroutine w base h2).call_rename + embed6to8_injective () hr (Or.inl rfl) hprep hb' hreg (fun _ _ => rfl) trivial + exact ⟨z, s', a, t, he, hstep, hdec, hrep, hmod, hframe, by omega⟩ + +/-- Two-operation sequence `[(), ()]` is demonically valid from `(false, false)`. -/ +theorem twoSlot_validOpSeq : + TwoSlotAlloc.ValidOpSeq (false, false) [(), ()] := by + refine ⟨Or.inl rfl, fun s₁ a₁ h₁ => ?_⟩ + rcases h₁ with ⟨rfl, _, rfl⟩ | ⟨rfl, _, rfl⟩ + · exact ⟨Or.inr rfl, fun _ _ _ => trivial⟩ + · exact ⟨Or.inl rfl, fun _ _ _ => trivial⟩ + +/-- Flat-program execution on fresh zeroed memory via `RelSubroutineDS.run_ops_exact_fresh`. -/ +theorem twoSlot_run_ops_fresh (w base : ℕ) (h2 : Fits w 2) + {d₀ : Data 6} (hd₀ : d₀.Bounded w) (hfr : d₀.regs 1 = base) + (hz : ∀ a ∈ (Block.mk base 0).footprint, d₀.mem a = 0) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w twoSlotCode.initFreshProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w twoSlotCode.initFreshProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base 0).footprint d₀.mem x₀.mem ∧ + (twoSlotSubroutine w base h2).RunsRelOps + (twoSlotSubroutine w base h2).prepareRegs (false, false) x₀ [(), ()] tr s_fin z + t_ops ∧ + tr.ops = [(), ()] ∧ + TwoSlotAlloc.ValidTranscript (false, false) tr s_fin ∧ + (twoSlotSubroutine w base h2).rep s_fin z ∧ + t_init ≤ 2 ∧ + t_ops ≤ TwoSlotAlloc.totalCost (fun _ _ => 4) (false, false) tr := by + obtain ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, ht_ops⟩ := + (twoSlotSubroutine w base h2).run_ops_exact_fresh + hd₀ hfr (fun j => j.elim0) hz [(), ()] twoSlot_validOpSeq + exact ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, by simpa using ht_ops⟩ + +/-- Flat-program execution on reusable dirty memory via `RelSubroutineDS.run_ops_exact_reusable`. -/ +theorem twoSlot_run_ops_reusable (w base : ℕ) (h2 : Fits w 2) + {d₀ : Data 6} (hd₀ : d₀.Bounded w) (hfr : d₀.regs 1 = base) : + ∃ pc₀ x₀ t_init tr s_fin z t_ops, + Runs w twoSlotCode.initClearProg ⟨0, d₀⟩ ⟨pc₀, x₀⟩ t_init ∧ + (∃ out, step w twoSlotCode.initClearProg ⟨pc₀, x₀⟩ = .halt out) ∧ + Modifies (Block.mk base 0).footprint d₀.mem x₀.mem ∧ + (twoSlotSubroutine w base h2).RunsRelOps + (twoSlotSubroutine w base h2).prepareRegs (false, false) x₀ [(), ()] tr s_fin z + t_ops ∧ + tr.ops = [(), ()] ∧ + TwoSlotAlloc.ValidTranscript (false, false) tr s_fin ∧ + (twoSlotSubroutine w base h2).rep s_fin z ∧ + t_init ≤ 8 ∧ + t_ops ≤ TwoSlotAlloc.totalCost (fun _ _ => 4) (false, false) tr := by + obtain ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, ht_ops⟩ := + (twoSlotSubroutine w base h2).run_ops_exact_reusable + hd₀ hfr (fun j => j.elim0) [(), ()] twoSlot_validOpSeq + exact ⟨pc₀, x₀, t_init, tr, s_fin, z, t_ops, hruns, hhalt, hmod, hops_run, hops, htr, hrep, + ht_init, by simpa using ht_ops⟩ + +/-! ## 4. Lifting a deterministic `SubroutineDS` via `SubroutineDS.toRelSubroutineDS` -/ + +/-- Deterministic specification of a bounded two-step counter (`0 → 1 → 2`). -/ +abbrev DetCounter2 : Spec where + State := ℕ + Op := Unit + Result _ := Fin 2 + valid n _ := n < 2 + step n _ := (n + 1, ⟨n % 2, Nat.mod_lt _ (by decide)⟩) + +/-- Deterministic `SubroutineDS` certificate for `twoSlotCode` over `DetCounter2`. -/ +abbrev detCounter2Subroutine (w base : ℕ) (h2 : Fits w 2) : + SubroutineDS twoSlotCode DetCounter2 w base Fin.elim0 0 where + space := 0 + spanTime := 1 + setupTime := 1 + initPotential := 0 + charge _ _ := 3 + rep n x := x.Bounded w ∧ x.regs 0 = base ∧ x.regs 5 = n ∧ n ≤ 2 + potential _ := 0 + encode _ _ := 0 + argMatch _ _ := True + decode _ regs := some ⟨regs 4 % 2, Nat.mod_lt _ (by decide)⟩ + bounded hr := hr.1 + rep_unique hr₁ hr₂ := hr₁.2.2.1.symm.trans hr₂.2.2.1 + arguments_fit _ _ _ _ := fits_zero w + encode_matches _ _ := trivial + rep_supported {s x x'} hregs _ hx' hr := by + refine ⟨hx', ?_, ?_, hr.2.2.2⟩ + · rw [hregs 0 rfl]; exact hr.2.1 + · rw [hregs 5 rfl]; exact hr.2.2.1 + potential_supported _ _ _ := rfl + clear_nodup := by decide + params_not_clear j := j.elim0 + incCur_eval {x} hone hfit := by + simp only [Op.eval_exactBin, BinOp.evalExact, hone, Option.bind_some, Data.put_of_fits hfit] + span_exec {x} _ hfr _ := by + refine ⟨x.set 0 (x.regs 1), ⟨1, Nat.le_refl 1, Exec.copy⟩, rfl, ?_, ?_, fun j => j.elim0⟩ + · simp [Data.set, hfr] + · simp [Data.set, hfr] + setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by + refine ⟨x₁.set 5 0, ⟨1, Nat.le_refl 1, Exec.const (fits_zero w)⟩, ?_, ?_, Nat.le_refl 0, + fun _ _ => rfl⟩ + · simp [Data.set, hfr₁] + · refine ⟨hx₁.set (fits_zero w), ?_, ?_, Nat.zero_le 2⟩ + · simp [Data.set, hb₁] + · simp [Data.set] + step_exec {s x} _ hr hv _ := by + have h1 : Fits w 1 := h2.of_le (by decide) + obtain ⟨hx, hb, h5, _⟩ := hr + let x₁ := x.set 4 (x.regs 5) + let x₂ := x₁.set 3 1 + let y := x₂.set 5 (x₂.regs 5 + x₂.regs 3) + have hfit : Fits w (x₂.regs 5 + x₂.regs 3) := by + have hfit2 : Fits w (s + 1) := h2.of_le hv + simpa [x₂, x₁, Data.set, h5] using hfit2 + have heval : (Op.exactBin .add (5 : Fin 6) 5 3).eval w x₂ = some y := by + simp only [Op.eval_exactBin, BinOp.evalExact, Option.bind_some, Data.put_of_fits hfit, y] + have hex : Exec w twoSlotCode.code x y 3 := + Exec.copy.seq ((Exec.const h1).seq (Exec.op heval)) + refine ⟨y, 3, hex, ?_, ⟨hex.bounded hx, ?_, ?_, hv⟩, fun _ _ => rfl, Nat.le_refl 3⟩ + · simp [y, x₂, x₁, Data.set, h5] + · simp [y, x₂, x₁, Data.set, hb] + · simp [y, x₂, x₁, Data.set, h5] + +/-- Lifting `detCounter2Subroutine` to `RelSubroutineDS` via `SubroutineDS.toRelSubroutineDS`. -/ +def detCounter2RelSubroutine (w base : ℕ) (h2 : Fits w 2) : + RelSubroutineDS twoSlotCode DetCounter2.toRelSpec w base Fin.elim0 0 := + (detCounter2Subroutine w base h2).toRelSubroutineDS + +end Cslib.WordRAM.Tests From 012511e6d019284380f7e13d76dcc4cf2cb5a0d9 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 16:33:45 +0000 Subject: [PATCH 02/18] feat(wordram): geometric Las Vegas loop expected-time bound and randomized SubroutineDS (#1634) --- Complexity.lean | 1 + .../WordRAM/Randomized/Expected.lean | 1738 ++++++++++++++++- .../WordRAM/Randomized/Subroutine.lean | 1046 ++++++++++ .../WordRAMRandomizedSubroutine.lean | 350 ++++ 4 files changed, 3133 insertions(+), 2 deletions(-) create mode 100644 Complexity/Computability/WordRAM/Randomized/Subroutine.lean create mode 100644 ComplexityTests/WordRAMRandomizedSubroutine.lean diff --git a/Complexity.lean b/Complexity.lean index 227ff8d31..b339fe51f 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -2009,6 +2009,7 @@ public import Complexity.Computability.WordRAM.Randomized.Defs public import Complexity.Computability.WordRAM.Randomized.Expected public import Complexity.Computability.WordRAM.Randomized.LoopSegment public import Complexity.Computability.WordRAM.Randomized.Structured +public import Complexity.Computability.WordRAM.Randomized.Subroutine public import Complexity.Computability.WordRAM.RegisterFrame public import Complexity.Computability.WordRAM.Sequence public import Complexity.Computability.WordRAM.Signed.Arithmetic diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index 7d9e1844a..acfe26e2d 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -47,6 +47,9 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, remaining instructions from a `RandCfg r` or initial `State r`. * `Cslib.WordRAM.expectedCodeTime`: `ℝ≥0∞`-valued expected instruction count of a structured `Code B r`. +* `Cslib.WordRAM.ExecCodeDist`: compositional structured execution certificate pairing an output + distribution `μ : PMF (Data r)` with an expected step bound `T : ℝ≥0∞`, closed under unbounded + almost-surely terminating geometric `while` loops (`ExecCodeDist.loop_geometric`). ## Main results @@ -63,6 +66,10 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, * `Cslib.WordRAM.expectedCodeTime_seq`, `Cslib.WordRAM.expectedCodeTime_branch_le`, `Cslib.WordRAM.expectedCodeTime_loop_le_of_drift`: compositional expected-time rules for structured `Code`. +* `Cslib.WordRAM.ExecCodeDist.loop_geometric`, + `Cslib.WordRAM.expectedStepsFrom_loop_le_of_geometric`, + `Cslib.WordRAM.ExecCodeDist.program_halts`: expected-time and almost-sure halting rules for + geometric Las Vegas `while` loops and structured programs. Discrete and conditional expectation principles are classical background; see [MitzenmacherUpfal2017], Sections 2.1 and 2.3. The particular native tail-sum/Bellman definitions, @@ -636,6 +643,1733 @@ theorem expectedCodeTime_loop_le_of_drift {w : ℕ} {cond : Cond r} {body : Code · exact ⟨Exists.intro _ (ExecRand.loop_exit hc), by rw [expectedCodeTime_loop_exit hc]; exact le_add_self⟩ -end WordRAM +/-! ## Distributional structured execution (`ExecCodeDist`) and geometric Las Vegas loops -/ -end Cslib +/-- Non-faulting structured randomized executions transfer instruction-for-instruction along any +defined-evaluation basis simulation `ψ : B.Simulates B'`. -/ +theorem ExecRand.mapSimulates {B' : Basis} (ψ : B.Simulates B') {w : ℕ} {c : Code B r} + {x : Data r} {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) : + ExecRand w (c.mapSimulates ψ) x m := by + induction h with + | op ho => exact .op (Op.eval_mapSimulates_of_some ψ ho) + | rand hfit => exact .rand hfit + | seq _ _ ih₁ ih₂ => exact .seq ih₁ ih₂ + | branch_pos hc _ ih => exact .branch_pos hc ih + | branch_neg hc _ ih => exact .branch_neg hc ih + | loop_exit hc => exact .loop_exit hc + | loop_iter hc _ _ ihb ihl => exact .loop_iter hc ihb ihl + +/-- Pointwise halting probability at `.halted v d` is monotone in the horizon `T`. -/ +theorem evalRand_halted_mono (w : ℕ) (p : Program B r) (s : State r) (v : ℕ) (d : Data r) : + Monotone fun T => evalRand w p T s (.halted v d) := by + refine monotone_nat_of_le_succ fun T => ?_ + have hstep : evalRand w p (T + 1) s = + (evalRand w p T s).bind (fun cfg => (stepRand w p cfg).eval) := by + rw [evalRand, runRand_add w p T 1, ProbComp.eval_bind] + simp only [evalRand, runRand_succ, runRand_zero, bind_pure] + rw [hstep, PMF.bind_apply] + calc + evalRand w p T s (.halted v d) + = evalRand w p T s (.halted v d) * (stepRand w p (.halted v d)).eval (.halted v d) := by + simp [stepRand_halted] + _ ≤ ∑' cfg, evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d) := + ENNReal.le_tsum + (f := fun cfg => evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d)) + (RandCfg.halted v d) + +private theorem finsetSum_iSup_of_monotone {α : Type*} (s : Finset α) {f : ℕ → α → ℝ≥0∞} + (hf : ∀ a, Monotone fun n => f n a) : + (∑ a ∈ s, ⨆ n, f n a) = ⨆ n, ∑ a ∈ s, f n a := by + classical + induction s using Finset.induction_on with + | empty => simp + | @insert a₀ s ha₀ ih => + rw [Finset.sum_insert ha₀, ih] + refine le_antisymm ?_ (iSup_le fun n => ?_) + · rw [ENNReal.add_iSup] + simp_rw [ENNReal.iSup_add] + refine iSup_le fun j => iSup_le fun i => le_iSup_of_le (max i j) ?_ + rw [Finset.sum_insert ha₀] + exact add_le_add (hf a₀ (le_max_left i j)) + (Finset.sum_le_sum fun a _ => hf a (le_max_right i j)) + · rw [Finset.sum_insert ha₀] + exact add_le_add (le_iSup (fun k => f k a₀) n) + (le_iSup (fun k => ∑ a ∈ s, f k a) n) + +private theorem tsum_iSup_of_monotone {α : Type*} {f : ℕ → α → ℝ≥0∞} + (hf : ∀ a, Monotone fun n => f n a) : + (∑' a, ⨆ n, f n a) = ⨆ n, ∑' a, f n a := by + rw [ENNReal.tsum_eq_iSup_sum] + simp_rw [finsetSum_iSup_of_monotone _ hf] + rw [iSup_comm] + simp_rw [← ENNReal.tsum_eq_iSup_sum] + +private theorem tsum_finsetSum {α β : Type*} (s : Finset β) (f : β → α → ℝ≥0∞) : + (∑' a, ∑ b ∈ s, f b a) = ∑ b ∈ s, ∑' a, f b a := by + classical + induction s using Finset.induction_on with + | empty => simp + | @insert b₀ s hb₀ ih => + simp only [Finset.sum_insert hb₀, ENNReal.tsum_add, ih] + +/-- Finite-horizon approximation of `expectedStepsFrom`: expected number of active (`.running`) +steps within the first `N` steps starting from configuration `cfg`. -/ +noncomputable def expectedStepsWithin (w : ℕ) (p : Program B r) (N : ℕ) (cfg : RandCfg r) : + ℝ≥0∞ := + ∑ t ∈ Finset.range N, (runRand w p t cfg).eval.toOuterMeasure {cfg' | ∃ u, cfg' = .running u} + +@[simp] +theorem expectedStepsWithin_zero (w : ℕ) (p : Program B r) (cfg : RandCfg r) : + expectedStepsWithin w p 0 cfg = 0 := by + simp [expectedStepsWithin] + +@[simp] +theorem expectedStepsWithin_halted (w : ℕ) (p : Program B r) (N v : ℕ) (d : Data r) : + expectedStepsWithin w p N (.halted v d) = 0 := by + simp [expectedStepsWithin, PMF.toOuterMeasure_pure_apply] + +@[simp] +theorem expectedStepsWithin_faulted (w : ℕ) (p : Program B r) (N : ℕ) : + expectedStepsWithin w p N .faulted = 0 := by + simp [expectedStepsWithin, PMF.toOuterMeasure_pure_apply] + +theorem expectedStepsWithin_mono (w : ℕ) (p : Program B r) (cfg : RandCfg r) : + Monotone fun N => expectedStepsWithin w p N cfg := by + intro N₁ N₂ hN + exact Finset.sum_le_sum_of_subset (Finset.range_mono hN) + +theorem iSup_expectedStepsWithin (w : ℕ) (p : Program B r) (cfg : RandCfg r) : + (⨆ N : ℕ, expectedStepsWithin w p N cfg) = expectedStepsFrom w p cfg := + (ENNReal.tsum_eq_iSup_nat).symm + +theorem expectedStepsWithin_le_expectedStepsFrom (w : ℕ) (p : Program B r) (N : ℕ) + (cfg : RandCfg r) : + expectedStepsWithin w p N cfg ≤ expectedStepsFrom w p cfg := by + rw [← iSup_expectedStepsWithin] + exact le_iSup (fun K => expectedStepsWithin w p K cfg) N + +/-- 1-step horizon recursion for `expectedStepsWithin`: at horizon `N + 1`, an active state `s` +incurs `1` step plus the `N`-step continuation expectation. -/ +theorem expectedStepsWithin_running_succ (w : ℕ) (p : Program B r) (N : ℕ) (s : State r) : + expectedStepsWithin w p (N + 1) (.running s) = + 1 + ProbComp.expect (stepRand w p (.running s)) (expectedStepsWithin w p N) := by + rw [expectedStepsWithin, Finset.sum_range_succ'] + have hzero : (runRand w p 0 (.running s)).eval.toOuterMeasure + {cfg' | ∃ u, cfg' = .running u} = 1 := by + simp [PMF.toOuterMeasure_pure_apply] + rw [hzero, add_comm] + congr 1 + have hsucc : ∀ t : ℕ, (runRand w p (t + 1) (.running s)).eval.toOuterMeasure + {cfg' | ∃ u, cfg' = .running u} = + ∑' cfg, (stepRand w p (.running s)).eval cfg * + (runRand w p t cfg).eval.toOuterMeasure {cfg' | ∃ u, cfg' = .running u} := by + intro t + rw [runRand_succ, ProbComp.eval_bind, PMF.toOuterMeasure_bind_apply] + simp only [hsucc] + rw [← tsum_finsetSum] + simp only [ProbComp.expect, expectedStepsWithin, Finset.mul_sum] + +theorem expectedStepsWithin_running_le (w : ℕ) (p : Program B r) (N : ℕ) (s : State r) : + expectedStepsWithin w p N (.running s) ≤ + 1 + ProbComp.expect (stepRand w p (.running s)) (expectedStepsWithin w p N) := by + cases N with + | zero => simp + | succ N => + rw [expectedStepsWithin_running_succ] + gcongr + exact ProbComp.expect_mono fun cfg => expectedStepsWithin_mono w p cfg (Nat.le_succ N) + +theorem ExecRand.expectedStepsWithin_assemble {w : ℕ} {c : Code B r} {x : Data r} + {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (N : ℕ) : + expectedStepsWithin w p N (.running ⟨pc, x⟩) ≤ + ProbComp.expect m (fun z => + (z.2 : ℝ≥0∞) + expectedStepsWithin w p N (.running ⟨pc + c.size, z.1⟩)) := by + induction h generalizing pc with + | @op o x y ho => + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_op (s := ⟨pc, x⟩) hp.head ho, ProbComp.expect_pure, ProbComp.expect_pure] + simp [Code.size] + | @rand dst src x hfit => + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, ProbComp.expect_bind, + ProbComp.expect_bind] + simp only [ProbComp.expect_pure, Code.size, Nat.cast_one] + rw [ProbComp.expect_add, ProbComp.expect_const] + | @seq c₁ c₂ x m₁ m₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine (ih₁ hp.append_left).trans ?_ + rw [ProbComp.expect_bind] + refine ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_ + refine (add_le_add le_rfl (ih₂ z₁ hz₁ hc₂)).trans (le_of_eq ?_) + rw [ProbComp.expect_bind, + ← ProbComp.expect_const (m₂ z₁.1) (z₁.2 : ℝ≥0∞), ← ProbComp.expect_add] + refine ProbComp.expect_congr_on_support fun z₂ _ => ?_ + simp only [ProbComp.expect_pure, Code.size, Nat.cast_add, add_assoc] + | @branch_pos cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + refine (add_le_add le_rfl (ih hc')).trans (le_of_eq ?_) + rw [ProbComp.expect_map, ← ProbComp.expect_const m 1, ← ProbComp.expect_add] + refine ProbComp.expect_congr_on_support fun z _ => ?_ + simp only [Code.size, Nat.cast_add, Nat.cast_one] + rw [show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + ac_rfl + | @branch_neg cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ih hp.cons_tail.append_left)).trans ?_ + rw [ProbComp.expect_map, ← ProbComp.expect_const m 1, ← ProbComp.expect_add] + refine ProbComp.expect_mono_on_support fun z _ => ?_ + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + refine (add_le_add le_rfl + (add_le_add le_rfl (expectedStepsWithin_running_le w p N ⟨pc + 1 + neg.size, z.1⟩))).trans + (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 1 + neg.size, z.1⟩) hj, ProbComp.expect_pure] + simp only [Code.size, Nat.cast_add, Nat.cast_ofNat] + rw [show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + set E := expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), z.1⟩) + calc + 1 + ((z.2 : ℝ≥0∞) + (1 + E)) = (z.2 : ℝ≥0∞) + (1 + 1) + E := by ac_rfl + _ = (z.2 : ℝ≥0∞) + 2 + E := by rw [one_add_one_eq_two] + | @loop_exit cond body x hc => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hj : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (expectedStepsWithin_running_le w p N ⟨pc + 1, x⟩)).trans + (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 1, x⟩) hj, ProbComp.expect_pure] + simp only [Code.size, Nat.cast_ofNat] + rw [← add_assoc, one_add_one_eq_two, show pc + body.size + 3 = pc + (body.size + 3) by omega] + | @loop_iter cond body x mb ml hc _ _ ihb ihl => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ihb hbody_contains)).trans ?_ + rw [ProbComp.expect_bind, ← ProbComp.expect_const mb 1, ← ProbComp.expect_add] + refine ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_ + have hjump_le : expectedStepsWithin w p N (.running ⟨pc + 2 + body.size, z₁.1⟩) ≤ + 1 + ProbComp.expect (ml z₁.1) (fun z => + (z.2 : ℝ≥0∞) + + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, z.1⟩)) := by + refine (expectedStepsWithin_running_le w p N ⟨pc + 2 + body.size, z₁.1⟩).trans ?_ + rw [stepRand_jump (s := ⟨pc + 2 + body.size, z₁.1⟩) hj, ProbComp.expect_pure] + exact add_le_add le_rfl (ihl z₁ hz₁ hp) + refine (add_le_add le_rfl (add_le_add le_rfl hjump_le)).trans (le_of_eq ?_) + rw [ProbComp.expect_bind] + have hconst : (1 : ℝ≥0∞) + ((z₁.2 : ℝ≥0∞) + + (1 + ProbComp.expect (ml z₁.1) fun z => + (z.2 : ℝ≥0∞) + + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, z.1⟩))) = + ProbComp.expect (ml z₁.1) (fun _ => (z₁.2 : ℝ≥0∞) + 2) + + ProbComp.expect (ml z₁.1) (fun z => + (z.2 : ℝ≥0∞) + + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, z.1⟩)) := by + rw [ProbComp.expect_const, ← one_add_one_eq_two] + ac_rfl + rw [hconst, ← ProbComp.expect_add] + refine ProbComp.expect_congr_on_support fun z₂ _ => ?_ + simp only [ProbComp.expect_pure, Nat.cast_add, Nat.cast_ofNat] + rw [← add_assoc ((z₁.2 : ℝ≥0∞) + 2) (z₂.2 : ℝ≥0∞), add_right_comm (z₁.2 : ℝ≥0∞) 2] + +/-- Along `ExecRand w c x m`, if no continuation from `⟨pc + c.size, y⟩` faults within `T` steps +for any supported outcome `y`, then running `p` from `⟨pc, x⟩` does not fault within `T` steps. -/ +theorem ExecRand.evalRand_faulted_eq_zero {w : ℕ} {c : Code B r} {x : Data r} + {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) {p : Program B r} {pc T : ℕ} + (hp : p.Contains pc (c.assemble pc)) + (hcont : ∀ z ∈ m.support, ∀ t ≤ T, evalRand w p t ⟨pc + c.size, z.1⟩ .faulted = 0) : + ∀ t ≤ T, evalRand w p t ⟨pc, x⟩ .faulted = 0 := by + induction h generalizing pc T with + | @op o x y ho => + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_op (s := ⟨pc, x⟩) hp.head ho, pure_bind] + simpa [evalRand, Code.size] using hcont (y, 1) rfl k (by omega) + | @rand dst src x hfit => + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, bind_assoc, + ProbComp.eval_bind, PMF.bind_apply, ENNReal.tsum_eq_zero] + intro sample + have hmem : (x.set dst sample.val, 1) ∈ + (ProbComp.unifFin (x.regs src) >>= fun s => pure (x.set dst s.val, 1)).support := + OracleComp.mem_supportWith_bind_iff.mpr ⟨sample, ProbComp.mem_support_unifFin _ sample, rfl⟩ + have hzero := hcont _ hmem k (by omega) + simp only [pure_bind] + exact mul_eq_zero_of_right _ hzero + | @seq c₁ c₂ x m₁ m₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine ih₁ hp.append_left fun z₁ hz₁ t ht => ?_ + refine ih₂ z₁ hz₁ hc₂ (T := t) (fun z₂ hz₂ t' ht' => ?_) t le_rfl + have hmem : (z₂.1, z₁.2 + z₂.2) ∈ + (m₁ >>= fun z₁ => m₂ z₁.1 >>= fun z₂ => pure (z₂.1, z₁.2 + z₂.2)).support := + OracleComp.mem_supportWith_bind_iff.mpr + ⟨z₁, hz₁, OracleComp.mem_supportWith_bind_iff.mpr ⟨z₂, hz₂, rfl⟩⟩ + simpa [Code.size, Nat.add_assoc] using hcont _ hmem t' (ht'.trans ht) + | @branch_pos cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hc' (T := k) (fun z hz t' ht' => ?_) k le_rfl + have hmem : (z.1, z.2 + 1) ∈ ((fun z : Data r × ℕ => (z.1, z.2 + 1)) <$> m).support := + OracleComp.mem_supportWith_map_iff.mpr ⟨z, hz, rfl⟩ + simpa [Code.size, show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + using hcont _ hmem t' (by omega) + | @branch_neg cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hp.cons_tail.append_left (T := k) (fun z hz t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 1 + neg.size, z.1⟩) hj, pure_bind] + have hmem : (z.1, z.2 + 2) ∈ ((fun z : Data r × ℕ => (z.1, z.2 + 2)) <$> m).support := + OracleComp.mem_supportWith_map_iff.mpr ⟨z, hz, rfl⟩ + simpa [evalRand, Code.size, + show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using hcont _ hmem k' (by omega) + | @loop_exit cond body x hc => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hj : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + cases k with + | zero => simp + | succ k' => + rw [runRand_succ, stepRand_jump (s := ⟨pc + 1, x⟩) hj, pure_bind] + simpa [evalRand, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using hcont (x, 2) rfl k' (by omega) + | @loop_iter cond body x mb ml hc _ _ ihb ihl => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ihb hbody_contains (T := k) (fun z₁ hz₁ t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 2 + body.size, z₁.1⟩) hj, pure_bind] + refine ihl z₁ hz₁ hp (T := k') (fun z₂ hz₂ t'' ht'' => ?_) k' le_rfl + have hmem : (z₂.1, z₁.2 + z₂.2 + 2) ∈ + (mb >>= fun z₁ => ml z₁.1 >>= fun z₂ => pure (z₂.1, z₁.2 + z₂.2 + 2)).support := + OracleComp.mem_supportWith_bind_iff.mpr + ⟨z₁, hz₁, OracleComp.mem_supportWith_bind_iff.mpr ⟨z₂, hz₂, rfl⟩⟩ + exact hcont _ hmem t'' (by omega) + +private theorem iSup_evalRand_halted_le_of_step {w : ℕ} {p : Program B r} {s s' : State r} + (hstep : stepRand w p (.running s) = pure (.running s')) (v : ℕ) (d : Data r) : + (⨆ T : ℕ, evalRand w p T s' (.halted v d)) ≤ ⨆ T : ℕ, evalRand w p T s (.halted v d) := by + refine iSup_le fun T => le_iSup_of_le (T + 1) ?_ + simp only [evalRand, runRand_succ, hstep, pure_bind, le_rfl] + +/-- Along `ExecRand w c x m`, the expected supremum halting probability from `⟨pc + c.size, y⟩` is a +lower bound on the supremum halting probability from `⟨pc, x⟩`. -/ +theorem ExecRand.le_iSup_evalRand_halted {w : ℕ} {c : Code B r} {x : Data r} + {m : ProbComp (Data r × ℕ)} (h : ExecRand w c x m) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (v : ℕ) (d : Data r) : + ProbComp.expect m (fun z => ⨆ T : ℕ, evalRand w p T ⟨pc + c.size, z.1⟩ (.halted v d)) ≤ + ⨆ T : ℕ, evalRand w p T ⟨pc, x⟩ (.halted v d) := by + induction h generalizing pc with + | @op o x y ho => + rw [ProbComp.expect_pure] + simpa [Code.size] using + iSup_evalRand_halted_le_of_step (stepRand_op (s := ⟨pc, x⟩) hp.head ho) v d + | @rand dst src x hfit => + rw [ProbComp.expect_bind] + simp only [ProbComp.expect_pure, Code.size] + change (∑' sample : Fin (x.regs src + 1), + (ProbComp.unifFin (x.regs src)).eval sample * + ⨆ T : ℕ, evalRand w p T ⟨pc + 1, x.set dst sample.val⟩ (.halted v d)) ≤ + ⨆ T : ℕ, evalRand w p T ⟨pc, x⟩ (.halted v d) + simp_rw [ENNReal.mul_iSup] + rw [tsum_iSup_of_monotone fun sample T₁ T₂ hT => + mul_le_mul_right (evalRand_halted_mono w p ⟨pc + 1, x.set dst sample.val⟩ v d hT) _] + refine iSup_le fun T => le_iSup_of_le (T + 1) ?_ + rw [evalRand, runRand_succ, stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, bind_assoc, + ProbComp.eval_bind, PMF.bind_apply] + refine le_of_eq (tsum_congr fun sample => ?_) + simp [evalRand] + | @seq c₁ c₂ x m₁ m₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_).trans (ih₁ hp.append_left) + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₂ _ => ?_).trans (ih₂ z₁ hz₁ hc₂) + simp only [ProbComp.expect_pure, Code.size, Nat.add_assoc, le_rfl] + | @branch_pos cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + neg.size + 2, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_map] + refine (ProbComp.expect_mono_on_support fun z _ => ?_).trans + ((ih hc').trans (iSup_evalRand_halted_le_of_step hstep v d)) + simp only [Code.size, + show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega, le_rfl] + | @branch_neg cond pos neg x m hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_map] + refine (ProbComp.expect_mono_on_support fun z _ => ?_).trans + ((ih hp.cons_tail.append_left).trans (iSup_evalRand_halted_le_of_step hstep v d)) + simpa [Code.size, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 1 + neg.size, z.1⟩) hj) v d + | @loop_exit cond body x hc => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hj : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_pure] + simpa [Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] using + (iSup_evalRand_halted_le_of_step (stepRand_jump (s := ⟨pc + 1, x⟩) hj) v d).trans + (iSup_evalRand_halted_le_of_step hstep v d) + | @loop_iter cond body x mb ml hc _ _ ihb ihl => + have hbr : p[pc]? = some (.branch cond (pc + 2)) := hp.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 2, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₁ hz₁ => ?_).trans + ((ihb hbody_contains).trans (iSup_evalRand_halted_le_of_step hstep v d)) + rw [ProbComp.expect_bind] + refine (ProbComp.expect_mono_on_support fun z₂ _ => ?_).trans + ((ihl z₁ hz₁ hp).trans + (iSup_evalRand_halted_le_of_step (stepRand_jump (s := ⟨pc + 2 + body.size, z₁.1⟩) hj) v d)) + simp only [ProbComp.expect_pure, le_rfl] + +/-! ### Absorbing Markov chain for geometric Las Vegas loops -/ + +/-- One step of the stopped loop Markov kernel on `Data r`: active states (`cond.Holds x`) step +according to `μ_body x`, while exiting states (`¬cond.Holds x`) are absorbing. -/ +noncomputable def loopStepPMF (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r) : + PMF (Data r) := + open Classical in + if cond.Holds x then μ_body x else PMF.pure x + +theorem loopStepPMF_of_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : cond.Holds x) : + loopStepPMF cond μ_body x = μ_body x := by + classical + simp [loopStepPMF, hc] + +theorem loopStepPMF_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : ¬cond.Holds x) : + loopStepPMF cond μ_body x = PMF.pure x := by + classical + simp [loopStepPMF, hc] + +/-- `K`-step iterate of `loopStepPMF`. -/ +noncomputable def loopIterPMF (cond : Cond r) (μ_body : Data r → PMF (Data r)) : + ℕ → Data r → PMF (Data r) + | 0, x => PMF.pure x + | K + 1, x => (loopStepPMF cond μ_body x).bind (loopIterPMF cond μ_body K) + +@[simp] +theorem loopIterPMF_zero (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r) : + loopIterPMF cond μ_body 0 x = PMF.pure x := rfl + +theorem loopIterPMF_succ (cond : Cond r) (μ_body : Data r → PMF (Data r)) (K : ℕ) (x : Data r) : + loopIterPMF cond μ_body (K + 1) x = + (loopStepPMF cond μ_body x).bind (loopIterPMF cond μ_body K) := rfl + +theorem loopIterPMF_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : ¬cond.Holds x) (K : ℕ) : + loopIterPMF cond μ_body K x = PMF.pure x := by + induction K with + | zero => rfl + | succ K ih => rw [loopIterPMF_succ, loopStepPMF_of_not_holds hc, PMF.pure_bind, ih] + +theorem loopIterPMF_succ_right (cond : Cond r) (μ_body : Data r → PMF (Data r)) (K : ℕ) + (x : Data r) : + loopIterPMF cond μ_body (K + 1) x = + (loopIterPMF cond μ_body K x).bind (loopStepPMF cond μ_body) := by + induction K generalizing x with + | zero => simp [loopIterPMF] + | succ K ih => + rw [loopIterPMF_succ (K := K), PMF.bind_bind] + exact congrArg ((loopStepPMF cond μ_body x).bind ·) (funext ih) + +theorem loopIterPMF_mono_of_not_holds (cond : Cond r) (μ_body : Data r → PMF (Data r)) + (x : Data r) {y : Data r} (hy : ¬cond.Holds y) : + Monotone fun K => loopIterPMF cond μ_body K x y := by + refine monotone_nat_of_le_succ fun K => ?_ + rw [loopIterPMF_succ_right, PMF.bind_apply] + calc + loopIterPMF cond μ_body K x y + = loopIterPMF cond μ_body K x y * loopStepPMF cond μ_body y y := by + rw [loopStepPMF_of_not_holds hy, PMF.pure_apply_self, mul_one] + _ ≤ ∑' z, loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y := + ENNReal.le_tsum + (f := fun z => loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y) y + +theorem loopIterPMF_indicator_mono (cond : Cond r) (μ_body : Data r → PMF (Data r)) + (x y : Data r) : + Monotone fun K => Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y := by + intro K₁ K₂ hK + dsimp only + by_cases hy : y ∈ {z | ¬cond.Holds z} + · rw [Set.indicator_of_mem hy, Set.indicator_of_mem hy] + exact loopIterPMF_mono_of_not_holds cond μ_body x hy hK + · rw [Set.indicator_of_notMem hy, Set.indicator_of_notMem hy] + +/-- Combined loop state invariant: either active (`cond.Holds x ∧ I x`) or exited +(`¬cond.Holds x ∧ Q x`). -/ +def LoopStateInv (cond : Cond r) (I Q : Data r → Prop) (x : Data r) : Prop := + (cond.Holds x ∧ I x) ∨ (¬cond.Holds x ∧ Q x) + +theorem loopIterPMF_support_inv {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (K : ℕ) {x : Data r} (hx : LoopStateInv cond I Q x) : + ∀ y ∈ (loopIterPMF cond μ_body K x).support, LoopStateInv cond I Q y := by + induction K generalizing x with + | zero => + intro y hy + rw [loopIterPMF_zero, PMF.support_pure, Set.mem_singleton_iff] at hy + rwa [hy] + | succ K ih => + intro y hy + rw [loopIterPMF_succ, PMF.mem_support_bind_iff] at hy + obtain ⟨z, hz, hyz⟩ := hy + rcases hx with ⟨hcx, hIx⟩ | ⟨hcx, hQx⟩ + · rw [loopStepPMF_of_holds hcx] at hz + exact ih (hinv x hIx hcx z hz) y hyz + · rw [loopStepPMF_of_not_holds hcx, PMF.support_pure, Set.mem_singleton_iff] at hz + subst hz + exact ih (Or.inr ⟨hcx, hQx⟩) y hyz + +private theorem pmf_toOuterMeasure_compl_add (μ : PMF (Data r)) (cond : Cond r) : + μ.toOuterMeasure {y | ¬cond.Holds y} + μ.toOuterMeasure {y | cond.Holds y} = 1 := by + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + convert μ.tsum_coe using 1 + congr 1 + funext y + by_cases hc : cond.Holds y <;> simp [hc] + +private theorem pmf_step_drift_bound (μ : PMF (Data r)) (cond : Cond r) + {C p_exit : ℝ≥0∞} (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hp_exit : p_exit ≤ μ.toOuterMeasure {y | ¬cond.Holds y}) : + C + ∑' y, μ y * (if cond.Holds y then C / p_exit else 0) ≤ C / p_exit := by + set M := C / p_exit + have hC : C = p_exit * M := by + rw [mul_comm, ENNReal.div_mul_cancel hp_pos.ne' hp_top] + have hact : (∑' y, μ y * (if cond.Holds y then M else 0)) = + μ.toOuterMeasure {y | cond.Holds y} * M := by + rw [PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] + refine tsum_congr fun y => ?_ + by_cases hc : cond.Holds y <;> simp [hc] + calc + C + ∑' y, μ y * (if cond.Holds y then M else 0) + = p_exit * M + μ.toOuterMeasure {y | cond.Holds y} * M := by rw [hC, hact] + _ ≤ μ.toOuterMeasure {y | ¬cond.Holds y} * M + + μ.toOuterMeasure {y | cond.Holds y} * M := by gcongr + _ = (μ.toOuterMeasure {y | ¬cond.Holds y} + + μ.toOuterMeasure {y | cond.Holds y}) * M := by rw [← add_mul] + _ = M := by rw [pmf_toOuterMeasure_compl_add, one_mul] + +/-- Expected number of active loop iterations within the first `N` steps of `loopIterPMF`. -/ +private noncomputable def loopActiveSteps (cond : Cond r) (μ_body : Data r → PMF (Data r)) + (N : ℕ) (x : Data r) : ℝ≥0∞ := + ∑ K ∈ Finset.range N, (loopIterPMF cond μ_body K x).toOuterMeasure {y | cond.Holds y} + +private theorem loopActiveSteps_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} + {x : Data r} (hc : ¬cond.Holds x) (N : ℕ) : + loopActiveSteps cond μ_body N x = 0 := by + unfold loopActiveSteps + refine Finset.sum_eq_zero fun K _ => ?_ + rw [loopIterPMF_of_not_holds hc, PMF.toOuterMeasure_pure_apply] + simp [hc] + +private theorem loopActiveSteps_succ_of_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} + {x : Data r} (hc : cond.Holds x) (N : ℕ) : + loopActiveSteps cond μ_body (N + 1) x = + 1 + ∑' z, μ_body x z * loopActiveSteps cond μ_body N z := by + rw [loopActiveSteps, Finset.sum_range_succ'] + have hzero : (loopIterPMF cond μ_body 0 x).toOuterMeasure {y | cond.Holds y} = 1 := by + simp [PMF.toOuterMeasure_pure_apply, hc] + rw [hzero, add_comm] + congr 1 + have hsucc : ∀ K : ℕ, (loopIterPMF cond μ_body (K + 1) x).toOuterMeasure {y | cond.Holds y} = + ∑' z, μ_body x z * (loopIterPMF cond μ_body K z).toOuterMeasure {y | cond.Holds y} := by + intro K + rw [loopIterPMF_succ, loopStepPMF_of_holds hc, PMF.toOuterMeasure_bind_apply] + simp only [hsucc] + rw [← tsum_finsetSum] + simp only [loopActiveSteps, Finset.mul_sum] + +private theorem loopActiveSteps_le {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + (N : ℕ) {x : Data r} (hx : LoopStateInv cond I Q x) : + loopActiveSteps cond μ_body N x ≤ if cond.Holds x then 1 / p_exit else 0 := by + induction N generalizing x with + | zero => simp [loopActiveSteps] + | succ N ih => + rcases hx with ⟨hcx, hIx⟩ | ⟨hcx, _⟩ + · simp only [hcx, ↓reduceIte] + rw [loopActiveSteps_succ_of_holds hcx] + refine (add_le_add le_rfl (ENNReal.tsum_le_tsum fun z => ?_)).trans + (pmf_step_drift_bound (μ_body x) cond hp_pos hp_top (hp_exit x hIx hcx)) + by_cases hz : z ∈ (μ_body x).support + · exact mul_le_mul_right (ih (hinv x hIx hcx z hz)) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hz, zero_mul, zero_mul] + · simp [loopActiveSteps_of_not_holds hcx, hcx] + +theorem hasSum_loopExitDensity {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + {x : Data r} (hx : LoopStateInv cond I Q x) : + HasSum (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1 := by + rw [ENNReal.summable.hasSum_iff, tsum_iSup_of_monotone (loopIterPMF_indicator_mono cond μ_body x)] + simp_rw [← PMF.toOuterMeasure_apply] + set b := ⨆ K : ℕ, (loopIterPMF cond μ_body K x).toOuterMeasure {z | ¬cond.Holds z} + have hb_le : b ≤ 1 := iSup_le fun K => Cslib.Probability.PMF.toOuterMeasure_le_one _ _ + have hsub_zero : 1 - b = 0 := by + by_contra hne + have hN_bound : ∀ N : ℕ, (N : ℝ≥0∞) * (1 - b) ≤ 1 / p_exit := by + intro N + have hterm : ∀ K : ℕ, 1 - b ≤ + (loopIterPMF cond μ_body K x).toOuterMeasure {z | cond.Holds z} := by + intro K + refine tsub_le_iff_left.mpr ?_ + rw [← pmf_toOuterMeasure_compl_add (loopIterPMF cond μ_body K x) cond] + exact add_le_add + (le_iSup (fun J => (loopIterPMF cond μ_body J x).toOuterMeasure {z | ¬cond.Holds z}) K) + le_rfl + calc + (N : ℝ≥0∞) * (1 - b) + = ∑ _K ∈ Finset.range N, (1 - b) := by simp + _ ≤ ∑ K ∈ Finset.range N, + (loopIterPMF cond μ_body K x).toOuterMeasure {z | cond.Holds z} := + Finset.sum_le_sum fun K _ => hterm K + _ ≤ if cond.Holds x then 1 / p_exit else 0 := + loopActiveSteps_le hp_pos hp_top hinv hp_exit N hx + _ ≤ 1 / p_exit := by split_ifs <;> simp + have htop : (⊤ : ℝ≥0∞) ≤ 1 / p_exit := by + rw [← ENNReal.top_mul hne, ← ENNReal.iSup_natCast, ENNReal.iSup_mul] + exact iSup_le hN_bound + exact ENNReal.div_ne_top ENNReal.one_ne_top hp_pos.ne' (top_le_iff.mp htop) + exact le_antisymm hb_le (tsub_eq_zero_iff_le.mp hsub_zero) + +/-- Limiting exit distribution of `Code.loop cond body` starting from `x` under one-step body +distribution family `μ_body`. -/ +noncomputable def loopExitPMF (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r) : + PMF (Data r) := + open Classical in + if h : HasSum (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1 then + ⟨_, h⟩ + else + PMF.pure x + +theorem loopExitPMF_apply_of_hasSum {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (h : HasSum (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1) + (y : Data r) : + loopExitPMF cond μ_body x y = + ⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y := by + classical + simp only [loopExitPMF, h, ↓reduceDIte] + rfl + +theorem loopExitPMF_of_not_holds {cond : Cond r} {μ_body : Data r → PMF (Data r)} {x : Data r} + (hc : ¬cond.Holds x) : + loopExitPMF cond μ_body x = PMF.pure x := by + have hfun : (fun y => ⨆ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) = PMF.pure x := by + funext y + have hK : ∀ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y = PMF.pure x y := by + intro K + rw [loopIterPMF_of_not_holds hc] + by_cases hyx : y = x + · subst hyx + simp [hc] + · simp [PMF.pure_apply, hyx] + simp [hK] + ext y + rw [loopExitPMF_apply_of_hasSum (hfun ▸ (PMF.pure x).2), congrFun hfun y] + +theorem loopExitPMF_of_holds {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + {x : Data r} (hIx : I x) (hcx : cond.Holds x) : + loopExitPMF cond μ_body x = (μ_body x).bind (loopExitPMF cond μ_body) := by + ext y + have hx : LoopStateInv cond I Q x := Or.inl ⟨hcx, hIx⟩ + rw [loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx), + PMF.bind_apply] + have hzero : Set.indicator {z | ¬cond.Holds z} (PMF.pure x) y = 0 := by + by_cases hy : ¬cond.Holds y + · have hyx : y ≠ x := fun h => hy (h ▸ hcx) + simp [hy, PMF.pure_apply, hyx] + · simp [Set.indicator_of_notMem, hy] + have hiSup_shift : + (⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) = + ⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body (K + 1) x) y := by + refine le_antisymm (iSup_le fun K => ?_) (iSup_le fun K => + le_iSup (fun J => Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body J x) y) (K + 1)) + cases K with + | zero => + rw [loopIterPMF_zero, hzero] + exact zero_le + | succ K => + exact le_iSup + (fun J => Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body (J + 1) x) y) K + rw [hiSup_shift] + have hstep : ∀ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body (K + 1) x) y = + ∑' z, μ_body x z * + Set.indicator {w' | ¬cond.Holds w'} (loopIterPMF cond μ_body K z) y := by + intro K + rw [loopIterPMF_succ, loopStepPMF_of_holds hcx] + by_cases hy : ¬cond.Holds y + · simp [Set.indicator_of_mem, hy, PMF.bind_apply] + · simp [Set.indicator_of_notMem, hy] + simp_rw [hstep] + rw [← tsum_iSup_of_monotone + (f := fun K z => μ_body x z * + Set.indicator {w' | ¬cond.Holds w'} (loopIterPMF cond μ_body K z) y) + (fun z _ _ hK => mul_le_mul_right (loopIterPMF_indicator_mono cond μ_body z y hK) _)] + refine tsum_congr fun z => ?_ + by_cases hz : z ∈ (μ_body x).support + · rw [← ENNReal.mul_iSup, + loopExitPMF_apply_of_hasSum + (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit (hinv x hIx hcx z hz))] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hz, zero_mul] + simp + +theorem mem_support_loopExitPMF {cond : Cond r} {I Q : Data r → Prop} + {μ_body : Data r → PMF (Data r)} {p_exit : ℝ≥0∞} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hinv : ∀ x, I x → cond.Holds x → ∀ y ∈ (μ_body x).support, LoopStateInv cond I Q y) + (hp_exit : ∀ x, I x → cond.Holds x → p_exit ≤ (μ_body x).toOuterMeasure {y | ¬cond.Holds y}) + {x : Data r} (hx : LoopStateInv cond I Q x) + {y : Data r} (hy : y ∈ (loopExitPMF cond μ_body x).support) : + (¬cond.Holds y ∧ Q y) ∧ ∃ K, y ∈ (loopIterPMF cond μ_body K x).support := by + rw [PMF.mem_support_iff, + loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx)] at hy + have ⟨K, hK⟩ : ∃ K : ℕ, + Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y ≠ 0 := by + by_contra! h + exact hy (by simp [h]) + have hnc : ¬cond.Holds y := by + by_contra hc + exact hK (by simp [Set.indicator_of_notMem, hc]) + rw [Set.indicator_of_mem hnc, ← PMF.mem_support_iff] at hK + rcases loopIterPMF_support_inv hinv K hx y hK with ⟨hcy, _⟩ | ⟨_, hQy⟩ + · exact absurd hcy hnc + · exact ⟨⟨hnc, hQy⟩, K, hK⟩ + +/-! ### Compositional distributional execution (`ExecCodeDist`) -/ + +/-- `ExecCodeDist w c x μ T`: at word size `w`, running structured program `c` from data `x` +terminates almost surely without faulting, realizes exit distribution `μ : PMF (Data r)` at the +fallthrough address `pc + c.size` of any containing flat program, and uses at most `T : ℝ≥0∞` +expected instructions. + +Unlike `ExecRand`, which requires a finite-depth computation tree, `ExecCodeDist` includes +unbounded Las Vegas `while` loops whose each active iteration exits with probability bounded below +by `p_exit > 0` (`ExecCodeDist.loop_geometric`). -/ +inductive ExecCodeDist {B : Basis} {r : ℕ} (w : ℕ) : Code B r → Data r → PMF (Data r) → + ℝ≥0∞ → Prop where + | of_execRand {c : Code B r} {x : Data r} {m : ProbComp (Data r × ℕ)} : + ExecRand w c x m → + ExecCodeDist w c x (Prod.fst <$> m).eval (ProbComp.expect m fun z => (z.2 : ℝ≥0∞)) + | mono {c : Code B r} {x : Data r} {μ : PMF (Data r)} {T T' : ℝ≥0∞} : + ExecCodeDist w c x μ T → T ≤ T' → ExecCodeDist w c x μ T' + | seq {c₁ c₂ : Code B r} {x : Data r} {μ₁ : PMF (Data r)} + {μ₂ : Data r → PMF (Data r)} {T₁ : ℝ≥0∞} {T₂ : Data r → ℝ≥0∞} : + ExecCodeDist w c₁ x μ₁ T₁ → + (∀ y ∈ μ₁.support, ExecCodeDist w c₂ y (μ₂ y) (T₂ y)) → + ExecCodeDist w (.seq c₁ c₂) x (μ₁.bind μ₂) (T₁ + ∑' y, μ₁ y * T₂ y) + | branch_pos {cond : Cond r} {pos neg : Code B r} {x : Data r} + {μ : PMF (Data r)} {T : ℝ≥0∞} : + cond.Holds x → ExecCodeDist w pos x μ T → + ExecCodeDist w (.branch cond pos neg) x μ (T + 1) + | branch_neg {cond : Cond r} {pos neg : Code B r} {x : Data r} + {μ : PMF (Data r)} {T : ℝ≥0∞} : + ¬cond.Holds x → ExecCodeDist w neg x μ T → + ExecCodeDist w (.branch cond pos neg) x μ (T + 2) + | loop_geometric {cond : Cond r} {body : Code B r} + (I Q : Data r → Prop) {μ_body : Data r → PMF (Data r)} {T_body : Data r → ℝ≥0∞} + {B_body p_exit : ℝ≥0∞} {x : Data r} : + 0 < p_exit → p_exit ≠ ⊤ → + (∀ z, I z → cond.Holds z → ExecCodeDist w body z (μ_body z) (T_body z)) → + (∀ z, I z → cond.Holds z → T_body z ≤ B_body) → + (∀ z, I z → cond.Holds z → + ∀ y ∈ (μ_body z).support, LoopStateInv cond I Q y) → + (∀ z, I z → cond.Holds z → + p_exit ≤ (μ_body z).toOuterMeasure {y | ¬cond.Holds y}) → + LoopStateInv cond I Q x → + ExecCodeDist w (.loop cond body) x (loopExitPMF cond μ_body x) + ((B_body + 2) / p_exit + 2) + +namespace ExecCodeDist + +variable {w : ℕ} {c c₁ c₂ pos neg body : Code B r} {cond : Cond r} {x y : Data r} + {μ μ₁ μ' : PMF (Data r)} {T T' T₁ T₂ : ℝ≥0∞} + +theorem congr_dist (h : ExecCodeDist w c x μ T) (hμ : μ = μ') (hT : T ≤ T') : + ExecCodeDist w c x μ' T' := by + subst hμ + exact h.mono hT + +/-- Every deterministic `Exec w c x y n` gives `ExecCodeDist w c x (PMF.pure y) n`. -/ +theorem of_exec {n : ℕ} (h : Exec w c x y n) : + ExecCodeDist w c x (PMF.pure y) n := by + simpa [ProbComp.expect_pure] using of_execRand h.execRand + +/-- Every deterministic `ExecWithin w c x y n` gives `ExecCodeDist w c x (PMF.pure y) n`. -/ +theorem of_execWithin {n : ℕ} (h : ExecWithin w c x y n) : + ExecCodeDist w c x (PMF.pure y) n := by + obtain ⟨m, hm, he⟩ := h + exact (of_exec he).mono (Nat.cast_le.mpr hm) + +/-- Single deterministic operation instruction. -/ +theorem op {o : Op B r} (ho : o.eval w x = some y) : + ExecCodeDist w (.op o) x (PMF.pure y) 1 := by + simpa using of_exec (Exec.op ho) + +/-- Single `rand dst src` instruction. -/ +theorem rand {dst src : Fin r} (hfit : Fits w (x.regs src)) : + ExecCodeDist (B := B) w (.rand dst src) x + ((ProbComp.unifFin (x.regs src)).eval.map fun k => x.set dst k.val) 1 := by + have h := of_execRand (ExecRand.rand (B := B) (dst := dst) (src := src) hfit) + refine h.congr_dist ?_ ?_ + · ext z + simp [ProbComp.eval_map, PMF.map_apply] + · simp [ProbComp.expect_map, ProbComp.expect_const] + +/-- Exiting a loop immediately when `¬cond.Holds x` takes `2` steps and leaves `x` unchanged. -/ +theorem loop_exit (hc : ¬cond.Holds x) : + ExecCodeDist w (.loop cond body) x (PMF.pure x) 2 := by + simpa using of_exec (Exec.loop_exit hc) + +/-- Sequencing with a uniform continuation step bound `T₂`. -/ +theorem seq_const {μ₂ : Data r → PMF (Data r)} {T₂ : Data r → ℝ≥0∞} {B₂ : ℝ≥0∞} + (h₁ : ExecCodeDist w c₁ x μ₁ T₁) + (h₂ : ∀ y ∈ μ₁.support, ExecCodeDist w c₂ y (μ₂ y) (T₂ y)) + (hB₂ : ∀ y ∈ μ₁.support, T₂ y ≤ B₂) : + ExecCodeDist w (.seq c₁ c₂) x (μ₁.bind μ₂) (T₁ + B₂) := by + refine (h₁.seq h₂).mono (add_le_add le_rfl ?_) + calc + (∑' y, μ₁ y * T₂ y) + ≤ ∑' y, μ₁ y * B₂ := by + refine ENNReal.tsum_le_tsum fun y => ?_ + by_cases hy : y ∈ μ₁.support + · exact mul_le_mul_right (hB₂ y hy) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] + _ = B₂ := by rw [ENNReal.tsum_mul_right, μ₁.tsum_coe, one_mul] + +/-- Sequencing after a deterministic first stage (`PMF.pure y`). -/ +theorem seq_pure (h₁ : ExecCodeDist w c₁ x (PMF.pure y) T₁) + (h₂ : ExecCodeDist w c₂ y μ T₂) : + ExecCodeDist w (.seq c₁ c₂) x μ (T₁ + T₂) := by + classical + have h := h₁.seq (c₂ := c₂) (μ₂ := fun z => if z = y then μ else PMF.pure z) + (T₂ := fun z => if z = y then T₂ else 0) fun z hz => by + rw [PMF.support_pure, Set.mem_singleton_iff] at hz + subst hz + simpa only [↓reduceIte] using h₂ + refine h.congr_dist ?_ ?_ + · simp + · rw [tsum_eq_single y] + · simp + · intro z hz + simp [PMF.pure_apply, hz] + +/-- Every state in the support of `μ` preserves `Data.Bounded w` when `x.Bounded w`. -/ +theorem bounded (h : ExecCodeDist w c x μ T) (hx : x.Bounded w) : + ∀ y ∈ μ.support, y.Bounded w := by + induction h with + | @of_execRand c x m hm => + intro y hy + rw [ProbComp.eval_map, PMF.mem_support_map_iff] at hy + obtain ⟨z, hz, rfl⟩ := hy + rw [ProbComp.support_eval] at hz + exact hm.bounded hx hz + | mono _ _ ih => exact ih hx + | seq _ _ ih₁ ih₂ => + intro y hy + rw [PMF.mem_support_bind_iff] at hy + obtain ⟨z, hz, hyz⟩ := hy + exact ih₂ z hz (ih₁ hx z hz) y hyz + | branch_pos _ _ ih => exact ih hx + | branch_neg _ _ ih => exact ih hx + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + intro y hy + obtain ⟨_, K, hK⟩ := mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx_inv hy + have hK_bdd : ∀ K : ℕ, ∀ x₀ : Data r, LoopStateInv cond I Q x₀ → x₀.Bounded w → + ∀ z ∈ (loopIterPMF cond μ_body K x₀).support, z.Bounded w := by + intro K + induction K with + | zero => + intro x₀ _ hx₀ z hz + rw [loopIterPMF_zero, PMF.support_pure, Set.mem_singleton_iff] at hz + rwa [hz] + | succ K ihK => + intro x₀ hx₀_inv hx₀ z hz + rw [loopIterPMF_succ, PMF.mem_support_bind_iff] at hz + obtain ⟨u, hu, hzu⟩ := hz + rcases hx₀_inv with ⟨hcx₀, hIx₀⟩ | ⟨hcx₀, hQx₀⟩ + · rw [loopStepPMF_of_holds hcx₀] at hu + exact ihK u (hinv x₀ hIx₀ hcx₀ u hu) (ih_body x₀ hIx₀ hcx₀ hx₀ u hu) z hzu + · rw [loopStepPMF_of_not_holds hcx₀, PMF.support_pure, Set.mem_singleton_iff] at hu + rw [hu] at hzu + exact ihK x₀ (Or.inr ⟨hcx₀, hQx₀⟩) hx₀ z hzu + exact hK_bdd K x hx_inv hx y hK + +/-- Any register `i` not written by `c` (`c.Preserves i`) is preserved across the support of `μ`. -/ +theorem regs_eq {i : Fin r} (h : ExecCodeDist w c x μ T) (hi : c.Preserves i) : + ∀ y ∈ μ.support, y.regs i = x.regs i := by + induction h with + | @of_execRand c x m hm => + intro y hy + rw [ProbComp.eval_map, PMF.mem_support_map_iff] at hy + obtain ⟨z, hz, rfl⟩ := hy + rw [ProbComp.support_eval] at hz + exact hm.regs_eq hi hz + | mono _ _ ih => exact ih hi + | seq _ _ ih₁ ih₂ => + intro y hy + rw [PMF.mem_support_bind_iff] at hy + obtain ⟨z, hz, hyz⟩ := hy + exact (ih₂ z hz hi.2 y hyz).trans (ih₁ hi.1 z hz) + | branch_pos _ _ ih => exact ih hi.1 + | branch_neg _ _ ih => exact ih hi.2 + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + intro y hy + obtain ⟨_, K, hK⟩ := mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx_inv hy + have hK_eq : ∀ K : ℕ, ∀ x₀ : Data r, LoopStateInv cond I Q x₀ → + ∀ z ∈ (loopIterPMF cond μ_body K x₀).support, z.regs i = x₀.regs i := by + intro K + induction K with + | zero => + intro x₀ _ z hz + rw [loopIterPMF_zero, PMF.support_pure, Set.mem_singleton_iff] at hz + rw [hz] + | succ K ihK => + intro x₀ hx₀_inv z hz + rw [loopIterPMF_succ, PMF.mem_support_bind_iff] at hz + obtain ⟨u, hu, hzu⟩ := hz + rcases hx₀_inv with ⟨hcx₀, hIx₀⟩ | ⟨hcx₀, hQx₀⟩ + · rw [loopStepPMF_of_holds hcx₀] at hu + exact (ihK u (hinv x₀ hIx₀ hcx₀ u hu) z hzu).trans (ih_body x₀ hIx₀ hcx₀ hi u hu) + · rw [loopStepPMF_of_not_holds hcx₀, PMF.support_pure, Set.mem_singleton_iff] at hu + rw [hu] at hzu + exact ihK x₀ (Or.inr ⟨hcx₀, hQx₀⟩) z hzu + exact hK_eq K x hx_inv y hK + +/-- Distributional executions transfer along any defined-evaluation basis simulation +`ψ : B.Simulates B'`. -/ +theorem mapSimulates {B' : Basis} (ψ : B.Simulates B') (h : ExecCodeDist w c x μ T) : + ExecCodeDist w (c.mapSimulates ψ) x μ T := by + induction h with + | of_execRand hm => exact of_execRand (hm.mapSimulates ψ) + | mono _ hle ih => exact ih.mono hle + | seq _ _ ih₁ ih₂ => exact ih₁.seq ih₂ + | branch_pos hc _ ih => exact branch_pos hc ih + | branch_neg hc _ ih => exact branch_neg hc ih + | loop_geometric I Q hp_pos hp_top _ hB hinv hp_exit hx ih_body => + exact loop_geometric I Q hp_pos hp_top ih_body hB hinv hp_exit hx + +private theorem extend_comap_eq {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (x : Data r') : + (x.comap f).extend f x = x := by + ext j + · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j + by_cases hj : ∃ i, f i = j + · obtain ⟨i, rfl⟩ := hj + exact hf.extend_apply _ _ i + · exact Function.extend_apply' _ _ _ hj + · rfl + +private theorem extend_left_injective {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) + (x : Data r') : Function.Injective (fun y : Data r => y.extend f x) := by + intro y₁ y₂ h + have h' := congrArg (Data.comap f) h + simpa [Data.comap_extend hf] using h' + +private theorem pmf_map_toOuterMeasure {α β : Type*} (p : PMF α) (g : α → β) (S : Set β) : + (p.map g).toOuterMeasure S = p.toOuterMeasure (g ⁻¹' S) := by + rw [PMF.map, PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] + refine tsum_congr fun a => ?_ + rw [Function.comp_apply, PMF.toOuterMeasure_pure_apply] + by_cases ha : g a ∈ S <;> simp [ha] + +private theorem pmf_map_apply_injective {α β : Type*} (p : PMF α) {g : α → β} + (hg : g.Injective) (a : α) : + p.map g (g a) = p a := by + rw [PMF.map_apply, tsum_eq_single a] + · simp + · intro a' ha' + have hne : g a ≠ g a' := hg.ne ha'.symm + simp [hne] + +private theorem pmf_map_apply_not_mem_range {α β : Type*} (p : PMF α) (g : α → β) {b : β} + (hb : b ∉ Set.range g) : + p.map g b = 0 := by + rw [PMF.map_apply, ENNReal.tsum_eq_zero] + intro a + have hne : b ≠ g a := fun h => hb ⟨a, h.symm⟩ + simp [hne] + +private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [ENNReal.tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +private theorem loopIterPMF_rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) + (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r') (K : ℕ) (u : Data r) : + loopIterPMF (cond.rename f) (fun x' => (μ_body (x'.comap f)).map (fun y => y.extend f x)) + K (u.extend f x) = + (loopIterPMF cond μ_body K u).map (fun y => y.extend f x) := by + induction K generalizing u with + | zero => simp [loopIterPMF] + | succ K ih => + rw [loopIterPMF_succ, loopIterPMF_succ] + by_cases hc : cond.Holds u + · have hc' : (cond.rename f).Holds (u.extend f x) := by + rw [Cond.holds_rename, Data.comap_extend hf] + exact hc + rw [loopStepPMF_of_holds hc', loopStepPMF_of_holds hc, Data.comap_extend hf, + PMF.bind_map, PMF.map_bind] + refine congrArg ((μ_body u).bind ·) (funext fun z => ih z) + · have hc' : ¬(cond.rename f).Holds (u.extend f x) := by + rw [Cond.holds_rename, Data.comap_extend hf] + exact hc + rw [loopStepPMF_of_not_holds hc', loopStepPMF_of_not_holds hc, PMF.pure_bind, + PMF.pure_bind, ih] + +/-- Renaming a distributional execution along an injective register map `f : Fin r → Fin r'` +transforms the output distribution via `Data.extend f x` and preserves the expected step bound. -/ +theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r'} + (h : ExecCodeDist w c (x.comap f) μ T) : + ExecCodeDist w (c.rename f) x (μ.map fun y => y.extend f x) T := by + generalize hx₀ : x.comap f = x₀ at h + induction h generalizing x with + | @of_execRand c _ m hm => + subst hx₀ + have hm' := of_execRand (hm.rename hf) + refine hm'.congr_dist ?_ ?_ + · rw [Functor.map_map, ProbComp.eval_map, ProbComp.eval_map, PMF.map_comp] + rfl + · rw [ProbComp.expect_map] + | mono _ hle ih => + exact (ih hx₀).mono hle + | @seq c₁ c₂ _ μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + subst hx₀ + have h₁' := ih₁ rfl + have hseq := h₁'.seq + (μ₂ := fun y' => (μ₂ (y'.comap f)).map fun z => z.extend f x) + (T₂ := fun y' => T₂ (y'.comap f)) + fun y' hy' => by + rw [PMF.mem_support_map_iff] at hy' + obtain ⟨y₁, hy₁, rfl⟩ := hy' + have h₂' := ih₂ y₁ hy₁ (x := y₁.extend f x) (Data.comap_extend hf y₁ x) + simpa only [Data.comap_extend hf, Data.extend_extend hf] using h₂' + refine hseq.congr_dist ?_ ?_ + · rw [PMF.map_bind, PMF.bind_map] + refine congrArg (μ₁.bind ·) (funext fun y₁ => ?_) + simp [Data.comap_extend hf] + · refine add_le_add le_rfl ?_ + rw [tsum_pmf_map] + simp [Data.comap_extend hf] + | branch_pos hc _ ih => + subst hx₀ + exact branch_pos ((Cond.holds_rename _ _).2 hc) (ih rfl) + | branch_neg hc _ ih => + subst hx₀ + exact branch_neg (fun h => hc ((Cond.holds_rename _ _).1 h)) (ih rfl) + | @loop_geometric cond body I Q μ_body T_body B_body p_exit _ + hp_pos hp_top _ hB hinv hp_exit hx_inv ih_body => + subst hx₀ + let g : Data r → Data r' := fun y => y.extend f x + have hg : g.Injective := extend_left_injective hf x + let I' : Data r' → Prop := fun x' => ∃ u, I u ∧ x' = g u + let Q' : Data r' → Prop := fun x' => ∃ u, Q u ∧ x' = g u + let μ_body' : Data r' → PMF (Data r') := fun x' => (μ_body (x'.comap f)).map g + let T_body' : Data r' → ℝ≥0∞ := fun x' => T_body (x'.comap f) + have hinv' : ∀ z', I' z' → (cond.rename f).Holds z' → + ∀ y' ∈ (μ_body' z').support, LoopStateInv (cond.rename f) I' Q' y' := by + rintro z' ⟨u, hIu, rfl⟩ hcz' y' hy' + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + simp only [μ_body', g, Data.comap_extend hf, PMF.mem_support_map_iff] at hy' + obtain ⟨y, hy, rfl⟩ := hy' + rcases hinv u hIu hcu y hy with ⟨hcy, hIy⟩ | ⟨hcy, hQy⟩ + · exact Or.inl ⟨by simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy, + ⟨y, hIy, rfl⟩⟩ + · exact Or.inr ⟨by simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy, + ⟨y, hQy, rfl⟩⟩ + have hp_exit' : ∀ z', I' z' → (cond.rename f).Holds z' → + p_exit ≤ (μ_body' z').toOuterMeasure {y' | ¬(cond.rename f).Holds y'} := by + rintro z' ⟨u, hIu, rfl⟩ hcz' + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + simp only [μ_body', g, Data.comap_extend hf, pmf_map_toOuterMeasure] + have hset : ((fun y : Data r => y.extend f x) ⁻¹' {y' | ¬(cond.rename f).Holds y'}) = + {y | ¬cond.Holds y} := by + ext y + simp [Cond.holds_rename, Data.comap_extend hf] + rw [hset] + exact hp_exit u hIu hcu + have hx_inv' : LoopStateInv (cond.rename f) I' Q' x := by + rcases hx_inv with ⟨hcx, hIx⟩ | ⟨hcx, hQx⟩ + · exact Or.inl ⟨(Cond.holds_rename _ _).2 hcx, ⟨x.comap f, hIx, (extend_comap_eq hf x).symm⟩⟩ + · exact Or.inr ⟨fun h => hcx ((Cond.holds_rename _ _).1 h), + ⟨x.comap f, hQx, (extend_comap_eq hf x).symm⟩⟩ + have hloop := loop_geometric (cond := cond.rename f) (body := body.rename f) + I' Q' hp_pos hp_top + (μ_body := μ_body') (T_body := T_body') (B_body := B_body) (x := x) + (fun z' ⟨u, hIu, hzu⟩ hcz' => by + subst hzu + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + have hb' := ih_body u hIu hcu (x := g u) (Data.comap_extend hf u x) + simpa only [μ_body', T_body', g, Data.comap_extend hf, Data.extend_extend hf] using hb') + (fun z' ⟨u, hIu, hzu⟩ hcz' => by + subst hzu + have hcu : cond.Holds u := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' + simpa only [T_body', g, Data.comap_extend hf] using hB u hIu hcu) + hinv' hp_exit' hx_inv' + refine hloop.congr_dist ?_ le_rfl + ext y' + rw [loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv' hp_exit' hx_inv')] + have hx_eq : x = g (x.comap f) := (extend_comap_eq hf x).symm + by_cases hy' : y' ∈ Set.range g + · obtain ⟨y, rfl⟩ := hy' + rw [pmf_map_apply_injective _ hg, + loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx_inv)] + refine iSup_congr fun K => ?_ + have hiter : loopIterPMF (cond.rename f) μ_body' K x = + (loopIterPMF cond μ_body K (x.comap f)).map g := by + conv_lhs => rw [hx_eq] + exact loopIterPMF_rename hf cond μ_body x K (x.comap f) + rw [hiter] + by_cases hcy : ¬cond.Holds y + · have hcy' : ¬(cond.rename f).Holds (g y) := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy + simp [Set.indicator_of_mem, hcy, hcy', pmf_map_apply_injective _ hg] + · have hcy' : ¬¬(cond.rename f).Holds (g y) := by + simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcy + simp [Set.indicator_of_notMem, hcy, hcy'] + · rw [pmf_map_apply_not_mem_range _ g hy'] + refine le_antisymm (iSup_le fun K => ?_) zero_le + have hiter : loopIterPMF (cond.rename f) μ_body' K x = + (loopIterPMF cond μ_body K (x.comap f)).map g := by + conv_lhs => rw [hx_eq] + exact loopIterPMF_rename hf cond μ_body x K (x.comap f) + rw [hiter] + by_cases hcy' : ¬(cond.rename f).Holds y' + · simp [Set.indicator_of_mem, hcy', pmf_map_apply_not_mem_range _ g hy'] + · simp [Set.indicator_of_notMem, hcy'] + +/-! ### Compilation of `ExecCodeDist` to flat `Program` semantics -/ + +/-- Finite-horizon Bellman inequality for `ExecCodeDist` inside any containing flat program `p`. -/ +theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (N : ℕ) : + expectedStepsWithin w p N (.running ⟨pc, x⟩) ≤ + T + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + c.size, y⟩) := by + induction h generalizing pc N with + | @of_execRand c x m hm => + refine (hm.expectedStepsWithin_assemble hp N).trans (le_of_eq ?_) + rw [ProbComp.expect_add, + ← ProbComp.expect_map Prod.fst m (fun y => + expectedStepsWithin w p N (.running ⟨pc + c.size, y⟩))] + rfl + | mono _ hle ih => + exact (ih hp N).trans (add_le_add hle le_rfl) + | @seq c₁ c₂ x μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine (ih₁ hp.append_left N).trans ?_ + have hstep : (∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩)) ≤ + (∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by + calc + (∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩)) + ≤ ∑' y, μ₁ y * (T₂ y + + ∑' z, (μ₂ y) z * + expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩)) := by + refine ENNReal.tsum_le_tsum fun y => ?_ + by_cases hy : y ∈ μ₁.support + · exact mul_le_mul_right (ih₂ y hy hc₂ N) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] + _ = (∑' y, μ₁ y * T₂ y) + + ∑' y, μ₁ y * ∑' z, (μ₂ y) z * + expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩) := by + simp_rw [mul_add, ENNReal.tsum_add] + _ = (∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by + congr 1 + simp only [PMF.bind_apply, Code.size, Nat.add_assoc] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + calc + T₁ + ∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩) + ≤ T₁ + ((∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩)) := + add_le_add le_rfl hstep + _ = (T₁ + ∑' y, μ₁ y * T₂ y) + + ∑' z, (μ₁.bind μ₂) z * + expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by + rw [add_assoc] + | @branch_pos cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ih hc' N)).trans (le_of_eq ?_) + simp only [Code.size, + show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + ac_rfl + | @branch_neg cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + refine (expectedStepsWithin_running_le w p N ⟨pc, x⟩).trans ?_ + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (ih hp.cons_tail.append_left N)).trans ?_ + have hjump : (∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) ≤ + 1 + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩) := by + calc + (∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) + ≤ ∑' y, μ y * (1 + + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩)) := by + refine ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _ + refine (expectedStepsWithin_running_le w p N ⟨pc + 1 + neg.size, y⟩).trans (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj, ProbComp.expect_pure, + show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + _ = 1 + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩) := by + simp_rw [mul_add, ENNReal.tsum_add, mul_one, μ.tsum_coe] + calc + 1 + (T + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) + ≤ 1 + (T + (1 + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩))) := + add_le_add le_rfl (add_le_add le_rfl hjump) + _ = (T + (1 + 1)) + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (Code.branch cond pos neg).size, y⟩) := by + simp only [Code.size] + ac_rfl + _ = (T + 2) + ∑' y, μ y * + expectedStepsWithin w p N (.running ⟨pc + (Code.branch cond pos neg).size, y⟩) := by + rw [one_add_one_eq_two] + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ hB hinv hp_exit hx_inv ih_body => + have hj_exit : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj_back : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + set K_cont : Data r → ℝ≥0∞ := fun y => + expectedStepsWithin w p N (.running ⟨pc + (Code.loop cond body).size, y⟩) + set M := (B_body + 2) / p_exit + have hmain : ∀ n ≤ N, ∀ z : Data r, LoopStateInv cond I Q z → + expectedStepsWithin w p n (.running ⟨pc, z⟩) ≤ + (if cond.Holds z then M else 0) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + intro n + induction n with + | zero => simp + | succ n ihn => + intro hn z hz_inv + rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ + · simp only [hcz, ↓reduceIte] + rw [expectedStepsWithin_running_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, ProbComp.expect_pure] + have hbody_step := (ih_body z hIz hcz hbody_contains n).trans + (add_le_add (hB z hIz hcz) le_rfl) + have hback_step : + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc + 2 + body.size, u⟩)) ≤ + 1 + ∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩) := by + calc + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc + 2 + body.size, u⟩)) + ≤ ∑' u, μ_body z u * (1 + expectedStepsWithin w p n (.running ⟨pc, u⟩)) := by + refine ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _ + refine (expectedStepsWithin_running_le w p n ⟨pc + 2 + body.size, u⟩).trans + (le_of_eq ?_) + rw [stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back, ProbComp.expect_pure] + _ = 1 + ∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩) := by + simp_rw [mul_add, ENNReal.tsum_add, mul_one, (μ_body z).tsum_coe] + have hbind_k : + (∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y) = + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz] + simp only [PMF.bind_apply] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + have hih_step : + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ + (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + calc + (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) + ≤ ∑' u, μ_body z u * ((if cond.Holds u then M else 0) + 2 + + ∑' y, loopExitPMF cond μ_body u y * K_cont y) := by + refine ENNReal.tsum_le_tsum fun u => ?_ + by_cases hu : u ∈ (μ_body z).support + · exact mul_le_mul_right + (ihn (Nat.le_of_succ_le hn) u (hinv z hIz hcz u hu)) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] + _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y := by + simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, (μ_body z).tsum_coe, + one_mul] + _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by + rw [hbind_k] + have hdrift := pmf_step_drift_bound (μ_body z) cond (C := B_body + 2) + hp_pos hp_top (hp_exit z hIz hcz) + calc + 1 + expectedStepsWithin w p n (.running ⟨pc + 2, z⟩) + ≤ 1 + (B_body + (1 + ∑' u, μ_body z u * + expectedStepsWithin w p n (.running ⟨pc, u⟩))) := + add_le_add le_rfl (hbody_step.trans (add_le_add le_rfl hback_step)) + _ = (B_body + (1 + 1)) + ∑' u, μ_body z u * + expectedStepsWithin w p n (.running ⟨pc, u⟩) := by ac_rfl + _ = (B_body + 2) + ∑' u, μ_body z u * + expectedStepsWithin w p n (.running ⟨pc, u⟩) := by rw [one_add_one_eq_two] + _ ≤ (B_body + 2) + ((∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y) := + add_le_add le_rfl hih_step + _ = ((B_body + 2) + ∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by ac_rfl + _ ≤ M + 2 + ∑' y, loopExitPMF cond μ_body z y * K_cont y := + add_le_add (add_le_add hdrift le_rfl) le_rfl + · simp only [hcz, ↓reduceIte, zero_add] + rw [loopExitPMF_of_not_holds hcz, tsum_eq_single z + (fun y hy => by simp [PMF.pure_apply, hy])] + simp only [PMF.pure_apply_self, one_mul] + rw [expectedStepsWithin_running_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, ProbComp.expect_pure] + refine (add_le_add le_rfl (expectedStepsWithin_running_le w p n ⟨pc + 1, z⟩)).trans ?_ + rw [stepRand_jump (s := ⟨pc + 1, z⟩) hj_exit, ProbComp.expect_pure, ← add_assoc, + one_add_one_eq_two] + refine add_le_add le_rfl ?_ + simpa [K_cont, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using expectedStepsWithin_mono w p + (.running ⟨pc + (body.size + 3), z⟩) (Nat.le_of_succ_le hn) + refine (hmain N le_rfl x hx_inv).trans ?_ + refine add_le_add (add_le_add ?_ le_rfl) le_rfl + split_ifs <;> simp + +/-- Full-horizon Bellman inequality for `ExecCodeDist` inside any containing flat program `p`. -/ +theorem expectedStepsFrom_assemble (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) : + expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ + T + ∑' y, μ y * expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) := by + rw [← iSup_expectedStepsWithin w p (.running ⟨pc, x⟩)] + refine iSup_le fun N => (h.expectedStepsWithin_assemble hp N).trans ?_ + refine add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _) + exact expectedStepsWithin_le_expectedStepsFrom w p N (.running ⟨pc + c.size, y⟩) + +/-- Continuation upper-bound form of `expectedStepsFrom_assemble`. -/ +theorem expectedStepsFrom_le (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) {k : Data r → ℝ≥0∞} + (hk : ∀ y ∈ μ.support, expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) ≤ k y) : + expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ T + ∑' y, μ y * k y := by + refine (h.expectedStepsFrom_assemble hp).trans + (add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => ?_)) + by_cases hy : y ∈ μ.support + · exact mul_le_mul_right (hk y hy) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] + +/-- Along `ExecCodeDist w c x μ T`, if no continuation from `⟨pc + c.size, y⟩` faults within +`T_hor` steps for any supported `y ∈ μ.support`, then running `p` from `⟨pc, x⟩` does not fault +within `T_hor` steps. -/ +theorem evalRand_faulted_eq_zero (h : ExecCodeDist w c x μ T) {p : Program B r} {pc T_hor : ℕ} + (hp : p.Contains pc (c.assemble pc)) + (hcont : ∀ y ∈ μ.support, ∀ t ≤ T_hor, evalRand w p t ⟨pc + c.size, y⟩ .faulted = 0) : + ∀ t ≤ T_hor, evalRand w p t ⟨pc, x⟩ .faulted = 0 := by + induction h generalizing pc T_hor with + | @of_execRand c x m hm => + refine hm.evalRand_faulted_eq_zero hp fun z hz t ht => ?_ + apply hcont z.1 + · rw [ProbComp.eval_map, PMF.mem_support_map_iff] + refine ⟨z, ?_, rfl⟩ + rwa [ProbComp.support_eval] + · exact ht + | mono _ _ ih => exact ih hp hcont + | @seq c₁ c₂ x μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + refine ih₁ hp.append_left fun y₁ hy₁ => ih₂ y₁ hy₁ hc₂ fun y₂ hy₂ t ht => ?_ + have hmem : y₂ ∈ (μ₁.bind μ₂).support := + (PMF.mem_support_bind_iff _ _ _).mpr ⟨y₁, hy₁, hy₂⟩ + simpa [Code.size, Nat.add_assoc] using hcont y₂ hmem t ht + | @branch_pos cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hc' (T_hor := k) (fun y hy t' ht' => ?_) k le_rfl + simpa [Code.size, show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + using hcont y hy t' (by omega) + | @branch_neg cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + intro t ht + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, x⟩) hbr] + simp only [hc, ↓reduceIte, pure_bind] + refine ih hp.cons_tail.append_left (T_hor := k) (fun y hy t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj, pure_bind] + simpa [evalRand, Code.size, + show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using hcont y hy k' (by omega) + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + have hj_exit : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj_back : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + have hmain : ∀ n ≤ T_hor, ∀ z : Data r, LoopStateInv cond I Q z → + (∀ y ∈ (loopExitPMF cond μ_body z).support, ∀ t ≤ T_hor, + evalRand w p t ⟨pc + (Code.loop cond body).size, y⟩ .faulted = 0) → + evalRand w p n ⟨pc, z⟩ .faulted = 0 := by + intro n + induction n using Nat.strong_induction_on with + | _ n ihn => + intro hn z hz_inv hcont_z + cases n with + | zero => simp [evalRand] + | succ k => + rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ + · rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, pure_bind] + refine ih_body z hIz hcz hbody_contains (T_hor := k) (fun u hu t' ht' => ?_) k le_rfl + cases t' with + | zero => simp [evalRand] + | succ k' => + rw [evalRand, runRand_succ, stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back, + pure_bind] + refine ihn k' (by omega) (by omega) u (hinv z hIz hcz u hu) fun y hy t'' ht'' => ?_ + apply hcont_z y _ t'' ht'' + rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz, PMF.mem_support_bind_iff] + exact ⟨u, hu, hy⟩ + · rw [evalRand, runRand_succ, stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp only [hcz, ↓reduceIte, pure_bind] + cases k with + | zero => simp + | succ k' => + rw [runRand_succ, stepRand_jump (s := ⟨pc + 1, z⟩) hj_exit, pure_bind] + have hz_mem : z ∈ (loopExitPMF cond μ_body z).support := by + simp [loopExitPMF_of_not_holds hcz] + simpa [evalRand, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using hcont_z z hz_mem k' (by omega) + intro t ht + exact hmain t ht x hx_inv hcont + +/-- If every continuation from `⟨pc + c.size, y⟩` (`y ∈ μ.support`) is fault-free at all horizons, +then running `p` from `⟨pc, x⟩` is fault-free at all horizons. -/ +theorem randFaultFree_assemble (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) + (hcont : ∀ y ∈ μ.support, RandFaultFree w p ⟨pc + c.size, y⟩) : + RandFaultFree w p ⟨pc, x⟩ := by + intro T_hor + rw [← support_evalRand, ← PMF.apply_eq_zero_iff] + exact h.evalRand_faulted_eq_zero hp + (fun y hy t _ => (hcont y hy).evalRand_faulted_eq_zero t) + T_hor le_rfl + +/-- Along `ExecCodeDist w c x μ T`, the `μ`-weighted supremum halting probability from +`⟨pc + c.size, y⟩` is a lower bound on the supremum halting probability from `⟨pc, x⟩`. -/ +theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} {pc : ℕ} + (hp : p.Contains pc (c.assemble pc)) (v : ℕ) (d : Data r) : + (∑' y, μ y * ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c.size, y⟩ (.halted v d)) ≤ + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc, x⟩ (.halted v d) := by + induction h generalizing pc with + | @of_execRand c x m hm => + refine (le_of_eq ?_).trans (hm.le_iSup_evalRand_halted hp v d) + exact ProbComp.expect_map Prod.fst m + (fun y => ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c.size, y⟩ (.halted v d)) + | mono _ _ ih => exact ih hp + | @seq c₁ c₂ x μ₁ μ₂ T₁ T₂ _ _ ih₁ ih₂ => + have hc₂ : p.Contains (pc + c₁.size) (c₂.assemble (pc + c₁.size)) := by + simpa [Code.length_assemble] using hp.append_right + have hbind : + (∑' y, (μ₁.bind μ₂) y * + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + (c₁.seq c₂).size, y⟩ (.halted v d)) = + ∑' y₁, μ₁ y₁ * ∑' y, (μ₂ y₁) y * + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c₁.size + c₂.size, y⟩ (.halted v d) := by + simp only [PMF.bind_apply, Code.size, Nat.add_assoc] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + rw [hbind] + refine (ENNReal.tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) + by_cases hy₁ : y₁ ∈ μ₁.support + · exact mul_le_mul_right (ih₂ y₁ hy₁ hc₂) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hy₁, zero_mul, zero_mul] + | @branch_pos cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hc' := (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).cons_tail + rw [Code.length_assemble, show pc + 1 + neg.size + 1 = pc + neg.size + 2 by omega] at hc' + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + neg.size + 2, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + refine (le_of_eq ?_).trans ((ih hc').trans (iSup_evalRand_halted_le_of_step hstep v d)) + refine tsum_congr fun y => ?_ + simp [Code.size, show pc + neg.size + 2 + pos.size = pc + (neg.size + pos.size + 2) by omega] + | @branch_neg cond pos neg x μ T hc _ ih => + have hbr : p[pc]? = some (.branch cond (pc + neg.size + 2)) := by + simpa [Nat.add_assoc, Nat.add_comm, Nat.add_left_comm] using hp.head + have hj : p[pc + 1 + neg.size]? = some (.jump (pc + neg.size + pos.size + 2)) := by + simpa using (hp.cons_tail.append_right (q₁ := neg.assemble (pc + 1))).head + have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by + rw [stepRand_branch (s := ⟨pc, x⟩) hbr] + simp [hc] + refine (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _).trans + ((ih hp.cons_tail.append_left).trans (iSup_evalRand_halted_le_of_step hstep v d)) + simpa [Code.size, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] + using iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj) v d + | @loop_geometric cond body I Q μ_body T_body B_body p_exit x + hp_pos hp_top _ _ hinv hp_exit hx_inv ih_body => + have hj_exit : p[pc + 1]? = some (.jump (pc + body.size + 3)) := hp.cons_tail.head + have hbody_contains : p.Contains (pc + 2) (body.assemble (pc + 2)) := by + simpa [Nat.add_assoc] using hp.cons_tail.cons_tail.append_left + have hj_back : p[pc + 2 + body.size]? = some (.jump pc) := by + simpa using hp.cons_tail.cons_tail.append_right.head + set H_loop : Data r → ℝ≥0∞ := fun z => ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc, z⟩ (.halted v d) + set H_exit : Data r → ℝ≥0∞ := fun z => + ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + (Code.loop cond body).size, z⟩ (.halted v d) + have hexit_le : ∀ z : Data r, ¬cond.Holds z → H_exit z ≤ H_loop z := by + intro z hcz + have hbr : stepRand w p (.running ⟨pc, z⟩) = pure (.running ⟨pc + 1, z⟩) := by + rw [stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp [hcz] + simpa [H_exit, H_loop, Code.size, show pc + body.size + 3 = pc + (body.size + 3) by omega] + using (iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 1, z⟩) hj_exit) v d).trans + (iSup_evalRand_halted_le_of_step hbr v d) + have hstep_le : ∀ z : Data r, I z → cond.Holds z → + (∑' u, μ_body z u * H_loop u) ≤ H_loop z := by + intro z hIz hcz + have hbr : stepRand w p (.running ⟨pc, z⟩) = pure (.running ⟨pc + 2, z⟩) := by + rw [stepRand_branch (s := ⟨pc, z⟩) hp.head] + simp [hcz] + refine (ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _).trans + ((ih_body z hIz hcz hbody_contains).trans (iSup_evalRand_halted_le_of_step hbr v d)) + exact iSup_evalRand_halted_le_of_step + (stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back) v d + have hiter_le : ∀ K : ℕ, ∀ z : Data r, LoopStateInv cond I Q z → + (∑' y, loopIterPMF cond μ_body K z y * H_loop y) ≤ H_loop z := by + intro K + induction K with + | zero => + intro z _ + rw [tsum_eq_single z (fun y hy => by simp [loopIterPMF_zero, PMF.pure_apply, hy])] + simp [H_loop] + | succ K ihK => + intro z hz_inv + rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ + · rw [loopIterPMF_succ, loopStepPMF_of_holds hcz] + have hbind : (∑' y, ((μ_body z).bind (loopIterPMF cond μ_body K)) y * H_loop y) = + ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := by + simp only [PMF.bind_apply] + simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] + exact ENNReal.tsum_comm + rw [hbind] + refine (ENNReal.tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) + by_cases hu : u ∈ (μ_body z).support + · exact mul_le_mul_right (ihK u (hinv z hIz hcz u hu)) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] + · rw [loopIterPMF_of_not_holds hcz, + tsum_eq_single z (fun y hy => by simp [PMF.pure_apply, hy])] + simp + have hrewrite_sum : + (∑' y, loopExitPMF cond μ_body x y * H_exit y) = + ⨆ K : ℕ, ∑' y, + Set.indicator {w' | ¬cond.Holds w'} (loopIterPMF cond μ_body K x) y * H_exit y := by + simp_rw [loopExitPMF_apply_of_hasSum + (hasSum_loopExitDensity hp_pos hp_top hinv hp_exit hx_inv), ENNReal.iSup_mul] + exact tsum_iSup_of_monotone fun y K₁ K₂ hK => + mul_le_mul_left (loopIterPMF_indicator_mono cond μ_body x y hK) (H_exit y) + rw [hrewrite_sum] + refine iSup_le fun K => (ENNReal.tsum_le_tsum fun y => ?_).trans (hiter_le K x hx_inv) + by_cases hcy : ¬cond.Holds y + · rw [Set.indicator_of_mem hcy] + exact mul_le_mul_right (hexit_le y hcy) _ + · simp [Set.indicator_of_notMem, hcy] + +/-- Compiling `ExecCodeDist w c x μ T` with `c.program out` yields: +1. `expectedTime w (c.program out) ⟨0, x⟩ ≤ T + 1`, +2. `RandFaultFree w (c.program out) ⟨0, x⟩`, +3. `AlmostSurelyHalts w (c.program out) ⟨0, x⟩` whenever `T ≠ ⊤`, and +4. for every `y : Data r`, + `μ y ≤ ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0, x⟩ (.halted (y.regs out) y)`. -/ +theorem program_halts (h : ExecCodeDist w c x μ T) (out : Fin r) : + expectedTime w (c.program out) ⟨0, x⟩ ≤ T + 1 ∧ + RandFaultFree w (c.program out) ⟨0, x⟩ ∧ + (T ≠ ⊤ → AlmostSurelyHalts w (c.program out) ⟨0, x⟩) ∧ + ∀ y : Data r, + μ y ≤ ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0, x⟩ (.halted (y.regs out) y) := by + have hp : (c.program out).Contains 0 (c.assemble 0) := + (Program.Contains.self (c.program out)).append_left + have hhalt : (c.program out)[0 + c.size]? = some (.halt out) := by + simp [Code.program] + have hexp : expectedTime w (c.program out) ⟨0, x⟩ ≤ T + 1 := by + rw [expectedTime_eq_expectedStepsFrom] + refine (h.expectedStepsFrom_le (k := fun _ => 1) hp fun y _ => ?_).trans ?_ + · rw [expectedStepsFrom_running, stepRand_halt (s := ⟨0 + c.size, y⟩) hhalt, + ProbComp.expect_pure, expectedStepsFrom_halted, add_zero] + · simp [μ.tsum_coe] + have hff : RandFaultFree w (c.program out) ⟨0, x⟩ := by + refine h.randFaultFree_assemble hp fun y _ t => ?_ + rw [← support_evalRand, ← PMF.apply_eq_zero_iff] + cases t with + | zero => simp [evalRand] + | succ k => + rw [evalRand, runRand_succ, stepRand_halt (s := ⟨0 + c.size, y⟩) hhalt, pure_bind, + runRand_halted] + simp [PMF.pure_apply] + refine ⟨hexp, hff, fun hT => ?_, fun y => ?_⟩ + · refine AlmostSurelyHalts.of_expectedTime_ne_top hff ?_ + exact ne_top_of_le_ne_top (ENNReal.add_ne_top.mpr ⟨hT, ENNReal.one_ne_top⟩) hexp + · have hle := h.le_iSup_evalRand_halted hp (y.regs out) y + refine (le_trans ?_ + (ENNReal.le_tsum + (f := fun z => μ z * + ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0 + c.size, z⟩ + (.halted (y.regs out) y)) y)).trans hle + have h1 : (1 : ℝ≥0∞) ≤ + ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0 + c.size, y⟩ (.halted (y.regs out) y) := by + refine le_iSup_of_le 1 ?_ + rw [evalRand, runRand_succ, stepRand_halt (s := ⟨0 + c.size, y⟩) hhalt, pure_bind, + runRand_zero, ProbComp.eval_pure, PMF.pure_apply_self] + simpa using mul_le_mul_right h1 (μ y) + +end ExecCodeDist + +/-- Geometric Las Vegas loop expected-step bound inside any flat program `p` containing +`(Code.loop cond body).assemble pc`. -/ +theorem expectedStepsFrom_loop_le_of_geometric {w : ℕ} {cond : Cond r} {body : Code B r} + (I Q : Data r → Prop) {μ_body : Data r → PMF (Data r)} {T_body : Data r → ℝ≥0∞} + {B_body p_exit : ℝ≥0∞} {x : Data r} + (hp_pos : 0 < p_exit) (hp_top : p_exit ≠ ⊤) + (hbody : ∀ z, I z → cond.Holds z → ExecCodeDist w body z (μ_body z) (T_body z)) + (hB : ∀ z, I z → cond.Holds z → T_body z ≤ B_body) + (hinv : ∀ z, I z → cond.Holds z → + ∀ y ∈ (μ_body z).support, LoopStateInv cond I Q y) + (hp_exit : ∀ z, I z → cond.Holds z → + p_exit ≤ (μ_body z).toOuterMeasure {y | ¬cond.Holds y}) + (hx_inv : LoopStateInv cond I Q x) + {p : Program B r} {pc : ℕ} (hp : p.Contains pc ((Code.loop cond body).assemble pc)) + {k : Data r → ℝ≥0∞} + (hk : ∀ y, ¬cond.Holds y → Q y → + expectedStepsFrom w p (.running ⟨pc + (Code.loop cond body).size, y⟩) ≤ k y) : + expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ + ((B_body + 2) / p_exit + 2) + ∑' y, loopExitPMF cond μ_body x y * k y := by + have hloop := ExecCodeDist.loop_geometric I Q hp_pos hp_top hbody hB hinv hp_exit hx_inv + refine hloop.expectedStepsFrom_le hp fun y hy => ?_ + obtain ⟨⟨hcy, hQy⟩, _⟩ := mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx_inv hy + exact hk y hcy hQy + +end Cslib.WordRAM + +end diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean new file mode 100644 index 000000000..f89efc199 --- /dev/null +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -0,0 +1,1046 @@ +/- +Copyright (c) 2026 Chaeyoung Lim. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Chaeyoung Lim +-/ + +module +-- upstream: candidate + +public import Complexity.Computability.WordRAM.Subroutine +public import Complexity.Computability.WordRAM.Randomized.Expected + +/-! +# Composable Randomized Word-RAM Data-Structure Subroutines + +This file extends the deterministic `Cslib.WordRAM.SubroutineDS` interface to **randomized data +structures and Las Vegas subroutines** on the Word-RAM — covering both structures initialized from a +random seed (such as universal hash tables, 2-choice hash tables, and cuckoo hash tables) and +structures whose operations make random choices via `Instr.rand` (such as treaps, skip lists, +reservoir samplers, or randomized rehashing) `[MitzenmacherUpfal2017]`. + +## Core design + +A `Cslib.WordRAM.Randomized.SubroutineDS P D w base params s₀` couples machine execution with a +latent seed type `Seed : Type` and reference distribution `seedLaw : PMF Seed` up to a bounded +density factor `seedSlack : ℝ≥0∞`: + +1. **Seed-coupled representation (`rep θ s x`) and potential (`potential θ x`):** + - Data structures without a persistent random seed set `Seed := Unit`, `seedLaw := PMF.pure ()`, + and `seedSlack := 1` (in particular, `Cslib.WordRAM.SubroutineDS.toRandomized` embeds every + deterministic `Cslib.WordRAM.SubroutineDS` directly). + - Hash tables that resample their hash family on rare table-full events maintain a joint + distribution `ν : PMF (Seed × Data r_ds)` whose marginal on `Seed` satisfies + `(ν.map Prod.fst) θ ≤ seedSlack * seedLaw θ`. + +2. **Distributional execution (`ExecCodeDist`):** + - `setup_exec` and `step_exec` use `ExecCodeDist` from + `Complexity.Computability.WordRAM.Randomized.Expected`, which supports both bounded + randomized code (`ExecRand`) and unbounded almost-surely terminating Las Vegas `while` loops + (`ExecCodeDist.loop_geometric`). + +3. **Addition-only `ℝ≥0∞` amortized-in-expectation bound:** + - Every admissible operation `o : D.Op` from abstract state `s : D.State` and coupled state + distribution `ν : PMF (Seed × Data r_ds)` produces a transition kernel + `μ_step : Seed × Data r_ds → PMF (Seed × Data r_ds)` and expected step count + `T_step : Seed × Data r_ds → ℝ≥0∞` satisfying + `∑' p, ν p * (T_step p + ∑' q, μ_step p q * Φ(q)) ≤ charge s o + ∑' p, ν p * Φ(p)`. + - Across any admissible finite operation sequence `ops : List D.Op`, `seq_expected_amortized` + telescopes these one-step bounds without `ℝ≥0∞` subtraction: + total expected step cost is at most `initTime + initPotential + D.totalCost charge s₀ ops`. + +## Main definitions + +* `Cslib.WordRAM.Randomized.SubroutineDS`: reusable randomized Word-RAM subroutine certificate for + a deterministic specification `D : Stateful.Spec`. +* `Cslib.WordRAM.SubroutineDS.toRandomized`: canonical embedding of any deterministic + `Cslib.WordRAM.SubroutineDS` into `Cslib.WordRAM.Randomized.SubroutineDS`. +* `Cslib.WordRAM.Randomized.SubroutineDS.mapSimulates`, `toStandard`: basis transport from + `Basis.exact` to `Basis.standard`. + +## Main results + +* `Cslib.WordRAM.Randomized.SubroutineDS.initFresh_exec`, `initFresh_rename`: expected-time + initialization on fresh zeroed memory. +* `Cslib.WordRAM.Randomized.SubroutineDS.zeroFillCode_exec`, `initClear_exec`, `initClear_rename`: + expected-time initialization on arbitrary dirty memory. +* `Cslib.WordRAM.Randomized.SubroutineDS.code_rename`, `call_rename`: inline subroutine invocation + on caller state space `Data r_caller` via register injection `f : Fin r_ds → Fin r_caller`. +* `Cslib.WordRAM.Randomized.SubroutineDS.frame_caller`, `frame_caller_dist`: preservation of `rep`, + `potential`, and the coupled `(Seed × Data r_caller)` distribution across arbitrary deterministic + or randomized caller steps that do not modify persistent DS registers (`P.input i = false`) or + `[base, base + space)`. +* `Cslib.WordRAM.Randomized.SubroutineDS.steps_expected_amortized`, `seq_expected_amortized`: + end-to-end telescope theorems bounding total expected time across any admissible operation + sequence `ops : List D.Op`. + +## References + +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Chapter 17][CLRS2009] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing: Randomization and Probabilistic + Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017] +-/ + +@[expose] public section + +namespace Cslib.WordRAM.Randomized + +open Cslib.WordRAM +open scoped ENNReal + +universe u v z + +/-- Unified standalone and inline subroutine specification/resource certificate for a randomized +Word-RAM data structure `P` realizing `D` at word width `w`, base address `base`, runtime +parameters `params`, and initial abstract state `s₀`. -/ +structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} + (P : SubroutineCode B r_ds k_init) (D : Stateful.Spec.{u, v, z}) + (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) : Type (max u v z 1) where + /-- Latent random seed type (use `Unit` when no persistent seed is needed). -/ + Seed : Type + /-- Reference distribution over latent seeds. -/ + seedLaw : PMF Seed + /-- Marginal density slack factor (`1` when the seed distribution is exact). -/ + seedSlack : ℝ≥0∞ := 1 + /-- The marginal density slack factor is finite. -/ + seedSlack_ne_top : seedSlack ≠ ⊤ := by simp + /-- Contiguous memory footprint length `[base, base + space)`. -/ + space : ℕ + /-- Worst-case instruction bound for `P.allocSpan`. -/ + spanTime : ℕ + /-- Expected instruction bound for `P.initSetup`. -/ + setupTime : ℝ≥0∞ + /-- Upper bound on initial concrete potential after initialization. -/ + initPotential : ℕ + /-- Amortized instruction charge per operation. -/ + charge : D.State → D.Op → ℕ + /-- Seed-indexed representation invariant linking `Seed`, `D.State`, and `Data r_ds`. -/ + rep : Seed → D.State → Data r_ds → Prop + /-- Seed-indexed nonnegative integer potential. -/ + potential : Seed → Data r_ds → ℕ + /-- Standalone register encoding for argument delivery. -/ + encode : D.Op → Fin r_ds → ℕ + /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ + argMatch : D.Op → (Fin r_ds → ℕ) → Prop + /-- Output decoder reading typed operation observations from final registers. -/ + decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) + /-- Represented configurations are word-bounded. -/ + bounded : ∀ {θ s x}, rep θ s x → x.Bounded w + /-- For a fixed seed `θ`, concrete machine data determines at most one abstract state. -/ + rep_unique : ∀ {θ s₁ s₂ x}, rep θ s₁ x → rep θ s₂ x → s₁ = s₂ + /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ + arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) + /-- Full standalone argument delivery satisfies `argMatch`. -/ + encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), + argMatch o (fun i => if P.input i then encode o i else regs i) + /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. + -/ + rep_supported : ∀ {θ s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + x'.Bounded w → rep θ s x → rep θ s x' + /-- `potential` depends only on persistent registers and `[base, base + space)` under `rep`. -/ + potential_supported : ∀ {θ s x x'}, + (∀ i, P.input i = false → x'.regs i = x.regs i) → + (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → + rep θ s x → potential θ x' = potential θ x + /-- Distinctness of registers used by `zeroFillCode` relative to `baseReg` and `frontierReg`. -/ + clear_nodup : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup + /-- Initialization parameter registers are not overwritten by `zeroFillCode`. -/ + params_not_clear : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg] + /-- `P.incCur` increments `P.curReg` by `1` when `P.oneReg = 1` and the sum fits in `w` bits. -/ + incCur_eval : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → + P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1)) + /-- Execution of `P.allocSpan`: records `baseReg = base` and advances `frontierReg`. -/ + span_exec : ∀ {x : Data r_ds}, + x.Bounded w → x.regs P.frontierReg = base → (∀ j, x.regs (P.initParams j) = params j) → + ∃ x₁, ExecWithin w P.allocSpan x x₁ spanTime ∧ x₁.mem = x.mem ∧ + x₁.regs P.baseReg = base ∧ x₁.regs P.frontierReg = base + space ∧ + (∀ j, x₁.regs (P.initParams j) = params j) + /-- Execution of `P.initSetup` once `[base, base + space)` holds zeros, producing a coupled + distribution `ν₀ : PMF (Seed × Data r_ds)` whose seed marginal is bounded by + `seedSlack * seedLaw`. -/ + setup_exec : ∀ {x₁ : Data r_ds}, + x₁.Bounded w → x₁.regs P.baseReg = base → x₁.regs P.frontierReg = base + space → + (∀ j, x₁.regs (P.initParams j) = params j) → + (∀ a ∈ (Block.mk base space).footprint, x₁.mem a = 0) → + ∃ ν₀ : PMF (Seed × Data r_ds), + ExecCodeDist w P.initSetup x₁ (ν₀.map Prod.snd) setupTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ seedSlack * seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + space ∧ + rep p.1 s₀ p.2 ∧ + potential p.1 p.2 ≤ initPotential ∧ + Modifies (Block.mk base space).footprint x₁.mem p.2.mem + /-- Coupled one-step execution of `P.code` for any admissible operation `o` from abstract state + `s` and coupled state distribution `ν : PMF (Seed × Data r_ds)` with seed marginal bounded by + `seedSlack * seedLaw`: produces a transition kernel `μ_step` and expected step count `T_step` + preserving the seed-marginal bound, realizing `D.step s o`, modifying only `[base, base + space)`, + and satisfying the addition-only `ℝ≥0∞` amortized-in-expectation inequality. -/ + step_exec : ∀ {s : D.State} (o : D.Op) (ν : PMF (Seed × Data r_ds)), + D.valid s o → + (∀ θ, (ν.map Prod.fst) θ ≤ seedSlack * seedLaw θ) → + (∀ p ∈ ν.support, rep p.1 s p.2 ∧ argMatch o p.2.regs) → + ∃ (μ_step : Seed × Data r_ds → PMF (Seed × Data r_ds)) + (T_step : Seed × Data r_ds → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w P.code p.2 ((μ_step p).map Prod.snd) (T_step p) ∧ + ∀ q ∈ (μ_step p).support, + decode o q.2.regs = some (D.step s o).2 ∧ + rep q.1 (D.step s o).1 q.2 ∧ + Modifies (Block.mk base space).footprint p.2.mem q.2.mem) ∧ + (∀ θ', ((ν.bind μ_step).map Prod.fst) θ' ≤ seedSlack * seedLaw θ') ∧ + (∑' p, ν p * (T_step p + ∑' q, μ_step p q * (potential q.1 q.2 : ℝ≥0∞))) ≤ + (charge s o : ℝ≥0∞) + ∑' p, ν p * (potential p.1 p.2 : ℝ≥0∞) + +end Cslib.WordRAM.Randomized + +namespace Cslib.WordRAM.SubroutineDS + +open scoped ENNReal + +universe u v z + +variable {B : Basis} {r_ds k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-- Canonical embedding of any deterministic `Cslib.WordRAM.SubroutineDS` certificate into +`Cslib.WordRAM.Randomized.SubroutineDS` (`Seed := Unit`, `seedLaw := PMF.pure ()`, +`seedSlack := 1`). -/ +noncomputable def toRandomized (C : SubroutineDS P D w base params s₀) : + Randomized.SubroutineDS P D w base params s₀ where + Seed := Unit + seedLaw := PMF.pure () + seedSlack := 1 + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep _ s x := C.rep s x + potential _ x := C.potential x + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + rep_unique := C.rep_unique + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec hx₁ hb hfr hpar hz := by + obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz + refine ⟨PMF.pure ((), y), ?_, ?_, ?_⟩ + · simpa using ExecCodeDist.of_execWithin he + · intro θ + cases θ + simp + · intro p hp + rw [PMF.support_pure, Set.mem_singleton_iff] at hp + subst hp + exact ⟨hfr', hrep, hpot, hmod⟩ + step_exec {s} o ν hv _ hsupp := by + classical + let y : Unit × Data r_ds → Data r_ds := fun p => + if hp : C.rep s p.2 ∧ C.argMatch o p.2.regs then + Classical.choose (C.step_exec o hp.1 hv hp.2) + else p.2 + let n : Unit × Data r_ds → ℕ := fun p => + if hp : C.rep s p.2 ∧ C.argMatch o p.2.regs then + Classical.choose (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) + else 0 + refine ⟨fun p => PMF.pure ((), y p), fun p => (n p : ℝ≥0∞), ?_, ?_, ?_⟩ + · intro p hp_mem + have hp : C.rep s p.2 ∧ C.argMatch o p.2.regs := hsupp p hp_mem + have hspec := Classical.choose_spec (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) + rcases hspec with ⟨he, hdec, hrep', hmod, _⟩ + have hy_eq : y p = Classical.choose (C.step_exec o hp.1 hv hp.2) := by simp [y, hp] + have hn_eq : n p = + Classical.choose (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) := by simp [n, hp] + refine ⟨?_, ?_⟩ + · simpa [hy_eq, hn_eq] using ExecCodeDist.of_exec he + · intro q hq + rw [PMF.support_pure, Set.mem_singleton_iff] at hq + subst hq + simpa only [hy_eq] using ⟨hdec, hrep', hmod⟩ + · intro θ' + cases θ' + have hconst : ((ν.bind fun p => PMF.pure ((), y p)).map Prod.fst) () = 1 := by + rw [PMF.map_bind] + simp + simp [hconst] + · calc + (∑' p, ν p * ((n p : ℝ≥0∞) + ∑' q, PMF.pure ((), y p) q * (C.potential q.2 : ℝ≥0∞))) + ≤ ∑' p, ν p * ((C.charge s o : ℝ≥0∞) + (C.potential p.2 : ℝ≥0∞)) := by + refine ENNReal.tsum_le_tsum fun p => ?_ + by_cases hp_mem : p ∈ ν.support + · refine mul_le_mul_right ?_ _ + have hp : C.rep s p.2 ∧ C.argMatch o p.2.regs := hsupp p hp_mem + have hspec := + Classical.choose_spec (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) + rcases hspec with ⟨_, _, _, _, hpot⟩ + have hy_eq : y p = Classical.choose (C.step_exec o hp.1 hv hp.2) := by simp [y, hp] + have hn_eq : n p = + Classical.choose (Classical.choose_spec (C.step_exec o hp.1 hv hp.2)) := by + simp [n, hp] + rw [tsum_eq_single ((), y p) (fun q hq => by simp [PMF.pure_apply, hq])] + simp only [PMF.pure_apply_self, one_mul, hy_eq, hn_eq] + exact_mod_cast hpot + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp_mem, zero_mul, zero_mul] + _ = (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.2 : ℝ≥0∞) := by + simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + +end Cslib.WordRAM.SubroutineDS + +namespace Cslib.WordRAM.Randomized.SubroutineDS + +open Cslib.WordRAM +open scoped ENNReal + +universe u v z + +variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-- Total expected instruction bound for `P.initFresh`. -/ +def initTime (C : SubroutineDS P D w base params s₀) : ℝ≥0∞ := + (C.spanTime : ℝ≥0∞) + C.setupTime + +/-! ### Projection 1: Basis lifting (`mapSimulates`, `toStandard`) -/ + +/-- Lift a randomized `SubroutineDS` certificate along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) : + SubroutineDS (P.mapSimulates φ) D w base params s₀ where + Seed := C.Seed + seedLaw := C.seedLaw + seedSlack := C.seedSlack + seedSlack_ne_top := C.seedSlack_ne_top + space := C.space + spanTime := C.spanTime + setupTime := C.setupTime + initPotential := C.initPotential + charge := C.charge + rep := C.rep + potential := C.potential + encode := C.encode + argMatch := C.argMatch + decode := C.decode + bounded := C.bounded + rep_unique := C.rep_unique + arguments_fit := C.arguments_fit + encode_matches := C.encode_matches + rep_supported := C.rep_supported + potential_supported := C.potential_supported + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval h1 hfit := Op.eval_mapSimulates_of_some φ (C.incCur_eval h1 hfit) + span_exec hx hf hp := + let ⟨x₁, he, hm, hb, hfr, hpar⟩ := C.span_exec hx hf hp + ⟨x₁, he.mapSimulates φ, hm, hb, hfr, hpar⟩ + setup_exec hx₁ hb hfr hpar hz := + let ⟨ν₀, he, hseed, hsupp⟩ := C.setup_exec hx₁ hb hfr hpar hz + ⟨ν₀, he.mapSimulates φ, hseed, hsupp⟩ + step_exec o ν hv hseed hsupp := + let ⟨μ_step, T_step, hstep, hseed', hamort⟩ := C.step_exec o ν hv hseed hsupp + ⟨μ_step, T_step, + fun p hp => ⟨(hstep p hp).1.mapSimulates φ, (hstep p hp).2⟩, + hseed', hamort⟩ + +/-- Lift an exact-basis randomized `SubroutineDS` certificate to `Basis.standard`. -/ +abbrev toStandard {P : SubroutineCode .exact r_ds k_init} + (C : SubroutineDS P D w base params s₀) : + SubroutineDS P.toStandard D w base params s₀ := + C.mapSimulates .exactStandard + +/-! ### Projection 2: Fresh initialization and register relocation (`initFresh_exec`, +`initFresh_rename`) -/ + +/-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` expected +instructions. -/ +theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ ν₀ : PMF (C.Seed × Data r_ds), + ExecCodeDist w P.initFresh x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + C.space ∧ + C.rep p.1 s₀ p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + have hx₁ : x₁.Bounded w := he₁.bounded hx + have hz₁ : ∀ a ∈ (Block.mk base C.space).footprint, x₁.mem a = 0 := by + intro a ha + rw [hm₁] + exact hz a ha + obtain ⟨ν₀, he₂, hseed, hsupp⟩ := C.setup_exec hx₁ hb₁ hfr₁ hpar₁ hz₁ + rw [hm₁] at hsupp + exact ⟨ν₀, (ExecCodeDist.of_execWithin he₁).seq_pure he₂, hseed, hsupp⟩ + +/-- Relocated fresh initialization along any injective register embedding `f`. -/ +theorem initFresh_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ ν₀ : PMF (C.Seed × Data r_caller), + ExecCodeDist w (P.initFresh.rename f) x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs (f P.frontierReg) = base + C.space ∧ + C.rep p.1 s₀ (p.2.comap f) ∧ + C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by + obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initFresh_exec hx hfr hpar hz + refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ + · have he' := he.rename hf + refine he'.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro θ + rw [PMF.map_comp] + exact hseed θ + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨p, hp, rfl⟩ := hq + obtain ⟨hfr', hrep, hpot, hmod⟩ := hsupp p hp + refine ⟨?_, ?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem p.2 x hj⟩ + · change ((p.2.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`zeroFillCode_exec`, +`initClear_exec`, `initClear_rename`) -/ + +private def zeroFillSpec : Stateful.Spec.{0, 0, 0} where + State := PUnit + Op := PEmpty + Result o := PEmpty.elim o + step _ o := PEmpty.elim o + +private noncomputable def toZeroFillDet (C : SubroutineDS P D w base params s₀) : + Cslib.WordRAM.SubroutineDS + { P with initSetup := .op (.copy P.zeroReg P.zeroReg) } + zeroFillSpec w base params PUnit.unit where + space := C.space + spanTime := C.spanTime + setupTime := 1 + initPotential := 0 + charge _ o := PEmpty.elim o + rep _ x := x.Bounded w + potential _ := 0 + encode o := PEmpty.elim o + argMatch o := PEmpty.elim o + decode o := PEmpty.elim o + bounded h := h + rep_unique _ _ := rfl + arguments_fit o := PEmpty.elim o + encode_matches o := PEmpty.elim o + rep_supported _ _ hb _ := hb + potential_supported _ _ _ := rfl + clear_nodup := C.clear_nodup + params_not_clear := C.params_not_clear + incCur_eval := C.incCur_eval + span_exec := C.span_exec + setup_exec {x₁} hx₁ _ hfr _ _ := by + refine ⟨x₁.set P.zeroReg (x₁.regs P.zeroReg), ExecWithin.copy, ?_, + hx₁.set (hx₁.1 P.zeroReg), Nat.le_refl 0, Modifies.refl _ _⟩ + simp [Data.set, hfr] + step_exec o := PEmpty.elim o + +/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` +instructions over any basis `B`. -/ +theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data r_ds} + (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) + (hfr₁ : x₁.regs P.frontierReg = base + C.space) + (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : + ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ + x₂.Bounded w ∧ + x₂.regs P.baseReg = base ∧ + x₂.regs P.frontierReg = base + C.space ∧ + (∀ j, x₂.regs (P.initParams j) = params j) ∧ + (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ + Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := + C.toZeroFillDet.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ + +/-- Execute `P.initClear` on **arbitrary dirty memory** in +`C.initTime + (4 * C.space + 6 : ℕ)` expected instructions. -/ +theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) : + ∃ ν₀ : PMF (C.Seed × Data r_ds), + ExecCodeDist w P.initClear x (ν₀.map Prod.snd) + (C.initTime + (4 * C.space + 6 : ℕ)) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + C.space ∧ + C.rep p.1 s₀ p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by + obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar + obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := + C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + obtain ⟨ν₀, he₃, hseed, hsupp⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ + rw [hm₁] at hmod₂ + have he_clear : ExecCodeDist w P.initClear x (ν₀.map Prod.snd) + ((C.spanTime : ℝ≥0∞) + ((4 * C.space + 6 : ℕ) + C.setupTime)) := + (ExecCodeDist.of_execWithin he₁).seq_pure ((ExecCodeDist.of_execWithin he₂).seq_pure he₃) + refine ⟨ν₀, he_clear.mono ?_, hseed, fun p hp => ?_⟩ + · unfold initTime + exact le_of_eq (by ac_rfl) + · obtain ⟨hfr₃, hrep, hpot, hmod₃⟩ := hsupp p hp + have hmod : Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by + simpa only [Finset.union_idempotent] using hmod₂.trans hmod₃ + exact ⟨hfr₃, hrep, hpot, hmod⟩ + +/-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ +theorem initClear_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : + ∃ ν₀ : PMF (C.Seed × Data r_caller), + ExecCodeDist w (P.initClear.rename f) x (ν₀.map Prod.snd) + (C.initTime + (4 * C.space + 6 : ℕ)) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs (f P.frontierReg) = base + C.space ∧ + C.rep p.1 s₀ (p.2.comap f) ∧ + C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by + obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initClear_exec hx hfr hpar + refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ + · have he' := he.rename hf + refine he'.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro θ + rw [PMF.map_comp] + exact hseed θ + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨p, hp, rfl⟩ := hq + obtain ⟨hfr', hrep, hpot, hmod⟩ := hsupp p hp + refine ⟨?_, ?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem p.2 x hj⟩ + · change ((p.2.extend f x).comap f).regs P.frontierReg = base + C.space + rw [Data.comap_extend hf] + exact hfr' + · rw [Data.comap_extend hf] + exact hrep + · rw [Data.comap_extend hf] + exact hpot + +/-! ### Helper lemmas for coupled PMF expectations -/ + +private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [ENNReal.tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +private theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : + (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by + simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] + exact ENNReal.tsum_comm + +/-! ### Projection 4: Caller-to-DS bidirectional framing (`frame_caller`, `frame_caller_dist`) -/ + +/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS +registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from +`[base, base + C.space)` preserves both `C.rep θ s (x'.comap f)` and +`C.potential θ (x'.comap f)`. -/ +theorem frame_caller (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} {θ : C.Seed} {s : D.State} {x x' : Data r_caller} + {F : Finset ℕ} + (hr : C.rep θ s (x.comap f)) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hmem : Modifies F x.mem x'.mem) : + C.rep θ s (x'.comap f) ∧ C.potential θ (x'.comap f) = C.potential θ (x.comap f) := by + have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => + hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) + exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ + +/-- Coupled caller distribution after a caller transition kernel `μ_caller` that does not resample +the DS seed `θ`. -/ +noncomputable def frameDist (C : SubroutineDS P D w base params s₀) + (ν : PMF (C.Seed × Data r_caller)) + (μ_caller : C.Seed × Data r_caller → PMF (Data r_caller)) : + PMF (C.Seed × Data r_caller) := + ν.bind fun p => (μ_caller p).map fun x' => (p.1, x') + +@[simp] +theorem frameDist_map_fst (C : SubroutineDS P D w base params s₀) + (ν : PMF (C.Seed × Data r_caller)) + (μ_caller : C.Seed × Data r_caller → PMF (Data r_caller)) : + (C.frameDist ν μ_caller).map Prod.fst = ν.map Prod.fst := by + unfold frameDist + rw [PMF.map_bind, PMF.map] + refine congrArg (ν.bind ·) (funext fun p => ?_) + ext θ + rw [PMF.map_comp] + simp + +/-- Distributional caller framing: any deterministic or randomized caller transition kernel +`μ_caller` (including caller steps that execute `Instr.rand`) that preserves +`(x'.comap f).Bounded w`, leaves persistent DS registers (`P.input i = false`) unchanged, and +writes only to a set `F` disjoint from `[base, base + C.space)` preserves the coupled seed +marginal bound, `C.rep`, and expected potential under `C.frameDist ν μ_caller`. -/ +theorem frame_caller_dist (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} {s : D.State} + {ν : PMF (C.Seed × Data r_caller)} + {μ_caller : C.Seed × Data r_caller → PMF (Data r_caller)} + {F : Finset ℕ} + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hrep : ∀ p ∈ ν.support, C.rep p.1 s (p.2.comap f)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hframe : ∀ p ∈ ν.support, ∀ x' ∈ (μ_caller p).support, + (x'.comap f).Bounded w ∧ + (∀ i, P.input i = false → x'.regs (f i) = p.2.regs (f i)) ∧ + Modifies F p.2.mem x'.mem) : + (∀ θ, ((C.frameDist ν μ_caller).map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ q ∈ (C.frameDist ν μ_caller).support, C.rep q.1 s (q.2.comap f)) ∧ + (∑' q, C.frameDist ν μ_caller q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + refine ⟨fun θ => by rw [C.frameDist_map_fst]; exact hseed θ, ?_, ?_⟩ + · intro q hq + rw [frameDist, PMF.mem_support_bind_iff] at hq + obtain ⟨p, hp, hq_map⟩ := hq + rw [PMF.mem_support_map_iff] at hq_map + obtain ⟨x', hx', rfl⟩ := hq_map + obtain ⟨hb', hreg, hmem⟩ := hframe p hp x' hx' + exact (C.frame_caller (hrep p hp) hb' hreg hdisj hmem).1 + · rw [frameDist, tsum_pmf_bind] + refine tsum_congr fun p => ?_ + by_cases hp : p ∈ ν.support + · rw [tsum_pmf_map] + have hpot_eq : (∑' x', μ_caller p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = + ∑' x', μ_caller p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + refine tsum_congr fun x' => ?_ + by_cases hx' : x' ∈ (μ_caller p).support + · obtain ⟨hb', hreg, hmem⟩ := hframe p hp x' hx' + rw [(C.frame_caller (hrep p hp) hb' hreg hdisj hmem).2] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hx', zero_mul, zero_mul] + rw [hpot_eq, ENNReal.tsum_mul_right, (μ_caller p).tsum_coe, one_mul] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + +/-! ### Projection 5: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ + +/-- Execute `P.code.rename f` inline when arguments already match `(p.2.comap f).regs` on the +support of a coupled caller distribution `ν : PMF (C.Seed × Data r_caller)`. -/ +theorem code_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} (o : D.Op) (ν : PMF (C.Seed × Data r_caller)) + (hv : D.valid s o) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hsupp : ∀ p ∈ ν.support, C.rep p.1 s (p.2.comap f) ∧ C.argMatch o (p.2.comap f).regs) : + ∃ (μ_caller : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller)) + (T_caller : C.Seed × Data r_caller → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (P.code.rename f) p.2 ((μ_caller p).map Prod.snd) (T_caller p) ∧ + ∀ q ∈ (μ_caller p).support, + C.decode o (q.2.comap f).regs = some (D.step s o).2 ∧ + C.rep q.1 (D.step s o).1 (q.2.comap f) ∧ + Modifies (Block.mk base C.space).footprint p.2.mem q.2.mem ∧ + (∀ j ∉ Set.range f, q.2.regs j = p.2.regs j)) ∧ + (∀ θ', ((ν.bind μ_caller).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ + (∑' p, ν p * (T_caller p + ∑' q, μ_caller p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) ≤ + (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + let proj : C.Seed × Data r_caller → C.Seed × Data r_ds := fun p => (p.1, p.2.comap f) + let ν_ds : PMF (C.Seed × Data r_ds) := ν.map proj + have hseed_ds : ∀ θ, (ν_ds.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + intro θ + rw [PMF.map_comp] + exact hseed θ + have hsupp_ds : ∀ p_ds ∈ ν_ds.support, C.rep p_ds.1 s p_ds.2 ∧ C.argMatch o p_ds.2.regs := by + intro p_ds hp_ds + rw [PMF.mem_support_map_iff] at hp_ds + obtain ⟨p, hp, rfl⟩ := hp_ds + exact hsupp p hp + obtain ⟨μ_ds, T_ds, hstep_ds, hseed_ds', hamort_ds⟩ := + C.step_exec o ν_ds hv hseed_ds hsupp_ds + let μ_caller : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller) := + fun p => (μ_ds (proj p)).map fun q => (q.1, q.2.extend f p.2) + let T_caller : C.Seed × Data r_caller → ℝ≥0∞ := fun p => T_ds (proj p) + refine ⟨μ_caller, T_caller, ?_, ?_, ?_⟩ + · intro p hp + have hp_ds : proj p ∈ ν_ds.support := (PMF.mem_support_map_iff _ _ _).mpr ⟨p, hp, rfl⟩ + obtain ⟨he_ds, hq_ds⟩ := hstep_ds (proj p) hp_ds + refine ⟨?_, ?_⟩ + · have he_ren := he_ds.rename hf (x := p.2) + refine he_ren.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨q_ds, hq_mem, rfl⟩ := hq + obtain ⟨hdec, hrep', hmod⟩ := hq_ds q_ds hq_mem + refine ⟨?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem q_ds.2 p.2 hj⟩ + · rw [Data.comap_extend hf] + exact hdec + · rw [Data.comap_extend hf] + exact hrep' + · intro θ' + have hmap_eq : (ν.bind μ_caller).map Prod.fst = (ν_ds.bind μ_ds).map Prod.fst := by + rw [PMF.map_bind, PMF.map_bind, PMF.bind_map] + refine congrArg (ν.bind ·) (funext fun p => ?_) + rw [PMF.map_comp] + rfl + rw [hmap_eq] + exact hseed_ds' θ' + · have hPot_inner : ∀ p : C.Seed × Data r_caller, + (∑' q, μ_caller p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞) := by + intro p + rw [tsum_pmf_map] + refine tsum_congr fun q_ds => ?_ + rw [Data.comap_extend hf] + simp_rw [hPot_inner] + have hLHS : + (∑' p, ν p * (T_caller p + + ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞))) = + ∑' p_ds, ν_ds p_ds * (T_ds p_ds + + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)) := + (tsum_pmf_map ν proj fun p_ds => + T_ds p_ds + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)).symm + have hRHS : + (∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞)) = + ∑' p_ds, ν_ds p_ds * (C.potential p_ds.1 p_ds.2 : ℝ≥0∞) := + (tsum_pmf_map ν proj fun p_ds => (C.potential p_ds.1 p_ds.2 : ℝ≥0∞)).symm + rw [hLHS, hRHS] + exact hamort_ds + +/-- Execute `.seq prep (P.code.rename f)` inline after a (deterministic or randomized) caller +argument-preparation block `prep` that preserves persistent DS registers and memory in +`[base, base + C.space)`. -/ +theorem call_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {s : D.State} {prep : Code B r_caller} {B_prep : ℝ≥0∞} + (o : D.Op) (ν : PMF (C.Seed × Data r_caller)) + {μ_prep : C.Seed × Data r_caller → PMF (Data r_caller)} + {T_prep : C.Seed × Data r_caller → ℝ≥0∞} + (hv : D.valid s o) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hrep : ∀ p ∈ ν.support, C.rep p.1 s (p.2.comap f)) + (hprep : ∀ p ∈ ν.support, + ExecCodeDist w prep p.2 (μ_prep p) (T_prep p) ∧ + T_prep p ≤ B_prep ∧ + ∀ x' ∈ (μ_prep p).support, + (x'.comap f).Bounded w ∧ + (∀ i, P.input i = false → x'.regs (f i) = p.2.regs (f i)) ∧ + (∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = p.2.mem a) ∧ + C.argMatch o (x'.comap f).regs) : + ∃ (μ_call : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller)) + (T_call : C.Seed × Data r_caller → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (.seq prep (P.code.rename f)) p.2 ((μ_call p).map Prod.snd) (T_call p) ∧ + ∀ q ∈ (μ_call p).support, + C.decode o (q.2.comap f).regs = some (D.step s o).2 ∧ + C.rep q.1 (D.step s o).1 (q.2.comap f)) ∧ + (∀ θ', ((ν.bind μ_call).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ + (∑' p, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) ≤ + B_prep + (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + let ν_prep := C.frameDist ν μ_prep + have hseed_prep : ∀ θ, (ν_prep.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + intro θ + rw [C.frameDist_map_fst] + exact hseed θ + have hsupp_prep : ∀ p' ∈ ν_prep.support, + C.rep p'.1 s (p'.2.comap f) ∧ C.argMatch o (p'.2.comap f).regs := by + intro p' hp' + simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] at hp' + obtain ⟨p, hp, hp_map⟩ := hp' + rw [PMF.mem_support_map_iff] at hp_map + obtain ⟨x', hx', rfl⟩ := hp_map + obtain ⟨_, _, hfr_x'⟩ := hprep p hp + obtain ⟨hb', hreg, hmem, harg⟩ := hfr_x' x' hx' + exact ⟨C.rep_supported (x := p.2.comap f) (x' := x'.comap f) hreg hmem hb' (hrep p hp), harg⟩ + obtain ⟨μ_code, T_code, hcode, hseed_code, hamort_code⟩ := + C.code_rename hf o ν_prep hv hseed_prep hsupp_prep + let μ_call : C.Seed × Data r_caller → PMF (C.Seed × Data r_caller) := + fun p => (μ_prep p).bind fun x' => μ_code (p.1, x') + let T_call : C.Seed × Data r_caller → ℝ≥0∞ := + fun p => T_prep p + ∑' x', μ_prep p x' * T_code (p.1, x') + refine ⟨μ_call, T_call, ?_, ?_, ?_⟩ + · intro p hp + obtain ⟨he_prep, _, hfr_x'⟩ := hprep p hp + have he_seq := he_prep.seq + (μ₂ := fun x' => (μ_code (p.1, x')).map Prod.snd) + (T₂ := fun x' => T_code (p.1, x')) + fun x' hx' => by + have hp' : (p.1, x') ∈ ν_prep.support := by + simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] + exact ⟨p, hp, (PMF.mem_support_map_iff _ _ _).mpr ⟨x', hx', rfl⟩⟩ + exact (hcode (p.1, x') hp').1 + refine ⟨he_seq.congr_dist ?_ le_rfl, ?_⟩ + · simp only [μ_call, PMF.map_bind] + · intro q hq + rw [PMF.mem_support_bind_iff] at hq + obtain ⟨x', hx', hq_code⟩ := hq + have hp' : (p.1, x') ∈ ν_prep.support := by + simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] + exact ⟨p, hp, (PMF.mem_support_map_iff _ _ _).mpr ⟨x', hx', rfl⟩⟩ + obtain ⟨hdec, hrep', _, _⟩ := (hcode (p.1, x') hp').2 q hq_code + exact ⟨hdec, hrep'⟩ + · intro θ' + have hbind_eq : ν.bind μ_call = ν_prep.bind μ_code := by + simp only [ν_prep, frameDist, μ_call, PMF.bind_bind, PMF.bind_map] + rfl + rw [hbind_eq] + exact hseed_code θ' + · have hpot_prep : + (∑' p', ν_prep p' * (C.potential p'.1 (p'.2.comap f) : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + simp only [ν_prep, frameDist, tsum_pmf_bind] + refine tsum_congr fun p => ?_ + by_cases hp : p ∈ ν.support + · rw [tsum_pmf_map] + have hpot_eq : (∑' x', μ_prep p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = + ∑' x', μ_prep p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + refine tsum_congr fun x' => ?_ + by_cases hx' : x' ∈ (μ_prep p).support + · obtain ⟨_, _, hfr_x'⟩ := hprep p hp + obtain ⟨_, hreg, hmem, _⟩ := hfr_x' x' hx' + rw [C.potential_supported (x := p.2.comap f) (x' := x'.comap f) hreg hmem (hrep p hp)] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hx', zero_mul, zero_mul] + rw [hpot_eq, ENNReal.tsum_mul_right, (μ_prep p).tsum_coe, one_mul] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + have hdecomp : + (∑' p, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) = + (∑' p, ν p * T_prep p) + + ∑' p', ν_prep p' * + (T_code p' + ∑' q, μ_code p' q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by + have hinner : ∀ p : C.Seed × Data r_caller, + T_call p + (∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + T_prep p + ∑' x', μ_prep p x' * + (T_code (p.1, x') + + ∑' q, μ_code (p.1, x') q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by + intro p + simp only [T_call, μ_call, tsum_pmf_bind, add_assoc] + congr 1 + rw [← ENNReal.tsum_add] + simp_rw [mul_add] + have hmul : ∀ p : C.Seed × Data r_caller, + ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = + ν p * T_prep p + ν p * ∑' x', μ_prep p x' * + (T_code (p.1, x') + + ∑' q, μ_code (p.1, x') q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by + intro p + rw [hinner p, mul_add] + simp_rw [hmul] + rw [ENNReal.tsum_add] + congr 1 + symm + dsimp only [ν_prep, frameDist] + rw [tsum_pmf_bind] + refine tsum_congr fun p => ?_ + rw [tsum_pmf_map] + have hprep_le : (∑' p, ν p * T_prep p) ≤ B_prep := by + calc + (∑' p, ν p * T_prep p) + ≤ ∑' p, ν p * B_prep := by + refine ENNReal.tsum_le_tsum fun p => ?_ + by_cases hp : p ∈ ν.support + · exact mul_le_mul_right (hprep p hp).2.1 _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + _ = B_prep := by rw [ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + rw [hdecomp] + calc + (∑' p, ν p * T_prep p) + + ∑' p', ν_prep p' * + (T_code p' + ∑' q, μ_code p' q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) + ≤ B_prep + ((C.charge s o : ℝ≥0∞) + + ∑' p', ν_prep p' * (C.potential p'.1 (p'.2.comap f) : ℝ≥0∞)) := + add_le_add hprep_le hamort_code + _ = B_prep + (C.charge s o : ℝ≥0∞) + + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by + rw [hpot_prep, add_assoc] + +/-! ### Projection 6: End-to-end expected-amortized sequence telescope (`steps_expected_amortized`, +`seq_expected_amortized`) -/ + +/-- Deliver standalone operation arguments for `o` without changing memory or persistent +registers (`P.input i = false`). -/ +def prepare (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : Data r_ds where + regs i := if P.input i then C.encode o i else x.regs i + mem := x.mem + +@[simp] +theorem prepare_mem (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : + (C.prepare o x).mem = x.mem := rfl + +theorem prepare_regs_of_not_input (C : SubroutineDS P D w base params s₀) + {i : Fin r_ds} (hi : P.input i = false) (o : D.Op) (x : Data r_ds) : + (C.prepare o x).regs i = x.regs i := by + simp [prepare, hi] + +theorem prepare_bounded (C : SubroutineDS P D w base params s₀) + {s : D.State} {o : D.Op} {x : Data r_ds} (hx : x.Bounded w) (hv : D.valid s o) : + (C.prepare o x).Bounded w := by + refine ⟨fun i => ?_, hx.2⟩ + change Fits w (if P.input i then C.encode o i else x.regs i) + split + · exact C.arguments_fit o hv i (by assumption) + · exact hx.1 i + +theorem prepare_rep (C : SubroutineDS P D w base params s₀) + {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} + (hr : C.rep θ s x) (hv : D.valid s o) : + C.rep θ s (C.prepare o x) := + C.rep_supported + (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) + (C.prepare_bounded (C.bounded hr) hv) + hr + +theorem prepare_potential (C : SubroutineDS P D w base params s₀) + {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} + (hr : C.rep θ s x) : + C.potential θ (C.prepare o x) = C.potential θ x := + C.potential_supported + (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) + hr + +theorem prepare_argMatch (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : + C.argMatch o (C.prepare o x).regs := + C.encode_matches o x.regs + +/-- **Multi-operation expected-amortized telescope (`steps_expected_amortized`).** +From any abstract state `s : D.State` and coupled state distribution `ν : PMF (C.Seed × Data r_ds)` +satisfying the seed-marginal bound and `C.rep p.1 s p.2` on `ν.support`, executing any admissible +operation sequence `ops : List D.Op` (`D.ValidHistory s ops`) produces a final coupled distribution +`ν_final` and total expected operation step count `T_ops : ℝ≥0∞` satisfying the addition-only +`ℝ≥0∞` telescope inequality +`T_ops + E_{ν_final}[Φ] ≤ D.totalCost C.charge s ops + E_ν[Φ]`. -/ +theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) + {s : D.State} (ops : List D.Op) (ν : PMF (C.Seed × Data r_ds)) + (hv : D.ValidHistory s ops) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) + (hrep : ∀ p ∈ ν.support, C.rep p.1 s p.2) : + ∃ (ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), + (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ p ∈ ν_final.support, C.rep p.1 (D.run s ops).1 p.2) ∧ + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ + (D.totalCost C.charge s ops : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by + induction ops generalizing s ν with + | nil => + exact ⟨ν, 0, hseed, hrep, by simp⟩ + | cons o os ih => + obtain ⟨hvo, hvos⟩ := hv + let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.prepare o p.2) + have hseed_prep : ∀ θ, (ν_prep.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + intro θ + rw [PMF.map_comp] + exact hseed θ + have hsupp_prep : ∀ p' ∈ ν_prep.support, C.rep p'.1 s p'.2 ∧ C.argMatch o p'.2.regs := by + intro p' hp' + rw [PMF.mem_support_map_iff] at hp' + obtain ⟨p, hp, rfl⟩ := hp' + exact ⟨C.prepare_rep (hrep p hp) hvo, C.prepare_argMatch o p.2⟩ + have hpot_prep : + (∑' p', ν_prep p' * (C.potential p'.1 p'.2 : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by + rw [tsum_pmf_map] + refine tsum_congr fun p => ?_ + by_cases hp : p ∈ ν.support + · rw [C.prepare_potential (hrep p hp)] + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + obtain ⟨μ_step, T_step, hstep, hseed₁, hamort₁⟩ := + C.step_exec o ν_prep hvo hseed_prep hsupp_prep + let ν₁ := ν_prep.bind μ_step + have hrep₁ : ∀ q ∈ ν₁.support, C.rep q.1 (D.step s o).1 q.2 := by + intro q hq + rw [PMF.mem_support_bind_iff] at hq + obtain ⟨p', hp', hq_step⟩ := hq + exact ((hstep p' hp').2 q hq_step).2.1 + obtain ⟨ν_final, T_rest, hseed_final, hrep_final, hamort_rest⟩ := + ih ν₁ hvos hseed₁ hrep₁ + let T₁ : ℝ≥0∞ := ∑' p', ν_prep p' * T_step p' + have hstep_split : + T₁ + (∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) = + ∑' p', ν_prep p' * (T_step p' + ∑' q, μ_step p' q * (C.potential q.1 q.2 : ℝ≥0∞)) := by + simp only [T₁, ν₁, tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] + refine ⟨ν_final, T₁ + T_rest, hseed_final, hrep_final, ?_⟩ + calc + (T₁ + T_rest) + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) + = T₁ + (T_rest + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl + _ ≤ T₁ + ((D.totalCost C.charge (D.step s o).1 os : ℝ≥0∞) + + ∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) := + add_le_add le_rfl hamort_rest + _ = (D.totalCost C.charge (D.step s o).1 os : ℝ≥0∞) + + (T₁ + ∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl + _ ≤ (D.totalCost C.charge (D.step s o).1 os : ℝ≥0∞) + + ((C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞)) := by + rw [hstep_split, ← hpot_prep] + exact add_le_add le_rfl hamort₁ + _ = (D.totalCost C.charge s (o :: os) : ℝ≥0∞) + + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by + simp only [Stateful.Spec.totalCost_cons, Nat.cast_add] + ac_rfl + +/-- **End-to-end expected-amortized theorem from fresh initialization (`seq_expected_amortized`).** +Starting from zeroed `[base, base + C.space)`, `P.initFresh` followed by any admissible operation +sequence `ops : List D.Op` (`D.ValidHistory s₀ ops`) has total expected instruction cost +`C.initTime + T_ops` bounded by +`C.initTime + C.initPotential + D.totalCost C.charge s₀ ops`. -/ +theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) + {x : Data r_ds} (ops : List D.Op) + (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) + (hpar : ∀ j, x.regs (P.initParams j) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) + (hv : D.ValidHistory s₀ ops) : + ∃ (ν₀ ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), + ExecCodeDist w P.initFresh x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ p ∈ ν_final.support, C.rep p.1 (D.run s₀ ops).1 p.2) ∧ + C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ + C.initTime + (C.initPotential : ℝ≥0∞) + (D.totalCost C.charge s₀ ops : ℝ≥0∞) := by + obtain ⟨ν₀, he_init, hseed₀, hsupp₀⟩ := C.initFresh_exec hx hfr hpar hz + have hrep₀ : ∀ p ∈ ν₀.support, C.rep p.1 s₀ p.2 := fun p hp => (hsupp₀ p hp).2.1 + obtain ⟨ν_final, T_ops, hseed_final, hrep_final, hamort⟩ := + C.steps_expected_amortized ops ν₀ hv hseed₀ hrep₀ + have hpot₀ : (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) ≤ (C.initPotential : ℝ≥0∞) := by + calc + (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) + ≤ ∑' p, ν₀ p * (C.initPotential : ℝ≥0∞) := by + refine ENNReal.tsum_le_tsum fun p => ?_ + by_cases hp : p ∈ ν₀.support + · exact mul_le_mul_right (Nat.cast_le.mpr (hsupp₀ p hp).2.2.1) _ + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + _ = (C.initPotential : ℝ≥0∞) := by + rw [ENNReal.tsum_mul_right, ν₀.tsum_coe, one_mul] + refine ⟨ν₀, ν_final, T_ops, he_init, hseed_final, hrep_final, ?_⟩ + calc + C.initTime + T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) + = C.initTime + (T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl + _ ≤ C.initTime + ((D.totalCost C.charge s₀ ops : ℝ≥0∞) + + ∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) := + add_le_add le_rfl hamort + _ ≤ C.initTime + ((D.totalCost C.charge s₀ ops : ℝ≥0∞) + (C.initPotential : ℝ≥0∞)) := + add_le_add le_rfl (add_le_add le_rfl hpot₀) + _ = C.initTime + (C.initPotential : ℝ≥0∞) + (D.totalCost C.charge s₀ ops : ℝ≥0∞) := by ac_rfl + +end Cslib.WordRAM.Randomized.SubroutineDS + +end diff --git a/ComplexityTests/WordRAMRandomizedSubroutine.lean b/ComplexityTests/WordRAMRandomizedSubroutine.lean new file mode 100644 index 000000000..d11f4056d --- /dev/null +++ b/ComplexityTests/WordRAMRandomizedSubroutine.lean @@ -0,0 +1,350 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Computability.WordRAM.Randomized.Expected +public import Complexity.Computability.WordRAM.Randomized.Subroutine + +/-! +# Randomized Word-RAM geometric loop and subroutine interface tests + +Verifies the distributional structured execution relation `ExecCodeDist`, the geometric Las Vegas +loop expected-time bound `ExecCodeDist.loop_geometric`, and the randomized Word-RAM subroutine +interface `Cslib.WordRAM.Randomized.SubroutineDS`: +* a concrete geometric coin-flipping Las Vegas loop (`while r0 == r1 do r0 ← rand r1` with + `r1 = 1` and exit probability `1 / 2` per iteration) almost surely halts with `r0 = 0` and + bounded expected step count; +* a concrete randomized data structure executing `Instr.rand` during its operations instantiates + `Cslib.WordRAM.Randomized.SubroutineDS` and exercises `toStandard`, `initFresh_exec`, + `initClear_exec`, `frame_caller_dist` (across a randomized caller step), `call_rename`, and + `seq_expected_amortized`. +-/ + +@[expose] public section + +namespace ComplexityTests.WordRAMRandomizedSubroutine + +open Cslib Cslib.WordRAM Cslib.WordRAM.Randomized +open Cslib.Probability.PMF (uniformOfFintype uniformOfFintype_apply) +open scoped ENNReal + +/-! ## 1. Geometric coin-flipping Las Vegas loop (`ExecCodeDist.loop_geometric`) -/ + +abbrev g0 : Fin 2 := 0 +abbrev g1 : Fin 2 := 1 + +/-- Geometric Las Vegas loop: while `r0 == r1` (with `r1 = 1`), sample `r0 ← rand r1` uniformly +from `{0, 1}`. Each iteration exits with probability `1 / 2` in `1` body instruction. -/ +def geomCoinLoop : Code .exact 2 := + .loop (.eq g0 g1) (.rand g0 g1) + +/-- Loop-head invariant: `x` is `w`-bounded and `r1 = 1`. -/ +def geomInv (w : ℕ) (x : Data 2) : Prop := + x.Bounded w ∧ x.regs g1 = 1 + +/-- Loop-exit postcondition: `x` is `w`-bounded, `r0 = 0`, and `r1 = 1`. -/ +def geomPost (w : ℕ) (x : Data 2) : Prop := + x.Bounded w ∧ x.regs g0 = 0 ∧ x.regs g1 = 1 + +/-- Single-iteration transition distribution of `r0 ← rand r1` when `r1 = 1`. -/ +noncomputable def geomBodyDist (x : Data 2) : PMF (Data 2) := + (uniformOfFintype (Fin (x.regs g1 + 1))).map fun k => x.set g0 k.val + +private theorem fits_one_of_pos {w : ℕ} (hw : 1 ≤ w) : Fits w 1 := + fits_one (by omega) + +theorem geomCoinLoop_execCodeDist {w : ℕ} (hw : 1 ≤ w) (x₀ : Data 2) + (hx₀ : geomInv w x₀ ∧ (Cond.eq g0 g1).Holds x₀) : + ∃ μ_exit : PMF (Data 2), + ExecCodeDist w geomCoinLoop x₀ μ_exit (((1 : ℝ≥0∞) + 2) / (2 : ℝ≥0∞)⁻¹ + 2) ∧ + ∀ y ∈ μ_exit.support, ¬(Cond.eq g0 g1).Holds y ∧ geomPost w y := by + have hp_pos : (0 : ℝ≥0∞) < (2 : ℝ≥0∞)⁻¹ := by simp + have hp_top : ((2 : ℝ≥0∞)⁻¹) ≠ ⊤ := by simp + have hbody : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → + ExecCodeDist (B := .exact) w (.rand g0 g1) z (geomBodyDist z) 1 := by + intro z ⟨hbz, _⟩ _ + simpa [geomBodyDist, ProbComp.eval_unifFin] using ExecCodeDist.rand (B := .exact) (hbz.1 g1) + have hB : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → (1 : ℝ≥0∞) ≤ 1 := + fun _ _ _ => le_rfl + have hinv : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → + ∀ y ∈ (geomBodyDist z).support, LoopStateInv (.eq g0 g1) (geomInv w) (geomPost w) y := by + intro x ⟨hbx, hr1⟩ _ y hy + rw [geomBodyDist, PMF.mem_support_map_iff] at hy + obtain ⟨k, _, rfl⟩ := hy + have hk_lt : k.val < 2 := by omega + have hby : (x.set g0 k.val).Bounded w := by + refine hbx.set ?_ + rcases Nat.lt_succ_iff_lt_or_eq.mp hk_lt with hk0 | hk1 + · have hk0' : k.val = 0 := Nat.lt_one_iff.mp hk0 + rw [hk0'] + exact fits_zero w + · rw [hk1] + exact fits_one_of_pos hw + have hy_r1 : (x.set g0 k.val).regs g1 = 1 := by + simp [Data.set, hr1] + rcases Nat.lt_succ_iff_lt_or_eq.mp hk_lt with hk0 | hk1 + · have hk0' : k.val = 0 := Nat.lt_one_iff.mp hk0 + right + refine ⟨?_, hby, ?_, hy_r1⟩ + · simp [Cond.Holds, Data.set, hr1, hk0'] + · simp [Data.set, hk0'] + · left + refine ⟨?_, hby, hy_r1⟩ + simp [Cond.Holds, Data.set, hr1, hk1] + have hp_exit : ∀ z, geomInv w z → (Cond.eq g0 g1).Holds z → + (2 : ℝ≥0∞)⁻¹ ≤ (geomBodyDist z).toOuterMeasure {y | ¬(Cond.eq g0 g1).Holds y} := by + intro x ⟨_, hr1⟩ _ + classical + rw [PMF.toOuterMeasure_apply] + have h_zero_lt : 0 < x.regs g1 + 1 := by omega + let k₀ : Fin (x.regs g1 + 1) := ⟨0, h_zero_lt⟩ + let y₀ : Data 2 := x.set g0 0 + have hy₀_not : ¬(Cond.eq g0 g1).Holds y₀ := by + simp [y₀, Cond.Holds, Data.set, hr1] + have h1 : (2 : ℝ≥0∞)⁻¹ ≤ geomBodyDist x y₀ := by + rw [geomBodyDist, PMF.map_apply] + have hle := ENNReal.le_tsum + (f := fun k : Fin (x.regs g1 + 1) => + if y₀ = x.set g0 k.val then uniformOfFintype (Fin (x.regs g1 + 1)) k else 0) k₀ + simpa [y₀, k₀, uniformOfFintype_apply, hr1] using hle + have h2 : geomBodyDist x y₀ ≤ + ∑' y, Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y := by + calc + geomBodyDist x y₀ + = Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y₀ := by + rw [Set.indicator_of_mem hy₀_not] + _ ≤ ∑' y, Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y := + ENNReal.le_tsum y₀ + exact le_trans h1 h2 + have hx₀_inv : LoopStateInv (.eq g0 g1) (geomInv w) (geomPost w) x₀ := Or.inl ⟨hx₀.2, hx₀.1⟩ + refine ⟨loopExitPMF (.eq g0 g1) geomBodyDist x₀, + ExecCodeDist.loop_geometric (geomInv w) (geomPost w) hp_pos hp_top hbody hB hinv hp_exit + hx₀_inv, + fun y hy => + (mem_support_loopExitPMF hp_pos hp_top hinv hp_exit hx₀_inv hy).1⟩ + +/-- Complete flat program for the geometric coin-flipping loop exercises `program_halts`: +it halts almost surely from `r0 = 1, r1 = 1` with finite expected step count. -/ +theorem geomCoinLoop_halts {w : ℕ} (hw : 1 ≤ w) : + let x₀ : Data 2 := ((Data.init [] : Data 2).set g0 1).set g1 1 + AlmostSurelyHalts w (geomCoinLoop.program g0) ⟨0, x₀⟩ ∧ + expectedTime w (geomCoinLoop.program g0) ⟨0, x₀⟩ ≤ + ((1 : ℝ≥0∞) + 2) / (2 : ℝ≥0∞)⁻¹ + 2 + 1 := by + intro x₀ + have hx₀_inv : geomInv w x₀ := by + refine ⟨?_, by simp [x₀, Data.set]⟩ + exact ((Data.Bounded.init (by simp [InputFits])).set + (fits_one_of_pos hw)).set (fits_one_of_pos hw) + have hx₀_cond : (Cond.eq g0 g1).Holds x₀ := by + simp [x₀, Cond.Holds, Data.set] + obtain ⟨μ_exit, hex, _⟩ := geomCoinLoop_execCodeDist hw x₀ ⟨hx₀_inv, hx₀_cond⟩ + obtain ⟨hExp, _, hAS, _⟩ := hex.program_halts g0 + have h12_ne : (1 : ℝ≥0∞) + 2 ≠ ⊤ := + ENNReal.add_ne_top.mpr ⟨ENNReal.one_ne_top, ENNReal.ofNat_ne_top⟩ + have hT_ne : ((1 : ℝ≥0∞) + 2) / (2 : ℝ≥0∞)⁻¹ + 2 ≠ ⊤ := + ENNReal.add_ne_top.mpr ⟨ENNReal.div_ne_top h12_ne (by simp), ENNReal.ofNat_ne_top⟩ + exact ⟨hAS hT_ne, hExp⟩ + +/-! ## 2. Concrete randomized data structure (`Cslib.WordRAM.Randomized.SubroutineDS`) -/ + +/-- Trivial stateful specification for a randomized sampler subroutine whose implementation invokes +`Instr.rand` during each operation. -/ +def coinSpec : Stateful.Spec.{0, 0, 0} where + State := PUnit + Op := PUnit + Result _ := PUnit + step _ _ := (PUnit.unit, PUnit.unit) + +/-- Subroutine code with 5 registers (`0` = `baseReg`, `1` = `frontierReg`, `2` = `zeroReg`, +`3` = `curReg` / scratch output, `4` = `oneReg`). Its operation body `code` executes +`r3 ← rand r2` (`Instr.rand`). -/ +def coinSubroutineCode : SubroutineCode .exact 5 0 where + allocSpan := .op (.copy 0 1) + initSetup := .op (.const 2 0) + code := .rand 3 2 + baseReg := 0 + frontierReg := 1 + zeroReg := 2 + oneReg := 4 + curReg := 3 + incCur := Op.exactBin .add 3 3 4 + initParams := Fin.elim0 + input i := i == 3 + output := 3 + +/-- Representation invariant for `coinSubroutineCode`: machine words are `w`-bounded, `baseReg` +holds `base`, and `zeroReg` holds `0`. -/ +def coinRep (w base : ℕ) (_θ : Fin 2) (_s : PUnit) (x : Data 5) : Prop := + x.Bounded w ∧ x.regs 0 = base ∧ x.regs 2 = 0 + +private theorem pmf_map_fst_const {α β γ : Type*} (q : PMF α) (b : β) (g : α → γ) : + (q.map fun a => (b, g a)).map Prod.fst = PMF.pure b := by + ext θ + classical + rw [PMF.map_comp] + by_cases h : θ = b <;> simp [PMF.map_apply, h, q.tsum_coe] + +private theorem pmf_map_snd_const {α β γ : Type*} (q : PMF α) (f : α → β) (c : γ) : + (q.map fun a => (f a, c)).map Prod.snd = PMF.pure c := by + ext y + classical + rw [PMF.map_comp] + by_cases h : y = c <;> simp [PMF.map_apply, h, q.tsum_coe] + +/-- Concrete randomized `SubroutineDS` certificate for `coinSubroutineCode` with latent seed type +`Fin 2` and uniform reference distribution `uniformOfFintype (Fin 2)`. -/ +noncomputable def coinSubroutineDS (w base : ℕ) : + Cslib.WordRAM.Randomized.SubroutineDS + coinSubroutineCode coinSpec w base Fin.elim0 PUnit.unit where + Seed := Fin 2 + seedLaw := uniformOfFintype (Fin 2) + seedSlack := 1 + seedSlack_ne_top := ENNReal.one_ne_top + space := 0 + spanTime := 1 + setupTime := 1 + initPotential := 0 + charge _ _ := 1 + rep := coinRep w base + potential _ _ := 0 + encode _ _ := 0 + argMatch _ _ := True + decode _ _ := some PUnit.unit + bounded h := h.1 + rep_unique _ _ := rfl + arguments_fit _ _ _ _ := fits_zero w + encode_matches _ _ := trivial + rep_supported hreg _ hb' hrep := + ⟨hb', + by rw [hreg 0 rfl, hrep.2.1], + by rw [hreg 2 rfl, hrep.2.2]⟩ + potential_supported _ _ _ := rfl + clear_nodup := by decide + params_not_clear j := Fin.elim0 j + incCur_eval {x} h1 hfit := by + change Data.put w x 3 (x.regs 3 + x.regs 4) = + some (x.set coinSubroutineCode.curReg (x.regs coinSubroutineCode.curReg + 1)) + change x.regs 4 = 1 at h1 + rw [h1] + exact Data.put_of_fits hfit + span_exec hx hfr _ := by + refine ⟨_, ExecWithin.copy, rfl, ?_, ?_, fun j => Fin.elim0 j⟩ + · simpa [coinSubroutineCode, Data.set] using hfr + · simpa [coinSubroutineCode, Data.set] using hfr + setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by + let x₂ : Data 5 := x₁.set 2 0 + let ν₀ : PMF (Fin 2 × Data 5) := (uniformOfFintype (Fin 2)).map fun θ => (θ, x₂) + refine ⟨ν₀, ?_, ?_, ?_⟩ + · have hex : ExecCodeDist (B := .exact) w (.op (.const 2 0)) x₁ (PMF.pure x₂) (1 : ℕ) := + ExecCodeDist.of_exec (Exec.const (fits_zero w)) + exact hex.congr_dist (pmf_map_snd_const (uniformOfFintype (Fin 2)) id x₂).symm (by simp) + · intro θ + rw [PMF.map_comp] + simp + · intro p hp + rw [PMF.mem_support_map_iff] at hp + obtain ⟨θ, _, rfl⟩ := hp + refine ⟨?_, ?_, Nat.le_refl 0, Modifies.refl _ _⟩ + · simpa [coinSubroutineCode, x₂, Data.set] using hfr₁ + · refine ⟨hx₁.set (fits_zero w), ?_, ?_⟩ + · simp [x₂, Data.set, ← hb₁, coinSubroutineCode] + · simp [x₂, Data.set] + step_exec _ ν _ hseed hsupp := by + let μ_step : Fin 2 × Data 5 → PMF (Fin 2 × Data 5) := fun p => + (uniformOfFintype (Fin (p.2.regs 2 + 1))).map fun k => (p.1, p.2.set 3 k.val) + refine ⟨μ_step, fun _ => 1, ?_, ?_, ?_⟩ + · intro p hp + obtain ⟨⟨hbp, hb0, hz2⟩, _⟩ := hsupp p hp + refine ⟨?_, ?_⟩ + · have hr := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 3) (src := 2) + (hbp.1 2) + refine hr.congr_dist ?_ le_rfl + rw [ProbComp.eval_unifFin, PMF.map_comp] + rfl + · intro q hq + rw [PMF.mem_support_map_iff] at hq + obtain ⟨k, _, rfl⟩ := hq + have hk0 : k.val = 0 := by omega + refine ⟨rfl, ⟨?_, ?_, ?_⟩, Modifies.refl _ _⟩ + · refine hbp.set ?_ + rw [hk0] + exact fits_zero w + · simp [Data.set, hb0] + · simp [Data.set, hz2] + · intro θ' + have hmap : (ν.bind μ_step).map Prod.fst = ν.map Prod.fst := by + rw [PMF.map_bind, PMF.map] + refine congrArg (ν.bind ·) (funext fun p => ?_) + exact pmf_map_fst_const (uniformOfFintype (Fin (p.2.regs 2 + 1))) p.1 + (fun k => p.2.set 3 k.val) + rw [hmap] + exact hseed θ' + · simp [ν.tsum_coe] + +/-- Standard-basis lifting of `coinSubroutineDS`. -/ +noncomputable def coinSubroutineDS_standard (w base : ℕ) : + Cslib.WordRAM.Randomized.SubroutineDS + coinSubroutineCode.toStandard coinSpec w base Fin.elim0 PUnit.unit := + (coinSubroutineDS w base).toStandard + +/-- Fresh initialization + multi-operation expected-amortized sequence bound for +`coinSubroutineDS`. -/ +theorem coinSubroutineDS_seq_amortized (w base : ℕ) + (ops : List PUnit) {x₀ : Data 5} + (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : + let C := coinSubroutineDS w base + ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), + ExecCodeDist w coinSubroutineCode.initFresh x₀ (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ p ∈ ν_final.support, C.rep p.1 (coinSpec.run PUnit.unit ops).1 p.2) ∧ + C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ + C.initTime + (C.initPotential : ℝ≥0∞) + + (coinSpec.totalCost C.charge PUnit.unit ops : ℝ≥0∞) := by + intro C + have hvalid : coinSpec.ValidHistory PUnit.unit ops := by + induction ops with + | nil => trivial + | cons _ _ ih => exact ⟨trivial, ih⟩ + have hz : ∀ a ∈ (Block.mk base C.space).footprint, x₀.mem a = 0 := by + change ∀ a ∈ (Block.mk base 0).footprint, x₀.mem a = 0 + simp [Block.footprint] + exact C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid + +/-- Caller framing across a randomized caller step: a caller with 6 registers embeds `Fin 5` via +`Fin.castSucc` and executes `r5 ← rand r2` on its own register `r5` (`5 : Fin 6`), preserving the +coupled seed marginal bound, `rep`, and expected potential via `frame_caller_dist`. -/ +theorem coinSubroutineDS_caller_rand_frame (w base : ℕ) + (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ + (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (hrep : ∀ p ∈ ν.support, + (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base + let μ_caller : C.Seed × Data 6 → PMF (Data 6) := fun p => + (uniformOfFintype (Fin 1)).map fun k => p.2.set 5 k.val + (∀ θ, ((C.frameDist ν μ_caller).map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + (∀ q ∈ (C.frameDist ν μ_caller).support, C.rep q.1 PUnit.unit (q.2.comap Fin.castSucc)) ∧ + (∑' q, C.frameDist ν μ_caller q * (C.potential q.1 (q.2.comap Fin.castSucc) : ℝ≥0∞)) = + ∑' p, ν p * (C.potential p.1 (p.2.comap Fin.castSucc) : ℝ≥0∞) := by + intro C μ_caller + refine C.frame_caller_dist (F := ∅) hseed hrep (Finset.disjoint_empty_right _) ?_ + intro p hp x' hx' + rw [PMF.mem_support_map_iff] at hx' + obtain ⟨k, _, rfl⟩ := hx' + have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) + refine ⟨?_, ?_, Modifies.refl _ _⟩ + · refine ⟨fun i => ?_, hbp.2⟩ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] + exact hbp.1 i + · intro i _ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp [Data.set, hne] + +end ComplexityTests.WordRAMRandomizedSubroutine From 8079947e5ee790950c5aab90f2073930f2a0fff9 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 17:00:40 +0000 Subject: [PATCH 03/18] Add ExecSteps trace to SubroutineDS and expand randomized subroutine tests --- .../WordRAM/Randomized/Expected.lean | 47 +++--- .../WordRAM/Randomized/Subroutine.lean | 159 +++++++++++------- .../WordRAMRandomizedSubroutine.lean | 67 ++++++++ 3 files changed, 188 insertions(+), 85 deletions(-) diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index acfe26e2d..60ca18eef 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -1431,6 +1431,23 @@ theorem mem_support_loopExitPMF {cond : Cond r} {I Q : Data r → Prop} · exact absurd hcy hnc · exact ⟨⟨hnc, hQy⟩, K, hK⟩ +/-- Expectation of `k : β → ℝ≥0∞` under `p.map g` equals expectation of `k ∘ g` under `p`. -/ +theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [ENNReal.tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +/-- Expectation of `k : β → ℝ≥0∞` under `p.bind f` decomposes as an iterated expectation. -/ +theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : + (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by + simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] + exact ENNReal.tsum_comm + /-! ### Compositional distributional execution (`ExecCodeDist`) -/ /-- `ExecCodeDist w c x μ T`: at word size `w`, running structured program `c` from data `x` @@ -1686,16 +1703,6 @@ private theorem pmf_map_apply_not_mem_range {α β : Type*} (p : PMF α) (g : α have hne : b ≠ g a := fun h => hb ⟨a, h.symm⟩ simp [hne] -private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : - (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by - simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] - refine tsum_congr fun a => ?_ - rw [tsum_eq_single (g a)] - · simp - · intro b hb - simp [hb] - private theorem loopIterPMF_rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (cond : Cond r) (μ_body : Data r → PMF (Data r)) (x : Data r') (K : ℕ) (u : Data r) : loopIterPMF (cond.rename f) (fun x' => (μ_body (x'.comap f)).map (fun y => y.extend f x)) @@ -1884,9 +1891,8 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program ∑' z, (μ₁.bind μ₂) z * expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by congr 1 - simp only [PMF.bind_apply, Code.size, Nat.add_assoc] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + simp only [Code.size, Nat.add_assoc] + exact (tsum_pmf_bind μ₁ μ₂ _).symm calc T₁ + ∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩) ≤ T₁ + ((∑' y, μ₁ y * T₂ y) + @@ -1985,9 +1991,7 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program (∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y) = ∑' y, loopExitPMF cond μ_body z y * K_cont y := by rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz] - simp only [PMF.bind_apply] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + exact (tsum_pmf_bind (μ_body z) (loopExitPMF cond μ_body) K_cont).symm have hih_step : (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + @@ -2196,9 +2200,8 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + (c₁.seq c₂).size, y⟩ (.halted v d)) = ∑' y₁, μ₁ y₁ * ∑' y, (μ₂ y₁) y * ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c₁.size + c₂.size, y⟩ (.halted v d) := by - simp only [PMF.bind_apply, Code.size, Nat.add_assoc] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + simp only [Code.size, Nat.add_assoc] + exact tsum_pmf_bind μ₁ μ₂ _ rw [hbind] refine (ENNReal.tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) by_cases hy₁ : y₁ ∈ μ₁.support @@ -2270,10 +2273,8 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} rcases hz_inv with ⟨hcz, hIz⟩ | ⟨hcz, _⟩ · rw [loopIterPMF_succ, loopStepPMF_of_holds hcz] have hbind : (∑' y, ((μ_body z).bind (loopIterPMF cond μ_body K)) y * H_loop y) = - ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := by - simp only [PMF.bind_apply] - simp_rw [← ENNReal.tsum_mul_left, ← ENNReal.tsum_mul_right, mul_assoc] - exact ENNReal.tsum_comm + ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := + tsum_pmf_bind (μ_body z) (loopIterPMF cond μ_body K) H_loop rw [hbind] refine (ENNReal.tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) by_cases hu : u ∈ (μ_body z).support diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean index f89efc199..38527898f 100644 --- a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -53,6 +53,8 @@ density factor `seedSlack : ℝ≥0∞`: * `Cslib.WordRAM.Randomized.SubroutineDS`: reusable randomized Word-RAM subroutine certificate for a deterministic specification `D : Stateful.Spec`. +* `Cslib.WordRAM.Randomized.SubroutineDS.ExecSteps`: multi-operation distributional execution trace + threading coupled state distributions and expected step costs across `ops : List D.Op`. * `Cslib.WordRAM.SubroutineDS.toRandomized`: canonical embedding of any deterministic `Cslib.WordRAM.SubroutineDS` into `Cslib.WordRAM.Randomized.SubroutineDS`. * `Cslib.WordRAM.Randomized.SubroutineDS.mapSimulates`, `toStandard`: basis transport from @@ -386,14 +388,19 @@ theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} rw [hm₁] at hsupp exact ⟨ν₀, (ExecCodeDist.of_execWithin he₁).seq_pure he₂, hseed, hsupp⟩ -/-- Relocated fresh initialization along any injective register embedding `f`. -/ -theorem initFresh_rename (C : SubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} - (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) - (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) - (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : +private theorem init_rename_of_exec (C : SubroutineDS P D w base params s₀) + {c : Code B r_ds} {T : ℝ≥0∞} {f : Fin r_ds → Fin r_caller} (hf : f.Injective) + {x : Data r_caller} + (hex : ∃ ν₀ : PMF (C.Seed × Data r_ds), + ExecCodeDist w c (x.comap f) (ν₀.map Prod.snd) T ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs P.frontierReg = base + C.space ∧ + C.rep p.1 s₀ p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint (x.comap f).mem p.2.mem) : ∃ ν₀ : PMF (C.Seed × Data r_caller), - ExecCodeDist w (P.initFresh.rename f) x (ν₀.map Prod.snd) C.initTime ∧ + ExecCodeDist w (c.rename f) x (ν₀.map Prod.snd) T ∧ (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ ∀ p ∈ ν₀.support, p.2.regs (f P.frontierReg) = base + C.space ∧ @@ -401,7 +408,7 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by - obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initFresh_exec hx hfr hpar hz + obtain ⟨ν₀, he, hseed, hsupp⟩ := hex refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ · have he' := he.rename hf refine he'.congr_dist ?_ le_rfl @@ -423,6 +430,23 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) · rw [Data.comap_extend hf] exact hpot +/-- Relocated fresh initialization along any injective register embedding `f`. -/ +theorem initFresh_rename (C : SubroutineDS P D w base params s₀) + {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} + (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) + (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) + (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : + ∃ ν₀ : PMF (C.Seed × Data r_caller), + ExecCodeDist w (P.initFresh.rename f) x (ν₀.map Prod.snd) C.initTime ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs (f P.frontierReg) = base + C.space ∧ + C.rep p.1 s₀ (p.2.comap f) ∧ + C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := + C.init_rename_of_exec hf (C.initFresh_exec hx hfr hpar hz) + /-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`zeroFillCode_exec`, `initClear_exec`, `initClear_rename`) -/ @@ -521,45 +545,8 @@ theorem initClear_rename (C : SubroutineDS P D w base params s₀) C.rep p.1 s₀ (p.2.comap f) ∧ C.potential p.1 (p.2.comap f) ≤ C.initPotential ∧ Modifies (Block.mk base C.space).footprint x.mem p.2.mem ∧ - (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := by - obtain ⟨ν₀, he, hseed, hsupp⟩ := C.initClear_exec hx hfr hpar - refine ⟨ν₀.map fun p => (p.1, p.2.extend f x), ?_, ?_, ?_⟩ - · have he' := he.rename hf - refine he'.congr_dist ?_ le_rfl - rw [PMF.map_comp, PMF.map_comp] - rfl - · intro θ - rw [PMF.map_comp] - exact hseed θ - · intro q hq - rw [PMF.mem_support_map_iff] at hq - obtain ⟨p, hp, rfl⟩ := hq - obtain ⟨hfr', hrep, hpot, hmod⟩ := hsupp p hp - refine ⟨?_, ?_, ?_, hmod, fun j hj => Data.extend_regs_of_notMem p.2 x hj⟩ - · change ((p.2.extend f x).comap f).regs P.frontierReg = base + C.space - rw [Data.comap_extend hf] - exact hfr' - · rw [Data.comap_extend hf] - exact hrep - · rw [Data.comap_extend hf] - exact hpot - -/-! ### Helper lemmas for coupled PMF expectations -/ - -private theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : - (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by - simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] - refine tsum_congr fun a => ?_ - rw [tsum_eq_single (g a)] - · simp - · intro b hb - simp [hb] - -private theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : - (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by - simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] - exact ENNReal.tsum_comm + (∀ j ∉ Set.range f, p.2.regs j = x.regs j) := + C.init_rename_of_exec hf (C.initClear_exec hx hfr hpar) /-! ### Projection 4: Caller-to-DS bidirectional framing (`frame_caller`, `frame_caller_dist`) -/ @@ -756,7 +743,10 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) ExecCodeDist w (.seq prep (P.code.rename f)) p.2 ((μ_call p).map Prod.snd) (T_call p) ∧ ∀ q ∈ (μ_call p).support, C.decode o (q.2.comap f).regs = some (D.step s o).2 ∧ - C.rep q.1 (D.step s o).1 (q.2.comap f)) ∧ + C.rep q.1 (D.step s o).1 (q.2.comap f) ∧ + ∃ x' ∈ (μ_prep p).support, + Modifies (Block.mk base C.space).footprint x'.mem q.2.mem ∧ + ∀ j ∉ Set.range f, q.2.regs j = x'.regs j) ∧ (∀ θ', ((ν.bind μ_call).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ (∑' p, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞))) ≤ B_prep + (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by @@ -800,8 +790,8 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) have hp' : (p.1, x') ∈ ν_prep.support := by simp only [ν_prep, frameDist, PMF.mem_support_bind_iff] exact ⟨p, hp, (PMF.mem_support_map_iff _ _ _).mpr ⟨x', hx', rfl⟩⟩ - obtain ⟨hdec, hrep', _, _⟩ := (hcode (p.1, x') hp').2 q hq_code - exact ⟨hdec, hrep'⟩ + obtain ⟨hdec, hrep', hmod, hregs⟩ := (hcode (p.1, x') hp').2 q hq_code + exact ⟨hdec, hrep', x', hx', hmod, hregs⟩ · intro θ' have hbind_eq : ν.bind μ_call = ν_prep.bind μ_code := by simp only [ν_prep, frameDist, μ_call, PMF.bind_bind, PMF.bind_map] @@ -926,12 +916,36 @@ theorem prepare_argMatch (C : SubroutineDS P D w base params s₀) (o : D.Op) (x C.argMatch o (C.prepare o x).regs := C.encode_matches o x.regs +/-- Inductive multi-operation distributional execution relation for a randomized subroutine `C`. +`C.ExecSteps s ops ν ν_final T_ops` witnesses that starting from abstract state `s` and coupled +state distribution `ν : PMF (C.Seed × Data r_ds)`, executing each operation `o` in `ops` by +delivering arguments via `C.prepare o` and running `P.code` under `ExecCodeDist` produces the +final coupled distribution `ν_final`, decodes the exact abstract return value `(D.step s o).2` at +every step, modifies memory only within `[base, base + C.space)`, and incurs total expected +operation step cost `T_ops`. -/ +inductive ExecSteps (C : SubroutineDS P D w base params s₀) : + D.State → List D.Op → PMF (C.Seed × Data r_ds) → PMF (C.Seed × Data r_ds) → ℝ≥0∞ → Prop where + | nil (s : D.State) (ν : PMF (C.Seed × Data r_ds)) : + C.ExecSteps s [] ν ν 0 + | cons {s : D.State} {o : D.Op} {ops : List D.Op} + {ν ν_final : PMF (C.Seed × Data r_ds)} + {μ_step : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds)} + {T_step : C.Seed × Data r_ds → ℝ≥0∞} {T_rest : ℝ≥0∞} : + (∀ p ∈ ν.support, + ExecCodeDist w P.code (C.prepare o p.2) ((μ_step p).map Prod.snd) (T_step p) ∧ + ∀ q ∈ (μ_step p).support, + C.decode o q.2.regs = some (D.step s o).2 ∧ + C.rep q.1 (D.step s o).1 q.2 ∧ + Modifies (Block.mk base C.space).footprint p.2.mem q.2.mem) → + C.ExecSteps (D.step s o).1 ops (ν.bind μ_step) ν_final T_rest → + C.ExecSteps s (o :: ops) ν ν_final ((∑' p, ν p * T_step p) + T_rest) + /-- **Multi-operation expected-amortized telescope (`steps_expected_amortized`).** From any abstract state `s : D.State` and coupled state distribution `ν : PMF (C.Seed × Data r_ds)` satisfying the seed-marginal bound and `C.rep p.1 s p.2` on `ν.support`, executing any admissible operation sequence `ops : List D.Op` (`D.ValidHistory s ops`) produces a final coupled distribution -`ν_final` and total expected operation step count `T_ops : ℝ≥0∞` satisfying the addition-only -`ℝ≥0∞` telescope inequality +`ν_final` and total expected operation step count `T_ops : ℝ≥0∞` witnessing +`C.ExecSteps s ops ν ν_final T_ops` and satisfying the addition-only `ℝ≥0∞` telescope inequality `T_ops + E_{ν_final}[Φ] ≤ D.totalCost C.charge s ops + E_ν[Φ]`. -/ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) {s : D.State} (ops : List D.Op) (ν : PMF (C.Seed × Data r_ds)) @@ -939,13 +953,14 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) (hrep : ∀ p ∈ ν.support, C.rep p.1 s p.2) : ∃ (ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), + C.ExecSteps s ops ν ν_final T_ops ∧ (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ p ∈ ν_final.support, C.rep p.1 (D.run s ops).1 p.2) ∧ T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ (D.totalCost C.charge s ops : ℝ≥0∞) + ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by induction ops generalizing s ν with | nil => - exact ⟨ν, 0, hseed, hrep, by simp⟩ + exact ⟨ν, 0, ExecSteps.nil s ν, hseed, hrep, by simp⟩ | cons o os ih => obtain ⟨hvo, hvos⟩ := hv let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.prepare o p.2) @@ -968,20 +983,39 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] obtain ⟨μ_step, T_step, hstep, hseed₁, hamort₁⟩ := C.step_exec o ν_prep hvo hseed_prep hsupp_prep - let ν₁ := ν_prep.bind μ_step + let μ_op : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds) := + fun p => μ_step (p.1, C.prepare o p.2) + let T_op : C.Seed × Data r_ds → ℝ≥0∞ := + fun p => T_step (p.1, C.prepare o p.2) + let ν₁ := ν.bind μ_op + have hν₁_eq : ν₁ = ν_prep.bind μ_step := by + simp only [ν₁, ν_prep, μ_op, PMF.bind_map] + rfl + have hseed₁' : ∀ θ, (ν₁.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by + rw [hν₁_eq] + exact hseed₁ have hrep₁ : ∀ q ∈ ν₁.support, C.rep q.1 (D.step s o).1 q.2 := by intro q hq - rw [PMF.mem_support_bind_iff] at hq + rw [hν₁_eq, PMF.mem_support_bind_iff] at hq obtain ⟨p', hp', hq_step⟩ := hq exact ((hstep p' hp').2 q hq_step).2.1 - obtain ⟨ν_final, T_rest, hseed_final, hrep_final, hamort_rest⟩ := - ih ν₁ hvos hseed₁ hrep₁ - let T₁ : ℝ≥0∞ := ∑' p', ν_prep p' * T_step p' + obtain ⟨ν_final, T_rest, hex_rest, hseed_final, hrep_final, hamort_rest⟩ := + ih ν₁ hvos hseed₁' hrep₁ + let T₁ : ℝ≥0∞ := ∑' p, ν p * T_op p + have hT₁_eq : T₁ = ∑' p', ν_prep p' * T_step p' := by + symm + exact tsum_pmf_map ν (fun p => (p.1, C.prepare o p.2)) T_step + have hex_cons : C.ExecSteps s (o :: os) ν ν_final (T₁ + T_rest) := by + refine ExecSteps.cons (μ_step := μ_op) (T_step := T_op) (fun p hp => ?_) hex_rest + have hp' : (p.1, C.prepare o p.2) ∈ ν_prep.support := + (PMF.mem_support_map_iff _ _ _).mpr ⟨p, hp, rfl⟩ + exact hstep (p.1, C.prepare o p.2) hp' have hstep_split : T₁ + (∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) = ∑' p', ν_prep p' * (T_step p' + ∑' q, μ_step p' q * (C.potential q.1 q.2 : ℝ≥0∞)) := by - simp only [T₁, ν₁, tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] - refine ⟨ν_final, T₁ + T_rest, hseed_final, hrep_final, ?_⟩ + rw [hT₁_eq, hν₁_eq] + simp only [tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] + refine ⟨ν_final, T₁ + T_rest, hex_cons, hseed_final, hrep_final, ?_⟩ calc (T₁ + T_rest) + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) = T₁ + (T_rest + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl @@ -1012,13 +1046,14 @@ theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) (hv : D.ValidHistory s₀ ops) : ∃ (ν₀ ν_final : PMF (C.Seed × Data r_ds)) (T_ops : ℝ≥0∞), ExecCodeDist w P.initFresh x (ν₀.map Prod.snd) C.initTime ∧ + C.ExecSteps s₀ ops ν₀ ν_final T_ops ∧ (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ p ∈ ν_final.support, C.rep p.1 (D.run s₀ ops).1 p.2) ∧ C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ C.initTime + (C.initPotential : ℝ≥0∞) + (D.totalCost C.charge s₀ ops : ℝ≥0∞) := by obtain ⟨ν₀, he_init, hseed₀, hsupp₀⟩ := C.initFresh_exec hx hfr hpar hz have hrep₀ : ∀ p ∈ ν₀.support, C.rep p.1 s₀ p.2 := fun p hp => (hsupp₀ p hp).2.1 - obtain ⟨ν_final, T_ops, hseed_final, hrep_final, hamort⟩ := + obtain ⟨ν_final, T_ops, hex_steps, hseed_final, hrep_final, hamort⟩ := C.steps_expected_amortized ops ν₀ hv hseed₀ hrep₀ have hpot₀ : (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) ≤ (C.initPotential : ℝ≥0∞) := by calc @@ -1030,7 +1065,7 @@ theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] _ = (C.initPotential : ℝ≥0∞) := by rw [ENNReal.tsum_mul_right, ν₀.tsum_coe, one_mul] - refine ⟨ν₀, ν_final, T_ops, he_init, hseed_final, hrep_final, ?_⟩ + refine ⟨ν₀, ν_final, T_ops, he_init, hex_steps, hseed_final, hrep_final, ?_⟩ calc C.initTime + T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) = C.initTime + (T_ops + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) := by ac_rfl diff --git a/ComplexityTests/WordRAMRandomizedSubroutine.lean b/ComplexityTests/WordRAMRandomizedSubroutine.lean index d11f4056d..1a335f4b5 100644 --- a/ComplexityTests/WordRAMRandomizedSubroutine.lean +++ b/ComplexityTests/WordRAMRandomizedSubroutine.lean @@ -301,6 +301,7 @@ theorem coinSubroutineDS_seq_amortized (w base : ℕ) let C := coinSubroutineDS w base ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), ExecCodeDist w coinSubroutineCode.initFresh x₀ (ν₀.map Prod.snd) C.initTime ∧ + C.ExecSteps PUnit.unit ops ν₀ ν_final T_ops ∧ (∀ θ, (ν_final.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ p ∈ ν_final.support, C.rep p.1 (coinSpec.run PUnit.unit ops).1 p.2) ∧ C.initTime + T_ops + (∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞)) ≤ @@ -316,6 +317,72 @@ theorem coinSubroutineDS_seq_amortized (w base : ℕ) simp [Block.footprint] exact C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid +/-- Dirty-memory initialization via `initClear_exec` for `coinSubroutineDS`. -/ +theorem coinSubroutineDS_initClear (w base : ℕ) {x₀ : Data 5} + (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : + let C := coinSubroutineDS w base + ∃ ν₀ : PMF (C.Seed × Data 5), + ExecCodeDist w coinSubroutineCode.initClear x₀ (ν₀.map Prod.snd) + (C.initTime + (4 * C.space + 6 : ℕ)) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs coinSubroutineCode.frontierReg = base + C.space ∧ + C.rep p.1 PUnit.unit p.2 ∧ + C.potential p.1 p.2 ≤ C.initPotential ∧ + Modifies (Block.mk base C.space).footprint x₀.mem p.2.mem := by + intro C + exact C.initClear_exec hx₀ hfr (fun j => Fin.elim0 j) + +/-- Inline caller subroutine invocation via `call_rename` for `coinSubroutineDS`: a caller with +6 registers embeds `Fin 5` via `Fin.castSucc`, executes `prep := .op (.const 5 0)` on its own +register `r5`, and invokes `coinSubroutineCode.code.rename Fin.castSucc`. -/ +theorem coinSubroutineDS_call_rename (w base : ℕ) + (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ + (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (hrep : ∀ p ∈ ν.support, + (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base + let prep : Code .exact 6 := .op (.const 5 0) + ∃ (μ_call : C.Seed × Data 6 → PMF (C.Seed × Data 6)) + (T_call : C.Seed × Data 6 → ℝ≥0∞), + (∀ p ∈ ν.support, + ExecCodeDist w (.seq prep (coinSubroutineCode.code.rename Fin.castSucc)) + p.2 ((μ_call p).map Prod.snd) (T_call p) ∧ + ∀ q ∈ (μ_call p).support, + C.decode PUnit.unit (q.2.comap Fin.castSucc).regs = + some (coinSpec.step PUnit.unit PUnit.unit).2 ∧ + C.rep q.1 (coinSpec.step PUnit.unit PUnit.unit).1 (q.2.comap Fin.castSucc) ∧ + ∃ x' ∈ (PMF.pure (p.2.set 5 0) : PMF (Data 6)).support, + Modifies (Block.mk base C.space).footprint x'.mem q.2.mem ∧ + ∀ j ∉ Set.range (Fin.castSucc : Fin 5 → Fin 6), q.2.regs j = x'.regs j) ∧ + (∀ θ', ((ν.bind μ_call).map Prod.fst) θ' ≤ C.seedSlack * C.seedLaw θ') ∧ + (∑' p, ν p * (T_call p + + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap Fin.castSucc) : ℝ≥0∞))) ≤ + 1 + (C.charge PUnit.unit PUnit.unit : ℝ≥0∞) + + ∑' p, ν p * (C.potential p.1 (p.2.comap Fin.castSucc) : ℝ≥0∞) := by + intro C prep + refine C.call_rename (Fin.castSucc_injective 5) PUnit.unit ν + (prep := prep) (B_prep := 1) + (μ_prep := fun p => PMF.pure (p.2.set 5 0)) (T_prep := fun _ => 1) + trivial hseed hrep ?_ + intro p hp + refine ⟨?_, le_rfl, ?_⟩ + · simpa [prep] using ExecCodeDist.of_exec (B := .exact) (w := w) (x := p.2) + (Exec.const (d := (5 : Fin 6)) (fits_zero w)) + · intro x' hx' + rw [PMF.mem_support_pure_iff] at hx' + subst hx' + have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) + refine ⟨?_, ?_, fun _ _ => rfl, trivial⟩ + · refine ⟨fun i => ?_, hbp.2⟩ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] + exact hbp.1 i + · intro i _ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp [Data.set, hne] + /-- Caller framing across a randomized caller step: a caller with 6 registers embeds `Fin 5` via `Fin.castSucc` and executes `r5 ← rand r2` on its own register `r5` (`5 : Fin 6`), preserving the coupled seed marginal bound, `rep`, and expected potential via `frame_caller_dist`. -/ From cefa21faff0d923eb0fac5dd8e2588777173b9f1 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 17:25:29 +0000 Subject: [PATCH 04/18] Update author header and MitzenmacherUpfal2017 section locators --- .../Computability/WordRAM/Randomized/Expected.lean | 8 ++++---- .../WordRAM/Randomized/Subroutine.lean | 14 ++++++++------ 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index 60ca18eef..ebfa3b1b7 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -72,14 +72,14 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, geometric Las Vegas `while` loops and structured programs. Discrete and conditional expectation principles are classical background; see -[MitzenmacherUpfal2017], Sections 2.1 and 2.3. The particular native tail-sum/Bellman definitions, -finite structured-execution-tree bridge and ranked-loop expected-cost formalization are original -library constructions in this module. +[MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. The particular native tail-sum/Bellman +definitions, finite structured-execution-tree bridge and ranked-loop expected-cost formalization are +original library constructions in this module. ## References * [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017], - Sections 2.1 and 2.3. + Sections 1.5, 2.1, and 2.3. -/ @[expose] public section diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean index 38527898f..acfd953f7 100644 --- a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -1,7 +1,7 @@ /- -Copyright (c) 2026 Chaeyoung Lim. All rights reserved. +Copyright (c) 2026 Renfei Zhou. All rights reserved. Released under Apache 2.0 license as described in the file LICENSE. -Authors: Chaeyoung Lim +Authors: Renfei Zhou -/ module @@ -17,7 +17,8 @@ This file extends the deterministic `Cslib.WordRAM.SubroutineDS` interface to ** structures and Las Vegas subroutines** on the Word-RAM — covering both structures initialized from a random seed (such as universal hash tables, 2-choice hash tables, and cuckoo hash tables) and structures whose operations make random choices via `Instr.rand` (such as treaps, skip lists, -reservoir samplers, or randomized rehashing) `[MitzenmacherUpfal2017]`. +reservoir samplers, or randomized rehashing); see [MitzenmacherUpfal2017], Sections 1.5, 2.1, and +2.3. ## Core design @@ -43,7 +44,7 @@ density factor `seedSlack : ℝ≥0∞`: - Every admissible operation `o : D.Op` from abstract state `s : D.State` and coupled state distribution `ν : PMF (Seed × Data r_ds)` produces a transition kernel `μ_step : Seed × Data r_ds → PMF (Seed × Data r_ds)` and expected step count - `T_step : Seed × Data r_ds → ℝ≥0∞` satisfying + `T_step : Seed × Data r_ds → ℝ≥0∞` satisfying (for `Φ(p) := potential p.1 p.2`) `∑' p, ν p * (T_step p + ∑' q, μ_step p q * Φ(q)) ≤ charge s o + ∑' p, ν p * Φ(p)`. - Across any admissible finite operation sequence `ops : List D.Op`, `seq_expected_amortized` telescopes these one-step bounds without `ℝ≥0∞` subtraction: @@ -81,7 +82,7 @@ density factor `seedSlack : ℝ≥0∞`: * [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, Chapter 17][CLRS2009] * [M. Mitzenmacher and E. Upfal, *Probability and Computing: Randomization and Probabilistic - Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017] + Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. -/ @[expose] public section @@ -946,7 +947,8 @@ satisfying the seed-marginal bound and `C.rep p.1 s p.2` on `ν.support`, execut operation sequence `ops : List D.Op` (`D.ValidHistory s ops`) produces a final coupled distribution `ν_final` and total expected operation step count `T_ops : ℝ≥0∞` witnessing `C.ExecSteps s ops ν ν_final T_ops` and satisfying the addition-only `ℝ≥0∞` telescope inequality -`T_ops + E_{ν_final}[Φ] ≤ D.totalCost C.charge s ops + E_ν[Φ]`. -/ +(with `Φ(p) := C.potential p.1 p.2`) +`T_ops + ∑' q, ν_final q * Φ(q) ≤ D.totalCost C.charge s ops + ∑' p, ν p * Φ(p)`. -/ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) {s : D.State} (ops : List D.Op) (ν : PMF (C.Seed × Data r_ds)) (hv : D.ValidHistory s ops) From a7461e87d2534a4b73d2f0b05e9e06c0f41c541c Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 17:53:41 +0000 Subject: [PATCH 05/18] refactor(stateful,word-ram): share Execution/DynamicProgram/SubroutineBase and drop aliases --- .../Computability/WordRAM/RelSubroutine.lean | 521 +++--------------- .../Computability/WordRAM/Subroutine.lean | 349 ++++++------ .../Semantics/Stateful/Realization.lean | 16 +- .../Semantics/Stateful/Relational.lean | 138 ++--- ComplexityTests/StatefulRelational.lean | 8 +- ComplexityTests/WordRAMRelSubroutine.lean | 10 +- 6 files changed, 311 insertions(+), 731 deletions(-) diff --git a/Complexity/Computability/WordRAM/RelSubroutine.lean b/Complexity/Computability/WordRAM/RelSubroutine.lean index 9b92e4dee..7e3dece99 100644 --- a/Complexity/Computability/WordRAM/RelSubroutine.lean +++ b/Complexity/Computability/WordRAM/RelSubroutine.lean @@ -12,39 +12,40 @@ public import Complexity.Computability.WordRAM.Subroutine public import Complexity.Foundations.Semantics.Stateful.Relational /-! -# Relational Word-RAM dynamic programs and composable data-structure subroutines +# Relational Word-RAM dynamic certificates and composable data-structure subroutines This file extends `Complexity.Computability.WordRAM.Dynamic` and `Complexity.Computability.WordRAM.Subroutine` from deterministic specifications (`Stateful.Spec`) to relational specifications (`Stateful.RelSpec`): -1. **Standalone relational fixed-program implementations (`RelDynamicProgram`, - `RelDynamicCertificate`):** Generalizes `DynamicProgram` and `DynamicCertificate` to - `D : Stateful.RelSpec`, where `execute` witnesses an angelic outcome `(s', a)` satisfying - `D.step s o s' a`. Because the Word-RAM structured semantics `Exec` is deterministic, - `RelDynamicCertificate.realization` establishes `Stateful.RelRealization` directly via - `Exec.unique`. +1. **Standalone relational fixed-program certificates (`RelDynamicCertificate`):** Generalizes + `DynamicCertificate` over `DynamicProgram D r` for `D : Stateful.RelSpec`, where `execute` + witnesses an angelic outcome `(s', a)` satisfying `D.step s o s' a`. Because the Word-RAM + structured semantics `Exec` is deterministic, `RelDynamicCertificate.realization` establishes + `Stateful.RelRealization` directly via `Exec.unique`. 2. **Unified standalone and inline relational subroutine certificates (`RelSubroutineDS`):** - Mirrors `SubroutineDS` over `D : Stateful.RelSpec`, omitting `rep_unique` (since concrete - potential `potential : Data r_ds → ℕ` depends only on machine data) and existentially - quantifying over `(s', a)` with `D.step s o s' a` in `step_exec`. + Extends `SubroutineBase P D.Result D.valid w base params s₀` over `D : Stateful.RelSpec`, + inheriting `toDynamicProgram`, `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, + `initClear_exec`, `initClear_rename`, `frame_caller`, and `prepareRegs` without duplication, + omitting `rep_unique` (since concrete potential `potential : Data r_ds → ℕ` depends only on + machine data), and existentially quantifying over `(s', a)` with `D.step s o s' a` in + `step_exec`. 3. **Zero-cost embedding of deterministic subroutines (`SubroutineDS.toRelSubroutineDS`):** Every `SubroutineDS P D w base params s₀` embeds into `RelSubroutineDS P D.toRelSpec w base params s₀`. 4. **Basis lifting, standalone `Stateful` projections, inline subroutine calls, and flat-program - operation sequences:** Provides `mapSimulates`, `toStandard`, `toRelDynamicProgram`, - `toRelCertificate`, `toRelAmortized`, `toRelBound`, `run_client_amortized`, `run_client_bound`, - `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, - `initClear_rename`, `code_rename`, `call_rename`, `frame_caller`, and flat-program sequence - execution (`RunsRelOps`, `seq_runsRelOps`, `run_ops_exact_fresh`, `run_ops_exact_reusable`). + operation sequences:** Provides `mapSimulates`, `toStandard`, `toRelCertificate`, + `toRelAmortized`, `toRelBound`, `run_client_amortized`, `run_client_bound`, `code_rename`, + `call_rename`, and flat-program sequence execution (`RunsRelOps`, `seq_runsRelOps`, + `run_ops_exact_fresh`, `run_ops_exact_reusable`). ## Main definitions -* `Cslib.WordRAM.RelDynamicProgram`, `Cslib.WordRAM.RelDynamicCertificate`: standalone fixed-program - realization and certificate for a relational specification `D : Stateful.RelSpec`. +* `Cslib.WordRAM.RelDynamicCertificate`: standalone fixed-program certificate for + `DynamicProgram D r` realizing a relational specification `D : Stateful.RelSpec`. * `Cslib.WordRAM.RelSubroutineDS`: unified standalone and inline subroutine specification and amortized resource certificate for `P : SubroutineCode B r_ds k_init` realizing - `D : Stateful.RelSpec` (without `rep_unique`). + `D : Stateful.RelSpec` (extending `SubroutineBase` without `rep_unique`). * `Cslib.WordRAM.SubroutineDS.toRelSubroutineDS`: embedding of a deterministic `SubroutineDS` into `RelSubroutineDS`. * `Cslib.WordRAM.RelSubroutineDS.RunsRelOps`: sequential flat-program execution of an operation list @@ -58,11 +59,9 @@ to relational specifications (`Stateful.RelSpec`): * `Cslib.WordRAM.RelSubroutineDS.toRelAmortized`, `Cslib.WordRAM.RelSubroutineDS.run_client_amortized`: standalone amortized potential certificate and adaptive-client bound over `Basis.standard`. -* `Cslib.WordRAM.RelSubroutineDS.initFresh_rename`, - `Cslib.WordRAM.RelSubroutineDS.initClear_rename`, - `Cslib.WordRAM.RelSubroutineDS.call_rename`, - `Cslib.WordRAM.RelSubroutineDS.frame_caller`: relocated inline initialization, inline operation - calls, and bidirectional framing. +* `Cslib.WordRAM.RelSubroutineDS.code_rename`, + `Cslib.WordRAM.RelSubroutineDS.call_rename`: relocated inline operation calls with angelic + relational step outcomes. * `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_fresh`, `Cslib.WordRAM.RelSubroutineDS.run_ops_exact_reusable`: end-to-end flat-program execution of any demonically admissible operation sequence `D.ValidOpSeq s₀ ops`. @@ -80,136 +79,13 @@ namespace Cslib.WordRAM universe u v z -/-! ## Standalone relational dynamic programs (`RelDynamicProgram`, `RelDynamicCertificate`) -/ +/-! ## Standalone relational dynamic certificates (`RelDynamicCertificate`) -/ variable {r : ℕ} {D : Stateful.RelSpec.{u, v, z}} -/-- A single `.standard`-basis Word-RAM program serving all operations of a relational -specification `D : Stateful.RelSpec` through a fixed register calling convention. -/ -structure RelDynamicProgram (D : Stateful.RelSpec.{u, v, z}) (r : ℕ) where - /-- The same structured program is executed for every operation. -/ - code : Code .standard r - /-- Register returned by the final `halt` instruction. -/ - output : Fin r - /-- Registers overwritten by externally supplied operation arguments. -/ - input : Fin r → Bool - /-- Encoded arguments, independent of the maintained machine state. -/ - encode : D.Op → Fin r → ℕ - /-- Interpretation of the final register tuple as the operation's typed observation. -/ - decode : (o : D.Op) → (Fin r → ℕ) → Option (D.Result o) - -/-- Embed a deterministic `DynamicProgram D r` as a `RelDynamicProgram D.toRelSpec r`. -/ -abbrev DynamicProgram.toRelDynamicProgram {D : Stateful.Spec.{u, v, z}} {r : ℕ} - (P : DynamicProgram D r) : RelDynamicProgram D.toRelSpec r where - code := P.code - output := P.output - input := P.input - encode := P.encode - decode := P.decode - -namespace RelDynamicProgram - -variable (P : RelDynamicProgram D r) - -/-- Deliver operation arguments without changing maintained memory or non-input registers. -/ -def prepare (o : D.Op) (x : Data r) : Data r where - regs i := if P.input i then P.encode o i else x.regs i - mem := x.mem - -/-- Delivering operation arguments preserves the machine memory unchanged. -/ -@[simp] -theorem prepare_mem (o : D.Op) (x : Data r) : (P.prepare o x).mem = x.mem := rfl - -/-- Registers outside the calling convention retain their contents. -/ -theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : D.Op) (x : Data r) : - (P.prepare o x).regs i = x.regs i := by - simp [prepare, hi] - -/-- Argument delivery preserves bounded words when the arguments fit. -/ -theorem prepare_bounded {w : ℕ} {o : D.Op} {x : Data r} (hx : x.Bounded w) - (ho : ∀ i, P.input i = true → Fits w (P.encode o i)) : (P.prepare o x).Bounded w := by - refine ⟨fun i => ?_, hx.2⟩ - change Fits w (if P.input i then P.encode o i else x.regs i) - split - · exact ho i (by assumption) - · exact hx.1 i - -/-- Evaluate one typed call, delivering its arguments and decoding its observation. The returned -cost includes the final `halt`. -/ -def eval (w fuel : ℕ) (o : D.Op) (x : Data r) : Option (Data r × D.Result o × ℕ) := - match P.code.eval w fuel (P.prepare o x) with - | none => none - | some (y, n) => (P.decode o y.regs).map fun a => (y, a, n + 1) - -/-- A bounded machine call with native execution evidence (`DynamicProgram.OperationTrace`). -/ -def ExecOp (w : ℕ) : Stateful.RelExecution D (Data r) DynamicProgram.OperationTrace := - fun x o a y trace => - (P.prepare o x).Bounded w ∧ ∃ n, TraceExec w P.code (P.prepare o x) y n trace.probes ∧ - P.decode o y.regs = some a ∧ trace.instructions = n + 1 - -/-- Successful typed evaluation supplies an exact operation trace when the delivered state is -bounded. -/ -theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result o} - (hb : (P.prepare o x).Bounded w) (h : P.eval w fuel o x = some (y, a, k)) : - ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := by - cases he : P.code.eval w fuel (P.prepare o x) with - | none => simp [eval, he] at h - | some yn => - obtain ⟨z, n⟩ := yn - cases hd : P.decode o z.regs with - | none => simp [eval, he, hd] at h - | some b => - simp only [eval, he, hd, Option.map_some, Option.some.injEq, Prod.mk.injEq] at h - obtain ⟨rfl, rfl, rfl⟩ := h - obtain ⟨probes, hp⟩ := (Exec.of_eval he).exists_trace - exact ⟨probes, hb, n, hp, hd, rfl⟩ - -/-- Every call is an execution of one fixed flat machine program. -/ -theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} - {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : - Executes w (P.code.program P.output) ⟨0, P.prepare o x⟩ (y.regs P.output) - trace.instructions := by - obtain ⟨_, n, he, _, ht⟩ := h - rw [ht] - exact he.exec.executes P.output - -/-- A call preserves bounded words. -/ -theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} - {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) : y.Bounded w := by - obtain ⟨hb, _, he, _, _⟩ := h - exact he.exec.bounded hb - -/-- Calls are deterministic in their final representation, observation, and instruction count. -/ -theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} - {trace trace' : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) - (h' : P.ExecOp w x o a' y' trace') : - y = y' ∧ a = a' ∧ trace.instructions = trace'.instructions := by - obtain ⟨_, _, he, ha, ht⟩ := h - obtain ⟨_, _, he', ha', ht'⟩ := h' - obtain ⟨rfl, rfl⟩ := he.exec.unique he'.exec - exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ - -/-- A call's final state and instruction count agree with any structured execution of the code. -/ -theorem ExecOp.eq_of_exec {P : RelDynamicProgram D r} {w : ℕ} {x y z : Data r} {o : D.Op} - {a : D.Result o} {trace : DynamicProgram.OperationTrace} {m : ℕ} - (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : - y = z ∧ trace.instructions = m + 1 := by - obtain ⟨_, _, ht, _, hn⟩ := he - obtain ⟨rfl, rfl⟩ := ht.exec.unique h - exact ⟨rfl, hn⟩ - -/-- The write transcript supplies a frame rule for a call, including register-only delivery. -/ -theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} - {trace : DynamicProgram.OperationTrace} (h : P.ExecOp w x o a y trace) {Q : HeapAssertion} - (hQ : Q.holds x.mem) (hd : Disjoint Q.footprint (Probe.writes trace.probes)) : - Q.holds y.mem := by - obtain ⟨_, _, he, _, _⟩ := h - exact he.frame hQ hd - -end RelDynamicProgram - -/-- Per-operation angelic machine proof obligations for a `RelDynamicProgram`. -/ -structure RelDynamicCertificate (P : RelDynamicProgram D r) (w : ℕ) where +/-- Per-operation angelic machine proof obligations for a `DynamicProgram D r` over a relational +specification `D : Stateful.RelSpec`. -/ +structure RelDynamicCertificate (P : DynamicProgram D r) (w : ℕ) where /-- Abstract state represented by the persistent machine data. -/ rep : D.State → Data r → Prop /-- Represented configurations contain only machine words. -/ @@ -225,7 +101,8 @@ structure RelDynamicCertificate (P : RelDynamicProgram D r) (w : ℕ) where /-- Embed a deterministic `DynamicCertificate` as a `RelDynamicCertificate`. -/ def DynamicCertificate.toRelDynamicCertificate {D : Stateful.Spec.{u, v, z}} {r w : ℕ} {P : DynamicProgram D r} (C : DynamicCertificate P w) : - RelDynamicCertificate P.toRelDynamicProgram w where + RelDynamicCertificate (D := D.toRelSpec) + ⟨P.code, P.output, P.input, P.encode, P.decode⟩ w where rep := C.rep bounded := C.bounded arguments_fit := C.arguments_fit @@ -235,14 +112,14 @@ def DynamicCertificate.toRelDynamicCertificate {D : Stateful.Spec.{u, v, z}} {r namespace RelDynamicCertificate -variable {P : RelDynamicProgram D r} {w : ℕ} (C : RelDynamicCertificate P w) +variable {P : DynamicProgram D r} {w : ℕ} (C : RelDynamicCertificate P w) /-- Turn a successful typed evaluation on a represented state and admissible operation into certified `ExecOp` evidence. -/ theorem execOp_of_eval {s : D.State} {x y : Data r} {o : D.Op} {a : D.Result o} {fuel k : ℕ} (hr : C.rep s x) (hv : D.valid s o) (h : P.eval w fuel o x = some (y, a, k)) : ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := - RelDynamicProgram.ExecOp.of_eval P + DynamicProgram.ExecOp.of_eval P (P.prepare_bounded (C.bounded hr) (C.arguments_fit o hv)) h /-- Fixed-program relational certificates realize `D` via `Stateful.RelRealization`. -/ @@ -291,7 +168,7 @@ theorem run_client_const {K : ℕ} {α : Type*} (p : D.Client α) {s : D.State} {x : Data r} (hr : C.rep s x) (hv : D.DemonicValid s p) : ∃ a y traces s' tr, - Stateful.RelExecutes (P.ExecOp w) p x a y traces ∧ + Stateful.Executes (P.ExecOp w) p x a y traces ∧ D.ClientRun s p a s' tr ∧ C.rep s' y ∧ traces.length = tr.length ∧ @@ -331,7 +208,7 @@ theorem executes_client_frame {j : ℕ} {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} {traces : List DynamicProgram.OperationTrace} (hr : C.rep s x) (hv : D.DemonicValid s p) - (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + (he : Stateful.Executes (P.ExecOp w) p x val y traces) : y.mem j = x.mem j := by induction he generalizing s with | pure _ _ => rfl @@ -348,7 +225,7 @@ theorem executes_client_modifies {F : Finset ℕ} {α : Type*} {p : D.Client α} {s : D.State} {x y : Data r} {val : α} {traces : List DynamicProgram.OperationTrace} (hr : C.rep s x) (hv : D.DemonicValid s p) - (he : Stateful.RelExecutes (P.ExecOp w) p x val y traces) : + (he : Stateful.Executes (P.ExecOp w) p x val y traces) : Modifies F x.mem y.mem := fun j hj => C.executes_client_frame (fun hr' hv' he' => hmod hr' hv' he' j hj) hr hv he @@ -388,70 +265,14 @@ end SubroutineCode `P : SubroutineCode B r_ds k_init` realizing a relational specification `D : Stateful.RelSpec` at word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. -In contrast to `SubroutineDS`, `rep_unique` is omitted (because concrete potential -`potential : Data r_ds → ℕ` depends only on machine data) and `step_exec` existentially -quantifies over a permitted outcome `(s', a)` satisfying `D.step s o s' a`. -/ +Extends `SubroutineBase P D.Result D.valid w base params s₀` (inheriting `toDynamicProgram`, +`initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, `initClear_rename`, +`frame_caller`, and `prepareRegs`), omits `rep_unique` (because concrete potential +`potential : Data r_ds → ℕ` depends only on machine data), and existentially quantifies over a +permitted outcome `(s', a)` satisfying `D.step s o s' a` in `step_exec`. -/ structure RelSubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) - (D : Stateful.RelSpec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) where - /-- Contiguous memory footprint length `[base, base + space)`. -/ - space : ℕ - /-- Instruction bound for `P.allocSpan`. -/ - spanTime : ℕ - /-- Instruction bound for `P.initSetup`. -/ - setupTime : ℕ - /-- Upper bound on initial concrete potential after initialization. -/ - initPotential : ℕ - /-- Amortized instruction charge per operation (excluding trailing standalone `halt`). -/ - charge : D.State → D.Op → ℕ - /-- Representation invariant linking mathematical state `s` to concrete `Data r_ds`. -/ - rep : D.State → Data r_ds → Prop - /-- Concrete integer potential depending only on `Data r_ds`. -/ - potential : Data r_ds → ℕ - /-- Standalone register encoding for `RelDynamicProgram.prepare`. -/ - encode : D.Op → Fin r_ds → ℕ - /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ - argMatch : D.Op → (Fin r_ds → ℕ) → Prop - /-- Output decoder reading typed operation observations from final registers. -/ - decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) - /-- Represented configurations are word-bounded. -/ - bounded : ∀ {s x}, rep s x → x.Bounded w - /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ - arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) - /-- Full standalone `prepare` delivery satisfies `argMatch`. -/ - encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), - argMatch o (fun i => if P.input i then encode o i else regs i) - /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. - -/ - rep_supported : ∀ {s x x'}, - (∀ i, P.input i = false → x'.regs i = x.regs i) → - (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → - x'.Bounded w → rep s x → rep s x' - /-- `potential` depends only on persistent registers and `[base, base + space)` under `rep`. -/ - potential_supported : ∀ {s x x'}, - (∀ i, P.input i = false → x'.regs i = x.regs i) → - (∀ a ∈ (Block.mk base space).footprint, x'.mem a = x.mem a) → - rep s x → potential x' = potential x - /-- Distinctness of registers used by `zeroFillCode` relative to `baseReg` and `frontierReg`. -/ - clear_nodup : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup - /-- Initialization parameter registers are not overwritten by `zeroFillCode`. -/ - params_not_clear : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg] - /-- `P.incCur` increments `P.curReg` by `1` when `P.oneReg = 1` and the sum fits in `w` bits. -/ - incCur_eval : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → - P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1)) - /-- Execution of `P.allocSpan`: records `baseReg = base` and advances `frontierReg`. -/ - span_exec : ∀ {x : Data r_ds}, - x.Bounded w → x.regs P.frontierReg = base → (∀ j, x.regs (P.initParams j) = params j) → - ∃ x₁, ExecWithin w P.allocSpan x x₁ spanTime ∧ x₁.mem = x.mem ∧ - x₁.regs P.baseReg = base ∧ x₁.regs P.frontierReg = base + space ∧ - (∀ j, x₁.regs (P.initParams j) = params j) - /-- Execution of `P.initSetup` once `[base, base + space)` holds zeros. -/ - setup_exec : ∀ {x₁ : Data r_ds}, - x₁.Bounded w → x₁.regs P.baseReg = base → x₁.regs P.frontierReg = base + space → - (∀ j, x₁.regs (P.initParams j) = params j) → - (∀ a ∈ (Block.mk base space).footprint, x₁.mem a = 0) → - ∃ y, ExecWithin w P.initSetup x₁ y setupTime ∧ y.regs P.frontierReg = base + space ∧ - rep s₀ y ∧ potential y ≤ initPotential ∧ - Modifies (Block.mk base space).footprint x₁.mem y.mem + (D : Stateful.RelSpec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) + extends SubroutineBase P D.Result D.valid w base params s₀ where /-- Angelic execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`, witnessing some permitted outcome `(s', a)` with `D.step s o s' a`. -/ step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), @@ -470,26 +291,7 @@ def SubroutineDS.toRelSubroutineDS {B : Basis} {r_ds k_init w base : ℕ} {params : Fin k_init → ℕ} {s₀ : D.State} (C : SubroutineDS P D w base params s₀) : RelSubroutineDS P D.toRelSpec w base params s₀ where - space := C.space - spanTime := C.spanTime - setupTime := C.setupTime - initPotential := C.initPotential - charge := C.charge - rep := C.rep - potential := C.potential - encode := C.encode - argMatch := C.argMatch - decode := C.decode - bounded := C.bounded - arguments_fit := C.arguments_fit - encode_matches := C.encode_matches - rep_supported := C.rep_supported - potential_supported := C.potential_supported - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval := C.incCur_eval - span_exec := C.span_exec - setup_exec := C.setup_exec + toSubroutineBase := C.toSubroutineBase step_exec o hr hv harg := by obtain ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg exact ⟨y, (D.step _ o).1, (D.step _ o).2, n, he, rfl, hdec, hrep, hmod, hpot⟩ @@ -500,39 +302,12 @@ variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} {P : SubroutineCode B r_ds k_init} {D : Stateful.RelSpec.{u, v, z}} {params : Fin k_init → ℕ} {s₀ : D.State} -/-- Total instruction bound for `P.initFresh`. -/ -def initTime (C : RelSubroutineDS P D w base params s₀) : ℕ := - C.spanTime + C.setupTime - -/-! ### Projection 1: Basis lifting and standalone `RelDynamicProgram` / `Stateful` theorems -/ +/-! ### Projection 1: Basis lifting and standalone `DynamicProgram` / `Stateful` theorems -/ /-- Lift a `RelSubroutineDS` certificate along any defined-evaluation basis simulation. -/ def mapSimulates (φ : B.Simulates B') (C : RelSubroutineDS P D w base params s₀) : RelSubroutineDS (P.mapSimulates φ) D w base params s₀ where - space := C.space - spanTime := C.spanTime - setupTime := C.setupTime - initPotential := C.initPotential - charge := C.charge - rep := C.rep - potential := C.potential - encode := C.encode - argMatch := C.argMatch - decode := C.decode - bounded := C.bounded - arguments_fit := C.arguments_fit - encode_matches := C.encode_matches - rep_supported := C.rep_supported - potential_supported := C.potential_supported - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval h1 hfit := Op.eval_mapSimulates_of_some φ (C.incCur_eval h1 hfit) - span_exec hx hf hp := - let ⟨x₁, he, hm, hb, hfr, hpar⟩ := C.span_exec hx hf hp - ⟨x₁, he.mapSimulates φ, hm, hb, hfr, hpar⟩ - setup_exec hx₁ hb hfr hpar hz := - let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz - ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ + toSubroutineBase := C.toSubroutineBase.mapSimulates φ step_exec o hr hv harg := let ⟨y, s', a, n, he, hstep, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg ⟨y, s', a, n, he.mapSimulates φ, hstep, hdec, hrep, hmod, hpot⟩ @@ -548,50 +323,42 @@ section Standard variable {P_std : SubroutineCode .standard r_ds k_init} (C : RelSubroutineDS P_std D w base params s₀) -/-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicProgram`. -/ -def toRelDynamicProgram : RelDynamicProgram D r_ds where - code := P_std.code - output := P_std.output - input := P_std.input - encode := C.encode - decode := C.decode - /-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `RelDynamicCertificate`. -/ -def toRelCertificate : RelDynamicCertificate C.toRelDynamicProgram w where +def toRelCertificate : RelDynamicCertificate (D := D) C.toDynamicProgram w where rep := C.rep bounded := C.bounded arguments_fit := C.arguments_fit execute {s x} o hr hv := by - have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := - C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := C.rep_supported - (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hb' hr - have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := C.encode_matches o x.regs obtain ⟨y, s', a, n, he, hstep, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg exact ⟨y, s', a, n, he, hstep, hdec, hrep⟩ /-- Project a `.standard`-basis `RelSubroutineDS` to a standalone `Stateful.RelAmortized` -certificate, charging `+ 1` per operation for `RelDynamicProgram`'s trailing `halt` instruction. -/ +certificate, charging `+ 1` per operation for `DynamicProgram`'s trailing `halt` instruction. -/ theorem toRelAmortized : Stateful.RelAmortized C.toRelCertificate.realization DynamicProgram.OperationTrace.instructions (fun _ x => C.potential x) (fun s o => C.charge s o + 1) where step {s x o a y trace} hr hv he := by - have hb' : (C.toRelDynamicProgram.prepare o x).Bounded w := - C.toRelDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toRelDynamicProgram.prepare o x) := + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := C.rep_supported - (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hb' hr - have hp_eq : C.potential (C.toRelDynamicProgram.prepare o x) = C.potential x := + have hp_eq : C.potential (C.toDynamicProgram.prepare o x) = C.potential x := C.potential_supported - (fun i hi => C.toRelDynamicProgram.prepare_regs_of_not_input hi o x) + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hr - have harg : C.argMatch o (C.toRelDynamicProgram.prepare o x).regs := + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := C.encode_matches o x.regs obtain ⟨z, s', a', m, hz, hstep, hdec, hrep, _, hpot⟩ := C.step_exec o hr' hv harg obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz @@ -603,9 +370,6 @@ theorem toRelAmortized : rw [hp_eq] at hpot omega -/-- Alias for `toRelAmortized` matching `SubroutineDS.toAmortized`. -/ -abbrev toAmortized := @toRelAmortized - /-- When `C.potential` is identically zero, `RelSubroutineDS` projects directly to a worst-case `Stateful.RelBound` certificate. -/ theorem toRelBound (hpot : ∀ x, C.potential x = 0) : @@ -617,14 +381,11 @@ theorem toRelBound (hpot : ∀ x, C.potential x = 0) : simp only [hpot] at h omega -/-- Alias for `toRelBound` matching `SubroutineDS.toBound`. -/ -abbrev toBound := @toRelBound - /-- Standalone adaptive-client amortized bound for any demonically admissible client `p`. -/ theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} (hr : C.rep s x) (hv : D.DemonicValid s p) : ∃ a y traces s' tr, - Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x a y traces ∧ D.ClientRun s p a s' tr ∧ C.rep s' y ∧ traces.length = tr.length ∧ @@ -636,7 +397,7 @@ theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : D theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} (hr : C.rep s x) (hv : D.DemonicValid s p) : ∃ a y traces s' tr, - Stateful.RelExecutes (C.toRelDynamicProgram.ExecOp w) p x a y traces ∧ + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x a y traces ∧ D.ClientRun s p a s' tr ∧ C.rep s' y ∧ traces.length = tr.length ∧ @@ -646,140 +407,7 @@ theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.C end Standard -/-! ### Projection 2: Fresh `O(1)` initialization and register relocation -/ - -/-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ -theorem initFresh_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} - (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) - (hpar : ∀ j, x.regs (P.initParams j) = params j) - (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : - ∃ y, ExecWithin w P.initFresh x y C.initTime ∧ - y.regs P.frontierReg = base + C.space ∧ - C.rep s₀ y ∧ - C.potential y ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem y.mem := by - obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar - have hx₁ : x₁.Bounded w := he₁.bounded hx - have hz₁ : ∀ a ∈ (Block.mk base C.space).footprint, x₁.mem a = 0 := by - intro a ha - rw [hm₁] - exact hz a ha - obtain ⟨y, he₂, hfr₂, hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb₁ hfr₁ hpar₁ hz₁ - rw [hm₁] at hmod - exact ⟨y, he₁.seq he₂, hfr₂, hrep, hpot, hmod⟩ - -/-- Relocated `O(1)` fresh initialization along any injective register embedding `f`. -/ -theorem initFresh_rename (C : RelSubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} - (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) - (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) - (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : - ∃ z, ExecWithin w (P.initFresh.rename f) x z C.initTime ∧ - z.regs (f P.frontierReg) = base + C.space ∧ - C.rep s₀ (z.comap f) ∧ - C.potential (z.comap f) ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ - (∀ j ∉ Set.range f, z.regs j = x.regs j) := by - obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initFresh_exec hx hfr hpar hz - refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, - fun j hj => Data.extend_regs_of_notMem y x hj⟩ - · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space - rw [Data.comap_extend hf] - exact hfr' - · rw [Data.comap_extend hf] - exact hrep - · rw [Data.comap_extend hf] - exact hpot - -/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`initClear`) -/ - -/-- Auxiliary deterministic `SubroutineDS` sharing `C`'s initialization fields so -`zeroFillCode_exec` is reused without duplicating its loop proof. -/ -private def toInitSubroutineDS (C : RelSubroutineDS P D w base params s₀) : - SubroutineDS P - { State := PUnit.{1}, Op := PUnit.{1}, Result := fun _ => PUnit.{1}, - step := fun _ _ => (⟨⟩, ⟨⟩), valid := fun _ _ => False } - w base params ⟨⟩ where - space := C.space - spanTime := C.spanTime - setupTime := C.setupTime - initPotential := C.initPotential - charge _ _ := 0 - rep _ x := C.rep s₀ x - potential := C.potential - encode _ _ := 0 - argMatch _ _ := True - decode _ _ := some ⟨⟩ - bounded := C.bounded - rep_unique _ _ := rfl - arguments_fit _ hv := False.elim hv - encode_matches _ _ := trivial - rep_supported := C.rep_supported - potential_supported := C.potential_supported - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval := C.incCur_eval - span_exec := C.span_exec - setup_exec := C.setup_exec - step_exec _ _ hv := False.elim hv - -/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` -instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : RelSubroutineDS P D w base params s₀) {x₁ : Data r_ds} - (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) - (hfr₁ : x₁.regs P.frontierReg = base + C.space) - (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : - ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ - x₂.Bounded w ∧ - x₂.regs P.baseReg = base ∧ - x₂.regs P.frontierReg = base + C.space ∧ - (∀ j, x₂.regs (P.initParams j) = params j) ∧ - (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ - Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := - C.toInitSubroutineDS.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ - -/-- Execute `P.initClear` on arbitrary dirty memory in `C.initTime + 4 * C.space + 6` -instructions. -/ -theorem initClear_exec (C : RelSubroutineDS P D w base params s₀) {x : Data r_ds} - (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) - (hpar : ∀ j, x.regs (P.initParams j) = params j) : - ∃ y, ExecWithin w P.initClear x y (C.initTime + 4 * C.space + 6) ∧ - y.regs P.frontierReg = base + C.space ∧ - C.rep s₀ y ∧ - C.potential y ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem y.mem := by - obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar - obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := - C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ - obtain ⟨y, he₃, hfr₃, hrep, hpot, hmod₃⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ - rw [hm₁] at hmod₂ - have hmod : Modifies (Block.mk base C.space).footprint x.mem y.mem := by - simpa only [Finset.union_idempotent] using hmod₂.trans hmod₃ - exact ⟨y, (he₁.seq (he₂.seq he₃)).mono (by unfold initTime; omega), hfr₃, hrep, hpot, hmod⟩ - -/-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ -theorem initClear_rename (C : RelSubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} - (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) - (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : - ∃ z, ExecWithin w (P.initClear.rename f) x z (C.initTime + 4 * C.space + 6) ∧ - z.regs (f P.frontierReg) = base + C.space ∧ - C.rep s₀ (z.comap f) ∧ - C.potential (z.comap f) ≤ C.initPotential ∧ - Modifies (Block.mk base C.space).footprint x.mem z.mem ∧ - (∀ j ∉ Set.range f, z.regs j = x.regs j) := by - obtain ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.initClear_exec hx hfr hpar - refine ⟨y.extend f x, he.rename hf, ?_, ?_, ?_, hmod, - fun j hj => Data.extend_regs_of_notMem y x hj⟩ - · change ((y.extend f x).comap f).regs P.frontierReg = base + C.space - rw [Data.comap_extend hf] - exact hfr' - · rw [Data.comap_extend hf] - exact hrep - · rw [Data.comap_extend hf] - exact hpot - -/-! ### Projection 4: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ +/-! ### Projection 2: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ /-- Execute `P.code.rename f` inline when arguments already match `(x.comap f).regs`, witnessing a permitted outcome `(s', a)` with `D.step s o s' a`. -/ @@ -831,30 +459,7 @@ theorem call_rename (C : RelSubroutineDS P D w base params s₀) exact ⟨z, s', a, t_prep + t_op, hprep.seq he_op, hstep, hdec, hrep_z, hmod_z, hregs_z, by rw [hp_eq] at hpot_z; omega⟩ -/-! ### Projection 5: Caller-to-DS bidirectional framing (`frame_caller`) -/ - -/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS -registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from -`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ -theorem frame_caller (C : RelSubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} {s : D.State} {x x' : Data r_caller} {F : Finset ℕ} - (hr : C.rep s (x.comap f)) - (hb' : (x'.comap f).Bounded w) - (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) - (hdisj : Disjoint (Block.mk base C.space).footprint F) - (hmem : Modifies F x.mem x'.mem) : - C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by - have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => - hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) - exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, - C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ - -/-! ### Projection 6: Flat-program operation-sequence execution (`RunsRelOps`, `run_ops_exact_*`) -/ - -/-- Default register-preparation function overwriting `P.input` registers with `C.encode o`. -/ -def prepareRegs (C : RelSubroutineDS P D w base params s₀) (o : D.Op) - (regs : Fin r_ds → ℕ) : Fin r_ds → ℕ := - fun i => if P.input i then C.encode o i else regs i +/-! ### Projection 3: Flat-program operation-sequence execution (`RunsRelOps`, `run_ops_exact_*`) -/ /-- Sequential flat-program execution of `ops : List D.Op` by repeated calls to `P.opProg`: each call prepares input registers via `prepRegs`, runs `P.opProg` from `pc = 0` to `halt`, @@ -937,12 +542,12 @@ theorem seq_runsRelOps (C : RelSubroutineDS P D w base params s₀) D.totalCost (fun st o => C.charge st o + 1) s tr + C.potential x := by refine C.seq_runsRelOps_of_prep C.prepareRegs ?_ ?_ C.encode_matches ops hr hv · intro s' o regs hv_o hregs i - simp only [prepareRegs] + simp only [SubroutineBase.prepareRegs] split_ifs with hi · exact C.arguments_fit o hv_o i hi · exact hregs i · intro o regs i hi - simp [prepareRegs, hi] + simp [SubroutineBase.prepareRegs, hi] /-- Standalone flat-program operation-sequence theorem with `O(1)` fresh initialization on zeroed `[base, base + C.space)`. -/ diff --git a/Complexity/Computability/WordRAM/Subroutine.lean b/Complexity/Computability/WordRAM/Subroutine.lean index 2dae6c8e7..bb0969dcc 100644 --- a/Complexity/Computability/WordRAM/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Subroutine.lean @@ -40,11 +40,12 @@ A data structure is packaged once as a `SubroutineCode B r_ds k_init` and certif ## Main definitions -* `Cslib.WordRAM.SubroutineCode`, `Cslib.WordRAM.SubroutineDS`: uniform relocatable machine code and - unified standalone / inline specification and resource certificate for a Word-RAM data structure. +* `Cslib.WordRAM.SubroutineCode`, `Cslib.WordRAM.SubroutineBase`, `Cslib.WordRAM.SubroutineDS`: + uniform relocatable machine code, step-independent initialization/framing bundle, and unified + standalone / inline specification and resource certificate for a Word-RAM data structure. * `Cslib.WordRAM.SubroutineCode.initFresh`, `Cslib.WordRAM.SubroutineCode.initClear`: `O(1)` fresh initialization on zeroed memory and generic `O(space)` dirty-memory re-initialization. -* `Cslib.WordRAM.SubroutineDS.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: +* `Cslib.WordRAM.SubroutineBase.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: standalone `.standard`-basis `DynamicProgram` and `DynamicCertificate` projections. ## Main results @@ -53,8 +54,8 @@ A data structure is packaged once as a `SubroutineCode B r_ds k_init` and certif * `Cslib.WordRAM.SubroutineDS.toAmortized`, `Cslib.WordRAM.SubroutineDS.toBound`, `Cslib.WordRAM.SubroutineDS.run_client_amortized`, `Cslib.WordRAM.SubroutineDS.run_client_bound`: standalone amortized and worst-case instruction bounds over `Basis.standard`. -* `Cslib.WordRAM.SubroutineDS.initFresh_rename`, `Cslib.WordRAM.SubroutineDS.initClear_rename`, - `Cslib.WordRAM.SubroutineDS.call_rename`, `Cslib.WordRAM.SubroutineDS.frame_caller`: relocated +* `Cslib.WordRAM.SubroutineBase.initFresh_rename`, `Cslib.WordRAM.SubroutineBase.initClear_rename`, + `Cslib.WordRAM.SubroutineDS.call_rename`, `Cslib.WordRAM.SubroutineBase.frame_caller`: relocated inline initialization, inline operation calls, and bidirectional register/memory framing. ## References @@ -209,10 +210,18 @@ end SubroutineCode universe u v z -/-- Unified standalone and subroutine specification/resource certificate for `P` realizing `D` at -word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. -/ -structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) - (D : Stateful.Spec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) where +/-- Step-independent specification, representation, calling-convention, and initialization bundle +for `P : SubroutineCode B r_ds k_init` over a state type `State`, operation type `Op`, observation +family `Result`, and admissibility predicate `valid` at word width `w`, base address `base`, +runtime parameters `params`, and initial state `s₀`. + +Extended by `SubroutineDS` (for deterministic `Stateful.Spec`) and `RelSubroutineDS` (for +relational `Stateful.RelSpec`) so that initialization (`initFresh`, `zeroFillCode`, `initClear`), +register relocation, standalone `toDynamicProgram`, and bidirectional framing (`frame_caller`) are +stated and proved once. -/ +structure SubroutineBase {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) + {State : Type u} {O : Type v} (Result : O → Type z) (valid : State → O → Prop) + (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : State) where /-- Contiguous memory footprint length `[base, base + space)`. -/ space : ℕ /-- Instruction bound for `P.allocSpan`. -/ @@ -222,25 +231,23 @@ structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_d /-- Upper bound on initial concrete potential after initialization. -/ initPotential : ℕ /-- Amortized instruction charge per operation (excluding trailing standalone `halt`). -/ - charge : D.State → D.Op → ℕ + charge : State → O → ℕ /-- Representation invariant linking mathematical state `s` to concrete `Data r_ds`. -/ - rep : D.State → Data r_ds → Prop + rep : State → Data r_ds → Prop /-- Concrete integer potential depending only on `Data r_ds`. -/ potential : Data r_ds → ℕ /-- Standalone register encoding for `DynamicProgram.prepare`. -/ - encode : D.Op → Fin r_ds → ℕ + encode : O → Fin r_ds → ℕ /-- Partial argument-matching predicate for inline callers (refined by `encode`). -/ - argMatch : D.Op → (Fin r_ds → ℕ) → Prop + argMatch : O → (Fin r_ds → ℕ) → Prop /-- Output decoder reading typed operation observations from final registers. -/ - decode : (o : D.Op) → (Fin r_ds → ℕ) → Option (D.Result o) + decode : (o : O) → (Fin r_ds → ℕ) → Option (Result o) /-- Represented configurations are word-bounded. -/ bounded : ∀ {s x}, rep s x → x.Bounded w - /-- Concrete machine data determines at most one represented abstract state. -/ - rep_unique : ∀ {s₁ s₂ x}, rep s₁ x → rep s₂ x → s₁ = s₂ /-- Standalone encoded arguments fit in `w`-bit words on admissible calls. -/ - arguments_fit : ∀ {s} (o), D.valid s o → ∀ i, P.input i = true → Fits w (encode o i) + arguments_fit : ∀ {s} (o), valid s o → ∀ i, P.input i = true → Fits w (encode o i) /-- Full standalone `prepare` delivery satisfies `argMatch`. -/ - encode_matches : ∀ (o : D.Op) (regs : Fin r_ds → ℕ), + encode_matches : ∀ (o : O) (regs : Fin r_ds → ℕ), argMatch o (fun i => if P.input i then encode o i else regs i) /-- `rep` depends only on persistent registers (`P.input i = false`) and `[base, base + space)`. -/ @@ -274,30 +281,25 @@ structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_d ∃ y, ExecWithin w P.initSetup x₁ y setupTime ∧ y.regs P.frontierReg = base + space ∧ rep s₀ y ∧ potential y ≤ initPotential ∧ Modifies (Block.mk base space).footprint x₁.mem y.mem - /-- Execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`. -/ - step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), - rep s x → D.valid s o → argMatch o x.regs → - ∃ y n, Exec w P.code x y n ∧ - decode o y.regs = some (D.step s o).2 ∧ - rep (D.step s o).1 y ∧ - Modifies (Block.mk base space).footprint x.mem y.mem ∧ - n + potential y ≤ charge s o + potential x -namespace SubroutineDS +namespace SubroutineBase variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} - {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} - {params : Fin k_init → ℕ} {s₀ : D.State} + {P : SubroutineCode B r_ds k_init} {State : Type u} {O : Type v} + {Result : O → Type z} {valid : State → O → Prop} {params : Fin k_init → ℕ} {s₀ : State} /-- Total instruction bound for `P.initFresh`. -/ -def initTime (C : SubroutineDS P D w base params s₀) : ℕ := +def initTime (C : SubroutineBase P Result valid w base params s₀) : ℕ := C.spanTime + C.setupTime -/-! ### Projection 1: Basis lifting and standalone `DynamicProgram` / `Stateful` theorems -/ +/-- Default register-preparation function overwriting `P.input` registers with `C.encode o`. -/ +def prepareRegs (C : SubroutineBase P Result valid w base params s₀) (o : O) + (regs : Fin r_ds → ℕ) : Fin r_ds → ℕ := + fun i => if P.input i then C.encode o i else regs i -/-- Lift a `SubroutineDS` certificate along any defined-evaluation basis simulation. -/ -def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) : - SubroutineDS (P.mapSimulates φ) D w base params s₀ where +/-- Lift a `SubroutineBase` bundle along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : SubroutineBase P Result valid w base params s₀) : + SubroutineBase (P.mapSimulates φ) Result valid w base params s₀ where space := C.space spanTime := C.spanTime setupTime := C.setupTime @@ -309,7 +311,6 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) argMatch := C.argMatch decode := C.decode bounded := C.bounded - rep_unique := C.rep_unique arguments_fit := C.arguments_fit encode_matches := C.encode_matches rep_supported := C.rep_supported @@ -323,110 +324,19 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) setup_exec hx₁ hb hfr hpar hz := let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ - step_exec o hr hv harg := - let ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg - ⟨y, n, he.mapSimulates φ, hdec, hrep, hmod, hpot⟩ - -/-- Lift an exact-basis `SubroutineDS` certificate to `Basis.standard`. -/ -abbrev toStandard {P : SubroutineCode .exact r_ds k_init} - (C : SubroutineDS P D w base params s₀) : - SubroutineDS P.toStandard D w base params s₀ := - C.mapSimulates .exactStandard - -section Standard - -variable {P_std : SubroutineCode .standard r_ds k_init} - (C : SubroutineDS P_std D w base params s₀) -/-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicProgram`. -/ -def toDynamicProgram : DynamicProgram D r_ds where +/-- Project a `.standard`-basis `SubroutineBase` to a standalone `DynamicProgram`. -/ +def toDynamicProgram {P_std : SubroutineCode .standard r_ds k_init} + (C : SubroutineBase P_std Result valid w base params s₀) : + DynamicProgram { State := Empty, Op := O, Result := Result, step := Empty.elim } r_ds where code := P_std.code output := P_std.output input := P_std.input encode := C.encode decode := C.decode -/-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicCertificate`. -/ -def toCertificate : DynamicCertificate C.toDynamicProgram w where - rep := C.rep - bounded := C.bounded - arguments_fit := C.arguments_fit - execute {s x} o hr hv := by - have hb' : (C.toDynamicProgram.prepare o x).Bounded w := - C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toDynamicProgram.prepare o x) := - C.rep_supported - (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) - (fun _ _ => rfl) hb' hr - have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := - C.encode_matches o x.regs - obtain ⟨y, n, he, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg - exact ⟨y, n, he, hdec, hrep⟩ - -/-- Project a `.standard`-basis `SubroutineDS` to a standalone `Stateful.Amortized` certificate, -charging `+ 1` per operation for `DynamicProgram`'s trailing `halt` instruction. -/ -theorem toAmortized : - Stateful.Amortized C.toCertificate.realization - DynamicProgram.OperationTrace.instructions - (fun _ x => C.potential x) - (fun s o => C.charge s o + 1) where - step {s x o a y trace} hr hv he := by - have hb' : (C.toDynamicProgram.prepare o x).Bounded w := - C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) - have hr' : C.rep s (C.toDynamicProgram.prepare o x) := - C.rep_supported - (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) - (fun _ _ => rfl) hb' hr - have hp_eq : C.potential (C.toDynamicProgram.prepare o x) = C.potential x := - C.potential_supported - (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) - (fun _ _ => rfl) hr - have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := - C.encode_matches o x.regs - obtain ⟨z, m, hz, _, _, _, hpot⟩ := C.step_exec o hr' hv harg - obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz - rw [hinst] - rw [hp_eq] at hpot - omega - -/-- When `C.potential` is identically zero, `SubroutineDS` projects directly to a worst-case -`Stateful.Bound` certificate. -/ -theorem toBound (hpot : ∀ x, C.potential x = 0) : - Stateful.Bound C.toCertificate.realization - DynamicProgram.OperationTrace.instructions - (fun s o => C.charge s o + 1) := by - intro s x o a y trace hr hv he - have h := C.toAmortized.step hr hv he - simp only [hpot] at h - omega - -/-- Standalone adaptive-client amortized bound for any admissible client program `p`. -/ -theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} - (hr : C.rep s x) (hv : D.ValidHistory s (D.runClient s p).2.2) : - ∃ y traces, - Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ - C.rep (D.runClient s p).2.1 y ∧ - Stateful.total DynamicProgram.OperationTrace.instructions traces + C.potential y ≤ - D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 + C.potential x := - C.toAmortized.exists_executes p hr hv - -/-- Standalone adaptive-client worst-case bound when `C.potential` is identically zero. -/ -theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) - {s : D.State} {x : Data r_ds} (hr : C.rep s x) - (hv : D.ValidHistory s (D.runClient s p).2.2) : - ∃ y traces, - Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ - C.rep (D.runClient s p).2.1 y ∧ - Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ - D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 := - Stateful.Bound.exists_executes (C.toBound hpot) p hr hv - -end Standard - -/-! ### Projection 2: Fresh `O(1)` initialization and register relocation -/ - /-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ -theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} +theorem initFresh_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) (hpar : ∀ j, x.regs (P.initParams j) = params j) (hz : ∀ a ∈ (Block.mk base C.space).footprint, x.mem a = 0) : @@ -446,7 +356,7 @@ theorem initFresh_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} exact ⟨y, he₁.seq he₂, hfr₂, hrep, hpot, hmod⟩ /-- Relocated `O(1)` fresh initialization along any injective register embedding `f`. -/ -theorem initFresh_rename (C : SubroutineDS P D w base params s₀) +theorem initFresh_rename (C : SubroutineBase P Result valid w base params s₀) {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) @@ -468,11 +378,9 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) · rw [Data.comap_extend hf] exact hpot -/-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`initClear`) -/ - /-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data r_ds} +theorem zeroFillCode_exec (C : SubroutineBase P Result valid w base params s₀) {x₁ : Data r_ds} (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) (hfr₁ : x₁.regs P.frontierReg = base + C.space) (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : @@ -618,7 +526,7 @@ theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data /-- Execute `P.initClear` on **arbitrary dirty memory** in `C.initTime + 4 * C.space + 6` instructions. -/ -theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} +theorem initClear_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} (hx : x.Bounded w) (hfr : x.regs P.frontierReg = base) (hpar : ∀ j, x.regs (P.initParams j) = params j) : ∃ y, ExecWithin w P.initClear x y (C.initTime + 4 * C.space + 6) ∧ @@ -636,7 +544,7 @@ theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} exact ⟨y, (he₁.seq (he₂.seq he₃)).mono (by unfold initTime; omega), hfr₃, hrep, hpot, hmod⟩ /-- Relocated `O(space)` dirty-memory initialization along any injective register embedding `f`. -/ -theorem initClear_rename (C : SubroutineDS P D w base params s₀) +theorem initClear_rename (C : SubroutineBase P Result valid w base params s₀) {f : Fin r_ds → Fin r_caller} (hf : f.Injective) {x : Data r_caller} (hx : (x.comap f).Bounded w) (hfr : x.regs (f P.frontierReg) = base) (hpar : ∀ j, x.regs (f (P.initParams j)) = params j) : @@ -657,7 +565,154 @@ theorem initClear_rename (C : SubroutineDS P D w base params s₀) · rw [Data.comap_extend hf] exact hpot -/-! ### Projection 4: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ +/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS +registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from +`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ +theorem frame_caller (C : SubroutineBase P Result valid w base params s₀) + {f : Fin r_ds → Fin r_caller} {s : State} {x x' : Data r_caller} {F : Finset ℕ} + (hr : C.rep s (x.comap f)) + (hb' : (x'.comap f).Bounded w) + (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) + (hdisj : Disjoint (Block.mk base C.space).footprint F) + (hmem : Modifies F x.mem x'.mem) : + C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by + have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => + hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) + exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, + C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ + +end SubroutineBase + +/-- Unified standalone and subroutine specification/resource certificate for `P` realizing `D` at +word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. -/ +structure SubroutineDS {B : Basis} {r_ds k_init : ℕ} (P : SubroutineCode B r_ds k_init) + (D : Stateful.Spec.{u, v, z}) (w base : ℕ) (params : Fin k_init → ℕ) (s₀ : D.State) + extends SubroutineBase P D.Result D.valid w base params s₀ where + /-- Concrete machine data determines at most one represented abstract state. -/ + rep_unique : ∀ {s₁ s₂ x}, rep s₁ x → rep s₂ x → s₁ = s₂ + /-- Execution of `P.code` for any admissible operation `o` whose arguments match `x.regs`. -/ + step_exec : ∀ {s : D.State} {x : Data r_ds} (o : D.Op), + rep s x → D.valid s o → argMatch o x.regs → + ∃ y n, Exec w P.code x y n ∧ + decode o y.regs = some (D.step s o).2 ∧ + rep (D.step s o).1 y ∧ + Modifies (Block.mk base space).footprint x.mem y.mem ∧ + n + potential y ≤ charge s o + potential x + +namespace SubroutineDS + +variable {B B' : Basis} {r_ds r_caller k_init w base : ℕ} + {P : SubroutineCode B r_ds k_init} {D : Stateful.Spec.{u, v, z}} + {params : Fin k_init → ℕ} {s₀ : D.State} + +/-! ### Projection 1: Basis lifting and standalone `DynamicProgram` / `Stateful` theorems -/ + +/-- Lift a `SubroutineDS` certificate along any defined-evaluation basis simulation. -/ +def mapSimulates (φ : B.Simulates B') (C : SubroutineDS P D w base params s₀) : + SubroutineDS (P.mapSimulates φ) D w base params s₀ where + toSubroutineBase := C.toSubroutineBase.mapSimulates φ + rep_unique := C.rep_unique + step_exec o hr hv harg := + let ⟨y, n, he, hdec, hrep, hmod, hpot⟩ := C.step_exec o hr hv harg + ⟨y, n, he.mapSimulates φ, hdec, hrep, hmod, hpot⟩ + +/-- Lift an exact-basis `SubroutineDS` certificate to `Basis.standard`. -/ +abbrev toStandard {P : SubroutineCode .exact r_ds k_init} + (C : SubroutineDS P D w base params s₀) : + SubroutineDS P.toStandard D w base params s₀ := + C.mapSimulates .exactStandard + +section Standard + +variable {P_std : SubroutineCode .standard r_ds k_init} + (C : SubroutineDS P_std D w base params s₀) + +/-- Project a `.standard`-basis `SubroutineDS` to its underlying standalone `DynamicProgram`. -/ +def toDynamicProgram : DynamicProgram D r_ds where + code := P_std.code + output := P_std.output + input := P_std.input + encode := C.encode + decode := C.decode + +/-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicCertificate`. -/ +def toCertificate : DynamicCertificate C.toDynamicProgram w where + rep := C.rep + bounded := C.bounded + arguments_fit := C.arguments_fit + execute {s x} o hr hv := by + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨y, n, he, hdec, hrep, _, _⟩ := C.step_exec o hr' hv harg + exact ⟨y, n, he, hdec, hrep⟩ + +/-- Project a `.standard`-basis `SubroutineDS` to a standalone `Stateful.Amortized` certificate, +charging `+ 1` per operation for `DynamicProgram`'s trailing `halt` instruction. -/ +theorem toAmortized : + Stateful.Amortized C.toCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun _ x => C.potential x) + (fun s o => C.charge s o + 1) where + step {s x o a y trace} hr hv he := by + have hb' : (C.toDynamicProgram.prepare o x).Bounded w := + C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit o hv) + have hr' : C.rep s (C.toDynamicProgram.prepare o x) := + C.rep_supported + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hb' hr + have hp_eq : C.potential (C.toDynamicProgram.prepare o x) = C.potential x := + C.potential_supported + (fun i hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) + (fun _ _ => rfl) hr + have harg : C.argMatch o (C.toDynamicProgram.prepare o x).regs := + C.encode_matches o x.regs + obtain ⟨z, m, hz, _, _, _, hpot⟩ := C.step_exec o hr' hv harg + obtain ⟨rfl, hinst⟩ := he.eq_of_exec hz + rw [hinst] + rw [hp_eq] at hpot + omega + +/-- When `C.potential` is identically zero, `SubroutineDS` projects directly to a worst-case +`Stateful.Bound` certificate. -/ +theorem toBound (hpot : ∀ x, C.potential x = 0) : + Stateful.Bound C.toCertificate.realization + DynamicProgram.OperationTrace.instructions + (fun s o => C.charge s o + 1) := by + intro s x o a y trace hr hv he + have h := C.toAmortized.step hr hv he + simp only [hpot] at h + omega + +/-- Standalone adaptive-client amortized bound for any admissible client program `p`. -/ +theorem run_client_amortized {α : Type*} (p : D.Client α) {s : D.State} {x : Data r_ds} + (hr : C.rep s x) (hv : D.ValidHistory s (D.runClient s p).2.2) : + ∃ y traces, + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ + C.rep (D.runClient s p).2.1 y ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces + C.potential y ≤ + D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 + C.potential x := + C.toAmortized.exists_executes p hr hv + +/-- Standalone adaptive-client worst-case bound when `C.potential` is identically zero. -/ +theorem run_client_bound (hpot : ∀ x, C.potential x = 0) {α : Type*} (p : D.Client α) + {s : D.State} {x : Data r_ds} (hr : C.rep s x) + (hv : D.ValidHistory s (D.runClient s p).2.2) : + ∃ y traces, + Stateful.Executes (C.toDynamicProgram.ExecOp w) p x (D.runClient s p).1 y traces ∧ + C.rep (D.runClient s p).2.1 y ∧ + Stateful.total DynamicProgram.OperationTrace.instructions traces ≤ + D.totalCost (fun s o => C.charge s o + 1) s (D.runClient s p).2.2 := + Stateful.Bound.exists_executes (C.toBound hpot) p hr hv + +end Standard + +/-! ### Projection 2: Relocated inline subroutine calls (`code_rename`, `call_rename`) -/ /-- Execute `P.code.rename f` inline when arguments already match `(x.comap f).regs`. -/ theorem code_rename (C : SubroutineDS P D w base params s₀) @@ -706,24 +761,6 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) exact ⟨z, t_prep + t_op, hprep.seq he_op, hdec, hrep_z, hmod_z, hregs_z, by rw [hp_eq] at hpot_z; omega⟩ -/-! ### Projection 5: Caller-to-DS bidirectional framing (`frame_caller`) -/ - -/-- Any caller state transition that preserves `(x'.comap f).Bounded w`, leaves persistent DS -registers (`P.input i = false`) unchanged, and writes only to a set `F` disjoint from -`[base, base + C.space)` preserves both `C.rep s (x'.comap f)` and `C.potential (x'.comap f)`. -/ -theorem frame_caller (C : SubroutineDS P D w base params s₀) - {f : Fin r_ds → Fin r_caller} {s : D.State} {x x' : Data r_caller} {F : Finset ℕ} - (hr : C.rep s (x.comap f)) - (hb' : (x'.comap f).Bounded w) - (hreg : ∀ i, P.input i = false → x'.regs (f i) = x.regs (f i)) - (hdisj : Disjoint (Block.mk base C.space).footprint F) - (hmem : Modifies F x.mem x'.mem) : - C.rep s (x'.comap f) ∧ C.potential (x'.comap f) = C.potential (x.comap f) := by - have hag : ∀ a ∈ (Block.mk base C.space).footprint, x'.mem a = x.mem a := fun a ha => - hmem a (fun hf => Finset.disjoint_left.mp hdisj ha hf) - exact ⟨C.rep_supported (x := x.comap f) (x' := x'.comap f) hreg hag hb' hr, - C.potential_supported (x := x.comap f) (x' := x'.comap f) hreg hag hr⟩ - end SubroutineDS end Cslib.WordRAM diff --git a/Complexity/Foundations/Semantics/Stateful/Realization.lean b/Complexity/Foundations/Semantics/Stateful/Realization.lean index 43afd738d..5761f7fb1 100644 --- a/Complexity/Foundations/Semantics/Stateful/Realization.lean +++ b/Complexity/Foundations/Semantics/Stateful/Realization.lean @@ -39,14 +39,16 @@ namespace Cslib.Stateful universe u v w x y -variable {D E : Spec.{u, v, w}} {C τ : Type x} {α β : Type y} +variable {S : PFunctor.{v, w}} {D E : Spec.{u, v, w}} {C τ : Type x} {α β : Type y} + +instance : Coe Spec.{u, v, w} PFunctor.{v, w} := ⟨Spec.signature⟩ /-- A backend chooses its own execution evidence, independently of a resource measure. -/ -abbrev Execution (D : Spec.{u, v, w}) (C Trace : Type x) := - C → (o : D.Op) → D.Result o → C → Trace → Prop +abbrev Execution (S : PFunctor.{v, w}) (C Trace : Type x) := + C → (o : S.A) → S.B o → C → Trace → Prop /-- Clients collect native execution evidence in call order; no costs occur in this semantics. -/ -inductive Executes (exec : Execution D C τ) : D.Client α → C → α → C → List τ → Prop +inductive Executes (exec : Execution S C τ) : S.FreeM α → C → α → C → List τ → Prop | pure (a c) : Executes exec (.pure a) c a c [] | call {o k c a c' trace b c'' traces} : exec c o a c' trace → Executes exec (k a) c' b c'' traces → @@ -97,12 +99,12 @@ end Realization /-- Execution of a sequential composition splits its native trace at the call boundary. -/ -theorem executes_bind_iff (exec : Execution D C τ) (p : D.Client α) - (k : α → D.Client β) {c c' : C} {b : β} {ts : List τ} : +theorem executes_bind_iff (exec : Execution S C τ) (p : S.FreeM α) + (k : α → S.FreeM β) {c c' : C} {b : β} {ts : List τ} : Executes exec (p.bind k) c b c' ts ↔ ∃ a mid front back, Executes exec p c a mid front ∧ Executes exec (k a) mid b c' back ∧ ts = front ++ back := by - induction p generalizing c ts with + induction p using PFunctor.FreeM.recOn generalizing c ts with | pure a => constructor · intro h; exact ⟨a, c, [], ts, .pure a c, h, rfl⟩ diff --git a/Complexity/Foundations/Semantics/Stateful/Relational.lean b/Complexity/Foundations/Semantics/Stateful/Relational.lean index 40d7761e7..722f77f02 100644 --- a/Complexity/Foundations/Semantics/Stateful/Relational.lean +++ b/Complexity/Foundations/Semantics/Stateful/Relational.lean @@ -52,9 +52,8 @@ deterministic counterparts on `D`. * `Cslib.Stateful.RelSpec.ClientRun`, `Cslib.Stateful.RelSpec.DemonicValid`, `Cslib.Stateful.RelSpec.Hoare`, `Cslib.Stateful.RelSpec.CostHoare`: adaptive client runs, demonic admissibility, and compositional Hoare / CostHoare triples. -* `Cslib.Stateful.RelExecution`, `Cslib.Stateful.RelExecutes`, - `Cslib.Stateful.RelRealization`, `Cslib.Stateful.RelBound`, `Cslib.Stateful.RelAmortized`: - native execution traces, angelic representation simulation, and potential-method certificates. +* `Cslib.Stateful.RelRealization`, `Cslib.Stateful.RelBound`, `Cslib.Stateful.RelAmortized`: + angelic representation simulation and potential-method certificates over `Execution` / `Executes`. ## Main results @@ -127,6 +126,19 @@ theorem exists_step [D.Total] {s : D.State} {o : D.Op} (hv : D.valid s o) : /-- The polynomial operation signature `⟨D.Op, D.Result⟩`. -/ abbrev signature : PFunctor.{v, w} := ⟨D.Op, D.Result⟩ +instance : Coe RelSpec.{u, v, w} PFunctor.{v, w} := ⟨RelSpec.signature⟩ + +/-- Project the operation signature `(D.Op, D.Result)` of a relational specification to a +canonical `Stateful.Spec` with `State := Empty`, allowing signature-only constructions over +`Stateful.Spec` (such as `WordRAM.DynamicProgram`) to apply directly to `D : RelSpec`. -/ +abbrev toSpec : Spec.{0, v, w} where + State := Empty + Op := D.Op + Result := D.Result + step s := s.elim + +instance : Coe RelSpec.{u, v, w} Spec.{0, v, w} := ⟨RelSpec.toSpec⟩ + /-- An adaptive client over `D`: each continuation may branch on the observed operation result. -/ abbrev Client (α : Type y) : Type (max v w y) := D.signature.FreeM α @@ -159,8 +171,6 @@ abbrev Spec.toRelSpec (D : Spec.{u, v, w}) : RelSpec.{u, v, w} where step s o s' a := (s', a) = D.step s o valid := D.valid -instance : Coe Spec.{u, v, w} RelSpec.{u, v, w} := ⟨Spec.toRelSpec⟩ - namespace Spec variable (D : Spec.{u, v, w}) @@ -858,74 +868,14 @@ theorem toRelSpec_costHoare_iff (charge : D.State → D.Op → ℕ) (pre : D.Sta end Spec -/-! ## Native backend executions (`RelExecution`, `RelExecutes`), `RelRealization`, `RelAmortized` --/ - -/-- A concrete backend execution relation for a relational specification `D : RelSpec`. -/ -abbrev RelExecution (D : RelSpec.{u, v, w}) (C Trace : Type x) := - C → (o : D.Op) → D.Result o → C → Trace → Prop - -/-- Adaptive client execution collecting native execution evidence in call order over a -relational specification `D : RelSpec`. -/ -inductive RelExecutes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - (exec : RelExecution D C τ) : D.Client α → C → α → C → List τ → Prop - | pure (a c) : RelExecutes exec (.pure a) c a c [] - | call {o k c a c' trace b c'' traces} : - exec c o a c' trace → RelExecutes exec (k a) c' b c'' traces → - RelExecutes exec (.liftBind o k) c b c'' (trace :: traces) - -/-- Sequential client execution splits its native trace at the call boundary. -/ -theorem relExecutes_bind_iff {D : RelSpec.{u, v, w}} {C τ : Type x} {α β : Type y} - (exec : RelExecution D C τ) (p : D.Client α) (k : α → D.Client β) - {c c' : C} {b : β} {ts : List τ} : - RelExecutes exec (p.bind k) c b c' ts ↔ - ∃ a mid front back, RelExecutes exec p c a mid front ∧ - RelExecutes exec (k a) mid b c' back ∧ ts = front ++ back := by - induction p generalizing c ts with - | pure a => - constructor - · intro h; exact ⟨a, c, [], ts, .pure a c, h, rfl⟩ - · rintro ⟨a', mid, front, back, hp, hk, ht⟩ - cases hp - simpa only [ht, List.nil_append, PFunctor.FreeM.bind] using hk - | liftBind o cont ih => - change RelExecutes exec (.liftBind o fun a => (cont a).bind k) c b c' ts ↔ _ - constructor - · intro h - cases h with - | call ho rest => - obtain ⟨a, mid, front, back, hp, hk, ht⟩ := (ih _).mp rest - exact ⟨a, mid, _, back, .call ho hp, hk, by simp [ht]⟩ - · rintro ⟨a, mid, front, back, hp, hk, ht⟩ - cases hp with - | call ho rest => - have hrest := (ih _).mpr ⟨a, mid, _, back, rest, hk, rfl⟩ - have h := RelExecutes.call (k := fun a => (cont a).bind k) ho hrest - simpa only [ht, List.cons_append] using h - -private theorem Spec.toRelSpec_relExecutes_imp {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : Execution D C τ} {p : D.toRelSpec.Client α} {c : C} {a : α} {c' : C} - {traces : List τ} (h : RelExecutes (D := D.toRelSpec) exec p c a c' traces) : - Executes exec p c a c' traces := by - induction h with - | pure a c => exact .pure a c - | call ho _ ih => exact .call ho ih - -/-- On `D.toRelSpec`, `RelExecutes` coincides with `Executes`. -/ -theorem Spec.toRelSpec_relExecutes_iff {D : Spec.{u, v, w}} {C τ : Type x} {α : Type y} - (exec : Execution D C τ) (p : D.Client α) (c : C) (a : α) (c' : C) (traces : List τ) : - RelExecutes (D := D.toRelSpec) exec p c a c' traces ↔ Executes exec p c a c' traces := by - refine ⟨Spec.toRelSpec_relExecutes_imp, fun h => ?_⟩ - induction h with - | pure a c => exact RelExecutes.pure (D := D.toRelSpec) a c - | call ho _ ih => exact RelExecutes.call (D := D.toRelSpec) ho ih +/-! ## Angelic backend realizations (`RelRealization`, `RelBound`, `RelAmortized`) -/ /-- Angelic semantic realization of a relational specification `D : RelSpec`: * `total`: every represented state and admissible operation admits a native execution; * `correct`: every native execution from a represented state on an admissible operation realizes *some* permitted abstract post-state `s'` with `D.step s o s' a ∧ rep s' c'`. -/ structure RelRealization (D : RelSpec.{u, v, w}) {C τ : Type x} - (exec : RelExecution D C τ) where + (exec : Execution D C τ) where /-- Representation invariant linking abstract states to concrete backend states. -/ rep : D.State → C → Prop /-- Existence of a native execution for every represented state and admissible operation. -/ @@ -937,13 +887,13 @@ structure RelRealization (D : RelSpec.{u, v, w}) {C τ : Type x} namespace RelRealization variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} (R : RelRealization D exec) + {exec : Execution D C τ} (R : RelRealization D exec) /-- From related states, every native execution of a demonically admissible client realizes a valid `ClientRun` ending in a related state. -/ theorem correct_client {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List τ} (hr : R.rep s c) (hv : D.DemonicValid s p) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by induction he generalizing s with | pure a c => @@ -959,7 +909,7 @@ theorem correct_client {p : D.Client α} {s : D.State} {c c' : C} {a : α} valid `ClientRun` ending in a related state. -/ theorem exists_executes (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces s' tr, Executes exec p c a c' traces ∧ D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length := by induction hv generalizing c with | pure s a => @@ -986,17 +936,17 @@ def Realization.toRelRealization {D : Spec.{u, v, w}} {C τ : Type x} /-- Every native execution of a verified demonic `Hoare` client satisfies its abstract postcondition in a related final state. -/ theorem RelSpec.Hoare.realize {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} - {exec : RelExecution D C Trace} (R : RelRealization D exec) + {exec : Execution D C Trace} (R : RelRealization D exec) {pre : D.State → Prop} {post : α → D.State → Prop} {p : D.Client α} (h : D.Hoare pre p post) {s : D.State} {c c' : C} {a : α} {traces : List Trace} - (hp : pre s) (hr : R.rep s c) (he : RelExecutes exec p c a c' traces) : + (hp : pre s) (hr : R.rep s c) (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' := by obtain ⟨hv, hpost⟩ := h s hp obtain ⟨s', tr, hrun, hr', _⟩ := R.correct_client hr hv he exact ⟨s', tr, hrun, hr', hpost a s' tr hrun⟩ /-- Per-operation worst-case resource certificate over a relational realization `R`. -/ -def RelBound {D : RelSpec.{u, v, w}} {C τ : Type x} {exec : RelExecution D C τ} +def RelBound {D : RelSpec.{u, v, w}} {C τ : Type x} {exec : Execution D C τ} (R : RelRealization D exec) (weight : τ → ℕ) (charge : D.State → D.Op → ℕ) : Prop := ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → weight trace ≤ charge s o @@ -1013,7 +963,7 @@ for every native operation execution from `R.rep s c` on an admissible `o`, ther permitted abstract post-state `s'` satisfying `D.step s o s' a`, `R.rep s' c'`, and the single-step potential inequality `weight trace + potential s' c' ≤ charge s o + potential s c`. -/ structure RelAmortized {D : RelSpec.{u, v, w}} {C Trace : Type x} - {exec : RelExecution D C Trace} (R : RelRealization D exec) (weight : Trace → ℕ) + {exec : Execution D C Trace} (R : RelRealization D exec) (weight : Trace → ℕ) (potential : D.State → C → ℕ) (charge : D.State → D.Op → ℕ) : Prop where /-- Angelic single-step potential inequality witnessing a permitted post-state `s'`. -/ step : ∀ {s c o a c' trace}, R.rep s c → D.valid s o → exec c o a c' trace → @@ -1023,19 +973,19 @@ structure RelAmortized {D : RelSpec.{u, v, w}} {C Trace : Type x} namespace RelAmortized variable {D : RelSpec.{u, v, w}} {C Trace : Type x} {α : Type y} - {exec : RelExecution D C Trace} {R : RelRealization D exec} + {exec : Execution D C Trace} {R : RelRealization D exec} {weight : Trace → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} (A : RelAmortized R weight potential charge) include A in /-- **Amortized telescoping along adaptive clients (`executes_le`).** -For any demonically admissible client `p` and native execution `RelExecutes exec p c a c' traces`, +For any demonically admissible client `p` and native execution `Executes exec p c a c' traces`, there exist a final abstract state `s'` and transcript `tr` such that `D.ClientRun s p a s' tr`, `R.rep s' c'`, and `total weight traces + potential s' c' ≤ D.totalCost charge s tr + potential s c`. -/ theorem executes_le {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List Trace} (hr : R.rep s c) (hv : D.DemonicValid s p) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces + potential s' c' ≤ @@ -1052,16 +1002,13 @@ theorem executes_le {p : D.Client α} {s : D.State} {c c' : C} {a : α} simp only [total_cons, RelSpec.totalCost_cons] omega -/-- Alias for `RelAmortized.executes_le` matching `Amortized.client`. -/ -abbrev client := @executes_le - include A in /-- **Existence, correctness, and amortized bound (`run_client_amortized`).** From related initial states, every demonically admissible client `p` admits a native execution satisfying `D.ClientRun`, `R.rep`, and the telescoping potential bound. -/ theorem run_client_amortized (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces s' tr, Executes exec p c a c' traces ∧ D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces + potential s' c' ≤ @@ -1070,17 +1017,6 @@ theorem run_client_amortized (p : D.Client α) {s : D.State} {c : C} obtain ⟨s', tr, hrun, hr', hlen, hpot⟩ := A.executes_le hr hv he exact ⟨a, c', traces, s', tr, he, hrun, hr', hlen, hpot⟩ -include A in -/-- Alias for `RelAmortized.run_client_amortized` matching `Amortized.exists_executes`. -/ -theorem exists_executes (p : D.Client α) {s : D.State} {c : C} - (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ - D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ - traces.length = tr.length ∧ - total weight traces + potential s' c' ≤ - D.totalCost charge s tr + potential s c := - A.run_client_amortized p hr hv - /-- A worst-case `RelBound` is a zero-potential `RelAmortized` certificate. -/ theorem of_relBound (B : RelBound R weight charge) : RelAmortized R weight (fun _ _ => 0) charge where @@ -1104,7 +1040,7 @@ theorem Amortized.toRelAmortized {D : Spec.{u, v, w}} {C Trace : Type x} namespace RelBound variable {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} (B : RelBound R weight charge) @@ -1112,7 +1048,7 @@ include B in /-- Under a worst-case `RelBound`, every native execution of a demonically admissible client has total weight at most `D.totalCost charge s tr` for some valid `ClientRun` transcript `tr`. -/ theorem client {p : D.Client α} {s : D.State} {c c' : C} {a : α} {traces : List τ} - (hr : R.rep s c) (hv : D.DemonicValid s p) (he : RelExecutes exec p c a c' traces) : + (hr : R.rep s c) (hv : D.DemonicValid s p) (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces ≤ D.totalCost charge s tr := by @@ -1124,7 +1060,7 @@ include B in meeting the total charge bound. -/ theorem exists_executes (p : D.Client α) {s : D.State} {c : C} (hr : R.rep s c) (hv : D.DemonicValid s p) : - ∃ a c' traces s' tr, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces s' tr, Executes exec p c a c' traces ∧ D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ traces.length = tr.length ∧ total weight traces ≤ D.totalCost charge s tr := by @@ -1137,13 +1073,13 @@ end RelBound /-- Transport a demonic `CostHoare` triple through a `RelAmortized` certificate to bound native execution weight plus final potential by `budget s + potential s c`. -/ theorem RelSpec.CostHoare.realize_amortized {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {potential : D.State → C → ℕ} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} (A : RelAmortized R weight potential charge) (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : ∃ s' tr, D.ClientRun s p a s' tr ∧ R.rep s' c' ∧ post a s' ∧ total weight traces + potential s' c' ≤ budget s + potential s c := by obtain ⟨hv, hpost⟩ := h.1 s hs @@ -1153,13 +1089,13 @@ theorem RelSpec.CostHoare.realize_amortized {D : RelSpec.{u, v, w}} {C τ : Type /-- Transport a demonic `CostHoare` triple through a worst-case `RelBound` certificate. -/ theorem RelSpec.CostHoare.realize {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} (B : RelBound R weight charge) (h : D.CostHoare charge pre p post budget) {s : D.State} {c c' : C} {a : α} {traces : List τ} (hs : pre s) (hr : R.rep s c) - (he : RelExecutes exec p c a c' traces) : + (he : Executes exec p c a c' traces) : (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by obtain ⟨s', _, _, hr', hpost, hpot⟩ := h.realize_amortized (RelAmortized.of_relBound B) hs hr he @@ -1167,13 +1103,13 @@ theorem RelSpec.CostHoare.realize {D : RelSpec.{u, v, w}} {C τ : Type x} {α : /-- Existence of a native execution satisfying a demonic `CostHoare` contract under `RelBound`. -/ theorem RelSpec.CostHoare.exists_executes {D : RelSpec.{u, v, w}} {C τ : Type x} {α : Type y} - {exec : RelExecution D C τ} {R : RelRealization D exec} + {exec : Execution D C τ} {R : RelRealization D exec} {weight : τ → ℕ} {charge : D.State → D.Op → ℕ} {pre : D.State → Prop} {post : α → D.State → Prop} {budget : D.State → ℕ} {p : D.Client α} (B : RelBound R weight charge) (h : D.CostHoare charge pre p post budget) {s : D.State} {c : C} (hs : pre s) (hr : R.rep s c) : - ∃ a c' traces, RelExecutes exec p c a c' traces ∧ + ∃ a c' traces, Executes exec p c a c' traces ∧ (∃ s', R.rep s' c' ∧ post a s') ∧ total weight traces ≤ budget s := by obtain ⟨a, c', traces, _, _, he, _, _, _⟩ := R.exists_executes p hr (h.1 s hs).1 exact ⟨a, c', traces, he, h.realize B hs hr he⟩ diff --git a/ComplexityTests/StatefulRelational.lean b/ComplexityTests/StatefulRelational.lean index 8d3373331..1b6980b9c 100644 --- a/ComplexityTests/StatefulRelational.lean +++ b/ComplexityTests/StatefulRelational.lean @@ -21,7 +21,7 @@ backend realization and amortized analysis (`RelRealization`, `RelBound`, `RelAm @[expose] public section -namespace Cslib.Stateful.Tests +namespace ComplexityTests.StatefulRelational open Cslib.Stateful @@ -144,7 +144,7 @@ theorem allocTwoAndFreeFirst_hoare : /-- Concrete first-fit implementation of `TwoSlotAlloc`: `alloc` chooses slot `0` when free and slot `1` otherwise, recording `1` unit of execution trace cost per operation. -/ -def firstFitExec : RelExecution TwoSlotAlloc (Bool × Bool) ℕ +def firstFitExec : Execution TwoSlotAlloc (Bool × Bool) ℕ | c, .alloc, i, c', n => n = 1 ∧ if c.1 = false then i = 0 ∧ c' = (true, c.2) @@ -199,7 +199,7 @@ theorem firstFitAmortized : `RelAmortized.run_client_amortized` and `RelSpec.CostHoare.realize_amortized`. -/ theorem firstFit_allocTwoAndFreeFirst_executes : ∃ v c' traces s' tr, - RelExecutes firstFitExec allocTwoAndFreeFirst (false, false) v c' traces ∧ + Executes firstFitExec allocTwoAndFreeFirst (false, false) v c' traces ∧ TwoSlotAlloc.ClientRun (false, false) allocTwoAndFreeFirst v s' tr ∧ firstFitRealization.rep s' c' ∧ (v.1 ≠ v.2 ∧ (if v.1 = 0 then s'.1 = false else s'.2 = false)) ∧ @@ -238,4 +238,4 @@ theorem counterTwice_relHoare : subst hs exact ⟨⟨trivial, trivial, trivial⟩, rfl, rfl⟩ -end Cslib.Stateful.Tests +end ComplexityTests.StatefulRelational diff --git a/ComplexityTests/WordRAMRelSubroutine.lean b/ComplexityTests/WordRAMRelSubroutine.lean index 1eecc6baa..b8aeee2fd 100644 --- a/ComplexityTests/WordRAMRelSubroutine.lean +++ b/ComplexityTests/WordRAMRelSubroutine.lean @@ -13,7 +13,7 @@ public import Complexity.Computability.WordRAM.RelSubroutine /-! # Tests for Relational Word-RAM Subroutine Realizations -Concrete non-vacuous tests for `Cslib.WordRAM.RelSubroutineDS`, `RelDynamicProgram`, +Concrete non-vacuous tests for `Cslib.WordRAM.RelSubroutineDS`, `DynamicProgram`, `RelDynamicCertificate`, `SubroutineDS.toRelSubroutineDS`, `RelSubroutineDS.toStandard`, `RelSubroutineDS.call_rename`, `RelSubroutineDS.run_client_amortized`, `RelSubroutineDS.run_ops_exact_fresh`, and `RelSubroutineDS.run_ops_exact_reusable` on a @@ -22,7 +22,7 @@ genuinely nondeterministic two-slot allocator specification. @[expose] public section -namespace Cslib.WordRAM.Tests +namespace ComplexityTests.WordRAMRelSubroutine open Cslib Cslib.Stateful Cslib.WordRAM @@ -176,8 +176,8 @@ theorem allocPair_demonicValid : theorem twoSlot_run_client_amortized (w base : ℕ) (h2 : Fits w 2) {x : Data 6} (hr : (twoSlotSubroutine w base h2).rep (false, false) x) : ∃ a y traces s' tr, - Stateful.RelExecutes - ((twoSlotSubroutine w base h2).toStandard.toRelDynamicProgram.ExecOp w) + Stateful.Executes + ((twoSlotSubroutine w base h2).toStandard.toDynamicProgram.ExecOp w) allocPair x a y traces ∧ TwoSlotAlloc.ClientRun (false, false) allocPair a s' tr ∧ (twoSlotSubroutine w base h2).rep s' y ∧ @@ -348,4 +348,4 @@ def detCounter2RelSubroutine (w base : ℕ) (h2 : Fits w 2) : RelSubroutineDS twoSlotCode DetCounter2.toRelSpec w base Fin.elim0 0 := (detCounter2Subroutine w base h2).toRelSubroutineDS -end Cslib.WordRAM.Tests +end ComplexityTests.WordRAMRelSubroutine From 888918d31bf556a04215a38559542b30a557bc44 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:10:19 -0400 Subject: [PATCH 06/18] chore(deps): track CSLib main and Lean rc2 Advance CSLib to 8a8ad748 and synchronize Mathlib, the documentation generator, Verso, and all three lockfiles with Lean v4.36.0-rc2. Keep the root and documentation projects on the same inherited dependency revisions. --- docbuild/lake-manifest.json | 34 +++++++++++++++++----------------- docbuild/lakefile.toml | 2 +- guide/lake-manifest.json | 30 +++++++++++++++--------------- guide/lakefile.toml | 2 +- lake-manifest.json | 26 +++++++++++++------------- lakefile.toml | 2 +- lean-toolchain | 2 +- 7 files changed, 49 insertions(+), 49 deletions(-) diff --git a/docbuild/lake-manifest.json b/docbuild/lake-manifest.json index 3609d26b5..a1f5e4692 100644 --- a/docbuild/lake-manifest.json +++ b/docbuild/lake-manifest.json @@ -5,10 +5,10 @@ "type": "git", "subDir": null, "scope": "leanprover", - "rev": "0656b7ebd305f089c8d6bd8684a0797f19414cc2", + "rev": "20afd4ddabd2ef38ffb1a8b930ee0f375e3f75e8", "name": "«doc-gen4»", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": false, "configFile": "lakefile.lean"}, {"type": "path", @@ -23,7 +23,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "e1faac1842cb10b3a68c644643b6e3568766343c", + "rev": "f582e4bc64d6f43e6e711f89b09567d2088fab8d", "name": "leansqlite", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -33,7 +33,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "34d2efffcb6dc8d5f3537cf6a7f88bfaa999ec49", + "rev": "c09cb1e4f1e6e1c231a1175a12b681b075054a1b", "name": "Cli", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -43,7 +43,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "a24bcde05a7971144ba0c0b5b1d60c3a0f0bc4af", + "rev": "91ece4f8d8f93aac566e247874f6b27b57249f1b", "name": "UnicodeBasic", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -53,7 +53,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "78f28e27259a4b2f5a8199800334cebd92730f3d", + "rev": "cdf8963f8b4c9a5593c5aaa9fa4f2759f8dec1c1", "name": "BibtexQuery", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -73,27 +73,27 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "rev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "name": "cslib", "manifestFile": "lake-manifest.json", - "inputRev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "inputRev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "inherited": true, "configFile": "lakefile.toml"}, {"url": "https://github.com/leanprover-community/mathlib4", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "625d8914199bc567cca9b91f90e8e5afafa5ae51", + "rev": "9d581dfff2e41de989072ea63abf3daf69311520", "name": "mathlib", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.lean"}, {"url": "https://github.com/leanprover-community/plausible", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "c2d7840c65673533e15b499f25beecdc6227f7d8", + "rev": "129d1ba9a16bc0660f46c79e9fb041ef5d5d80de", "name": "plausible", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -103,7 +103,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "9d39f9bb0c30aa2f6200808805ff16e6ce950890", + "rev": "c61286b7b63cbfd563b28e11e81d0a18233a044a", "name": "LeanSearchClient", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -113,7 +113,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "b3cc36c295aa5e3e3c0197b09a6556298e710447", + "rev": "b8cc5dd0a36233b88a6ed5a7dbf1207d09338c60", "name": "importGraph", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -123,7 +123,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "17137fad38a2cc782ff00b323a13c3a991d085ab", + "rev": "16259c0d07d61ee37d999d05b5cf96338b8daa11", "name": "proofwidgets", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -133,7 +133,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "cbdde3aa2665915fb286c48bcd4cdf885c629511", + "rev": "7d11bfa88558f8d3d90afdf6bd877ed70001498d", "name": "aesop", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -143,7 +143,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "6d26f8bb4b2bb78f929ad160feac508866821316", + "rev": "b0256621cb88178e9ad932a53633077b3029e621", "name": "Qq", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -153,7 +153,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "4453b52d2468f4c346a8711d728c0742e7483c9b", + "rev": "d4c39b228d4a06f3b00748c2c445a7b9f056a6e3", "name": "batteries", "manifestFile": "lake-manifest.json", "inputRev": "main", diff --git a/docbuild/lakefile.toml b/docbuild/lakefile.toml index ed4689727..b9c4092df 100644 --- a/docbuild/lakefile.toml +++ b/docbuild/lakefile.toml @@ -12,4 +12,4 @@ path = ".." [[require]] scope = "leanprover" name = "doc-gen4" -rev = "v4.36.0-rc1" +rev = "v4.36.0-rc2" diff --git a/guide/lake-manifest.json b/guide/lake-manifest.json index b8da10f21..3f19786a0 100644 --- a/guide/lake-manifest.json +++ b/guide/lake-manifest.json @@ -5,10 +5,10 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f7a4dc6867f8df27778b2b320763ec9e3b7be070", + "rev": "71a2035b44e7f54a797ce799fd1b03cdd3a44684", "name": "verso", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": false, "configFile": "lakefile.lean"}, {"type": "path", @@ -23,7 +23,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "34d2efffcb6dc8d5f3537cf6a7f88bfaa999ec49", + "rev": "c09cb1e4f1e6e1c231a1175a12b681b075054a1b", "name": "Cli", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -33,7 +33,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "d22e76ef3d2a925132315af902ea432233b4a44a", + "rev": "758bff3509b8e90ca82e37aa5de81de3c9e70fc0", "name": "illuminate", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -43,7 +43,7 @@ "type": "git", "subDir": null, "scope": "", - "rev": "c2d7840c65673533e15b499f25beecdc6227f7d8", + "rev": "129d1ba9a16bc0660f46c79e9fb041ef5d5d80de", "name": "plausible", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -73,27 +73,27 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "rev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "name": "cslib", "manifestFile": "lake-manifest.json", - "inputRev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "inputRev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "inherited": true, "configFile": "lakefile.toml"}, {"url": "https://github.com/leanprover-community/mathlib4", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "625d8914199bc567cca9b91f90e8e5afafa5ae51", + "rev": "9d581dfff2e41de989072ea63abf3daf69311520", "name": "mathlib", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.lean"}, {"url": "https://github.com/leanprover-community/LeanSearchClient", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "9d39f9bb0c30aa2f6200808805ff16e6ce950890", + "rev": "c61286b7b63cbfd563b28e11e81d0a18233a044a", "name": "LeanSearchClient", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -103,7 +103,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "b3cc36c295aa5e3e3c0197b09a6556298e710447", + "rev": "b8cc5dd0a36233b88a6ed5a7dbf1207d09338c60", "name": "importGraph", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -113,7 +113,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "17137fad38a2cc782ff00b323a13c3a991d085ab", + "rev": "16259c0d07d61ee37d999d05b5cf96338b8daa11", "name": "proofwidgets", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -123,7 +123,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "cbdde3aa2665915fb286c48bcd4cdf885c629511", + "rev": "7d11bfa88558f8d3d90afdf6bd877ed70001498d", "name": "aesop", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -133,7 +133,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "6d26f8bb4b2bb78f929ad160feac508866821316", + "rev": "b0256621cb88178e9ad932a53633077b3029e621", "name": "Qq", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -143,7 +143,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "4453b52d2468f4c346a8711d728c0742e7483c9b", + "rev": "d4c39b228d4a06f3b00748c2c445a7b9f056a6e3", "name": "batteries", "manifestFile": "lake-manifest.json", "inputRev": "main", diff --git a/guide/lakefile.toml b/guide/lakefile.toml index fb7601692..f62d2a91a 100644 --- a/guide/lakefile.toml +++ b/guide/lakefile.toml @@ -21,7 +21,7 @@ path = ".." [[require]] name = "verso" git = "https://github.com/leanprover/verso" -rev = "v4.36.0-rc1" +rev = "v4.36.0-rc2" [[lean_lib]] name = "ComplexityGuide" diff --git a/lake-manifest.json b/lake-manifest.json index a4892b4fc..a2c1eaffd 100644 --- a/lake-manifest.json +++ b/lake-manifest.json @@ -5,27 +5,27 @@ "type": "git", "subDir": null, "scope": "", - "rev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "rev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "name": "cslib", "manifestFile": "lake-manifest.json", - "inputRev": "f4a472560f78c59bbedd795fe041ec507f7d0582", + "inputRev": "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad", "inherited": false, "configFile": "lakefile.toml"}, {"url": "https://github.com/leanprover-community/mathlib4", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "625d8914199bc567cca9b91f90e8e5afafa5ae51", + "rev": "9d581dfff2e41de989072ea63abf3daf69311520", "name": "mathlib", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.lean"}, {"url": "https://github.com/leanprover-community/plausible", "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "c2d7840c65673533e15b499f25beecdc6227f7d8", + "rev": "129d1ba9a16bc0660f46c79e9fb041ef5d5d80de", "name": "plausible", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -35,7 +35,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "9d39f9bb0c30aa2f6200808805ff16e6ce950890", + "rev": "c61286b7b63cbfd563b28e11e81d0a18233a044a", "name": "LeanSearchClient", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -45,7 +45,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "b3cc36c295aa5e3e3c0197b09a6556298e710447", + "rev": "b8cc5dd0a36233b88a6ed5a7dbf1207d09338c60", "name": "importGraph", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -55,7 +55,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "17137fad38a2cc782ff00b323a13c3a991d085ab", + "rev": "16259c0d07d61ee37d999d05b5cf96338b8daa11", "name": "proofwidgets", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -65,7 +65,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "cbdde3aa2665915fb286c48bcd4cdf885c629511", + "rev": "7d11bfa88558f8d3d90afdf6bd877ed70001498d", "name": "aesop", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -75,7 +75,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "6d26f8bb4b2bb78f929ad160feac508866821316", + "rev": "b0256621cb88178e9ad932a53633077b3029e621", "name": "Qq", "manifestFile": "lake-manifest.json", "inputRev": "master", @@ -85,7 +85,7 @@ "type": "git", "subDir": null, "scope": "leanprover-community", - "rev": "4453b52d2468f4c346a8711d728c0742e7483c9b", + "rev": "d4c39b228d4a06f3b00748c2c445a7b9f056a6e3", "name": "batteries", "manifestFile": "lake-manifest.json", "inputRev": "main", @@ -95,10 +95,10 @@ "type": "git", "subDir": null, "scope": "leanprover", - "rev": "34d2efffcb6dc8d5f3537cf6a7f88bfaa999ec49", + "rev": "c09cb1e4f1e6e1c231a1175a12b681b075054a1b", "name": "Cli", "manifestFile": "lake-manifest.json", - "inputRev": "v4.36.0-rc1", + "inputRev": "v4.36.0-rc2", "inherited": true, "configFile": "lakefile.toml"}], "name": "complexitylib", diff --git a/lakefile.toml b/lakefile.toml index ce7b8cd77..00f09a874 100644 --- a/lakefile.toml +++ b/lakefile.toml @@ -33,7 +33,7 @@ autoImplicit = false [[require]] name = "cslib" git = "https://github.com/leanprover/cslib" -rev = "f4a472560f78c59bbedd795fe041ec507f7d0582" +rev = "8a8ad7489cdccb007ad7279bfb9d03bbda2229ad" [[lean_lib]] name = "Complexity" diff --git a/lean-toolchain b/lean-toolchain index 2afd38716..0fc433692 100644 --- a/lean-toolchain +++ b/lean-toolchain @@ -1 +1 @@ -leanprover/lean4:v4.36.0-rc1 +leanprover/lean4:v4.36.0-rc2 From 9a1b75416b7723a073411f0ee9a7d2ccfdd49ec9 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:10:24 -0400 Subject: [PATCH 07/18] refactor(probability): use generic infinite-sum lemmas Replace 187 deprecated ENNReal wrapper uses across 55 proof files with their generic counterparts. The exact mapping is: - ENNReal.tsum_le_tsum -> tsum_le_tsum (94) - ENNReal.tsum_add -> tsum_add (54) - ENNReal.tsum_eq_zero -> tsum_eq_zero_iff (14) - ENNReal.summable -> CompleteLattice.summable (6) - ENNReal.tsum_comm -> tsum_comm (6) - ENNReal.tsum_prod' -> tsum_prod' (5) - ENNReal.le_tsum -> le_tsum (5) - ENNReal.tsum_eq_add_tsum_ite -> tsum_eq_add_tsum_ite (2) - ENNReal.tsum_sigma -> tsum_sigma (1) The old wrappers invoke these same theorems specialized to ENNReal. All five le_tsum calls already pass the function by name. No statements or definitions change. --- .../Algorithms/WordRAM/Karger/FastCut.lean | 2 +- .../Algorithms/WordRAM/Karger/Graph.lean | 6 +-- .../Algorithms/WordRAM/KargerMinCut.lean | 2 +- .../Algorithms/WordRAM/KargerStein.lean | 6 +-- .../Computability/Complexity/Classes/IP.lean | 2 +- .../Classes/IP/GraphNonIsomorphism.lean | 2 +- .../Complexity/Classes/IP/Guessing.lean | 6 +-- .../Complexity/Classes/IP/OptimalProver.lean | 8 ++-- .../Complexity/Classes/IP/Repetition.lean | 6 +-- .../LogSpace/Randomized/Amplification.lean | 2 +- .../Complexity/Classes/MIP/OptimalProver.lean | 4 +- .../Complexity/FineGrained/PPSZ/Basic.lean | 2 +- .../Complexity/FineGrained/PPZ.lean | 2 +- .../Computability/Interactive/Plain.lean | 4 +- .../Computability/Kolmogorov/AverageCase.lean | 4 +- .../Computability/PCP/TwoProverGame.lean | 2 +- .../WordRAM/Randomized/Expected.lean | 12 ++--- .../CramerShoup/ProjectiveHash.lean | 4 +- .../Computational/CramerShoup/Scheme.lean | 2 +- .../Crypto/Computational/DiffieHellman.lean | 16 +++---- .../Computational/DiffieHellman/Order.lean | 4 +- .../Encryption/Authenticated.lean | 14 +++--- .../Encryption/EncryptThenMAC.lean | 8 ++-- .../Crypto/Computational/GMW/Soundness.lean | 6 +-- Complexity/Crypto/Computational/MAC.lean | 8 ++-- .../Computational/NaorReingold/Security.lean | 10 ++-- .../Computational/OneWay/LiuPassConverse.lean | 2 +- .../Computational/OneWayToSignature.lean | 6 +-- .../Crypto/Computational/PRGToOneWay.lean | 2 +- .../Computational/PedersenCommitment.lean | 8 ++-- .../Pseudoentropy/MaskedExtraction.lean | 2 +- Complexity/Crypto/Game.lean | 20 ++++---- Complexity/Crypto/Game/UntilBad.lean | 16 +++---- Complexity/Crypto/Primitives/PRG/NextBit.lean | 4 +- .../Crypto/Protocols/Cipher/EvenMansour.lean | 6 +-- .../Crypto/Protocols/Cipher/LubyRackoff.lean | 48 +++++++++---------- .../Crypto/Protocols/MAC/CarterWegman.lean | 16 +++---- .../Protocols/SigmaProtocol/FiatShamir.lean | 6 +-- .../Probabilistic/BoundedUniform.lean | 6 +-- .../Languages/Probabilistic/Expect.lean | 8 ++-- .../Probabilistic/Oracle/Interaction.lean | 2 +- .../Probabilistic/RandomPermutation.lean | 8 ++-- .../Languages/Probabilistic/UntilBad.lean | 10 ++-- .../Languages/Probabilistic/Vector.lean | 2 +- Complexity/Probability/Conditioning.lean | 6 +-- .../Probability/CorrelatedSampling.lean | 2 +- Complexity/Probability/Forking.lean | 2 +- Complexity/Probability/IID.lean | 2 +- Complexity/Probability/MinEntropy.lean | 2 +- Complexity/Probability/Minimax.lean | 4 +- Complexity/Probability/PMF/Bind.lean | 14 +++--- Complexity/Probability/PMF/Event.lean | 16 +++---- Complexity/Probability/PMF/Map.lean | 2 +- .../ProductConditioning/SideInformation.lean | 2 +- Complexity/Probability/RejectionSampling.lean | 4 +- 55 files changed, 186 insertions(+), 186 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/Karger/FastCut.lean b/Complexity/Algorithms/WordRAM/Karger/FastCut.lean index a216c2ae8..30a2afa92 100644 --- a/Complexity/Algorithms/WordRAM/Karger/FastCut.lean +++ b/Complexity/Algorithms/WordRAM/Karger/FastCut.lean @@ -658,7 +658,7 @@ theorem two_branch_hit_prob_ge {α β γ : Type} (m₁ : ProbComp α) (m₂ : α have ha : a ∈ m₁.support := (ProbComp.support_eval m₁) ▸ ha_eval have ha_hit : a ∈ hit₁ := not_not.mp ha_not_miss rw [PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_eq_zero.mpr fun b => ?_ + refine tsum_eq_zero_iff.mpr fun b => ?_ by_cases hb_eval : b ∈ (m₂ a).eval.support · have hb : b ∈ (m₂ a).support := (ProbComp.support_eval (m₂ a)) ▸ hb_eval have hmem : f a b ∈ hit := hprop₁ a ha ha_hit b hb diff --git a/Complexity/Algorithms/WordRAM/Karger/Graph.lean b/Complexity/Algorithms/WordRAM/Karger/Graph.lean index 9afa1d805..c03da4013 100644 --- a/Complexity/Algorithms/WordRAM/Karger/Graph.lean +++ b/Complexity/Algorithms/WordRAM/Karger/Graph.lean @@ -1107,7 +1107,7 @@ theorem toOuterMeasure_bind_ge_mul_ennreal {α β : Type*} (p : PMF α) (kernel s₁ * s₂ ≤ p.toOuterMeasure pre * s₂ := mul_le_mul_of_nonneg_right hp zero_le _ ≤ (p.bind kernel).toOuterMeasure event := by rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ pre · simp only [Set.indicator_of_mem ha] by_cases hs : a ∈ p.support @@ -1131,7 +1131,7 @@ theorem toOuterMeasure_bind_le_mul_ennreal {α β : Type*} (p : PMF α) (kernel (p.bind kernel).toOuterMeasure failure ≤ p.toOuterMeasure stillFailing * q₂ := by rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases hs : a ∈ p.support · by_cases ha : a ∈ stillFailing · simp only [Set.indicator_of_mem ha] @@ -1146,7 +1146,7 @@ valid on arbitrary types without measurability hypotheses. -/ theorem toOuterMeasure_add_compl_ennreal {α : Type*} (p : PMF α) (s : Set α) : p.toOuterMeasure s + p.toOuterMeasure sᶜ = 1 := by classical - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] have hpt : (fun x => s.indicator p x + sᶜ.indicator p x) = p := by ext x by_cases hx : x ∈ s <;> simp [hx] diff --git a/Complexity/Algorithms/WordRAM/KargerMinCut.lean b/Complexity/Algorithms/WordRAM/KargerMinCut.lean index b3ef003a5..c07c84a3c 100644 --- a/Complexity/Algorithms/WordRAM/KargerMinCut.lean +++ b/Complexity/Algorithms/WordRAM/KargerMinCut.lean @@ -1181,7 +1181,7 @@ theorem ampLoop_spec {w sb n R : ℕ} {W : Fin n → Fin n → ℕ} {mem₀ : obtain ⟨m_r, hm_r, _, hsup_r, _⟩ := hnext z₁ hz₁ rw [hml_def, hm_r.codeComp_eq, PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - convert ENNReal.tsum_eq_zero.mpr _ + convert tsum_eq_zero_iff.mpr _ intro z₂ by_cases hz₂_eval : z₂ ∈ m_r.eval.support · have hz₂ : z₂ ∈ m_r.support := (ProbComp.support_eval m_r) ▸ hz₂_eval diff --git a/Complexity/Algorithms/WordRAM/KargerStein.lean b/Complexity/Algorithms/WordRAM/KargerStein.lean index 92368fc36..5a2a42a1a 100644 --- a/Complexity/Algorithms/WordRAM/KargerStein.lean +++ b/Complexity/Algorithms/WordRAM/KargerStein.lean @@ -168,7 +168,7 @@ private theorem toOuterMeasure_mono_support {α : Type} (m : ProbComp α) {S T : m.eval.toOuterMeasure S ≤ m.eval.toOuterMeasure T := by classical rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ S · simp only [Set.indicator_of_mem ha] by_cases hs : a ∈ m.support @@ -2684,7 +2684,7 @@ theorem fastCutNode_spec {n w : ℕ} {W₀ : Fin n → Fin n → ℕ} have hcompl0 : (ms2_fn z₂.1).eval.toOuterMeasure {zs | zs.1.regs retVal = minCut W₀}ᶜ = 0 := by rw [PMF.toOuterMeasure_apply] - refine ENNReal.tsum_eq_zero.mpr fun zs => ?_ + refine tsum_eq_zero_iff.mpr fun zs => ?_ by_cases hzs_ev : zs ∈ (ms2_fn z₂.1).eval.support · have hzs : zs ∈ (ms2_fn z₂.1).support := (ProbComp.support_eval (ms2_fn z₂.1)) ▸ hzs_ev @@ -3679,7 +3679,7 @@ theorem fastCutAmpLoop_spec {w sb n R : ℕ} {W : Fin n → Fin n → ℕ} {mem have hbv₁ : z₁.1.regs bestVal = minCut W := not_not.mp hhit obtain ⟨m_r, hm_r, _, hsup_r, _⟩ := hnext z₁ hz₁ rw [hml_def, hm_r.codeComp_eq, PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_eq_zero.mpr fun z₂ => ?_ + refine tsum_eq_zero_iff.mpr fun z₂ => ?_ by_cases hz₂_eval : z₂ ∈ m_r.eval.support · have hz₂ : z₂ ∈ m_r.support := (ProbComp.support_eval m_r) ▸ hz₂_eval have hbv₂ : z₂.1.regs bestVal = minCut W := diff --git a/Complexity/Computability/Complexity/Classes/IP.lean b/Complexity/Computability/Complexity/Classes/IP.lean index d2c73a5bc..d583e0f90 100644 --- a/Complexity/Computability/Complexity/Classes/IP.lean +++ b/Complexity/Computability/Complexity/Classes/IP.lean @@ -148,7 +148,7 @@ theorem acceptProb_eq_acceptProbWith (verifier : WordOracleComp Operation Bool) /-- The acceptance probability sums the masses of the accepting outcomes over final states. -/ theorem acceptProb_eq_tsum (verifier : WordOracleComp Operation Bool) (P : Prover Operation) : acceptProb verifier P = ∑' s, P.run verifier P.init (true, s) := by - rw [acceptProb, PMF.map_apply, ENNReal.tsum_prod', tsum_bool] + rw [acceptProb, PMF.map_apply, tsum_prod', tsum_bool] simp /-- Private coins drawn before any message average the acceptance probabilities of the diff --git a/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean b/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean index dfce7de33..659f372bf 100644 --- a/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean +++ b/Complexity/Computability/Complexity/Classes/IP/GraphNonIsomorphism.lean @@ -413,7 +413,7 @@ private theorem card_filter_not_injective_div_le (n : ℕ) : private theorem tsum_map_mul {β γ : Type*} (μ : PMF β) (m : β → γ) (f : γ → ENNReal) : ∑' c, μ.map m c * f c = ∑' b, μ b * f (m b) := by simp only [PMF.map_apply, ENNReal.tsum_mul_right.symm] - rw [ENNReal.tsum_comm] + rw [tsum_comm] refine tsum_congr fun b => ?_ rw [tsum_eq_single (m b)] · simp diff --git a/Complexity/Computability/Complexity/Classes/IP/Guessing.lean b/Complexity/Computability/Complexity/Classes/IP/Guessing.lean index 6eaedc8da..02e8929d3 100644 --- a/Complexity/Computability/Complexity/Classes/IP/Guessing.lean +++ b/Complexity/Computability/Complexity/Classes/IP/Guessing.lean @@ -77,8 +77,8 @@ noncomputable def Prover.guessProb (P : Prover Unit) (s : P.State) (m : List Boo theorem Prover.guessProb_add_le_one (P : Prover Unit) (s : P.State) (m : List Bool) : P.guessProb s m false + P.guessProb s m true ≤ 1 := by simp only [Prover.guessProb, PMF.map_apply] - rw [← ENNReal.tsum_add, ← PMF.tsum_coe ((P.respond ((), m)).run s)] - refine ENNReal.tsum_le_tsum fun y => ?_ + rw [← tsum_add, ← PMF.tsum_coe ((P.respond ((), m)).run s)] + refine tsum_le_tsum fun y => ?_ by_cases h : List.take 1 y.1 = [false] · simp [h] · by_cases h' : List.take 1 y.1 = [true] <;> simp [h, h'] @@ -120,7 +120,7 @@ theorem acceptProb_guessingRound_le {message : Bool → ProbComp (List Bool)} { exact hclose _ fun m => PMF.coe_le_one _ _ _ = ∑' m, ProbComp.eval (message false) m * (P.guessProb P.init m false + P.guessProb P.init m true) + δ := by - rw [← add_assoc, ← ENNReal.tsum_add] + rw [← add_assoc, ← tsum_add] simp only [mul_add] _ ≤ 1 + δ := by gcongr diff --git a/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean b/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean index 3ad366c14..7fd4eec66 100644 --- a/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean +++ b/Complexity/Computability/Complexity/Classes/IP/OptimalProver.lean @@ -304,7 +304,7 @@ private theorem sum_acceptFrom_after_le (hF : IsBounded F D L) (P : Prover Unit) lia rw [Finset.sum_eq_zero fun m _ => ?_, add_zero] · split_ifs <;> simp - · refine ENNReal.tsum_eq_zero.mpr fun ys => ?_ + · refine tsum_eq_zero_iff.mpr fun ys => ?_ rw [after_append] unfold outcome at hnil rcases hah : after (F r) h with _ | ⟨b⟩ | ⟨q, next⟩ @@ -320,12 +320,12 @@ private theorem sum_acceptFrom_after_le (hF : IsBounded F D L) (P : Prover Unit) exact (Finset.sum_congr rfl fun r _ => by split_ifs <;> simp).le rw [Finset.sum_comm] refine Finset.sum_le_sum fun m _ => ?_ - rw [← Summable.tsum_finsetSum fun r _ => ENNReal.summable] + rw [← Summable.tsum_finsetSum fun r _ => CompleteLattice.summable] calc ∑' ys : List Bool × P.State, ∑ r, (P.respond ((), m)).run s ys * acceptFrom P ys.2 (after (F r) (h ++ [(m, ys.1)])) ≤ ∑' ys : List Bool × P.State, (P.respond ((), m)).run s ys * ((BitString.shortWords L).sup fun y => value F L n (h ++ [(m, y)]) : ℕ) := by - refine ENNReal.tsum_le_tsum fun ys => ?_ + refine tsum_le_tsum fun ys => ?_ rw [← Finset.mul_sum] refine mul_le_mul_right ?_ _ refine (ih (h ++ [(m, ys.1)]) (by @@ -395,7 +395,7 @@ private theorem sum_acceptFrom_optimalProver_after (hF : IsBounded F D L) (n : lia rw [Finset.sum_eq_zero fun m _ => ?_, add_zero] · split_ifs <;> simp - · refine ENNReal.tsum_eq_zero.mpr fun ys => ?_ + · refine tsum_eq_zero_iff.mpr fun ys => ?_ rw [after_append] unfold outcome at hnil rcases hah : after (F r) h with _ | ⟨b⟩ | ⟨q, next⟩ diff --git a/Complexity/Computability/Complexity/Classes/IP/Repetition.lean b/Complexity/Computability/Complexity/Classes/IP/Repetition.lean index 66023d528..77f8fb503 100644 --- a/Complexity/Computability/Complexity/Classes/IP/Repetition.lean +++ b/Complexity/Computability/Complexity/Classes/IP/Repetition.lean @@ -64,7 +64,7 @@ private theorem acceptProb_bind_ite (P : Prover Operation) acceptProb (do let ok ← round; if ok then rest else pure false) P = ∑' s, P.run round P.init (true, s) * acceptProb rest { P with init := s } := by simp only [acceptProb, Prover.run_bind, PMF.map_bind] - rw [PMF.bind_apply, ENNReal.tsum_prod', tsum_bool] + rw [PMF.bind_apply, tsum_prod', tsum_bool] simp only [Bool.false_eq_true, Bool.true_eq_false, ↓reduceIte, Prover.run_pure, PMF.pure_map, PMF.pure_apply, mul_zero, tsum_zero, zero_add] rfl @@ -81,7 +81,7 @@ theorem acceptProb_sequentialRepeat_le {round : WordOracleComp Operation Bool} { calc ∑' s, P.run round P.init (true, s) * acceptProb (sequentialRepeat k round) { P with init := s } ≤ ∑' s, P.run round P.init (true, s) * ε ^ k := - ENNReal.tsum_le_tsum fun s => mul_le_mul' le_rfl (ih _) + tsum_le_tsum fun s => mul_le_mul' le_rfl (ih _) _ = (∑' s, P.run round P.init (true, s)) * ε ^ k := ENNReal.tsum_mul_right _ ≤ ε * ε ^ k := by gcongr @@ -103,7 +103,7 @@ theorem le_acceptProb_sequentialRepeat {round : WordOracleComp Operation Bool} { gcongr exact (h s).trans_eq (acceptProb_eq_tsum round _) _ = ∑' s', P.run round s (true, s') * c ^ k := ENNReal.tsum_mul_right.symm - _ ≤ _ := ENNReal.tsum_le_tsum fun s' => mul_le_mul' le_rfl (ih s') + _ ≤ _ := tsum_le_tsum fun s' => mul_le_mul' le_rfl (ih s') /-- A constant number of sequential runs of a strict oracle PPT verifier is strict oracle PPT. -/ theorem isOraclePPTOn_sequentialRepeat [DecidableEq Operation] [Finite Operation] diff --git a/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean b/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean index 1c3606355..56fd01273 100644 --- a/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean +++ b/Complexity/Computability/Complexity/Classes/LogSpace/Randomized/Amplification.lean @@ -126,7 +126,7 @@ theorem eval_runClosed_orElse_apply (h₁ : first.toNTM.RunsInTime input t₁) exact congrArg _ (PMF.pure_apply_self _) · rw [add_zero] _ = q + ProbComp.eval (first.runClosed t₁ input) [true] * (1 - q) := by - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, tsum_ite_eq] + rw [tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, tsum_ite_eq] /-- **Rejections multiply under `orElse`.** Counted on coin strings, the probability that `first.orElse second` does not accept is the product of the probabilities that `first` and diff --git a/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean b/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean index ebf6834a9..0f1a57869 100644 --- a/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean +++ b/Complexity/Computability/Complexity/Classes/MIP/OptimalProver.lean @@ -581,7 +581,7 @@ private theorem tsum_mul_le_bestOfList (μ : PMF α) (g : α → ℝ≥0∞) (de ∑' x, μ x * g x ≤ g (bestOfList g default l) := by calc ∑' x, μ x * g x ≤ ∑' x, μ x * g (bestOfList g default l) := by - refine ENNReal.tsum_le_tsum fun x => ?_ + refine tsum_le_tsum fun x => ?_ by_cases hx : x ∈ μ.support · exact mul_le_mul_right (le_bestOfList g default l (hsupp x hx)) (μ x) · rw [PMF.mem_support_iff, not_not] at hx @@ -618,7 +618,7 @@ theorem exists_deterministic_prover_family [Finite ι] [DecidableEq ι] {γ : Ty rw [run_eq_bind_piPresample (hF c) P (fun _ => d) (fun _ => le_rfl) (Prover.independent P).init, PMF.bind_apply] simp only [hc, ← ENNReal.tsum_mul_left] - rw [← Summable.tsum_finsetSum fun _ _ => ENNReal.summable] + rw [← Summable.tsum_finsetSum fun _ _ => CompleteLattice.summable] refine tsum_congr fun f => ?_ rw [Finset.mul_sum] exact Finset.sum_congr rfl fun c _ => by rw [mul_left_comm] diff --git a/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean b/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean index e47a1f654..a73035873 100644 --- a/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean +++ b/Complexity/Computability/Complexity/FineGrained/PPSZ/Basic.lean @@ -276,7 +276,7 @@ theorem ppz_round_apply_le {φ : CNF (Fin n)} {w : ℕ} (hw : 1 ≤ w) {σ : Fin (hσ : φ.eval σ = true) : PPZ.round φ (fun x => some (σ x)) ≤ round φ w (fun x => some (σ x)) := by rw [PPZ.round, round, PPZ.roundWith, PPZ.roundWith, PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun π => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun π => mul_le_mul_right ?_ _ rw [← PPZ.restrict_prefixSet_self σ π] rw [PPZ.runWith_apply_restrict (O := PPZ.forced φ) (fun _ hτ _ _ h => PPZ.forced_eq hσ hτ h) π le_rfl, PPZ.runWith_apply_restrict (O := implied φ w) (fun _ hτ _ _ h => implied_eq hσ hτ h) π diff --git a/Complexity/Computability/Complexity/FineGrained/PPZ.lean b/Complexity/Computability/Complexity/FineGrained/PPZ.lean index 29084952a..a7c4fa2c3 100644 --- a/Complexity/Computability/Complexity/FineGrained/PPZ.lean +++ b/Complexity/Computability/Complexity/FineGrained/PPZ.lean @@ -458,7 +458,7 @@ theorem runWith_apply_restrict {O : (Fin n → Option Bool) → Fin n → Option by_cases hsupp : τ ∈ (runWith O π i).support · rw [PMF.map_apply] convert mul_zero _ - refine ENNReal.tsum_eq_zero.mpr fun b => ite_eq_right fun e => hτ ?_ + refine tsum_eq_zero_iff.mpr fun b => ite_eq_right fun e => hτ ?_ funext y by_cases hy : y = x · subst hy diff --git a/Complexity/Computability/Interactive/Plain.lean b/Complexity/Computability/Interactive/Plain.lean index 23a157b84..822083780 100644 --- a/Complexity/Computability/Interactive/Plain.lean +++ b/Complexity/Computability/Interactive/Plain.lean @@ -201,9 +201,9 @@ private theorem rbrFrom_le : ∀ (s : Spec) (smp : PerChallenge ProbComp s) st' calc ∑' c, ProbComp.eval smp.1 c * _ ≤ ∑' c, ProbComp.eval smp.1 c * (A.indicator 1 c + PerChallenge.sum s ε.2) := - ENNReal.tsum_le_tsum fun c => by gcongr; exact hinner c + tsum_le_tsum fun c => by gcongr; exact hinner c _ = (ProbComp.eval smp.1).toOuterMeasure A + PerChallenge.sum s ε.2 := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun c => ?_ diff --git a/Complexity/Computability/Kolmogorov/AverageCase.lean b/Complexity/Computability/Kolmogorov/AverageCase.lean index 0ead74cf0..86083ba86 100644 --- a/Complexity/Computability/Kolmogorov/AverageCase.lean +++ b/Complexity/Computability/Kolmogorov/AverageCase.lean @@ -277,7 +277,7 @@ private theorem pmf_map_true_le_one_sub_of_imp_false {α : Type*} (q : PMF α) ( ((q.map P) true).toReal ≤ 1 - ((q.map Q) true).toReal := by have hle : (q.map P) true ≤ (q.map Q) false := by rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun v => ?_ + refine tsum_le_tsum fun v => ?_ by_cases hp : P v = true · have hq : Q v = false := hdisj v hp simp [hp, hq] @@ -292,7 +292,7 @@ private theorem pmf_map_true_le_of_imp_true {α : Type*} (q : PMF α) (P Q : α ((q.map P) true).toReal ≤ ((q.map Q) true).toReal := by apply ENNReal.toReal_mono (PMF.apply_ne_top _ _) rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun v => ?_ + refine tsum_le_tsum fun v => ?_ by_cases hp : P v = true · have hq : Q v = true := himp v hp simp [hp, hq] diff --git a/Complexity/Computability/PCP/TwoProverGame.lean b/Complexity/Computability/PCP/TwoProverGame.lean index d918d1eba..e742d8fc8 100644 --- a/Complexity/Computability/PCP/TwoProverGame.lean +++ b/Complexity/Computability/PCP/TwoProverGame.lean @@ -117,7 +117,7 @@ theorem winProb_shared_le_value [Nonempty A] [Nonempty B] {R : Type*} (ρ : PMF exact ENNReal.ofReal_le_ofReal (G.winProb_le_value (fun x => ha x r) (fun y => hb y r)) calc ∑' r, ρ r * (G.questions.map fun q => (q, ha q.1 r, hb q.2 r)).toOuterMeasure {z | G.accepts z.1.1 z.1.2 z.2.1 z.2.2} - ≤ ∑' r, ρ r * ENNReal.ofReal G.value := ENNReal.tsum_le_tsum fun r => by gcongr; exact hr r + ≤ ∑' r, ρ r * ENNReal.ofReal G.value := tsum_le_tsum fun r => by gcongr; exact hr r _ = ENNReal.ofReal G.value := by rw [ENNReal.tsum_mul_right, ρ.tsum_coe, one_mul] end Finite diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index 7d9e1844a..4f1881163 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -117,7 +117,7 @@ theorem evalRand_haltedSet_mono (w : ℕ) (p : Program B r) (s : State r) : rw [evalRand, runRand_add w p T 1, ProbComp.eval_bind] simp only [evalRand, runRand_succ, runRand_zero, bind_pure] rw [hstep, PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] - refine ENNReal.tsum_le_tsum fun cfg => ?_ + refine tsum_le_tsum fun cfg => ?_ by_cases hcfg : cfg ∈ RandCfg.haltedSet r · have hmem := hcfg obtain ⟨v, d, rfl⟩ := hcfg @@ -139,7 +139,7 @@ theorem RandHaltsWithin.almostSurelyHalts {w T : ℕ} {p : Program B r} {s : Sta exact hnot (h cfg hmem) have hsum : (evalRand w p T s).toOuterMeasure (RandCfg.haltedSet r) + (evalRand w p T s).toOuterMeasure (RandCfg.haltedSet r)ᶜ = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert (evalRand w p T s).tsum_coe using 1 congr 1 funext cfg @@ -172,7 +172,7 @@ theorem expectedStepsFrom_faulted (w : ℕ) (p : Program B r) : private theorem tsum_nat_eq_zero_add (g : ℕ → ℝ≥0∞) : ∑' t : ℕ, g t = g 0 + ∑' t : ℕ, g (t + 1) := by - rw [ENNReal.tsum_eq_add_tsum_ite (f := g) 0] + rw [tsum_eq_add_tsum_ite (f := g) 0] congr 1 have hsh := Function.Injective.tsum_eq Nat.succ_injective (f := fun t => if t = 0 then 0 else g t) fun t ht => by @@ -199,7 +199,7 @@ theorem expectedStepsFrom_running (w : ℕ) (p : Program B r) (s : State r) : intro t rw [runRand_succ, ProbComp.eval_bind, PMF.toOuterMeasure_bind_apply] simp only [hsucc] - rw [ENNReal.tsum_comm] + rw [tsum_comm] simp only [ProbComp.expect, expectedStepsFrom, ENNReal.tsum_mul_left] /-- Under `RandFaultFree`, every horizon partitions probability `1` between `.running` and @@ -208,7 +208,7 @@ theorem RandFaultFree.running_add_halted_eq_one {w : ℕ} {p : Program B r} {s : (hf : RandFaultFree w p s) (t : ℕ) : (evalRand w p t s).toOuterMeasure {cfg | ∃ u, cfg = .running u} + (evalRand w p t s).toOuterMeasure (RandCfg.haltedSet r) = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert (evalRand w p t s).tsum_coe using 1 congr 1 funext cfg @@ -242,7 +242,7 @@ theorem AlmostSurelyHalts.of_expectedTime_ne_top {w : ℕ} {p : Program B r} {s have hinf : (⊤ : ℝ≥0∞) ≤ expectedTime w p s := by calc (⊤ : ℝ≥0∞) = ∑' _ : ℕ, (1 - supH) := (ENNReal.tsum_const_eq_top_of_ne_zero hpos.ne').symm - _ ≤ expectedTime w p s := ENNReal.tsum_le_tsum hle_run + _ ≤ expectedTime w p s := tsum_le_tsum hle_run exact htop (top_unique hinf) /-- Under `RandHaltsWithin w p s T`, the active-configuration tail sum vanishes from `T` onward and diff --git a/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean b/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean index 09e298a66..c52fe95e6 100644 --- a/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean +++ b/Complexity/Crypto/Computational/CramerShoup/ProjectiveHash.lean @@ -555,8 +555,8 @@ theorem uniform_projHash2_fiber_any_le (d : F.Desc) [DecidableEq (F.Carrier d)] have hor : (μ.map (fun τ₂ => (q :: qs).any (fun q' => hit q' τ₂))) true ≤ (μ.map (hit q)) true + (μ.map (fun τ₂ => qs.any (fun q' => hit q' τ₂))) true := by - simp only [List.any_cons, PMF.map_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun τ₂ => ?_ + simp only [List.any_cons, PMF.map_apply, ← tsum_add] + refine tsum_le_tsum fun τ₂ => ?_ cases hit q τ₂ <;> cases qs.any (fun q' => hit q' τ₂) <;> simp have hsingle : (μ.map (hit q)) true = (F.order d : ℝ≥0∞)⁻¹ := by have heq : diff --git a/Complexity/Crypto/Computational/CramerShoup/Scheme.lean b/Complexity/Crypto/Computational/CramerShoup/Scheme.lean index 58a4a0937..0e6744c8c 100644 --- a/Complexity/Crypto/Computational/CramerShoup/Scheme.lean +++ b/Complexity/Crypto/Computational/CramerShoup/Scheme.lean @@ -1008,7 +1008,7 @@ private theorem tsum_and_isCramerShoupBadDH_le rw [hpre, Bool.true_and, ← hvElem, ← hwElem, encodeProjHash2_encode, he_equiv τ] rw [hbad_eq] cases cond <;> cases F.encode d (e_equiv τ • F.generator d) == parts.2.2.2 <;> simp - exact (ENNReal.tsum_le_tsum fun τ => + exact (tsum_le_tsum fun τ => mul_le_mul_of_nonneg_left (hle τ) zero_le).trans (F.tsum_uniform_smul_generator_eq e_equiv parts.2.2.2) · simp only [Bool.not_eq_true] at hpre diff --git a/Complexity/Crypto/Computational/DiffieHellman.lean b/Complexity/Crypto/Computational/DiffieHellman.lean index 00a002fc8..2f8a35c45 100644 --- a/Complexity/Crypto/Computational/DiffieHellman.lean +++ b/Complexity/Crypto/Computational/DiffieHellman.lean @@ -173,14 +173,14 @@ theorem succProbability_discreteLogGame_le_cdhGame (setup : ℕ → PMF F.Desc) simp only [discreteLogGame, cdhGame, dlToCDHAdversary, ProbComp.eval_bind, ProbComp.eval_pure, PMF.monad_bind_eq_bind, PMF.monad_pure_eq_pure, PMF.bind_bind, PMF.pure_bind] rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ zero_le rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le rw [← PMF.bind_const (PMF.uniformOfFintype (ZMod (F.order d))) ((ProbComp.eval (adversary n (d, F.encode d (α • F.generator d)))).bind _)] rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le - refine ENNReal.tsum_le_tsum fun guess => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun guess => mul_le_mul_of_nonneg_left ?_ zero_le by_cases hα : (ofBits guess : ZMod (F.order d)) = α · simp only [hα, F.smul_encode, smul_smul, decide_true, le_rfl] · simp only [hα, decide_false, PMF.pure_apply, Bool.true_eq_false, ↓reduceIte, zero_le] @@ -301,7 +301,7 @@ theorem ddhIdealGame_cdhToDDHDistinguisher_true_le (setup : ℕ → PMF F.Desc) fun guess => PMF.pure (decide (guess = F.encode d (γ • F.generator d)))) true ≤ ∑' d, setup n d * (m : ℝ≥0∞)⁻¹ := by - refine ENNReal.tsum_le_tsum fun d => ?_ + refine tsum_le_tsum fun d => ?_ by_cases hd : d ∈ (setup n).support · refine mul_le_mul_of_nonneg_left ?_ zero_le have hord : (F.order d : ℝ≥0∞)⁻¹ ≤ (m : ℝ≥0∞)⁻¹ := @@ -315,8 +315,8 @@ theorem ddhIdealGame_cdhToDDHDistinguisher_true_le (setup : ℕ → PMF F.Desc) PMF.pure (decide (guess = F.encode d (γ • F.generator d)))) true ≤ ∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * ∑' β, PMF.uniformOfFintype (ZMod (F.order d)) β * (m : ℝ≥0∞)⁻¹ := by - refine ENNReal.tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le - refine ENNReal.tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun α => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun β => mul_le_mul_of_nonneg_left ?_ zero_le rw [PMF.bind_comm, PMF.bind_apply] calc ∑' guess, ProbComp.eval (adversary n (d, F.encode d (α • F.generator d), F.encode d (β • F.generator d))) guess * @@ -325,7 +325,7 @@ theorem ddhIdealGame_cdhToDDHDistinguisher_true_le (setup : ℕ → PMF F.Desc) ≤ ∑' guess, ProbComp.eval (adversary n (d, F.encode d (α • F.generator d), F.encode d (β • F.generator d))) guess * (m : ℝ≥0∞)⁻¹ := by - refine ENNReal.tsum_le_tsum fun guess => + refine tsum_le_tsum fun guess => mul_le_mul_of_nonneg_left ((F.uniform_decide_eq_encode_le d guess).trans hord) zero_le _ = (m : ℝ≥0∞)⁻¹ := by rw [ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] diff --git a/Complexity/Crypto/Computational/DiffieHellman/Order.lean b/Complexity/Crypto/Computational/DiffieHellman/Order.lean index b0a3dbafa..478cd484b 100644 --- a/Complexity/Crypto/Computational/DiffieHellman/Order.lean +++ b/Complexity/Crypto/Computational/DiffieHellman/Order.lean @@ -138,7 +138,7 @@ private theorem expected_inv_order_le_two_mul_advantage_ddh (F : PrimeOrderGroup (((F.order d - 1 : ℕ) : ℝ≥0∞) * ((F.order d : ℝ≥0∞)⁻¹ * (F.order d : ℝ≥0∞)⁻¹))) := by dsimp only [ddhRealGame, ddhZeroDistinguisher] simp only [ProbComp.eval_pure, PMF.monad_bind_eq_bind, PMF.bind_apply] - rw [← ENNReal.tsum_add] + rw [← tsum_add] refine tsum_congr fun d => ?_ have hstep : (∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * @@ -164,7 +164,7 @@ private theorem expected_inv_order_le_two_mul_advantage_ddh (F : PrimeOrderGroup 2 * ∑' d, setup n d * (((F.order d - 1 : ℕ) : ℝ≥0∞) * ((F.order d : ℝ≥0∞)⁻¹ * (F.order d : ℝ≥0∞)⁻¹)) := by rw [← ENNReal.tsum_mul_left] - refine ENNReal.tsum_le_tsum fun d => ?_ + refine tsum_le_tsum fun d => ?_ rw [← mul_assoc, mul_comm (2 : ℝ≥0∞) (setup n d), mul_assoc] refine mul_le_mul_of_nonneg_left ?_ zero_le have hq2 : 2 ≤ F.order d := (F.prime_order d).two_le diff --git a/Complexity/Crypto/Computational/Encryption/Authenticated.lean b/Complexity/Crypto/Computational/Encryption/Authenticated.lean index a177011ae..951e29dce 100644 --- a/Complexity/Crypto/Computational/Encryption/Authenticated.lean +++ b/Complexity/Crypto/Computational/Encryption/Authenticated.lean @@ -1662,7 +1662,7 @@ private theorem ennreal_tsum_finsetSum {α β : Type*} induction s using Finset.induction_on with | empty => simp | insert b s hb ih => - simp only [Finset.sum_insert hb, ENNReal.tsum_add, ih] + simp only [Finset.sum_insert hb, tsum_add, ih] private theorem eval_ccaRejectWithBadOracle_false (E : SymmetricCipher) (n : ℕ) (k : List Bool) (bit : Bool) (request : Bool × List Bool) (queried : List (List Bool)) : @@ -1761,7 +1761,7 @@ private theorem bad_ccaRejectWithBad_le_sum_ciCombined {α : Type} (E : Symmetri (fun x => PMF.pure (!x.2.2.contains x.2.1.2.2 && (E.decrypt n k x.2.1.2.2).isSome)) true := - ENNReal.tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (ih a queried) zero_le + tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (ih a queried) zero_le _ = _ := by simp only [Finset.mul_sum] rw [ennreal_tsum_finsetSum] @@ -1818,7 +1818,7 @@ private theorem bad_ccaRejectWithBad_le_sum_ciCombined {α : Type} (E : Symmetri PMF.pure (!x.2.2.contains x.2.1.2.2 && (E.decrypt n k x.2.1.2.2).isSome)) true := - ENNReal.tsum_le_tsum fun ct => + tsum_le_tsum fun ct => mul_le_mul_of_nonneg_left (ih ct (ct :: queried)) zero_le _ = _ := by simp only [Finset.mul_sum] @@ -1993,8 +1993,8 @@ private theorem advantage_ccaGame_ccaRejectGame_le_sum (E : SymmetricCipher) (ct ProbOracleComp.eval_run_interpret, hrejEq] exact PMF.bind_apply _ _ _ have hle1 : pCCA ≤ pRej + pSum := by - rw [hCCA_eq, hRej_eq, hsum_eq, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun k => ?_ + rw [hCCA_eq, hRej_eq, hsum_eq, ← tsum_add] + refine tsum_le_tsum fun k => ?_ rw [← mul_add] by_cases hk : k ∈ (ProbComp.eval (E.keyGen n)).support · refine mul_le_mul_of_nonneg_left ?_ zero_le @@ -2007,8 +2007,8 @@ private theorem advantage_ccaGame_ccaRejectGame_le_sum (E : SymmetricCipher) (ct rw [eval_ciGame_ccaToCIAtIndexAdversary_key E ctBound j bit adversary n k hk hctBound] · rw [(PMF.apply_eq_zero_iff _ _).mpr hk, zero_mul, zero_mul] have hle2 : pRej ≤ pCCA + pSum := by - rw [hCCA_eq, hRej_eq, hsum_eq, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun k => ?_ + rw [hCCA_eq, hRej_eq, hsum_eq, ← tsum_add] + refine tsum_le_tsum fun k => ?_ rw [← mul_add] by_cases hk : k ∈ (ProbComp.eval (E.keyGen n)).support · refine mul_le_mul_of_nonneg_left ?_ zero_le diff --git a/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean b/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean index d8d240fb8..e33925650 100644 --- a/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean +++ b/Complexity/Crypto/Computational/Encryption/EncryptThenMAC.lean @@ -1629,7 +1629,7 @@ private theorem step_bad_prfCipherForgetfulWithBadOracle_le (n : ℕ) (bit : Boo {x | x.2.2 = true} ≤ ∑' r, (BitString.uniformList n) r * {r : List Bool | (cache.lookup r).isSome}.indicator 1 r := by - refine ENNReal.tsum_le_tsum fun r => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun r => mul_le_mul_of_nonneg_left ?_ zero_le by_cases hr : (cache.lookup r).isSome = true · rw [Set.indicator_of_mem (show r ∈ {r : List Bool | (cache.lookup r).isSome} from hr), Pi.one_apply] @@ -1697,7 +1697,7 @@ private theorem bad_prfCipherForgetfulWithBadOracle_le {α : Type} (n : ℕ) (bi (cont y.1)).run y.2).toOuterMeasure {x | x.2.2 = true} ≤ ∑' y, ProbOracleComp.handler (fun m => StateT.lift (ProbComp.unifOracle m)) (prfCipherForgetfulWithBadOracle n bit) (Sum.inl m) (cache, false) y * B := by - refine ENNReal.tsum_le_tsum fun y => ?_ + refine tsum_le_tsum fun y => ?_ by_cases hy : y ∈ (ProbOracleComp.handler (fun m => StateT.lift (ProbComp.unifOracle m)) (prfCipherForgetfulWithBadOracle n bit) (Sum.inl m) (cache, false)).support @@ -1749,10 +1749,10 @@ private theorem bad_prfCipherForgetfulWithBadOracle_le {α : Type} (n : ℕ) (bi ≤ ∑' y, prfCipherForgetfulWithBadOracle n bit request (cache, false) y * ({x : List Bool × (List (List Bool × List Bool) × Bool) | x.2.2 = true}.indicator 1 y + K) := - ENNReal.tsum_le_tsum hterm + tsum_le_tsum hterm _ = (prfCipherForgetfulWithBadOracle n bit request (cache, false)).toOuterMeasure {x | x.2.2 = true} + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun y => ?_ diff --git a/Complexity/Crypto/Computational/GMW/Soundness.lean b/Complexity/Crypto/Computational/GMW/Soundness.lean index 7841dd8ec..99dcd2c2f 100644 --- a/Complexity/Crypto/Computational/GMW/Soundness.lean +++ b/Complexity/Crypto/Computational/GMW/Soundness.lean @@ -562,7 +562,7 @@ theorem acceptProb_gmwVerifier_le {α : Type} {generator : List Bool → List Bo P.init x * ProbOracleComp.acceptProbWith P.respond (step2 r x.1) x.2 ≤ ∑' x, ProbOracleComp.runWith P.respond (List.take (K * m) <$> ProbOracleComp.query ((), r)) P.init x * (1 / 8) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_right (hstep2_le r x.1 x.2 hbind) _ + tsum_le_tsum fun x => mul_le_mul_right (hstep2_le r x.1 x.2 hbind) _ _ = (∑' x, ProbOracleComp.runWith P.respond (List.take (K * m) <$> ProbOracleComp.query ((), r)) P.init x) * (1 / 8) := ENNReal.tsum_mul_right @@ -591,9 +591,9 @@ theorem acceptProb_gmwVerifier_le {α : Type} {generator : List Bool → List Bo calc ∑' r, BitString.uniformList m r * acceptProb (step1 r) P ≤ ∑' r, (({r | ¬ NaorBinding generator s r}.indicator (BitString.uniformList m) r) + - BitString.uniformList m r * (1 / 8)) := ENNReal.tsum_le_tsum hterm_le + BitString.uniformList m r * (1 / 8)) := tsum_le_tsum hterm_le _ = (∑' r, {r | ¬ NaorBinding generator s r}.indicator (BitString.uniformList m) r) + - ∑' r, BitString.uniformList m r * (1 / 8) := ENNReal.tsum_add + ∑' r, BitString.uniformList m r * (1 / 8) := tsum_add _ = naorBindingProb generator length s + 1 / 8 := by rw [← PMF.toOuterMeasure_apply, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] rfl diff --git a/Complexity/Crypto/Computational/MAC.lean b/Complexity/Crypto/Computational/MAC.lean index 5b35fad27..5d8f4e36f 100644 --- a/Complexity/Crypto/Computational/MAC.lean +++ b/Complexity/Crypto/Computational/MAC.lean @@ -797,7 +797,7 @@ private theorem pmf_bind_pure_bool_le {α : Type} (p : PMF α) (f g : α → Boo (p.bind (fun a => PMF.pure (f a))) true ≤ (p.bind (fun a => PMF.pure (g a))) true := by rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ p.support · refine mul_le_mul_of_nonneg_left ?_ zero_le cases hf : f a @@ -816,7 +816,7 @@ theorem succProbability_eufCmaGame_le_sufCmaGame (M : MACScheme) simp only [Game.succProbability, sufCmaGame, ProbComp.eval_bind, ProbComp.eval_pure] refine ENNReal.toNNReal_mono (PMF.apply_ne_top _ true) ?_ rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun sk => ?_ + refine tsum_le_tsum fun sk => ?_ refine mul_le_mul_of_nonneg_left (pmf_bind_pure_bool_le _ _ _ fun out _ hout => ?_) zero_le simp only [Bool.and_eq_true, Bool.not_eq_true'] at hout ⊢ refine ⟨⟨hout.1.1, ?_⟩, hout.2⟩ @@ -1450,7 +1450,7 @@ theorem succProbability_prfIdealGame_macToPrfDistinguisher_le (adversary : Forge zero_le] have htotal : (phase1.bind phase2) true ≤ (1 : ℝ≥0∞) / 2 ^ n := by rw [PMF.bind_apply] - refine (ENNReal.tsum_le_tsum fun a => ?_).trans + refine (tsum_le_tsum fun a => ?_).trans (PMF.tsum_mul_le_of_forall_le phase1 fun _ => le_rfl) by_cases ha : a ∈ phase1.support · exact mul_le_mul_right (hstep_bound a ha) _ @@ -2012,7 +2012,7 @@ theorem succProbability_eufCmaGame_le_sufCmaGame (M : NonceMACScheme) simp only [Game.succProbability, eufCmaGame, sufCmaGame, ProbComp.eval_bind, ProbComp.eval_pure] refine ENNReal.toNNReal_mono (PMF.apply_ne_top _ true) ?_ rw [PMF.bind_apply, PMF.bind_apply] - refine ENNReal.tsum_le_tsum fun sk => ?_ + refine tsum_le_tsum fun sk => ?_ refine mul_le_mul_of_nonneg_left (MACScheme.pmf_bind_pure_bool_le _ _ _ fun out _ hout => ?_) zero_le simp only [Bool.and_eq_true, Bool.not_eq_true'] at hout ⊢ diff --git a/Complexity/Crypto/Computational/NaorReingold/Security.lean b/Complexity/Crypto/Computational/NaorReingold/Security.lean index ad05595ed..bcc1e4022 100644 --- a/Complexity/Crypto/Computational/NaorReingold/Security.lean +++ b/Complexity/Crypto/Computational/NaorReingold/Security.lean @@ -2995,7 +2995,7 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} refine ne_of_lt (lt_of_le_of_lt ?_ ENNReal.one_lt_top) calc (∑' d, setup n d * (F.order d : ℝ≥0∞)⁻¹) ≤ ∑' d, setup n d * 1 := by - refine ENNReal.tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ (by positivity) + refine tsum_le_tsum fun d => mul_le_mul_of_nonneg_left ?_ (by positivity) exact ENNReal.inv_le_one.mpr (by exact_mod_cast (F.prime_order d).pos) _ = 1 := by simp [(setup n).tsum_coe] have hε_pos : 0 < ε n := by @@ -3007,7 +3007,7 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} ENNReal.mul_pos hd₀_ne hord_ne exact ENNReal.toNNReal_pos (ne_of_gt (lt_of_lt_of_le hterm_pos - (ENNReal.le_tsum (f := fun d => setup n d * (F.order d : ℝ≥0∞)⁻¹) d₀))) + (le_tsum (f := fun d => setup n d * (F.order d : ℝ≥0∞)⁻¹) d₀))) hexp_top let τ : ℝ≥0 := NNReal.sqrt (ε n) have hτ_pos : 0 < τ := NNReal.sqrt_pos.mpr hε_pos @@ -3079,7 +3079,7 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} have hsum_err : (∑' d, setup n d * errTerm d) = ((lhlBound n : ℝ≥0) : ℝ≥0∞) := by dsimp only [errTerm, lhlBound] - simp_rw [mul_add, ENNReal.tsum_add, ← mul_assoc] + simp_rw [mul_add, tsum_add, ← mul_assoc] rw [ENNReal.tsum_mul_right, ENNReal.tsum_mul_right, (setup n).tsum_coe, one_mul, ENNReal.coe_add] congr 1 @@ -3095,8 +3095,8 @@ theorem DDH.negligible_naorReingoldHashedGame_const {F : PrimeOrderGroupFamily} (hfg : ∀ d ∈ (setup n).support, f d true ≤ g d true + ((Game.advantage (g1 d) (g2 d) : ℝ≥0) : ℝ≥0∞)) : ((setup n).bind f) true ≤ ((setup n).bind g) true + ((lhlBound n : ℝ≥0) : ℝ≥0∞) := by - rw [← hsum_err, PMF.bind_apply, PMF.bind_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun d => ?_ + rw [← hsum_err, PMF.bind_apply, PMF.bind_apply, ← tsum_add] + refine tsum_le_tsum fun d => ?_ by_cases hd : d ∈ (setup n).support · rw [← mul_add] exact mul_le_mul_of_nonneg_left ((hfg d hd).trans (add_le_add le_rfl (hpoint d hd))) diff --git a/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean b/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean index 3c997b6a5..2350d6382 100644 --- a/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean +++ b/Complexity/Crypto/Computational/OneWay/LiuPassConverse.lean @@ -610,7 +610,7 @@ private theorem heuristic_fail_mul_le_family_fail {t : ℕ → ℕ} (ht : ∀ n, rw [hfailA_eq] dsimp only [failH] rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun coins => ?_ + refine tsum_le_tsum fun coins => ?_ split_ifs with h1 h2 · rfl · exfalso diff --git a/Complexity/Crypto/Computational/OneWayToSignature.lean b/Complexity/Crypto/Computational/OneWayToSignature.lean index ce784ba7d..a3cc11acb 100644 --- a/Complexity/Crypto/Computational/OneWayToSignature.lean +++ b/Complexity/Crypto/Computational/OneWayToSignature.lean @@ -663,7 +663,7 @@ private theorem map_bool_apply_true {α : Type*} (p : PMF α) (P : α → Bool) private theorem ennreal_tsum_finsetSum {α ι : Type*} (s : Finset ι) (g : ι → α → ℝ≥0∞) : (∑' a, ∑ i ∈ s, g i a) = ∑ i ∈ s, ∑' a, g i a := - Summable.tsum_finsetSum fun _ _ => ENNReal.summable + Summable.tsum_finsetSum fun _ _ => CompleteLattice.summable /-- **Lamport's one-time signature theorem**: for any one-way function `f`, `lamportScheme f` is existentially unforgeable under one-time adaptive chosen-message attack (`OneTimeEUFCMA`). -/ @@ -817,7 +817,7 @@ theorem OneWay.oneTimeEUFCMA_lamport {f : List Bool → List Bool} (hf : OneWay ≤ ∑' out, ∑ j : Fin (dyadicSize (2 * n)), if j.val < 2 * n then runWith (pkOf w) (dummySkOf w j.val) out * (if invWin w j.val out then 1 else 0) - else 0 := ENNReal.tsum_le_tsum (hpointwise w hw) + else 0 := tsum_le_tsum (hpointwise w hw) _ = ∑ j : Fin (dyadicSize (2 * n)), ∑' out, if j.val < 2 * n then runWith (pkOf w) (dummySkOf w j.val) out * (if invWin w j.val out then 1 else 0) @@ -845,7 +845,7 @@ theorem OneWay.oneTimeEUFCMA_lamport {f : List Bool → List Bool} (hf : OneWay if j.val < 2 * n then ((runWith (pkOf w) (dummySkOf w j.val)).map (invWin w j.val)) true else 0 := by - refine ENNReal.tsum_le_tsum fun w => ?_ + refine tsum_le_tsum fun w => ?_ by_cases hw : w ∈ μ.support · exact mul_le_mul_right (hw_sum w hw) (μ w) · simp [(PMF.apply_eq_zero_iff μ w).mpr hw] diff --git a/Complexity/Crypto/Computational/PRGToOneWay.lean b/Complexity/Crypto/Computational/PRGToOneWay.lean index 1fb8c7c47..70e7f7aea 100644 --- a/Complexity/Crypto/Computational/PRGToOneWay.lean +++ b/Complexity/Crypto/Computational/PRGToOneWay.lean @@ -107,7 +107,7 @@ theorem succProbability_uniformList_test_le {generator : List Bool → List Bool split_ifs with hr · exact PMF.coe_le_one _ _ simp only [test, ProbComp.eval_bind, ProbComp.eval_pure, PMF.bind_apply, PMF.pure_apply] - refine le_of_eq (ENNReal.tsum_eq_zero.2 fun x => ?_) + refine le_of_eq (tsum_eq_zero_iff.2 fun x => ?_) have hne : generator x ≠ List.ofFn r := fun hx => hr <| mem_image.2 ⟨x, hP x (by rw [← hlen x, hx, List.length_ofFn]), hx⟩ simp [hne] diff --git a/Complexity/Crypto/Computational/PedersenCommitment.lean b/Complexity/Crypto/Computational/PedersenCommitment.lean index e77853f5a..09842b79b 100644 --- a/Complexity/Crypto/Computational/PedersenCommitment.lean +++ b/Complexity/Crypto/Computational/PedersenCommitment.lean @@ -405,11 +405,11 @@ theorem pedersenHidingGame_apply_true_le (setup : ℕ → PMF F.Desc) ≤ ∑' α, (PMF.uniformOfFintype (ZMod (F.order d)) α * F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n true + if α = 0 then (F.order d : ℝ≥0∞)⁻¹ else 0) := - ENNReal.tsum_le_tsum hα + tsum_le_tsum hα _ = (∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n true) + (F.order d : ℝ≥0∞)⁻¹ := by - rw [ENNReal.tsum_add, tsum_ite_eq] + rw [tsum_add, tsum_ite_eq] _ ≤ (∑' α, PMF.uniformOfFintype (ZMod (F.order d)) α * F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n true) + (m : ℝ≥0∞)⁻¹ := add_le_add le_rfl hord @@ -423,7 +423,7 @@ theorem pedersenHidingGame_apply_true_le (setup : ℕ → PMF F.Desc) (((PMF.uniformOfFintype (ZMod (F.order d))).bind fun α => F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n) true + (m : ℝ≥0∞)⁻¹) := by - refine ENNReal.tsum_le_tsum fun d => ?_ + refine tsum_le_tsum fun d => ?_ by_cases hd : d ∈ (setup n).support · exact mul_le_mul_of_nonneg_left (hd_bound d hd) zero_le · have hd0 : setup n d = 0 := by rwa [PMF.mem_support_iff, not_not] at hd @@ -432,7 +432,7 @@ theorem pedersenHidingGame_apply_true_le (setup : ℕ → PMF F.Desc) ((PMF.uniformOfFintype (ZMod (F.order d))).bind fun α => F.pedersenScheme.hidingForKey ⟨d, α • F.generator d⟩ adversary b₂ n) true) + (m : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, (setup n).tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, (setup n).tsum_coe, one_mul] /-- **Unconditional hiding bound for Pedersen commitments**: when every supported group at parameter `n` has order at least `m > 0`, any (even computationally unbounded) diff --git a/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean b/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean index 9a46827f4..ffd5f9b0b 100644 --- a/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean +++ b/Complexity/Crypto/Computational/Pseudoentropy/MaskedExtraction.lean @@ -109,7 +109,7 @@ theorem maskedSample_mass_ge {α β : Type} (source : ProbComp α) rw [← ENNReal.toReal_mul] apply ENNReal.toReal_mono (PMF.apply_ne_top _ _) rw [PMF.bind_apply] - exact ENNReal.le_tsum (f := fun x => ProbComp.eval source x * + exact le_tsum (f := fun x => ProbComp.eval source x * ((ProbComp.eval (maskBit (mask x) (truth x))).map (observe x, ·)) (observe value, bit)) value exact (mul_le_mul (hsource value hvalue) (hbitMass.trans hmap) diff --git a/Complexity/Crypto/Game.lean b/Complexity/Crypto/Game.lean index e289aabc6..4f37d2714 100644 --- a/Complexity/Crypto/Game.lean +++ b/Complexity/Crypto/Game.lean @@ -295,16 +295,16 @@ theorem advantage_bind_le {α : Type*} (d : PMF α) (game1 game2 : α → Game) rw [PMF.bind_apply, PMF.bind_apply] calc ∑' a, d a * game1 a true ≤ ∑' a, d a * (game2 a true + ENNReal.ofReal ε) := - ENNReal.tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h1 a) zero_le + tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h1 a) zero_le _ = (∑' a, d a * game2 a true) + ENNReal.ofReal ε := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] have hsum2 : (d.bind game2) true ≤ (d.bind game1) true + ENNReal.ofReal ε := by rw [PMF.bind_apply, PMF.bind_apply] calc ∑' a, d a * game2 a true ≤ ∑' a, d a * (game1 a true + ENNReal.ofReal ε) := - ENNReal.tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h2 a) zero_le + tsum_le_tsum fun a => mul_le_mul_of_nonneg_left (h2 a) zero_le _ = (∑' a, d a * game1 a true) + ENNReal.ofReal ε := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, d.tsum_coe, one_mul] have hadv := advantage_le_of_apply_le ENNReal.ofReal_ne_top hsum1 hsum2 rwa [ENNReal.toReal_ofReal hε0] at hadv @@ -324,8 +324,8 @@ theorem advantage_bind_uniformOfFintype_of_ne_le {α : Type*} [Fintype α] [None have hle₁ : ((PMF.uniformOfFintype α).bind g) true ≤ p true + (Fintype.card α : ℝ≥0∞)⁻¹ := by rw [← PMF.bind_const (PMF.uniformOfFintype α) p, ← hsingle, PMF.bind_apply, PMF.bind_apply, - PMF.bind_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun a => ?_ + PMF.bind_apply, ← tsum_add] + refine tsum_le_tsum fun a => ?_ rw [← mul_add] refine mul_le_mul_of_nonneg_left ?_ zero_le by_cases ha : a = a₀ @@ -335,8 +335,8 @@ theorem advantage_bind_uniformOfFintype_of_ne_le {α : Type*} [Fintype α] [None have hle₂ : p true ≤ ((PMF.uniformOfFintype α).bind g) true + (Fintype.card α : ℝ≥0∞)⁻¹ := by nth_rw 1 [← PMF.bind_const (PMF.uniformOfFintype α) p] - rw [← hsingle, PMF.bind_apply, PMF.bind_apply, PMF.bind_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun a => ?_ + rw [← hsingle, PMF.bind_apply, PMF.bind_apply, PMF.bind_apply, ← tsum_add] + refine tsum_le_tsum fun a => ?_ rw [← mul_add] refine mul_le_mul_of_nonneg_left ?_ zero_le by_cases ha : a = a₀ @@ -363,8 +363,8 @@ theorem bind_apply_le_add_bind_apply {β : Type*} (μ : PMF β) (μ.bind game2) b ≤ (μ.bind game1) b + (μ.bind err) b := by have h1 (f g : β → Game) (hle : ∀ x, f x b ≤ g x b + err x b) : (μ.bind f) b ≤ (μ.bind g) b + (μ.bind err) b := by - simp only [PMF.bind_apply, ← ENNReal.tsum_add, ← mul_add] - exact ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le + simp only [PMF.bind_apply, ← tsum_add, ← mul_add] + exact tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le exact ⟨h1 game1 game2 fun x => (h x).1, h1 game2 game1 fun x => (h x).2⟩ /-- Averaging pointwise acceptance-probability bounds `game1 x true ≤ game2 x true + err x true` diff --git a/Complexity/Crypto/Game/UntilBad.lean b/Complexity/Crypto/Game/UntilBad.lean index 52f3c2892..f80d1e46d 100644 --- a/Complexity/Crypto/Game/UntilBad.lean +++ b/Complexity/Crypto/Game/UntilBad.lean @@ -164,9 +164,9 @@ theorem runStatelessOracle_bind_le {T α : Type} (μ : PMF T) (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * g x) + (Q : ℝ≥0∞) * δ := by calc ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * f x ≤ ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * (g x + (Q : ℝ≥0∞) * δ) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hle x) zero_le _ = (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * g x) + (Q : ℝ≥0∞) * δ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] exact ⟨hsum _ _ (fun x => (ih x).1), hsum _ _ (fun x => (ih x).2)⟩ · simp only [Sum.elim_inr] have hinr₁ : @@ -184,13 +184,13 @@ theorem runStatelessOracle_bind_le {T α : Type} (μ : PMF T) simp only [PMF.bind_apply] calc ∑' τ, μ τ * f τ b ≤ ∑' τ, (μ τ * g τ b + μ τ * if isBad τ req.2 then 1 else 0) := by - refine ENNReal.tsum_le_tsum fun τ => ?_ + refine tsum_le_tsum fun τ => ?_ by_cases hb : isBad τ req.2 = true · simp only [hb, ↓reduceIte, mul_one] exact le_add_of_le_right (mul_le_of_le_one_right zero_le (PMF.coe_le_one _ _)) · simp only [Bool.not_eq_true] at hb rw [hfg τ hb]; simp [hb] - _ = (∑' τ, μ τ * g τ b) + ∑' τ, μ τ * if isBad τ req.2 then 1 else 0 := ENNReal.tsum_add + _ = (∑' τ, μ τ * g τ b) + ∑' τ, μ τ * if isBad τ req.2 then 1 else 0 := tsum_add _ ≤ (∑' τ, μ τ * g τ b) + δ := add_le_add le_rfl (hbad req.2) have hstep1 := hstep (fun τ => runStatelessOracle (o₁ τ) (k (o₁ τ req.2)) (next τ)) @@ -372,20 +372,20 @@ theorem runStatelessOracle_le_runFinderOracle {α : Type} ≤ ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * (runStatelessOracle o₂ (k x) next b + runFinderOracle o₂ isBad extract (k x) none isGood true) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).1 zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).1 zero_le _ = (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runStatelessOracle o₂ (k x) next b) + ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runFinderOracle o₂ isBad extract (k x) none isGood true := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] · calc ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runStatelessOracle o₂ (k x) next b ≤ ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * (runStatelessOracle o₁ (k x) next b + runFinderOracle o₂ isBad extract (k x) none isGood true) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).2 zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (ih x).2 zero_le _ = (∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runStatelessOracle o₁ (k x) next b) + ∑' x, PMF.uniformOfFintype (Fin (m + 1)) x * runFinderOracle o₂ isBad extract (k x) none isGood true := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] · rw [runStatelessOracle_query_inr, runStatelessOracle_query_inr, runFinderOracle_query_inr] by_cases hb : isBad req.2 = true · simp only [hb, ↓reduceIte] diff --git a/Complexity/Crypto/Primitives/PRG/NextBit.lean b/Complexity/Crypto/Primitives/PRG/NextBit.lean index 886b68555..76fe10070 100644 --- a/Complexity/Crypto/Primitives/PRG/NextBit.lean +++ b/Complexity/Crypto/Primitives/PRG/NextBit.lean @@ -121,9 +121,9 @@ private theorem bind_apply_add_of_forall {α β : Type*} (μ : PMF α) {f g h : {c : ℝ≥0∞} (hfg : ∀ x, f x b + g x b = h x b + c) : μ.bind f b + μ.bind g b = μ.bind h b + c := by simp only [PMF.bind_apply] - rw [← ENNReal.tsum_add] + rw [← tsum_add] simp_rw [← mul_add, hfg, mul_add] - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] + rw [tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] /-- **The distinguisher/predictor lemma** [Boneh and Shoup, Lemma 3.5][BonehShoup2023]. Draw `x` from `μ`. The guess `guessOfDistinguisher (d x)` equals `target x` with probability `1 / 2`, plus diff --git a/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean b/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean index b88c043a5..ca41828a6 100644 --- a/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean +++ b/Complexity/Crypto/Protocols/Cipher/EvenMansour.lean @@ -1955,7 +1955,7 @@ private theorem idealGame_apply_le_realGame_add_of_keyDist (keyDist : PMF (G × calc (∑' res : Bool × (Cache G × Cache G), M_ideal res * keyDist.toOuterMeasure {k | isBadEvenMansourKey res.2.1 res.2.2 k = true}) ≤ ∑' res : Bool × (Cache G × Cache G), M_ideal res * ε := by - refine ENNReal.tsum_le_tsum fun res => ?_ + refine tsum_le_tsum fun res => ?_ by_cases hmem : res ∈ M_ideal.support · refine mul_le_mul_of_nonneg_left (hbad_le res.2.1 res.2.2 ?_ ?_) zero_le · simpa using length_fst_idealLazyHandler_le hcall.1 hmem @@ -1968,14 +1968,14 @@ private theorem idealGame_apply_le_realGame_add_of_keyDist (keyDist : PMF (G × ≤ ∑' k : G × G, keyDist k * ((((OracleComp.interpret (realLazyHandler k) program).run []).map Prod.fst) b + M_ideal.toOuterMeasure {res | isBadEvenMansourKey res.2.1 res.2.2 k = true}) := - ENNReal.tsum_le_tsum fun k => mul_le_mul_of_nonneg_left (hpoint k) zero_le + tsum_le_tsum fun k => mul_le_mul_of_nonneg_left (hpoint k) zero_le _ = (∑' k : G × G, keyDist k * (((OracleComp.interpret (realLazyHandler k) program).run []).map Prod.fst) b) + (∑' k : G × G, keyDist k * M_ideal.toOuterMeasure {res | isBadEvenMansourKey res.2.1 res.2.2 k = true}) := by - simp_rw [mul_add, ENNReal.tsum_add] + simp_rw [mul_add, tsum_add] _ ≤ _ := add_le_add le_rfl hsum_bad private theorem advantage_le_of_forall_bool_le {g₁ g₂ : Game} {ε : ℝ≥0∞} (hε : ε ≠ ⊤) diff --git a/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean b/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean index ac6f10219..a46a9a06e 100644 --- a/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean +++ b/Complexity/Crypto/Protocols/Cipher/LubyRackoff.lean @@ -1013,7 +1013,7 @@ private theorem tsum_seed_toOuterMeasure_hasCollision_le {seed₁ : PMF K₁} {h ∑' x, idealRun x * ∑' k₁, ({k₁ : K₁ | hasUCollision (h₁ k₁) x.2 = true}.indicator seed₁ k₁) := by simp only [← ENNReal.tsum_mul_left] - rw [ENNReal.tsum_comm] + rw [tsum_comm] refine tsum_congr fun x => tsum_congr fun k₁ => ?_ by_cases hx : hasUCollision (h₁ k₁) x.2 = true · rw [Set.indicator_of_mem hx, Set.indicator_of_mem hx, mul_comm] @@ -1022,7 +1022,7 @@ private theorem tsum_seed_toOuterMeasure_hasCollision_le {seed₁ : PMF K₁} {h calc ∑' x, idealRun x * ∑' k₁, ({k₁ : K₁ | hasUCollision (h₁ k₁) x.2 = true}.indicator seed₁ k₁) ≤ ∑' x, idealRun x * (Q.choose 2 * ε) := by - refine ENNReal.tsum_le_tsum fun x => ?_ + refine tsum_le_tsum fun x => ?_ by_cases hmem : x ∈ idealRun.support · refine mul_le_mul_of_nonneg_left ?_ zero_le rw [← PMF.toOuterMeasure_apply] @@ -1045,10 +1045,10 @@ private theorem tsum_seed_toOuterMeasure_hasCollision_le {seed₁ : PMF K₁} {h ≤ ∑' k₁, seed₁ k₁ * (idealRun.toOuterMeasure {x | hasUCollision (h₁ k₁) x.2 = true} + idealRun.toOuterMeasure {x | hasVCollision x.2 = true}) := - ENNReal.tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (hsplit k₁) zero_le + tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (hsplit k₁) zero_le _ = (∑' k₁, seed₁ k₁ * idealRun.toOuterMeasure {x | hasUCollision (h₁ k₁) x.2 = true}) + (∑' k₁, seed₁ k₁ * idealRun.toOuterMeasure {x | hasVCollision x.2 = true}) := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] _ ≤ Q.choose 2 * ε + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ := add_le_add hu hv /-- For every fixed `f₁`, the acceptance probability of `feistel3FixedGame f₁ program` and @@ -1092,13 +1092,13 @@ theorem feistel3HashIdealGame_apply_le {seed₁ : PMF K₁} {h₁ : K₁ → G ≤ ∑' k₁, seed₁ k₁ * (randomFunctionGame program b + ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true}) := - ENNReal.tsum_le_tsum fun k₁ => + tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (feistel3FixedGame_apply_le (h₁ k₁) program b) zero_le _ = randomFunctionGame program b + ∑' k₁, seed₁ k₁ * ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, seed₁.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, seed₁.tsum_coe, one_mul] _ ≤ randomFunctionGame program b + (Q.choose 2 * ε + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹) := add_le_add le_rfl (tsum_seed_toOuterMeasure_hasCollision_le hh₁ hQ) @@ -1120,14 +1120,14 @@ theorem randomFunctionGame_apply_le_feistel3HashIdealGame {seed₁ : PMF K₁} { ≤ ∑' k₁, seed₁ k₁ * (feistel3FixedGame (h₁ k₁) program b + ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true}) := - ENNReal.tsum_le_tsum fun k₁ => + tsum_le_tsum fun k₁ => mul_le_mul_of_nonneg_left (randomFunctionGame_apply_le_feistel3FixedGame (h₁ k₁) program b) zero_le _ = (∑' k₁, seed₁ k₁ * feistel3FixedGame (h₁ k₁) program b) + ∑' k₁, seed₁ k₁ * ((OracleComp.interpret (memoize (fun _ : G × G => PMF.uniformOfFintype (G × G))) program).run []).toOuterMeasure {x | hasCollision (h₁ k₁) x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] _ ≤ (∑' k₁, seed₁ k₁ * feistel3FixedGame (h₁ k₁) program b) + (Q.choose 2 * ε + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹) := add_le_add le_rfl (tsum_seed_toOuterMeasure_hasCollision_le hh₁ hQ) @@ -1619,7 +1619,7 @@ theorem le_idealStrongGame_feistel3CCAAttack_false {l₀ l₁ l₂ r₀ : G} _ ≤ ∑' π : Equiv.Perm (G × G), PMF.uniformOfFintype (Equiv.Perm (G × G)) π * OracleComp.interpret (feistelStrongOracle π) (feistel3CCAAttack l₀ l₁ r₀) false := - ENNReal.le_tsum (f := fun π : Equiv.Perm (G × G) => + le_tsum (f := fun π : Equiv.Perm (G × G) => PMF.uniformOfFintype (Equiv.Perm (G × G)) π * OracleComp.interpret (feistelStrongOracle π) (feistel3CCAAttack l₀ l₁ r₀) false) π₀ @@ -1875,9 +1875,9 @@ theorem idealStrongGame_feistel3CCAAttack_true_le {l₀ l₁ : G} (r₀ : G) (h0 (fun x => decide (-vw₀.1 + x.1.2 = -vw₁.1 + r₀))) true ≤ ∑' vw₁, ((PMF.uniformOfFintype (G × G)) vw₁ * (Fintype.card G : ℝ≥0∞)⁻¹ + ({q₃} : Set (G × G)).indicator (PMF.uniformOfFintype (G × G)) vw₁) := - ENNReal.tsum_le_tsum hvw₁ + tsum_le_tsum hvw₁ _ = (Fintype.card G : ℝ≥0∞)⁻¹ + (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + rw [tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, ← PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply_singleton, PMF.uniformOfFintype_apply] calc ∑' vw₀, (PMF.uniformOfFintype (G × G)) vw₀ * @@ -1888,7 +1888,7 @@ theorem idealStrongGame_feistel3CCAAttack_true_le {l₀ l₁ : G} (r₀ : G) (h0 (fun x => decide (-vw₀.1 + x.1.2 = -vw₁.1 + r₀))) true ≤ ∑' vw₀, (PMF.uniformOfFintype (G × G)) vw₀ * ((Fintype.card G : ℝ≥0∞)⁻¹ + (Fintype.card (G × G) : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun vw₀ => mul_le_mul_of_nonneg_left (hvw₀ vw₀) zero_le + tsum_le_tsum fun vw₀ => mul_le_mul_of_nonneg_left (hvw₀ vw₀) zero_le _ = (Fintype.card G : ℝ≥0∞)⁻¹ + (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by rw [ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] have hchoose : ((Nat.choose 3 2 : ℕ) : ℝ≥0∞) = 3 := rfl @@ -3043,7 +3043,7 @@ private theorem tsum_toOuterMeasure_of_pointwise_le {α β : Type} (μ : PMF α) (∑' a, μ a * ∑' y, {y | P a y}.indicator d y) = ∑' y, d y * ∑' a, {a | P a y}.indicator μ a := by simp only [← ENNReal.tsum_mul_left] - rw [ENNReal.tsum_comm] + rw [tsum_comm] refine tsum_congr fun y => tsum_congr fun a => ?_ by_cases h : P a y · rw [Set.indicator_of_mem h, Set.indicator_of_mem h, mul_comm] @@ -3051,7 +3051,7 @@ private theorem tsum_toOuterMeasure_of_pointwise_le {α β : Type} (μ : PMF α) rw [hcomm] calc ∑' y, d y * ∑' a, {a | P a y}.indicator μ a ≤ ∑' y, d y * M := - ENNReal.tsum_le_tsum fun y => mul_le_mul_of_nonneg_left (by + tsum_le_tsum fun y => mul_le_mul_of_nonneg_left (by rw [← PMF.toOuterMeasure_apply] exact hM y) zero_le _ = M := by rw [ENNReal.tsum_mul_right, d.tsum_coe, one_mul] @@ -3155,14 +3155,14 @@ private theorem tsum_uniform_toOuterMeasure_hasCollision4_le d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasVCollision4 f₄ x.2 = true} + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun f₄ => + tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left (toOuterMeasure_hasCollision4_le f₁ f₄ hQ) zero_le _ = d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasUCollision f₁ x.2 = true} + (∑' f₄, unif f₄ * d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasVCollision4 f₄ x.2 = true}) + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] _ ≤ _ := add_le_add (add_le_add le_rfl (tsum_uniform_toOuterMeasure_hasVCollision4_le d Q)) le_rfl calc ∑' f₁, unif f₁ * ∑' f₄, unif f₄ * d.toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true} @@ -3171,12 +3171,12 @@ private theorem tsum_uniform_toOuterMeasure_hasCollision4_le {x | IsGoodCache4 Q x.2 ∧ hasUCollision f₁ x.2 = true} + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left (hinner f₁) zero_le + tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left (hinner f₁) zero_le _ = (∑' f₁, unif f₁ * d.toOuterMeasure {x | IsGoodCache4 Q x.2 ∧ hasUCollision f₁ x.2 = true}) + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] _ ≤ Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + Q.choose 2 * (Fintype.card G : ℝ≥0∞)⁻¹ + (Q.choose 2 : ℝ≥0∞) * (Fintype.card (G × G) : ℝ≥0∞)⁻¹ := @@ -3253,13 +3253,13 @@ theorem feistel4StrongRandomRoundsGame_apply_le (forgetVal + ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true}) := - ENNReal.tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left - (ENNReal.tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left + tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left + (tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left (feistel4StrongFixedGame_apply_le f₁ f₄ program b) zero_le) zero_le _ = forgetVal + ∑' f₁, unif f₁ * ∑' f₄, unif f₄ * ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, unif.tsum_coe, one_mul] _ ≤ _ := add_le_add le_rfl (tsum_uniform_toOuterMeasure_hasCollision4_le hQ') have hperm := RandomPermutation.forgetfulAnswer_bind_apply_le_uniformPerm hQ' PMF.pure b simp only [Probability.PMF.monad_bind_eq_bind, PMF.bind_pure] at hperm @@ -3303,14 +3303,14 @@ theorem idealStrongGame_apply_le_feistel4StrongRandomRoundsGame (feistel4StrongFixedGame f₁ f₄ program b + ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true}) := - ENNReal.tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left - (ENNReal.tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left + tsum_le_tsum fun f₁ => mul_le_mul_of_nonneg_left + (tsum_le_tsum fun f₄ => mul_le_mul_of_nonneg_left (forgetfulAnswer_apply_le_feistel4StrongFixedGame f₁ f₄ program b) zero_le) zero_le _ = (∑' f₁, unif f₁ * ∑' f₄, unif f₄ * feistel4StrongFixedGame f₁ f₄ program b) + ∑' f₁, unif f₁ * ∑' f₄, unif f₄ * ((OracleComp.interpret RandomPermutation.forgetfulAnswer prog).run []).toOuterMeasure {x | hasCollision4 f₁ f₄ x.2 = true} := by - simp only [mul_add, ENNReal.tsum_add] + simp only [mul_add, tsum_add] _ ≤ _ := add_le_add le_rfl (tsum_uniform_toOuterMeasure_hasCollision4_le hQ') have hperm := RandomPermutation.uniformPerm_bind_apply_le_forgetful hQ' PMF.pure b simp only [Probability.PMF.monad_bind_eq_bind, PMF.bind_pure] at hperm diff --git a/Complexity/Crypto/Protocols/MAC/CarterWegman.lean b/Complexity/Crypto/Protocols/MAC/CarterWegman.lean index dd32115b4..4b4f2bb30 100644 --- a/Complexity/Crypto/Protocols/MAC/CarterWegman.lean +++ b/Complexity/Crypto/Protocols/MAC/CarterWegman.lean @@ -3802,10 +3802,10 @@ theorem randomNonceSimHandler_not_nodup_le {α : Type} [Fintype N] [Nonempty N] PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true ≤ ∑' x, PMF.uniformOfFintype N x * ((PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le _ = ((PMF.uniformOfFintype N).bind fun x => PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, (PMF.uniformOfFintype N).tsum_coe, one_mul, PMF.bind_apply] _ ≤ (tr.length : ℝ≥0∞) / Fintype.card N + B := add_le_add hmem_le le_rfl _ = ((∑ i ∈ Finset.range (Q + 1), (tr.length + i) : ℕ) : ℝ≥0∞) / Fintype.card N := by @@ -4029,11 +4029,11 @@ theorem randomNonceStrongGame_le_sum_add_max {seed : PMF K} {hash : K → M → ≤ ∑' res, simPMF res * ((PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le + tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le _ = (simPMF.bind fun res => PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, PMF.bind_apply] _ ≤ ((∑ i ∈ Finset.range Q, i : ℕ) : ℝ≥0∞) / Fintype.card N + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by @@ -4485,10 +4485,10 @@ theorem probOracleComp_randomNonceSimHandler_not_nodup_le {α : Type} PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true ≤ ∑' x, PMF.uniformOfFintype N x * ((PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B) := - ENNReal.tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le + tsum_le_tsum fun x => mul_le_mul_of_nonneg_left (hx_bound x) zero_le _ = ((PMF.uniformOfFintype N).bind fun x => PMF.pure (decide (x ∈ tr.map (fun e => e.1.2)))) true + B := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, (PMF.uniformOfFintype N).tsum_coe, one_mul, PMF.bind_apply] _ ≤ (tr.length : ℝ≥0∞) / Fintype.card N + B := add_le_add hmem_le le_rfl _ = ((∑ i ∈ Finset.range (Q + 1), (tr.length + i) : ℕ) : ℝ≥0∞) / Fintype.card N := by @@ -4610,11 +4610,11 @@ theorem seed_uniform_probOracleComp_checkRandomNonceSUF_le {α : Type} ≤ ∑' res, simPMF res * ((PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹) := - ENNReal.tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le + tsum_le_tsum fun res => mul_le_mul_of_nonneg_left (hpoint res) zero_le _ = (simPMF.bind fun res => PMF.pure (!decide (res.2.map (fun e => e.1.2)).Nodup)) true + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, simPMF.tsum_coe, one_mul, PMF.bind_apply] _ ≤ ((∑ i ∈ Finset.range Q, i : ℕ) : ℝ≥0∞) / Fintype.card N + max ε (Fintype.card Y : ℝ≥0∞)⁻¹ := by diff --git a/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean b/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean index 2075b932a..39a83a0cb 100644 --- a/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean +++ b/Complexity/Crypto/Protocols/SigmaProtocol/FiatShamir.lean @@ -391,7 +391,7 @@ private theorem toOuterMeasure_flag_le (P : SigmaProtocol X Y T C Z) (key : X × {x | x.2.2 = true} ≤ ∑' ts, (P.commit key.1 key.2) ts * {ts : T × P.State | (st.1.1.lookup (m, ts.1)).isSome}.indicator 1 ts := by - refine ENNReal.tsum_le_tsum fun ts => mul_le_mul_of_nonneg_left ?_ zero_le + refine tsum_le_tsum fun ts => mul_le_mul_of_nonneg_left ?_ zero_le by_cases hts : (st.1.1.lookup (m, ts.1)).isSome · rw [Set.indicator_of_mem (show ts ∈ {ts : T × P.State | (st.1.1.lookup (m, ts.1)).isSome} from hts), Pi.one_apply] @@ -828,12 +828,12 @@ private theorem bind_apply_le_add {α β : Type} {μ : PMF α} {f g : α → PMF (μ.bind f) b ≤ (μ.bind g) b + K := by simp only [PMF.bind_apply] calc ∑' a, μ a * f a b ≤ ∑' a, μ a * (g a b + K) := by - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ μ.support · exact mul_le_mul_of_nonneg_left (h a ha) zero_le · rw [(PMF.apply_eq_zero_iff _ _).mpr ha, zero_mul, zero_mul] _ = ∑' a, μ a * g a b + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul] omit [DecidableEq C] in /-- **The game hops** for one key and one choice of the forger's coins: the experiment wins at diff --git a/Complexity/Languages/Probabilistic/BoundedUniform.lean b/Complexity/Languages/Probabilistic/BoundedUniform.lean index 6fa41f00f..a4166310d 100644 --- a/Complexity/Languages/Probabilistic/BoundedUniform.lean +++ b/Complexity/Languages/Probabilistic/BoundedUniform.lean @@ -187,7 +187,7 @@ private theorem retry_some (bound value : ℕ) (next : PMF (Option ℕ)) : simp [hib, hiv] change (∑' i, μ i * (if i < bound then PMF.pure (some i) else next) (some value)) = _ simp_rw [hterm] - rw [ENNReal.tsum_add] + rw [tsum_add] simp only [tsum_ite_eq] rw [show μ bound = (intervalFailure bound : ℝ≥0∞) from eval_sampleBoundedIndex_sentinel bound] by_cases hv : value < bound @@ -342,7 +342,7 @@ private theorem uniformFin_nat_apply (bound value : ℕ) [NeZero bound] : · intro i hi have : value ≠ i.val := fun h => hi (Fin.ext h.symm) simp [this] - · rw [ENNReal.tsum_eq_zero.mpr] + · rw [tsum_eq_zero_iff.mpr] · simp [hv] · intro i have : value ≠ i.val := by omega @@ -364,7 +364,7 @@ theorem eval_sampleUniformInterval_some_eq (bound attempts value : ℕ) [NeZero private theorem tsum_option_eq {α : Type} (f : Option α → ℝ≥0∞) : ∑' x, f x = f none + ∑' x, f (some x) := by rw [← (Equiv.optionEquivSumPUnit.{0, 0} α).symm.tsum_eq f] - rw [ENNReal.summable.tsum_sum ENNReal.summable] + rw [CompleteLattice.summable.tsum_sum CompleteLattice.summable] simp [add_comm] private theorem optional_map_density {α β : Type} (μ : PMF (Option α)) (ν : PMF α) diff --git a/Complexity/Languages/Probabilistic/Expect.lean b/Complexity/Languages/Probabilistic/Expect.lean index 3785b103e..6b22d82f8 100644 --- a/Complexity/Languages/Probabilistic/Expect.lean +++ b/Complexity/Languages/Probabilistic/Expect.lean @@ -70,7 +70,7 @@ theorem expect_zero (m : ProbComp α) : expect m (fun _ => 0) = 0 := theorem expect_add (m : ProbComp α) (f g : α → ℝ≥0∞) : expect m (fun a => f a + g a) = expect m f + expect m g := by - simp only [expect, mul_add, ENNReal.tsum_add] + simp only [expect, mul_add, tsum_add] theorem expect_mul_left (m : ProbComp α) (c : ℝ≥0∞) (f : α → ℝ≥0∞) : expect m (fun a => c * f a) = c * expect m f := by @@ -86,7 +86,7 @@ theorem expect_mul_right (m : ProbComp α) (f : α → ℝ≥0∞) (c : ℝ≥0 theorem expect_bind (m : ProbComp α) (k : α → ProbComp β) (f : β → ℝ≥0∞) : expect (m >>= k) f = expect m (fun a => expect (k a) f) := by simp only [expect, eval_bind, PMF.bind_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] + rw [tsum_comm] congr 1 funext a simp only [mul_assoc, ENNReal.tsum_mul_left] @@ -107,7 +107,7 @@ theorem expect_congr_on_support {m : ProbComp α} {f g : α → ℝ≥0∞} theorem expect_mono_on_support {m : ProbComp α} {f g : α → ℝ≥0∞} (h : ∀ a ∈ m.support, f a ≤ g a) : expect m f ≤ expect m g := by - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ m.support · exact mul_le_mul_right (h a ha) _ · have hzero : (eval m) a = 0 := (PMF.apply_eq_zero_iff (eval m) a).mpr (by @@ -129,7 +129,7 @@ theorem expect_eq_top_of_mem_support {m : ProbComp α} {f : α → ℝ≥0∞} { refine top_unique ?_ calc (⊤ : ℝ≥0∞) = (eval m) a * f a := by rw [hf, ENNReal.mul_top hpos] - _ ≤ expect m f := ENNReal.le_tsum (f := fun b => (eval m) b * f b) a + _ ≤ expect m f := le_tsum (f := fun b => (eval m) b * f b) a theorem expect_unifFin (n : ℕ) (f : Fin (n + 1) → ℝ≥0∞) : expect (unifFin n) f = (∑ k : Fin (n + 1), f k) / (n + 1 : ℝ≥0∞) := by diff --git a/Complexity/Languages/Probabilistic/Oracle/Interaction.lean b/Complexity/Languages/Probabilistic/Oracle/Interaction.lean index 6aa1215f3..d822a739c 100644 --- a/Complexity/Languages/Probabilistic/Oracle/Interaction.lean +++ b/Complexity/Languages/Probabilistic/Oracle/Interaction.lean @@ -168,7 +168,7 @@ theorem acceptProbWith_bind_ite_le (program : ProbOracleComp spec Bool) acceptProbWith oracle program s := by conv_rhs => rw [← bind_pure program, acceptProbWith_bind] rw [acceptProbWith_bind] - refine ENNReal.tsum_le_tsum fun x => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun x => mul_le_mul_right ?_ _ cases x.1 · simp only [Bool.false_eq_true, ↓reduceIte, le_refl] · simp only [↓reduceIte, acceptProbWith_pure] diff --git a/Complexity/Languages/Probabilistic/RandomPermutation.lean b/Complexity/Languages/Probabilistic/RandomPermutation.lean index f8b175e6d..153ac0a84 100644 --- a/Complexity/Languages/Probabilistic/RandomPermutation.lean +++ b/Complexity/Languages/Probabilistic/RandomPermutation.lean @@ -1175,9 +1175,9 @@ theorem toOuterMeasure_bad_choose_two_le {ι σ β : Type} {spec : ι → Type} calc ∑' y, (handler q s) y * ((OracleComp.interpret handler (cont y.1)).run y.2).toOuterMeasure {x | bad x.2} ≤ ∑' y, (handler q s) y * ({x : spec q × σ | bad x.2}.indicator 1 y + K) := - ENNReal.tsum_le_tsum hterm + tsum_le_tsum hterm _ = (handler q s).toOuterMeasure {x | bad x.2} + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun y => ?_ @@ -1413,8 +1413,8 @@ theorem toOuterMeasure_forgetfulAnswer_le_pairwise_and_length_le_add {β : Type} {x | (x.2.Pairwise (fun a b => a.1 ≠ b.1) ∧ x.2.Pairwise (fun a b => a.2 ≠ b.2) ∧ x.2.length ≤ Q) ∧ P x.2}) + q.toOuterMeasure {x | x.2.2 = true} - simp only [PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun ⟨a, c, b⟩ => ?_ + simp only [PMF.toOuterMeasure_apply, ← tsum_add] + refine tsum_le_tsum fun ⟨a, c, b⟩ => ?_ by_cases hmem : (a, (c, b)) ∈ q.support · by_cases hP : P c · cases b with diff --git a/Complexity/Languages/Probabilistic/UntilBad.lean b/Complexity/Languages/Probabilistic/UntilBad.lean index 7bcd7e202..ac26422b6 100644 --- a/Complexity/Languages/Probabilistic/UntilBad.lean +++ b/Complexity/Languages/Probabilistic/UntilBad.lean @@ -129,8 +129,8 @@ theorem bind_apply_le (h : IdenticalUntilBad first second bad) (program : Oracle ((interpret first program).run s).toOuterMeasure {x | bad x.2} := by classical simp only [PMF.bind_apply, PMF.toOuterMeasure_apply] - rw [← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun x => ?_ + rw [← tsum_add] + refine tsum_le_tsum fun x => ?_ by_cases hx : bad x.2 · have hbad : x ∈ {x : α × σ | bad x.2} := hx rw [Set.indicator_of_mem hbad] @@ -153,7 +153,7 @@ private theorem toOuterMeasure_bad_add_good (p : PMF (α × σ)) : p.toOuterMeasure {x | bad x.2} + p.toOuterMeasure {x | ¬bad x.2} = 1 := by classical simp only [PMF.toOuterMeasure_apply] - rw [← ENNReal.tsum_add, ← p.tsum_coe] + rw [← tsum_add, ← p.tsum_coe] refine tsum_congr fun x => ?_ by_cases hx : bad x.2 · rw [Set.indicator_of_mem (show x ∈ {x : α × σ | bad x.2} from hx), @@ -244,9 +244,9 @@ theorem toOuterMeasure_bad_le {handler : (q : ι) → StateT σ PMF (spec q)} {b calc ∑' y, (handler q s) y * ((interpret handler (cont y.1)).run y.2).toOuterMeasure {x | bad x.2} ≤ ∑' y, (handler q s) y * ({x : spec q × σ | bad x.2}.indicator 1 y + K) := - ENNReal.tsum_le_tsum hterm + tsum_le_tsum hterm _ = (handler q s).toOuterMeasure {x | bad x.2} + K := by - simp only [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, + simp only [mul_add, tsum_add, ENNReal.tsum_mul_right, PMF.tsum_coe, one_mul, PMF.toOuterMeasure_apply] congr 1 refine tsum_congr fun y => ?_ diff --git a/Complexity/Languages/Probabilistic/Vector.lean b/Complexity/Languages/Probabilistic/Vector.lean index 5e43aaaa5..4f3e7ead7 100644 --- a/Complexity/Languages/Probabilistic/Vector.lean +++ b/Complexity/Languages/Probabilistic/Vector.lean @@ -274,7 +274,7 @@ theorem eval_pi {m : ℕ} {α : Fin m → Type} [∀ i, Finite (α i)] funext i simpa using (congrFun hEq (Fin.succ i)).symm · intro a ha - simp only [mul_eq_zero, ENNReal.tsum_eq_zero] + simp only [mul_eq_zero, tsum_eq_zero_iff] right intro b rw [ite_eq_right] diff --git a/Complexity/Probability/Conditioning.lean b/Complexity/Probability/Conditioning.lean index faca6d699..340e70ef4 100644 --- a/Complexity/Probability/Conditioning.lean +++ b/Complexity/Probability/Conditioning.lean @@ -48,7 +48,7 @@ variable {α β : Type*} theorem map_fst_apply (joint : PMF (α × β)) (a : α) : (joint.map Prod.fst) a = ∑' b, joint (a, b) := by classical - rw [PMF.map_apply, ENNReal.tsum_prod'] + rw [PMF.map_apply, tsum_prod'] rw [tsum_eq_single a] · simp · intro x hx @@ -72,9 +72,9 @@ theorem conditionalSnd_apply (joint : PMF (α × β)) (a : α) conditionalSnd joint a b = joint (a, b) / (joint.map Prod.fst) a := by classical have hpost : a ∈ (joint.bind (fun pair => PMF.pure pair.1)).support := ha - simp only [conditionalSnd, hpost, dite_true, PMF.map_apply, ENNReal.tsum_prod', + simp only [conditionalSnd, hpost, dite_true, PMF.map_apply, tsum_prod', posteriorDist_apply, bind_pair_apply, PMF.pure_apply] - simp [ENNReal.tsum_prod', div_eq_mul_inv] + simp [tsum_prod', div_eq_mul_inv] /-- Multiplying by the marginal recovers the joint mass, also at zero-probability observations. -/ theorem marginal_mul_conditionalSnd (joint : PMF (α × β)) (a : α) (b : β) : diff --git a/Complexity/Probability/CorrelatedSampling.lean b/Complexity/Probability/CorrelatedSampling.lean index 5e9f61775..ae9ebd647 100644 --- a/Complexity/Probability/CorrelatedSampling.lean +++ b/Complexity/Probability/CorrelatedSampling.lean @@ -171,7 +171,7 @@ theorem map_firstIn_weightedOrderAux (w : Ω → ℝ≥0) : rcases o with _ | c · rw [PMF.bind_apply, PMF.map_apply] simp only [reduceCtorEq, ite_false, tsum_zero] - refine ENNReal.tsum_eq_zero.mpr fun a => ?_ + refine tsum_eq_zero_iff.mpr fun a => ?_ split_ifs <;> simp [PMF.map_apply] rw [PMF.bind_apply, tsum_fintype, hrhs] set du := weightedDraw w U hU0 diff --git a/Complexity/Probability/Forking.lean b/Complexity/Probability/Forking.lean index 960ea5e2e..b13827992 100644 --- a/Complexity/Probability/Forking.lean +++ b/Complexity/Probability/Forking.lean @@ -215,7 +215,7 @@ private theorem bind_match_apply {X : Type} (μ : PMF X) (index : X → Option ( | some j => decide (event j x))) true = ∑ j, (μ.bind fun x => PMF.pure (decide (index x = some j ∧ event j x))) true := by simp only [PMF.bind_apply, PMF.pure_apply] - rw [← Summable.tsum_finsetSum fun _ _ => ENNReal.summable] + rw [← Summable.tsum_finsetSum fun _ _ => CompleteLattice.summable] refine tsum_congr fun x => ?_ rw [← Finset.mul_sum] congr 1 diff --git a/Complexity/Probability/IID.lean b/Complexity/Probability/IID.lean index 74388f5b5..0a600cf4a 100644 --- a/Complexity/Probability/IID.lean +++ b/Complexity/Probability/IID.lean @@ -73,7 +73,7 @@ noncomputable def iid (μ : PMF α) : (k : ℕ) → PMF (Fin k → α) intro h exact absurd (by rw [h, Fin.tail_cons]) hw · intro a ha - rw [PMF.map_apply, ENNReal.tsum_eq_zero.mpr, mul_zero] + rw [PMF.map_apply, tsum_eq_zero_iff.mpr, mul_zero] intro w simp only [ite_eq_right_iff] intro h diff --git a/Complexity/Probability/MinEntropy.lean b/Complexity/Probability/MinEntropy.lean index 15120ffd6..29fd28b12 100644 --- a/Complexity/Probability/MinEntropy.lean +++ b/Complexity/Probability/MinEntropy.lean @@ -183,7 +183,7 @@ theorem le_minEntropy_bind (μ : PMF β) (f : β → PMF α) {k : ℝ} apply ENNReal.toReal_le_of_le_ofReal hk calc μ.bind f a = ∑' b, μ b * f b a := PMF.bind_apply μ f a - _ ≤ ∑' b, μ b * ENNReal.ofReal (2 ^ (-k)) := ENNReal.tsum_le_tsum hcomponent + _ ≤ ∑' b, μ b * ENNReal.ofReal (2 ^ (-k)) := tsum_le_tsum hcomponent _ = ENNReal.ofReal (2 ^ (-k)) := by rw [ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] /-! ### Weak random sources -/ diff --git a/Complexity/Probability/Minimax.lean b/Complexity/Probability/Minimax.lean index e34129b86..ba395a192 100644 --- a/Complexity/Probability/Minimax.lean +++ b/Complexity/Probability/Minimax.lean @@ -213,10 +213,10 @@ theorem exists_mem_support_tsum_mul_le (μ : PMF α) (ν : PMF β) (loss : α calc ∑' a, μ a * ∑' b, ν b * loss a b = ∑' b, ν b * ∑' a, μ a * loss a b := by simp only [← ENNReal.tsum_mul_left] - rw [ENNReal.tsum_comm] + rw [tsum_comm] exact tsum_congr fun b => tsum_congr fun a => by ring _ ≤ ∑' b, ν b * ε := by - refine ENNReal.tsum_le_tsum fun b => ?_ + refine tsum_le_tsum fun b => ?_ by_cases hb : b ∈ ν.support · gcongr exact h b hb diff --git a/Complexity/Probability/PMF/Bind.lean b/Complexity/Probability/PMF/Bind.lean index fcdd9502b..c3b9858bc 100644 --- a/Complexity/Probability/PMF/Bind.lean +++ b/Complexity/Probability/PMF/Bind.lean @@ -53,7 +53,7 @@ theorem bind_sigma_apply {β : α → Type*} (p : PMF α) (f : (a : α) → PMF (p.bind (fun a => (f a).map (Sigma.mk a))) ⟨a, b⟩ = p a * f a b := by classical simp only [PMF.bind_apply, PMF.map_apply, ← ENNReal.tsum_mul_left, mul_ite, mul_zero] - rw [← ENNReal.tsum_sigma] + rw [← tsum_sigma] simp /-- A continuation only needs to agree on outcomes the preceding distribution can produce. -/ @@ -76,7 +76,7 @@ theorem le_bind_apply_on_support (μ : PMF α) {f : α → PMF β} {b : β} {c : (h : ∀ a ∈ μ.support, c ≤ f a b) : c ≤ μ.bind f b := by rw [PMF.bind_apply] calc c = ∑' a, μ a * c := by rw [ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] - _ ≤ ∑' a, μ a * f a b := ENNReal.tsum_le_tsum fun a => by + _ ≤ ∑' a, μ a * f a b := tsum_le_tsum fun a => by by_cases ha : a ∈ μ.support · exact mul_le_mul_right (h a ha) _ · simp [(PMF.apply_eq_zero_iff μ a).2 ha] @@ -87,7 +87,7 @@ theorem bind_apply_le_on_support (μ : PMF α) {f : α → PMF β} {b : β} {c : (h : ∀ a ∈ μ.support, f a b ≤ c) : μ.bind f b ≤ c := by rw [PMF.bind_apply] calc ∑' a, μ a * f a b - _ ≤ ∑' a, μ a * c := ENNReal.tsum_le_tsum fun a => by + _ ≤ ∑' a, μ a * c := tsum_le_tsum fun a => by by_cases ha : a ∈ μ.support · exact mul_le_mul_right (h a ha) _ · simp [(PMF.apply_eq_zero_iff μ a).2 ha] @@ -98,7 +98,7 @@ probability in the whole experiment. The sample spaces may be infinite. -/ theorem bind_apply_mono_on_support (μ : PMF α) {f g : α → PMF β} (b : β) (h : ∀ a ∈ μ.support, f a b ≤ g a b) : μ.bind f b ≤ μ.bind g b := by simp only [PMF.bind_apply] - apply ENNReal.tsum_le_tsum + apply tsum_le_tsum intro a by_cases ha : a ∈ μ.support · exact mul_le_mul_right (h a ha) _ @@ -117,7 +117,7 @@ theorem exists_bind_apply_le [Finite α] (μ : PMF α) (f : α → PMF β) (b : /-- An average of values at most `B` is at most `B`. -/ theorem tsum_mul_le_of_forall_le (μ : PMF α) {f : α → ℝ≥0∞} {B : ℝ≥0∞} (h : ∀ a, f a ≤ B) : ∑' a, μ a * f a ≤ B := - calc ∑' a, μ a * f a ≤ ∑' a, μ a * B := ENNReal.tsum_le_tsum fun a => mul_le_mul_right (h a) _ + calc ∑' a, μ a * f a ≤ ∑' a, μ a * B := tsum_le_tsum fun a => mul_le_mul_right (h a) _ _ = B := by rw [ENNReal.tsum_mul_right, μ.tsum_coe, one_mul] /-- Values at most `1` on a finite set `T` and at most `B` elsewhere average to at most @@ -126,9 +126,9 @@ theorem tsum_mul_le_card_mul_add [DecidableEq α] (μ : PMF α) {ε : ℝ≥0∞ (T : Finset α) {f : α → ℝ≥0∞} {B : ℝ≥0∞} (h : ∀ a, f a ≤ (if a ∈ T then 1 else 0) + B) : ∑' a, μ a * f a ≤ T.card * ε + B := by calc ∑' a, μ a * f a ≤ ∑' a, (μ a * (if a ∈ T then 1 else 0) + μ a * B) := - ENNReal.tsum_le_tsum fun a => (mul_le_mul_right (h a) _).trans_eq (mul_add _ _ _) + tsum_le_tsum fun a => (mul_le_mul_right (h a) _).trans_eq (mul_add _ _ _) _ = ∑ a ∈ T, μ a + B := by - rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul, + rw [tsum_add, ENNReal.tsum_mul_right, μ.tsum_coe, one_mul, tsum_eq_sum (s := T) fun a ha => by simp only [ha, ↓reduceIte, mul_zero]] congr 1 exact Finset.sum_congr rfl fun a ha => by simp only [ha, ↓reduceIte, mul_one] diff --git a/Complexity/Probability/PMF/Event.lean b/Complexity/Probability/PMF/Event.lean index ce673d7ce..16a45d6e8 100644 --- a/Complexity/Probability/PMF/Event.lean +++ b/Complexity/Probability/PMF/Event.lean @@ -73,7 +73,7 @@ theorem toOuterMeasure_ne_top (p : PMF α) (event : Set α) : p.toOuterMeasure e /-- An event has probability at most one. -/ theorem toOuterMeasure_le_one (p : PMF α) (event : Set α) : p.toOuterMeasure event ≤ 1 := by rw [PMF.toOuterMeasure_apply, ← p.tsum_coe] - exact ENNReal.tsum_le_tsum (fun a => Set.indicator_apply_le (fun _ => le_rfl)) + exact tsum_le_tsum (fun a => Set.indicator_apply_le (fun _ => le_rfl)) /-- The event that `f a` deviates from `m` by at least `ε` is covered by the upper tail `m + ε ≤ f a` and the lower tail `f a ≤ m - ε`. -/ @@ -111,7 +111,7 @@ theorem toOuterMeasure_apply_toReal [Fintype α] (p : PMF α) (event : Set α) : theorem toOuterMeasure_toReal_add_compl (p : PMF α) (event : Set α) : (p.toOuterMeasure event).toReal + (p.toOuterMeasure eventᶜ).toReal = 1 := by have h : p.toOuterMeasure event + p.toOuterMeasure eventᶜ = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert p.tsum_coe using 1 congr 1 funext a @@ -154,7 +154,7 @@ theorem toOuterMeasure_bind_failure_le (p : PMF α) (kernel : α → PMF β) rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] calc _ ≤ ∑' a, ((preᶜ).indicator p a + p a * error) := by - apply ENNReal.tsum_le_tsum + apply tsum_le_tsum intro a by_cases ha : a ∈ pre · simp only [Set.indicator_of_notMem (show a ∉ preᶜ by simpa), zero_add] @@ -165,7 +165,7 @@ theorem toOuterMeasure_bind_failure_le (p : PMF α) (kernel : α → PMF β) · simp only [Set.indicator_of_mem (show a ∈ preᶜ from ha)] exact (mul_le_mul_of_nonneg_left (toOuterMeasure_le_one (kernel a) failure) zero_le).trans (by simp) - _ = _ := by rw [ENNReal.tsum_add, ENNReal.tsum_mul_right, p.tsum_coe, one_mul] + _ = _ := by rw [tsum_add, ENNReal.tsum_mul_right, p.tsum_coe, one_mul] /-- The real-valued error rule for a randomized continuation. -/ theorem toOuterMeasure_bind_failure_toReal_le (p : PMF α) (kernel : α → PMF β) @@ -193,7 +193,7 @@ theorem toOuterMeasure_bind_ge_mul (p : PMF α) (kernel : α → PMF β) classical have hle : p.toOuterMeasure pre * ENNReal.ofReal s₂ ≤ (p.bind kernel).toOuterMeasure event := by rw [PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_mul_right] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ pre · simp only [Set.indicator_of_mem ha] by_cases hs : a ∈ p.support @@ -242,7 +242,7 @@ theorem map_true_mono (p : PMF α) {E F : α → Bool} (h : ∀ a ∈ p.support, E a = true → F a = true) : (p.map E) true ≤ (p.map F) true := by rw [PMF.map_apply, PMF.map_apply] - refine ENNReal.tsum_le_tsum fun a => ?_ + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ p.support · by_cases hE : true = E a · simp [← hE, h a ha hE.symm] @@ -253,8 +253,8 @@ theorem map_true_mono (p : PMF α) {E F : α → Bool} theorem map_true_le_add (p : PMF α) {E A B : α → Bool} (h : ∀ a ∈ p.support, E a = true → A a = true ∨ B a = true) : (p.map E) true ≤ (p.map A) true + (p.map B) true := by - rw [PMF.map_apply, PMF.map_apply, PMF.map_apply, ← ENNReal.tsum_add] - refine ENNReal.tsum_le_tsum fun a => ?_ + rw [PMF.map_apply, PMF.map_apply, PMF.map_apply, ← tsum_add] + refine tsum_le_tsum fun a => ?_ by_cases ha : a ∈ p.support · by_cases hE : true = E a · rcases h a ha hE.symm with hA | hB diff --git a/Complexity/Probability/PMF/Map.lean b/Complexity/Probability/PMF/Map.lean index eed796d9b..4aad0a45d 100644 --- a/Complexity/Probability/PMF/Map.lean +++ b/Complexity/Probability/PMF/Map.lean @@ -40,7 +40,7 @@ variable {α β : Type*} theorem le_map_apply (p : PMF α) (f : α → β) (a : α) : p a ≤ (p.map f) (f a) := by classical rw [PMF.map_apply] - simpa using ENNReal.le_tsum (f := fun x => if f a = f x then p x else 0) a + simpa using le_tsum (f := fun x => if f a = f x then p x else 0) a /-- Relabeling a distribution by an equivalence preserves each corresponding point mass. -/ theorem map_equiv_apply (p : PMF α) (e : α ≃ β) (b : β) : diff --git a/Complexity/Probability/ProductConditioning/SideInformation.lean b/Complexity/Probability/ProductConditioning/SideInformation.lean index 316b7c63c..11c12f8fe 100644 --- a/Complexity/Probability/ProductConditioning/SideInformation.lean +++ b/Complexity/Probability/ProductConditioning/SideInformation.lean @@ -186,7 +186,7 @@ theorem sideJoint_apply (μ : PMF T) (J : T → PMF ((∀ i, β i) × V)) (t : T exact hx (Prod.ext (congrArg Prod.snd h).symm (congrArg (·.2) (congrArg Prod.fst h)).symm) · intro t' ht' rw [PMF.map_apply] - refine mul_eq_zero_of_right _ (ENNReal.tsum_eq_zero.mpr fun x => ite_eq_right fun h => ?_) + refine mul_eq_zero_of_right _ (tsum_eq_zero_iff.mpr fun x => ite_eq_right fun h => ?_) exact absurd (congrArg (·.1) (congrArg Prod.fst h)) (Ne.symm ht') /-- The pairs `(u, v)` that complete `(t, v)` to an outcome in `W`. -/ diff --git a/Complexity/Probability/RejectionSampling.lean b/Complexity/Probability/RejectionSampling.lean index d317c0d2f..83b85bf82 100644 --- a/Complexity/Probability/RejectionSampling.lean +++ b/Complexity/Probability/RejectionSampling.lean @@ -144,7 +144,7 @@ theorem shiftReject_apply_some (hv : ∀ z ∈ S, z - v ∈ B) (z : α) : · intro y hy have hyz : y + v ≠ z := fun h => hy (eq_sub_of_add_eq h) split_ifs <;> simp_all [eq_comm] - · refine ENNReal.tsum_eq_zero.mpr fun y => ?_ + · refine tsum_eq_zero_iff.mpr fun y => ?_ split_ifs with h₁ h₂ <;> simp_all /-- **Rejection sampling, aborts.** Under the same hypothesis, the sampler aborts with @@ -156,7 +156,7 @@ theorem shiftReject_apply_none (hv : ∀ z ∈ S, z - v ∈ B) : rw [tsum_eq_sum (s := S) fun z hz => by simp [hz], Finset.sum_ite_mem, Finset.inter_self, Finset.sum_const, nsmul_eq_mul, div_eq_mul_inv] have htotal := (shiftReject B S hB v).tsum_coe - rw [ENNReal.tsum_eq_add_tsum_ite none, ← (Option.some_injective α).tsum_eq] at htotal + rw [tsum_eq_add_tsum_ite none, ← (Option.some_injective α).tsum_eq] at htotal · simp only [reduceCtorEq, ite_false, hsome] at htotal exact ENNReal.eq_sub_of_add_eq (ENNReal.div_ne_top (ENNReal.natCast_ne_top _) (by simpa using hB.ne_empty)) htotal From d7dcd7985e43c0678127230df389fa8ea3e3ea61 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:21:52 -0400 Subject: [PATCH 08/18] fix(build): adapt proofs to the rc2 Mathlib API Qualify newly protected matrix and finite-sum lemmas, publicly import the relocated Matrix.Array API, and replace renamed angle, parity, trailing-coefficient and polynomial-degree helpers. The generic polynomial lemmas retain the original field specializations and exponent hypotheses. No theorem statement or definition body changes. Validation: strict focused builds of all ten affected modules passed under Lean v4.36.0-rc2. Source comparison against d6ccb24f shows all Lean edits are inside theorem proofs except the required Bird import. --- Complexity/Computability/Circuit/Bird.lean | 1 + Complexity/Computability/Circuit/Quantum/Basic.lean | 4 ++-- Complexity/Computability/Circuit/Rational/Wronskian.lean | 2 +- .../Foundations/BooleanAnalysis/LowDegree/Addition.lean | 2 +- .../Foundations/BooleanAnalysis/SparseEquations.lean | 2 +- .../FieldTheory/Finite/LinearizedPolynomial.lean | 6 +++--- Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean | 4 ++-- Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean | 2 +- .../Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean | 8 ++++---- Complexity/Foundations/Probability/Gaussian/Sheppard.lean | 2 +- 10 files changed, 17 insertions(+), 16 deletions(-) diff --git a/Complexity/Computability/Circuit/Bird.lean b/Complexity/Computability/Circuit/Bird.lean index 62b11531b..a39c61ae9 100644 --- a/Complexity/Computability/Circuit/Bird.lean +++ b/Complexity/Computability/Circuit/Bird.lean @@ -8,6 +8,7 @@ module -- upstream: candidate public import Complexity.Init +public import Mathlib.LinearAlgebra.Matrix.Array public import Mathlib.LinearAlgebra.Matrix.Determinant.Bird.Defs public import Mathlib.LinearAlgebra.Matrix.Determinant.Basic import Mathlib.Algebra.Order.BigOperators.Group.LocallyFinite diff --git a/Complexity/Computability/Circuit/Quantum/Basic.lean b/Complexity/Computability/Circuit/Quantum/Basic.lean index 7797874a4..c9265f0f4 100644 --- a/Complexity/Computability/Circuit/Quantum/Basic.lean +++ b/Complexity/Computability/Circuit/Quantum/Basic.lean @@ -172,7 +172,7 @@ theorem place_apply (A : Fin k ↪ Fin n) (U : unitaryGroup (Fin k → Bool) ℂ (place A U : Matrix _ _ ℂ) y x = if ∀ j, (¬∃ i, A i = j) → y j = x j then (U : Matrix _ _ ℂ) (y ∘ A) (x ∘ A) else 0 := by simp only [place, MonoidHom.coe_mk, OneHom.coe_mk, submatrix_apply, kroneckerMap_apply, - splitEquiv_apply_fst, splitEquiv_apply_snd, one_apply, funext_iff, Subtype.forall] + splitEquiv_apply_fst, splitEquiv_apply_snd, Matrix.one_apply, funext_iff, Subtype.forall] split_ifs <;> simp namespace PlacedGate @@ -281,7 +281,7 @@ theorem hadamardMatrix_mem_unitaryGroup : hadamardMatrix ∈ unitaryGroup Bool rw [mem_unitaryGroup_iff] ext b c cases b <;> cases c <;> - simp [hadamardMatrix, mul_apply, star_eq_conjTranspose, Complex.conj_ofReal, h2, h1] + simp [hadamardMatrix, Matrix.mul_apply, star_eq_conjTranspose, Complex.conj_ofReal, h2, h1] theorem star_hadamardMatrix : star hadamardMatrix = hadamardMatrix := by ext b c diff --git a/Complexity/Computability/Circuit/Rational/Wronskian.lean b/Complexity/Computability/Circuit/Rational/Wronskian.lean index f2ce76817..138a37718 100644 --- a/Complexity/Computability/Circuit/Rational/Wronskian.lean +++ b/Complexity/Computability/Circuit/Rational/Wronskian.lean @@ -148,7 +148,7 @@ lemma wronskian_order_of_distinct_orders {K : Type*} [Field K] [CharZero K] Matrix.det_mul_row (fun j => (f j).trailingCoeff) (Matrix.of fun i j : Fin h => ((f j).natTrailingDegree.descFactorial i.val : K))] exact mul_ne_zero (Finset.prod_ne_zero_iff.mpr fun j _ => - trailingCoeff_nonzero_iff_nonzero.mpr (hf j)) (det_descFactorial_ne_zero _ hd) + trailingCoeff_ne_zero.mpr (hf j)) (det_descFactorial_ne_zero _ hd) have hB0 : B.det ≠ 0 := fun hz => by simp [hz] at hBc have hA0 : A.det ≠ 0 := hdet ▸ mul_ne_zero (pow_ne_zero _ X_ne_zero) hB0 have hscale : A.det = X ^ (∑ i : Fin h, i.val) * wronskianFamily f := by diff --git a/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean b/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean index 905b6c11a..271991b3b 100644 --- a/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean +++ b/Complexity/Foundations/BooleanAnalysis/LowDegree/Addition.lean @@ -82,7 +82,7 @@ theorem hasSliceDecomposition_add (p q : ℕ) : have hexpand : (fun x y : ι → Bool => monomial K S.1 (x + y)) = ∑ T ∈ S.1.powerset, (fun x y => monomial K T x * monomial K (S.1 \ T) y) := by funext x y - simp only [sum_apply, monomial_add] + simp only [Finset.sum_apply, monomial_add] rw [hexpand] apply (sliceSpace (lowDegree K ι p) (lowDegree K ι q)).sum_mem intro T hT diff --git a/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean b/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean index d9daca94e..d1bc719e3 100644 --- a/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean +++ b/Complexity/Foundations/BooleanAnalysis/SparseEquations.lean @@ -143,7 +143,7 @@ is even. -/ theorem monomialSum_zmod_two_eq_zero_iff_even (𝓜 : Finset (Finset ι)) (x : ι → Bool) : monomialSum (ZMod 2) 𝓜 x = 0 ↔ Even (monomialSum ℤ 𝓜 x) := by rw [monomialSum_eq_card_filter (ZMod 2), monomialSum_eq_card_filter ℤ, - ZMod.natCast_eq_zero_iff_even, Int.even_coe_nat] + ZMod.natCast_eq_zero_iff_even, Int.even_natCast] /-- An assignment `x : ι → Bool` is a common zero of the family `{P i}_{i ∈ I}` over `𝔽₂` if every equation `P i` (`i ∈ I`) evaluates to `0` in `ZMod 2` at `x`. -/ diff --git a/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean b/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean index 09309146e..a27032be5 100644 --- a/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean +++ b/Complexity/Foundations/FieldTheory/Finite/LinearizedPolynomial.lean @@ -252,7 +252,7 @@ private theorem comp_eq_X_pow_card_sub_X [Fintype F] {P Q : F[X]} ⟨by rw [natDegree_comp, hdeg], hQm.comp hPm hPdeg⟩ have hX : IsMonicOfDegree (X ^ Fintype.card F - X : F[X]) (Fintype.card F) := by rw [isMonicOfDegree_iff] - refine ⟨(FiniteField.X_pow_card_sub_X_natDegree_eq F hq1).le, ?_⟩ + refine ⟨(natDegree_X_pow_sub_X (R := F) hq1).le, ?_⟩ have h1 : (1 : ℕ) ≠ Fintype.card F := hq1.ne simp [coeff_X, h1] rw [← sub_eq_zero] @@ -410,10 +410,10 @@ theorem exists_subfield_card_le [Finite F] {n k : ℕ} (hn : finrank (ZMod 2) F have h1 : 1 < 2 ^ k := Nat.one_lt_two_pow hk0.ne' have hcard : Fintype.card F = 2 ^ n := by rw [Module.card_eq_pow_finrank (K := ZMod 2) (V := F), ZMod.card, hn] - refine FiniteField.X_pow_card_sub_X_ne_zero F h1 + refine X_pow_sub_X_ne_zero (R := F) h1 (eq_zero_of_natDegree_lt_card_of_eval_eq_zero _ (f := id) Function.injective_id (fun x => by simp [hall x]) ?_) - rw [FiniteField.X_pow_card_sub_X_natDegree_eq F h1, hcard] + rw [natDegree_X_pow_sub_X (R := F) h1, hcard] exact Nat.pow_lt_pow_right (by norm_num) hkn · by_cases hW : ∃ y₀ ∈ W, y₀ ≠ 0 · obtain ⟨y₀, hy₀W, hy₀⟩ := hW diff --git a/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean b/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean index 11579b762..50ea8c7ef 100644 --- a/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean +++ b/Complexity/Foundations/LinearAlgebra/Matrix/ListProd.lean @@ -38,7 +38,7 @@ theorem list_prod_ofFn_apply : ∀ {d : ℕ} (M : Fin d → Matrix ι ι R) (a b if p 0 = a ∧ p (Fin.last d) = b then ∏ i : Fin d, M i (p i.castSucc) (p i.succ) else 0 | 0, M, a, b => by rw [← (Equiv.funUnique (Fin 1) ι).symm.sum_comp] - simp only [List.ofFn_zero, List.prod_nil, one_apply, Finset.univ_eq_empty, + simp only [List.ofFn_zero, List.prod_nil, Matrix.one_apply, Finset.univ_eq_empty, Finset.prod_empty, Equiv.funUnique_symm_apply, Fin.last_zero, uniqueElim_const] by_cases hab : a = b · subst hab @@ -51,7 +51,7 @@ theorem list_prod_ofFn_apply : ∀ {d : ℕ} (M : Fin d → Matrix ι ι R) (a b have hL : (List.ofFn M).prod a b = ∑ q : Fin (d + 1) → ι, if q (Fin.last d) = b then M 0 a (q 0) * ∏ i : Fin d, M i.succ (q i.castSucc) (q i.succ) else 0 := by - rw [List.ofFn_succ, List.prod_cons, mul_apply] + rw [List.ofFn_succ, List.prod_cons, Matrix.mul_apply] simp only [list_prod_ofFn_apply (fun i => M i.succ), Finset.mul_sum, mul_ite, mul_zero] rw [Finset.sum_comm] refine Finset.sum_congr rfl fun q _ => ?_ diff --git a/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean b/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean index 04e6d3604..de84069b6 100644 --- a/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean +++ b/Complexity/Foundations/LinearAlgebra/Matrix/Rank.lean @@ -87,7 +87,7 @@ theorem exists_eq_mul_rank (A : Matrix m n K) : (b.sum_repr (col j)) simp only [Submodule.coe_sum, Submodule.coe_smul, Finset.sum_apply, Pi.smul_apply, smul_eq_mul] at h - simpa [mul_apply, col, mul_comm] using h.symm + simpa [Matrix.mul_apply, col, mul_comm] using h.symm /-- The natural rank of a finite matrix agrees with the cardinal rank of its linear map. -/ theorem natCast_rank_eq_rank_toLin' [DecidableEq n] (M : Matrix m n K) : diff --git a/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean b/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean index 16c25ab37..7d6f58267 100644 --- a/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean +++ b/Complexity/Foundations/LinearAlgebra/Matrix/SubmatrixRobust.lean @@ -340,17 +340,17 @@ theorem add_blockRank_le_cutRank_add_transpose [Finite ι] (hfg.symm.mono inter_subset_right inter_subset_right) (fun i hi j hj hj' => by have hjY : j ∉ Y := fun h => hj' ⟨hj, h⟩ - simp only [add_apply, transpose_apply, hMY i j hjY, hMY j i (hnotY i hi.2), add_zero]) + simp only [Matrix.add_apply, transpose_apply, hMY i j hjY, hMY j i (hnotY i hi.2), add_zero]) (fun i hi j hj hj' => by have hjX : j ∉ X := fun h => hj' ⟨hj, h⟩ - simp only [add_apply, transpose_apply, hMX i j (hnotX i hi.2), hMX j i hjX, add_zero]) + simp only [Matrix.add_apply, transpose_apply, hMX i j (hnotX i hi.2), hMX j i hjX, add_zero]) have hb₁ : R.blockRank (f ⁻¹' U) (g ⁻¹' Uᶜ) ≤ (M + Mᵀ).blockRank (U ∩ X) Uᶜ := by calc R.blockRank (f ⁻¹' U) (g ⁻¹' Uᶜ) = M.blockRank (U ∩ X) (Uᶜ ∩ Y) := by rw [← hMR, M.blockRank_submatrix hf hg, image_preimage_eq_inter_range, image_preimage_eq_inter_range] _ = (M + Mᵀ).blockRank (U ∩ X) (Uᶜ ∩ Y) := blockRank_congr fun i hi j _ => by - simp only [add_apply, transpose_apply, hMY j i (hnotY i hi.2), add_zero] + simp only [Matrix.add_apply, transpose_apply, hMY j i (hnotY i hi.2), add_zero] _ ≤ (M + Mᵀ).blockRank (U ∩ X) Uᶜ := blockRank_mono_right _ _ inter_subset_left have hb₂ : R.blockRank (f ⁻¹' Uᶜ) (g ⁻¹' U) ≤ (M + Mᵀ).blockRank (U ∩ Y) Uᶜ := by calc R.blockRank (f ⁻¹' Uᶜ) (g ⁻¹' U) @@ -359,7 +359,7 @@ theorem add_blockRank_le_cutRank_add_transpose [Finite ι] image_preimage_eq_inter_range] _ = Mᵀ.blockRank (U ∩ Y) (Uᶜ ∩ X) := (M.blockRank_transpose _ _).symm _ = (M + Mᵀ).blockRank (U ∩ Y) (Uᶜ ∩ X) := blockRank_congr fun i hi j _ => by - simp only [add_apply, transpose_apply, hMX i j (hnotX i hi.2), zero_add] + simp only [Matrix.add_apply, transpose_apply, hMX i j (hnotX i hi.2), zero_add] _ ≤ (M + Mᵀ).blockRank (U ∩ Y) Uᶜ := blockRank_mono_right _ _ inter_subset_left unfold cutRank omega diff --git a/Complexity/Foundations/Probability/Gaussian/Sheppard.lean b/Complexity/Foundations/Probability/Gaussian/Sheppard.lean index e3c419ba0..e18ca8f6d 100644 --- a/Complexity/Foundations/Probability/Gaussian/Sheppard.lean +++ b/Complexity/Foundations/Probability/Gaussian/Sheppard.lean @@ -308,7 +308,7 @@ theorem measureReal_stdGaussian_mem_uIoc_le_angle {u v : E} (huv : ‖u‖ = ‖ simp only [hu, hv, inner_zero_left, uIoc_of_le le_rfl, Ioc_self, mem_empty_iff_false, ofPred_false, measureReal_empty] exact div_nonneg (InnerProductGeometry.angle_nonneg _ _) pi_pos.le - · rw [eq_neg_of_add_eq_zero_right hne, InnerProductGeometry.angle_self_neg_of_nonzero hu, + · rw [eq_neg_of_add_eq_zero_right hne, InnerProductGeometry.angle_self_neg_of_ne_zero hu, div_self pi_ne_zero] exact measureReal_le_one · rw [InnerProductGeometry.angle_eq_two_mul_arctan huv hne] From f5a70cb9826c88744c1dfbc560340d3efebbfb04 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:26:19 -0400 Subject: [PATCH 09/18] test(quantum): qualify the protected matrix lemma Preserve the existing non-unitary matrix counterexample under rc2 Mathlib by explicitly selecting Matrix.mul_apply, as in the library proof repairs. --- ComplexityTests/QuantumCircuit.lean | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ComplexityTests/QuantumCircuit.lean b/ComplexityTests/QuantumCircuit.lean index a440e1a4b..cca9dbe2c 100644 --- a/ComplexityTests/QuantumCircuit.lean +++ b/ComplexityTests/QuantumCircuit.lean @@ -158,7 +158,7 @@ example : ¬Circuit.IsOver {⟨3, Gate.toffoli⟩} [h₀] := fun h => example : !![(1 : ℂ), 1; 0, 1] ∉ unitaryGroup (Fin 2) ℂ := by intro h have := congrFun (congrFun (mem_unitaryGroup_iff.mp h) 0) 0 - simp [mul_apply, Fin.sum_univ_two, star_eq_conjTranspose] at this + simp [Matrix.mul_apply, Fin.sum_univ_two, star_eq_conjTranspose] at this -- A two-qubit gate cannot be placed on a one-qubit register: the wires must be distinct. example : IsEmpty (Fin 2 ↪ Fin 1) := From bb3e910dfc197e46f4b11e0c666b6546a075b7c8 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 15:46:20 -0400 Subject: [PATCH 10/18] fix(build): normalize goals before proof automation Use expected-type-directed refinement for the generic zero-sum lemma and expose the determinant variable-count function before fun_prop. Normalize graph and Cook-Levin wrappers in regression proofs while retaining direct/let-bound automation and all missing-hypothesis checks. Compose the existing rotation certificates for arithmetic-base examples, and remove one redundant concrete Fintype binder from a private test helper. Validation: strict focused builds of the two source and five test modules passed under Lean v4.36.0-rc2. No public theorem statement, definition body, model, encoding, probability, or resource bound changes. No heartbeat or fun_prop compatibility option was introduced. --- Complexity/Algorithms/WordRAM/KargerMinCut.lean | 2 +- .../Computability/Circuit/Arithmetic/Determinant.lean | 4 +++- ComplexityTests/ExpanderContraction.lean | 3 ++- ComplexityTests/ExpanderFamily.lean | 1 + ComplexityTests/ExplicitExpanders.lean | 8 ++++++-- ComplexityTests/Iteration/SparseDisplacement.lean | 2 +- ComplexityTests/MultiTapePolyTime.lean | 1 + 7 files changed, 15 insertions(+), 6 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/KargerMinCut.lean b/Complexity/Algorithms/WordRAM/KargerMinCut.lean index c07c84a3c..5cf511394 100644 --- a/Complexity/Algorithms/WordRAM/KargerMinCut.lean +++ b/Complexity/Algorithms/WordRAM/KargerMinCut.lean @@ -1181,7 +1181,7 @@ theorem ampLoop_spec {w sb n R : ℕ} {W : Fin n → Fin n → ℕ} {mem₀ : obtain ⟨m_r, hm_r, _, hsup_r, _⟩ := hnext z₁ hz₁ rw [hml_def, hm_r.codeComp_eq, PMF.toOuterMeasure_map_apply, PMF.toOuterMeasure_apply] - convert tsum_eq_zero_iff.mpr _ + refine tsum_eq_zero_iff.mpr ?_ intro z₂ by_cases hz₂_eval : z₂ ∈ m_r.eval.support · have hz₂ : z₂ ∈ m_r.support := (ProbComp.support_eval m_r) ▸ hz₂_eval diff --git a/Complexity/Computability/Circuit/Arithmetic/Determinant.lean b/Complexity/Computability/Circuit/Arithmetic/Determinant.lean index 1f231b4cf..01cf49231 100644 --- a/Complexity/Computability/Circuit/Arithmetic/Determinant.lean +++ b/Complexity/Computability/Circuit/Arithmetic/Determinant.lean @@ -272,7 +272,9 @@ generic determinants has `n * n` variables, degree at most `n` and arithmetic ci `3 * n ^ 4 + 4` gates. -/ theorem detFamily_mem_VP : detFamily K ∈ VP K := by choose c hc hsize using exists_circuit_det (K := K) - refine ⟨⟨by fun_prop, PolynomiallyBounded.id.mono_of_le fun n => + refine ⟨⟨?_, PolynomiallyBounded.id.mono_of_le fun n => totalDegree_det_genericMatrix_le n⟩, c, PolynomiallyBounded.mono_of_le (by fun_prop) hsize, hc⟩ + change PolynomiallyBounded (fun n : ℕ => n * n) + fun_prop end Cslib.Circuits.Arithmetic diff --git a/ComplexityTests/ExpanderContraction.lean b/ComplexityTests/ExpanderContraction.lean index ac59474d9..1bd9d7150 100644 --- a/ComplexityTests/ExpanderContraction.lean +++ b/ComplexityTests/ExpanderContraction.lean @@ -68,7 +68,7 @@ example : ComputableInPolyTime threeVertices.rot have hg : ComputableInPolyTime completeFive.rot (encPair (encFinEnum (Fin 5)) (encFinEnum (Fin 5))) (encPair (encFinEnum (Fin 5)) (encFinEnum (Fin 5))) := .of_finite _ - fun_prop [threeVertices, RotationMap.contractFin] + exact ComputableInPolyTime.rotationMap_contract he hd hg example {V W I D : Type} (G : RotationMap V D) (encode : V → W × I) (decode : W × I → Option V) (hinv : Function.IsPartialInv encode decode) @@ -80,6 +80,7 @@ example {V W I D : Type} (G : RotationMap V D) (encode : V → W × I) (fun p : (W × (I × D)) × Bool => ((RotationMap.contract encode decode hinv G).rot p.1, !p.2)) (encPair (encPair ew (encPair ei ed)) encBool) (encPair (encPair ew (encPair ei ed)) encBool) := by + dsimp only [RotationMap.contract, RotationMap.contractRot, RotationMap.contractDart] fail_if_success (clear he; fun_prop) fail_if_success (clear hd; fun_prop) fail_if_success (clear hg; fun_prop) diff --git a/ComplexityTests/ExpanderFamily.lean b/ComplexityTests/ExpanderFamily.lean index 0dd5ac6e9..51163712e 100644 --- a/ComplexityTests/ExpanderFamily.lean +++ b/ComplexityTests/ExpanderFamily.lean @@ -96,6 +96,7 @@ example {V W D : Type} (G : RotationMap V D) (e : V ≃ W) let next := (G.reindex e).rot p.1 (next, !p.2)) (encPair (encPair ew ed) encBool) (encPair (encPair ew ed) encBool) := by + dsimp only [RotationMap.reindex, RotationMap.reindexRot] fail_if_success (clear hf; fun_prop) fail_if_success (clear hb; fun_prop) fail_if_success (clear hg; fun_prop) diff --git a/ComplexityTests/ExplicitExpanders.lean b/ComplexityTests/ExplicitExpanders.lean index a912aabc8..c8d706921 100644 --- a/ComplexityTests/ExplicitExpanders.lean +++ b/ComplexityTests/ExplicitExpanders.lean @@ -64,7 +64,8 @@ example {K : Type} [Field K] (encode : K ↪ List Bool) ComputableInPolyTime (RotationMap.affinePlaneTowerBase K).rot (encPair (baseEncoding encode) (planeEncoding encode)) (encPair (baseEncoding encode) (planeEncoding encode)) := by - fun_prop [RotationMap.affinePlaneTowerBase] + have hplane := ComputableInPolyTime.rotationMap_affinePlane hmul hsub + exact ((hplane.rotationMap_tensor hplane).rotationMap_zigzag hplane).rotationMap_zigzag hplane -- Only the fixed field operations use the finite-domain theorem; graph products compose them. example (encode : ZMod 257 ↪ List Bool) : @@ -75,7 +76,8 @@ example (encode : ZMod 257 ↪ List Bool) : (planeEncoding encode) encode := .of_finite _ have hsub : ComputableInPolyTime (fun p : ZMod 257 × ZMod 257 => p.1 - p.2) (planeEncoding encode) encode := .of_finite _ - fun_prop [RotationMap.affinePlaneTowerBase] + have hplane := ComputableInPolyTime.rotationMap_affinePlane hmul hsub + exact ((hplane.rotationMap_tensor hplane).rotationMap_zigzag hplane).rotationMap_zigzag hplane -- Equivalent direct and let-bound spellings, with a supplied rotation on an arbitrary type. example {V D E : Type} (G : RotationMap V D) (H : RotationMap D E) @@ -86,6 +88,7 @@ example {V D E : Type} (G : RotationMap V D) (H : RotationMap D E) (fun p : ((V × D) × (E × E)) × Bool => ((G.zigzag H).rot p.1, !p.2)) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) := by + dsimp only [RotationMap.zigzag, RotationMap.zigzagRot] fail_if_success (clear hG; fun_prop) fail_if_success (clear hH; fun_prop) fun_prop @@ -100,6 +103,7 @@ example {V D E : Type} (G : RotationMap V D) (H : RotationMap D E) (next, !p.2)) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) (encPair (encPair (encPair vertex label) (encPair small small)) encBool) := by + dsimp only [RotationMap.zigzag, RotationMap.zigzagRot] fun_prop end ComplexityTests.ExplicitExpanders diff --git a/ComplexityTests/Iteration/SparseDisplacement.lean b/ComplexityTests/Iteration/SparseDisplacement.lean index 634e74f09..fbe746f4f 100644 --- a/ComplexityTests/Iteration/SparseDisplacement.lean +++ b/ComplexityTests/Iteration/SparseDisplacement.lean @@ -34,7 +34,7 @@ example : private def editedCounter : Equiv.Perm (BitVec 2) := (Equiv.swap 0 1).trans (Equiv.addRight 1) -private theorem editedCounter_disagreement [Fintype (BitVec 2)] : +private theorem editedCounter_disagreement : disagreement editedCounter (fun x => x + 1) = {0, 1} := by ext x simp only [mem_disagreement, Finset.mem_insert, Finset.mem_singleton] diff --git a/ComplexityTests/MultiTapePolyTime.lean b/ComplexityTests/MultiTapePolyTime.lean index 2042aace5..8fde4c4a0 100644 --- a/ComplexityTests/MultiTapePolyTime.lean +++ b/ComplexityTests/MultiTapePolyTime.lean @@ -139,6 +139,7 @@ example {α : Type} (enc : α ↪ List Bool) (p : Polynomial ℕ) (c d : ℕ) : ComputableTidilyInPolyTime (fun a => List.range (Cslib.Complexity.CookLevin.parameters p c d enc a).2.2) enc (encList encUnary) := by + dsimp only [Cslib.Complexity.CookLevin.parameters] fun_prop private def finish (move : SignType) (symbol : Bool) : Turing.MultiTapeTM 0 Bool Unit := From 1cdc80ed67756efebde20dfabe1570b5ad5bc678 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:07:03 -0400 Subject: [PATCH 11/18] test(wordram): exercise two-outcome randomized subroutines Initialize and preserve the source register at one, requiring 1 <= w, so each native rand samples Fin 2. Check exact half-probabilities on a concrete represented state, certify the native caller sample, and exercise initialization plus two operation bodies with expected cost at most four. Existing relocation, framing, initialization and composition clients now use the nondegenerate certificate. Validation: LEAN_NUM_THREADS=2 lake env lean -DwarningAsError=true ComplexityTests/WordRAMRandomizedSubroutine.lean; git diff --check. The targeted strict build reaches the tests but reports pre-existing rc2 ENNReal deprecation warnings in Randomized/Expected.lean and Randomized/Subroutine.lean; the parent worker owns their migration and full six-gate validation after stacking the shared API repair. No curated statement, scope or rendering changes. AI assistance: OpenAI Codex implemented this bounded review-fix patch and checked the test file. --- .../WordRAMRandomizedSubroutine.lean | 164 ++++++++++++------ 1 file changed, 109 insertions(+), 55 deletions(-) diff --git a/ComplexityTests/WordRAMRandomizedSubroutine.lean b/ComplexityTests/WordRAMRandomizedSubroutine.lean index 1a335f4b5..23f2c2292 100644 --- a/ComplexityTests/WordRAMRandomizedSubroutine.lean +++ b/ComplexityTests/WordRAMRandomizedSubroutine.lean @@ -109,7 +109,7 @@ theorem geomCoinLoop_execCodeDist {w : ℕ} (hw : 1 ≤ w) (x₀ : Data 2) simp [y₀, Cond.Holds, Data.set, hr1] have h1 : (2 : ℝ≥0∞)⁻¹ ≤ geomBodyDist x y₀ := by rw [geomBodyDist, PMF.map_apply] - have hle := ENNReal.le_tsum + have hle := le_tsum (f := fun k : Fin (x.regs g1 + 1) => if y₀ = x.set g0 k.val then uniformOfFintype (Fin (x.regs g1 + 1)) k else 0) k₀ simpa [y₀, k₀, uniformOfFintype_apply, hr1] using hle @@ -120,7 +120,8 @@ theorem geomCoinLoop_execCodeDist {w : ℕ} (hw : 1 ≤ w) (x₀ : Data 2) = Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y₀ := by rw [Set.indicator_of_mem hy₀_not] _ ≤ ∑' y, Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} (geomBodyDist x) y := - ENNReal.le_tsum y₀ + le_tsum (f := fun y => Set.indicator {y | ¬(Cond.eq g0 g1).Holds y} + (geomBodyDist x) y) y₀ exact le_trans h1 h2 have hx₀_inv : LoopStateInv (.eq g0 g1) (geomInv w) (geomPost w) x₀ := Or.inl ⟨hx₀.2, hx₀.1⟩ refine ⟨loopExitPMF (.eq g0 g1) geomBodyDist x₀, @@ -163,11 +164,11 @@ def coinSpec : Stateful.Spec.{0, 0, 0} where /-- Subroutine code with 5 registers (`0` = `baseReg`, `1` = `frontierReg`, `2` = `zeroReg`, `3` = `curReg` / scratch output, `4` = `oneReg`). Its operation body `code` executes -`r3 ← rand r2` (`Instr.rand`). -/ +`r3 ← rand r4` (`Instr.rand`), with `r4 = 1`, so it samples both `0` and `1`. -/ def coinSubroutineCode : SubroutineCode .exact 5 0 where allocSpan := .op (.copy 0 1) - initSetup := .op (.const 2 0) - code := .rand 3 2 + initSetup := .op (.const 4 1) + code := .rand 3 4 baseReg := 0 frontierReg := 1 zeroReg := 2 @@ -179,9 +180,9 @@ def coinSubroutineCode : SubroutineCode .exact 5 0 where output := 3 /-- Representation invariant for `coinSubroutineCode`: machine words are `w`-bounded, `baseReg` -holds `base`, and `zeroReg` holds `0`. -/ +holds `base`, and `oneReg` holds `1`. -/ def coinRep (w base : ℕ) (_θ : Fin 2) (_s : PUnit) (x : Data 5) : Prop := - x.Bounded w ∧ x.regs 0 = base ∧ x.regs 2 = 0 + x.Bounded w ∧ x.regs 0 = base ∧ x.regs 4 = 1 private theorem pmf_map_fst_const {α β γ : Type*} (q : PMF α) (b : β) (g : α → γ) : (q.map fun a => (b, g a)).map Prod.fst = PMF.pure b := by @@ -198,8 +199,9 @@ private theorem pmf_map_snd_const {α β γ : Type*} (q : PMF α) (f : α → β by_cases h : y = c <;> simp [PMF.map_apply, h, q.tsum_coe] /-- Concrete randomized `SubroutineDS` certificate for `coinSubroutineCode` with latent seed type -`Fin 2` and uniform reference distribution `uniformOfFintype (Fin 2)`. -/ -noncomputable def coinSubroutineDS (w base : ℕ) : +`Fin 2` and uniform reference distribution `uniformOfFintype (Fin 2)`. The native operation +samples from `Fin 2`; its randomness is separate from this latent coupling seed. -/ +noncomputable def coinSubroutineDS (w base : ℕ) (hw : 1 ≤ w) : Cslib.WordRAM.Randomized.SubroutineDS coinSubroutineCode coinSpec w base Fin.elim0 PUnit.unit where Seed := Fin 2 @@ -223,7 +225,7 @@ noncomputable def coinSubroutineDS (w base : ℕ) : rep_supported hreg _ hb' hrep := ⟨hb', by rw [hreg 0 rfl, hrep.2.1], - by rw [hreg 2 rfl, hrep.2.2]⟩ + by rw [hreg 4 rfl, hrep.2.2]⟩ potential_supported _ _ _ := rfl clear_nodup := by decide params_not_clear j := Fin.elim0 j @@ -238,11 +240,11 @@ noncomputable def coinSubroutineDS (w base : ℕ) : · simpa [coinSubroutineCode, Data.set] using hfr · simpa [coinSubroutineCode, Data.set] using hfr setup_exec {x₁} hx₁ hb₁ hfr₁ _ _ := by - let x₂ : Data 5 := x₁.set 2 0 + let x₂ : Data 5 := x₁.set 4 1 let ν₀ : PMF (Fin 2 × Data 5) := (uniformOfFintype (Fin 2)).map fun θ => (θ, x₂) refine ⟨ν₀, ?_, ?_, ?_⟩ - · have hex : ExecCodeDist (B := .exact) w (.op (.const 2 0)) x₁ (PMF.pure x₂) (1 : ℕ) := - ExecCodeDist.of_exec (Exec.const (fits_zero w)) + · have hex : ExecCodeDist (B := .exact) w (.op (.const 4 1)) x₁ (PMF.pure x₂) (1 : ℕ) := + ExecCodeDist.of_exec (Exec.const (fits_one_of_pos hw)) exact hex.congr_dist (pmf_map_snd_const (uniformOfFintype (Fin 2)) id x₂).symm (by simp) · intro θ rw [PMF.map_comp] @@ -252,53 +254,73 @@ noncomputable def coinSubroutineDS (w base : ℕ) : obtain ⟨θ, _, rfl⟩ := hp refine ⟨?_, ?_, Nat.le_refl 0, Modifies.refl _ _⟩ · simpa [coinSubroutineCode, x₂, Data.set] using hfr₁ - · refine ⟨hx₁.set (fits_zero w), ?_, ?_⟩ + · refine ⟨hx₁.set (fits_one_of_pos hw), ?_, ?_⟩ · simp [x₂, Data.set, ← hb₁, coinSubroutineCode] · simp [x₂, Data.set] step_exec _ ν _ hseed hsupp := by let μ_step : Fin 2 × Data 5 → PMF (Fin 2 × Data 5) := fun p => - (uniformOfFintype (Fin (p.2.regs 2 + 1))).map fun k => (p.1, p.2.set 3 k.val) + (uniformOfFintype (Fin (p.2.regs 4 + 1))).map fun k => (p.1, p.2.set 3 k.val) refine ⟨μ_step, fun _ => 1, ?_, ?_, ?_⟩ · intro p hp - obtain ⟨⟨hbp, hb0, hz2⟩, _⟩ := hsupp p hp + obtain ⟨⟨hbp, hb0, ho4⟩, _⟩ := hsupp p hp refine ⟨?_, ?_⟩ - · have hr := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 3) (src := 2) - (hbp.1 2) + · have hr := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 3) (src := 4) + (hbp.1 4) refine hr.congr_dist ?_ le_rfl rw [ProbComp.eval_unifFin, PMF.map_comp] rfl · intro q hq rw [PMF.mem_support_map_iff] at hq obtain ⟨k, _, rfl⟩ := hq - have hk0 : k.val = 0 := by omega + have hk1 : k.val ≤ 1 := by omega refine ⟨rfl, ⟨?_, ?_, ?_⟩, Modifies.refl _ _⟩ - · refine hbp.set ?_ - rw [hk0] - exact fits_zero w + · exact hbp.set ((fits_one_of_pos hw).of_le hk1) · simp [Data.set, hb0] - · simp [Data.set, hz2] + · simp [Data.set, ho4] · intro θ' have hmap : (ν.bind μ_step).map Prod.fst = ν.map Prod.fst := by rw [PMF.map_bind, PMF.map] refine congrArg (ν.bind ·) (funext fun p => ?_) - exact pmf_map_fst_const (uniformOfFintype (Fin (p.2.regs 2 + 1))) p.1 + exact pmf_map_fst_const (uniformOfFintype (Fin (p.2.regs 4 + 1))) p.1 (fun k => p.2.set 3 k.val) rw [hmap] exact hseed θ' · simp [ν.tsum_coe] +/-- The concrete represented state with `oneReg = 1` executes the native sampler in one body +instruction, and its output register is exactly fair on both `0` and `1`. -/ +theorem coinSubroutineCode_two_outcomes {w : ℕ} (hw : 1 ≤ w) : + let x : Data 5 := (Data.init []).set 4 1 + let μ : PMF (Data 5) := (uniformOfFintype (Fin 2)).map fun k => x.set 3 k.val + coinRep w 0 0 PUnit.unit x ∧ + ExecCodeDist w coinSubroutineCode.code x μ 1 ∧ + (μ.map fun y => y.regs 3) 0 = (2 : ℝ≥0∞)⁻¹ ∧ + (μ.map fun y => y.regs 3) 1 = (2 : ℝ≥0∞)⁻¹ := by + intro x μ + have hx : x.Bounded w := + (Data.Bounded.init (by simp [InputFits])).set (fits_one_of_pos hw) + refine ⟨⟨hx, by simp [x, Data.set], by simp [x, Data.set]⟩, ?_, ?_, ?_⟩ + · have hsource : x.regs 4 = 1 := by simp [x, Data.set] + have he := ExecCodeDist.rand (B := .exact) (w := w) (x := x) (dst := 3) (src := 4) (hx.1 4) + rw [ProbComp.eval_unifFin, hsource] at he + exact he + · simp [μ, PMF.map_comp, Function.comp_def, PMF.map_apply, tsum_fintype, + Fin.sum_univ_two, uniformOfFintype_apply, Data.set] + · simp [μ, PMF.map_comp, Function.comp_def, PMF.map_apply, tsum_fintype, + Fin.sum_univ_two, uniformOfFintype_apply, Data.set] + /-- Standard-basis lifting of `coinSubroutineDS`. -/ -noncomputable def coinSubroutineDS_standard (w base : ℕ) : +noncomputable def coinSubroutineDS_standard (w base : ℕ) (hw : 1 ≤ w) : Cslib.WordRAM.Randomized.SubroutineDS coinSubroutineCode.toStandard coinSpec w base Fin.elim0 PUnit.unit := - (coinSubroutineDS w base).toStandard + (coinSubroutineDS w base hw).toStandard /-- Fresh initialization + multi-operation expected-amortized sequence bound for `coinSubroutineDS`. -/ -theorem coinSubroutineDS_seq_amortized (w base : ℕ) +theorem coinSubroutineDS_seq_amortized (w base : ℕ) (hw : 1 ≤ w) (ops : List PUnit) {x₀ : Data 5} (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : - let C := coinSubroutineDS w base + let C := coinSubroutineDS w base hw ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), ExecCodeDist w coinSubroutineCode.initFresh x₀ (ν₀.map Prod.snd) C.initTime ∧ C.ExecSteps PUnit.unit ops ν₀ ν_final T_ops ∧ @@ -317,10 +339,34 @@ theorem coinSubroutineDS_seq_amortized (w base : ℕ) simp [Block.footprint] exact C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid +/-- From concrete empty memory, initialization and two genuine coin-sampling bodies have +expected cost at most four; external argument delivery is uncharged by `ExecSteps`. -/ +example : + let C := coinSubroutineDS 1 0 (by decide) + ∃ (ν₀ ν_final : PMF (C.Seed × Data 5)) (T_ops : ℝ≥0∞), + ExecCodeDist 1 coinSubroutineCode.initFresh (Data.init []) (ν₀.map Prod.snd) C.initTime ∧ + C.ExecSteps PUnit.unit [PUnit.unit, PUnit.unit] ν₀ ν_final T_ops ∧ + C.initTime + T_ops ≤ 4 := by + intro C + obtain ⟨ν₀, ν_final, T_ops, hinit, hsteps, _, _, hcost⟩ := + coinSubroutineDS_seq_amortized 1 0 (by decide) [PUnit.unit, PUnit.unit] + (x₀ := (Data.init [] : Data 5)) + (Data.Bounded.init (by simp [InputFits])) (by simp [Data.init]) + refine ⟨ν₀, ν_final, T_ops, hinit, hsteps, ?_⟩ + have hT : T_ops ≤ 2 := by + simpa [C, coinSubroutineDS, Cslib.WordRAM.Randomized.SubroutineDS.initTime, + Stateful.Spec.totalCost, coinSpec] using hcost + have hinitTime : C.initTime = 2 := by + norm_num [C, coinSubroutineDS, Cslib.WordRAM.Randomized.SubroutineDS.initTime] + rw [hinitTime] + calc + 2 + T_ops ≤ 2 + 2 := add_le_add le_rfl hT + _ = 4 := by norm_num + /-- Dirty-memory initialization via `initClear_exec` for `coinSubroutineDS`. -/ -theorem coinSubroutineDS_initClear (w base : ℕ) {x₀ : Data 5} +theorem coinSubroutineDS_initClear (w base : ℕ) (hw : 1 ≤ w) {x₀ : Data 5} (hx₀ : x₀.Bounded w) (hfr : x₀.regs 1 = base) : - let C := coinSubroutineDS w base + let C := coinSubroutineDS w base hw ∃ ν₀ : PMF (C.Seed × Data 5), ExecCodeDist w coinSubroutineCode.initClear x₀ (ν₀.map Prod.snd) (C.initTime + (4 * C.space + 6 : ℕ)) ∧ @@ -336,13 +382,13 @@ theorem coinSubroutineDS_initClear (w base : ℕ) {x₀ : Data 5} /-- Inline caller subroutine invocation via `call_rename` for `coinSubroutineDS`: a caller with 6 registers embeds `Fin 5` via `Fin.castSucc`, executes `prep := .op (.const 5 0)` on its own register `r5`, and invokes `coinSubroutineCode.code.rename Fin.castSucc`. -/ -theorem coinSubroutineDS_call_rename (w base : ℕ) - (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) +theorem coinSubroutineDS_call_rename (w base : ℕ) (hw : 1 ≤ w) + (ν : PMF ((coinSubroutineDS w base hw).Seed × Data 6)) (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ - (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (coinSubroutineDS w base hw).seedSlack * (coinSubroutineDS w base hw).seedLaw θ) (hrep : ∀ p ∈ ν.support, - (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : - let C := coinSubroutineDS w base + (coinSubroutineDS w base hw).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base hw let prep : Code .exact 6 := .op (.const 5 0) ∃ (μ_call : C.Seed × Data 6 → PMF (C.Seed × Data 6)) (T_call : C.Seed × Data 6 → ℝ≥0∞), @@ -384,34 +430,42 @@ theorem coinSubroutineDS_call_rename (w base : ℕ) simp [Data.set, hne] /-- Caller framing across a randomized caller step: a caller with 6 registers embeds `Fin 5` via -`Fin.castSucc` and executes `r5 ← rand r2` on its own register `r5` (`5 : Fin 6`), preserving the +`Fin.castSucc` and executes `r5 ← rand r4` on its own register `r5` (`5 : Fin 6`), preserving the coupled seed marginal bound, `rep`, and expected potential via `frame_caller_dist`. -/ -theorem coinSubroutineDS_caller_rand_frame (w base : ℕ) - (ν : PMF ((coinSubroutineDS w base).Seed × Data 6)) +theorem coinSubroutineDS_caller_rand_frame (w base : ℕ) (hw : 1 ≤ w) + (ν : PMF ((coinSubroutineDS w base hw).Seed × Data 6)) (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ - (coinSubroutineDS w base).seedSlack * (coinSubroutineDS w base).seedLaw θ) + (coinSubroutineDS w base hw).seedSlack * (coinSubroutineDS w base hw).seedLaw θ) (hrep : ∀ p ∈ ν.support, - (coinSubroutineDS w base).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : - let C := coinSubroutineDS w base + (coinSubroutineDS w base hw).rep p.1 PUnit.unit (p.2.comap Fin.castSucc)) : + let C := coinSubroutineDS w base hw let μ_caller : C.Seed × Data 6 → PMF (Data 6) := fun p => - (uniformOfFintype (Fin 1)).map fun k => p.2.set 5 k.val + (uniformOfFintype (Fin 2)).map fun k => p.2.set 5 k.val + (∀ p ∈ ν.support, ExecCodeDist (B := .exact) w (.rand 5 4) p.2 (μ_caller p) 1) ∧ (∀ θ, ((C.frameDist ν μ_caller).map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ) ∧ (∀ q ∈ (C.frameDist ν μ_caller).support, C.rep q.1 PUnit.unit (q.2.comap Fin.castSucc)) ∧ (∑' q, C.frameDist ν μ_caller q * (C.potential q.1 (q.2.comap Fin.castSucc) : ℝ≥0∞)) = ∑' p, ν p * (C.potential p.1 (p.2.comap Fin.castSucc) : ℝ≥0∞) := by intro C μ_caller - refine C.frame_caller_dist (F := ∅) hseed hrep (Finset.disjoint_empty_right _) ?_ - intro p hp x' hx' - rw [PMF.mem_support_map_iff] at hx' - obtain ⟨k, _, rfl⟩ := hx' - have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) - refine ⟨?_, ?_, Modifies.refl _ _⟩ - · refine ⟨fun i => ?_, hbp.2⟩ - have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i - simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] - exact hbp.1 i - · intro i _ - have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i - simp [Data.set, hne] + refine ⟨?_, ?_⟩ + · intro p hp + have hsource : p.2.regs 4 = 1 := (hrep p hp).2.2 + have hfit : Fits w (p.2.regs 4) := by rw [hsource]; exact fits_one_of_pos hw + have he := ExecCodeDist.rand (B := .exact) (w := w) (x := p.2) (dst := 5) (src := 4) hfit + rw [ProbComp.eval_unifFin, hsource] at he + exact he + · refine C.frame_caller_dist (F := ∅) hseed hrep (Finset.disjoint_empty_right _) ?_ + intro p hp x' hx' + rw [PMF.mem_support_map_iff] at hx' + obtain ⟨k, _, rfl⟩ := hx' + have hbp : (p.2.comap Fin.castSucc).Bounded w := C.bounded (hrep p hp) + refine ⟨?_, ?_, Modifies.refl _ _⟩ + · refine ⟨fun i => ?_, hbp.2⟩ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp only [Data.comap_regs, Data.set_regs, Function.update_of_ne hne] + exact hbp.1 i + · intro i _ + have hne : Fin.castSucc i ≠ (5 : Fin 6) := Fin.castSucc_ne_last i + simp [Data.set, hne] end ComplexityTests.WordRAMRandomizedSubroutine From e62c359d802b888d1576882481152dfe650bce0d Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:09:05 -0400 Subject: [PATCH 12/18] refactor(wordram): index dynamic programs by operation signatures Remove the implicit relational-to-Empty-state deterministic specification. Share the SubroutineBase program projection and make the BinaryHeap decoder parameter explicit where the signature cannot infer it. Validation: scripts/check.sh passed all six gates under the published rc2 support candidate 0d38523e. Hygiene, changed-file lints and the tidy-surface audit passed. Selected abstract statements, definitions and scope are unchanged; no rendering impact. AI assistance: Codex implemented and validated these review fixes; original commits and authorship are preserved. --- .../WordRAM/PriorityQueue/BinaryHeap.lean | 2 +- Complexity/Computability/WordRAM/Dynamic.lean | 38 ++++++++++--------- .../Computability/WordRAM/Subroutine.lean | 10 +---- .../Semantics/Stateful/Relational.lean | 11 ------ 4 files changed, 22 insertions(+), 39 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean b/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean index f0605c71b..b9de086eb 100644 --- a/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean +++ b/Complexity/Algorithms/WordRAM/PriorityQueue/BinaryHeap.lean @@ -691,7 +691,7 @@ def dynamicProgram (n K_max : ℕ) : DynamicProgram (IndexedMinPQ.spec n K_max) output := pqCode.output input := pqCode.input encode := pqCode.encode - decode := pqCode.decode + decode := pqCode.decode (K_max := K_max) /-- Standalone `.standard`-basis `DynamicCertificate` for the binary min-heap. -/ def certificate {n K_max w base : ℕ} (hS : Fits w (base + space n)) (hK : Fits w K_max) : diff --git a/Complexity/Computability/WordRAM/Dynamic.lean b/Complexity/Computability/WordRAM/Dynamic.lean index ab80464d6..3121bebc1 100644 --- a/Complexity/Computability/WordRAM/Dynamic.lean +++ b/Complexity/Computability/WordRAM/Dynamic.lean @@ -74,8 +74,10 @@ universe u v z variable {r : ℕ} {D : Stateful.Spec.{u, v, z}} -/-- A single program serving all operations through a fixed register calling convention. -/ -structure DynamicProgram (D : Stateful.Spec.{u, v, z}) (r : ℕ) where +/-- A single program serving an operation signature through a fixed register calling convention. +Deterministic and relational stateful specifications supply their signatures by coercion; +the program itself makes no assumption about abstract states or permitted transitions. -/ +structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) where /-- The same code is executed for every operation. -/ code : Code .standard r /-- Register returned by the final halt instruction. -/ @@ -83,30 +85,30 @@ structure DynamicProgram (D : Stateful.Spec.{u, v, z}) (r : ℕ) where /-- Registers overwritten by externally supplied operation arguments. -/ input : Fin r → Bool /-- Encoded arguments, independent of the maintained machine state. -/ - encode : D.Op → Fin r → ℕ + encode : S.A → Fin r → ℕ /-- Interpretation of the final register tuple as the operation's typed observation. -/ - decode : (o : D.Op) → (Fin r → ℕ) → Option (D.Result o) + decode : (o : S.A) → (Fin r → ℕ) → Option (S.B o) namespace DynamicProgram -variable (P : DynamicProgram D r) +variable {S : PFunctor.{v, z}} (P : DynamicProgram S r) /-- Deliver operation arguments without changing maintained memory or non-input registers. -/ -def prepare (o : D.Op) (x : Data r) : Data r where +def prepare (o : S.A) (x : Data r) : Data r where regs i := if P.input i then P.encode o i else x.regs i mem := x.mem /-- Delivering operation arguments preserves the machine memory unchanged. -/ @[simp] -theorem prepare_mem (o : D.Op) (x : Data r) : (P.prepare o x).mem = x.mem := rfl +theorem prepare_mem (o : S.A) (x : Data r) : (P.prepare o x).mem = x.mem := rfl /-- Registers outside the calling convention retain their contents. -/ -theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : D.Op) (x : Data r) : +theorem prepare_regs_of_not_input {i : Fin r} (hi : P.input i = false) (o : S.A) (x : Data r) : (P.prepare o x).regs i = x.regs i := by simp [prepare, hi] /-- Argument delivery preserves bounded words when the arguments fit. -/ -theorem prepare_bounded {w : ℕ} {o : D.Op} {x : Data r} (hx : x.Bounded w) +theorem prepare_bounded {w : ℕ} {o : S.A} {x : Data r} (hx : x.Bounded w) (ho : ∀ i, P.input i = true → Fits w (P.encode o i)) : (P.prepare o x).Bounded w := by refine ⟨fun i => ?_, hx.2⟩ change Fits w (if P.input i then P.encode o i else x.regs i) @@ -120,7 +122,7 @@ instructions; exhaustion, a machine fault, or decoder failure returns `none`. This interpreter does not check the initial state's global word bound or operation admissibility. Use `DynamicCertificate.execOp_of_eval` to connect a successful call to certified execution. -/ -def eval (w fuel : ℕ) (o : D.Op) (x : Data r) : Option (Data r × D.Result o × ℕ) := +def eval (w fuel : ℕ) (o : S.A) (x : Data r) : Option (Data r × S.B o × ℕ) := match P.code.eval w fuel (P.prepare o x) with | none => none | some (y, n) => (P.decode o y.regs).map fun a => (y, a, n + 1) @@ -133,13 +135,13 @@ structure OperationTrace where probes : List Probe /-- A bounded machine call with native execution evidence independent of any resource bound. -/ -def ExecOp (w : ℕ) : Stateful.Execution D (Data r) OperationTrace := fun x o a y trace => +def ExecOp (w : ℕ) : Stateful.Execution S (Data r) OperationTrace := fun x o a y trace => (P.prepare o x).Bounded w ∧ ∃ n, TraceExec w P.code (P.prepare o x) y n trace.probes ∧ P.decode o y.regs = some a ∧ trace.instructions = n + 1 /-- Successful typed evaluation supplies an exact operation trace when the delivered state is bounded. Evaluator success alone does not imply a bound on untouched memory. -/ -theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result o} +theorem ExecOp.of_eval {w fuel k : ℕ} {o : S.A} {x y : Data r} {a : S.B o} (hb : (P.prepare o x).Bounded w) (h : P.eval w fuel o x = some (y, a, k)) : ∃ probes, P.ExecOp w x o a y ⟨k, probes⟩ := by cases he : P.code.eval w fuel (P.prepare o x) with @@ -156,7 +158,7 @@ theorem ExecOp.of_eval {w fuel k : ℕ} {o : D.Op} {x y : Data r} {a : D.Result /-- Every call is an execution of one fixed flat machine program. The returned register tuple is observed at the final state; the designated output register is also the halt value. -/ -theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} +theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} (h : P.ExecOp w x o a y trace) : Executes w (P.code.program P.output) ⟨0, P.prepare o x⟩ (y.regs P.output) trace.instructions := by @@ -165,13 +167,13 @@ theorem ExecOp.executes {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} exact he.exec.executes P.output /-- A call preserves bounded words, including every word in the observed register tuple. -/ -theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} +theorem ExecOp.bounded {w : ℕ} {x y : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} (h : P.ExecOp w x o a y trace) : y.Bounded w := by obtain ⟨hb, n, he, _, _⟩ := h exact he.exec.bounded hb /-- Calls are deterministic in their final representation, observation, and instruction count. -/ -theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} +theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : S.A} {a a' : S.B o} {trace trace' : OperationTrace} (h : P.ExecOp w x o a y trace) (h' : P.ExecOp w x o a' y' trace') : y = y' ∧ a = a' ∧ trace.instructions = trace'.instructions := by @@ -181,8 +183,8 @@ theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : D.Op} {a a' : D.Result o} exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ /-- A call's final state and instruction count agree with any structured execution of the code. -/ -theorem ExecOp.eq_of_exec {P : DynamicProgram D r} {w : ℕ} {x y z : Data r} {o : D.Op} - {a : D.Result o} {trace : OperationTrace} {m : ℕ} (he : P.ExecOp w x o a y trace) +theorem ExecOp.eq_of_exec {P : DynamicProgram S r} {w : ℕ} {x y z : Data r} {o : S.A} + {a : S.B o} {trace : OperationTrace} {m : ℕ} (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : y = z ∧ trace.instructions = m + 1 := by obtain ⟨_, n, ht, _, hn⟩ := he @@ -190,7 +192,7 @@ theorem ExecOp.eq_of_exec {P : DynamicProgram D r} {w : ℕ} {x y z : Data r} {o exact ⟨rfl, hn⟩ /-- The write transcript supplies a frame rule for a call, including register-only delivery. -/ -theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : D.Op} {a : D.Result o} +theorem ExecOp.frame {w : ℕ} {x y : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} (h : P.ExecOp w x o a y trace) {Q : HeapAssertion} (hQ : Q.holds x.mem) (hd : Disjoint Q.footprint (Probe.writes trace.probes)) : Q.holds y.mem := by diff --git a/Complexity/Computability/WordRAM/Subroutine.lean b/Complexity/Computability/WordRAM/Subroutine.lean index bb0969dcc..feffa199e 100644 --- a/Complexity/Computability/WordRAM/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Subroutine.lean @@ -328,7 +328,7 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineBase P Result valid w base /-- Project a `.standard`-basis `SubroutineBase` to a standalone `DynamicProgram`. -/ def toDynamicProgram {P_std : SubroutineCode .standard r_ds k_init} (C : SubroutineBase P_std Result valid w base params s₀) : - DynamicProgram { State := Empty, Op := O, Result := Result, step := Empty.elim } r_ds where + DynamicProgram ⟨O, Result⟩ r_ds where code := P_std.code output := P_std.output input := P_std.input @@ -627,14 +627,6 @@ section Standard variable {P_std : SubroutineCode .standard r_ds k_init} (C : SubroutineDS P_std D w base params s₀) -/-- Project a `.standard`-basis `SubroutineDS` to its underlying standalone `DynamicProgram`. -/ -def toDynamicProgram : DynamicProgram D r_ds where - code := P_std.code - output := P_std.output - input := P_std.input - encode := C.encode - decode := C.decode - /-- Project a `.standard`-basis `SubroutineDS` to a standalone `DynamicCertificate`. -/ def toCertificate : DynamicCertificate C.toDynamicProgram w where rep := C.rep diff --git a/Complexity/Foundations/Semantics/Stateful/Relational.lean b/Complexity/Foundations/Semantics/Stateful/Relational.lean index 722f77f02..6a709a6d1 100644 --- a/Complexity/Foundations/Semantics/Stateful/Relational.lean +++ b/Complexity/Foundations/Semantics/Stateful/Relational.lean @@ -128,17 +128,6 @@ abbrev signature : PFunctor.{v, w} := ⟨D.Op, D.Result⟩ instance : Coe RelSpec.{u, v, w} PFunctor.{v, w} := ⟨RelSpec.signature⟩ -/-- Project the operation signature `(D.Op, D.Result)` of a relational specification to a -canonical `Stateful.Spec` with `State := Empty`, allowing signature-only constructions over -`Stateful.Spec` (such as `WordRAM.DynamicProgram`) to apply directly to `D : RelSpec`. -/ -abbrev toSpec : Spec.{0, v, w} where - State := Empty - Op := D.Op - Result := D.Result - step s := s.elim - -instance : Coe RelSpec.{u, v, w} Spec.{0, v, w} := ⟨RelSpec.toSpec⟩ - /-- An adaptive client over `D`: each continuation may branch on the observed operation result. -/ abbrev Client (α : Type y) : Type (max v w y) := D.signature.FreeM α From a115e42b53774978d96e2202cb74ea05bea5f7e7 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:14:24 -0400 Subject: [PATCH 13/18] refactor(wordram): share basis-preserving subroutine plumbing Keep the standard DynamicProgram specialization while allowing an explicit basis. Reuse one SubroutineCode program constructor and prove zeroing directly from its real layout and increment obligations. Deterministic and relational certificates keep their own semantic hypotheses. Validation: scripts/check.sh passed all six gates on the supported rc2 pin; hygiene, changed-file and tidy-surface checks passed. Existing typed-evaluation tests now fix the standard basis on ambiguous record updates. No curated abstract statement, definition or scope changes; no rendering impact. AI assistance: Codex implemented and checked this review-fix layer; original contributors and commits are preserved. --- Complexity/Computability/WordRAM/Dynamic.lean | 12 +- .../Computability/WordRAM/RelSubroutine.lean | 4 +- .../Computability/WordRAM/Subroutine.lean | 327 +++++++++--------- ComplexityTests/WordRAMDynamic.lean | 6 +- 4 files changed, 184 insertions(+), 165 deletions(-) diff --git a/Complexity/Computability/WordRAM/Dynamic.lean b/Complexity/Computability/WordRAM/Dynamic.lean index 3121bebc1..ace6fd08a 100644 --- a/Complexity/Computability/WordRAM/Dynamic.lean +++ b/Complexity/Computability/WordRAM/Dynamic.lean @@ -14,6 +14,10 @@ public import Complexity.Computability.WordRAM.Trace /-! # Fixed-program implementations of dynamic data structures +`DynamicProgram` depends only on the existing `PFunctor` operation signature and its instruction +basis, which defaults to `.standard`. Deterministic and relational specifications supply their +signatures by coercion; correctness certificates retain the actual specification separately. + An implementation has one structured program, one output register, and a register-only calling convention. Each call replaces designated input registers with encoded words, preserving all other registers and the entire memory. Argument encoding, external delivery, and result decoding @@ -77,9 +81,9 @@ variable {r : ℕ} {D : Stateful.Spec.{u, v, z}} /-- A single program serving an operation signature through a fixed register calling convention. Deterministic and relational stateful specifications supply their signatures by coercion; the program itself makes no assumption about abstract states or permitted transitions. -/ -structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) where +structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) (B : Basis := .standard) where /-- The same code is executed for every operation. -/ - code : Code .standard r + code : Code B r /-- Register returned by the final halt instruction. -/ output : Fin r /-- Registers overwritten by externally supplied operation arguments. -/ @@ -91,7 +95,7 @@ structure DynamicProgram (S : PFunctor.{v, z}) (r : ℕ) where namespace DynamicProgram -variable {S : PFunctor.{v, z}} (P : DynamicProgram S r) +variable {S : PFunctor.{v, z}} {B : Basis} (P : DynamicProgram S r B) /-- Deliver operation arguments without changing maintained memory or non-input registers. -/ def prepare (o : S.A) (x : Data r) : Data r where @@ -183,7 +187,7 @@ theorem ExecOp.unique {w : ℕ} {x y y' : Data r} {o : S.A} {a a' : S.B o} exact ⟨rfl, Option.some.inj (ha.symm.trans ha'), ht.trans ht'.symm⟩ /-- A call's final state and instruction count agree with any structured execution of the code. -/ -theorem ExecOp.eq_of_exec {P : DynamicProgram S r} {w : ℕ} {x y z : Data r} {o : S.A} +theorem ExecOp.eq_of_exec {P : DynamicProgram S r B} {w : ℕ} {x y z : Data r} {o : S.A} {a : S.B o} {trace : OperationTrace} {m : ℕ} (he : P.ExecOp w x o a y trace) (h : Exec w P.code (P.prepare o x) z m) : y = z ∧ trace.instructions = m + 1 := by diff --git a/Complexity/Computability/WordRAM/RelSubroutine.lean b/Complexity/Computability/WordRAM/RelSubroutine.lean index 7e3dece99..ba187f24c 100644 --- a/Complexity/Computability/WordRAM/RelSubroutine.lean +++ b/Complexity/Computability/WordRAM/RelSubroutine.lean @@ -25,7 +25,7 @@ to relational specifications (`Stateful.RelSpec`): `Stateful.RelRealization` directly via `Exec.unique`. 2. **Unified standalone and inline relational subroutine certificates (`RelSubroutineDS`):** Extends `SubroutineBase P D.Result D.valid w base params s₀` over `D : Stateful.RelSpec`, - inheriting `toDynamicProgram`, `initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, + inheriting `toDynamicProgram`, `initFresh_exec`, `initFresh_rename`, `initClear_exec`, `initClear_rename`, `frame_caller`, and `prepareRegs` without duplication, omitting `rep_unique` (since concrete potential `potential : Data r_ds → ℕ` depends only on machine data), and existentially quantifying over `(s', a)` with `D.step s o s' a` in @@ -266,7 +266,7 @@ end SubroutineCode at word width `w`, base address `base`, runtime parameters `params`, and initial state `s₀`. Extends `SubroutineBase P D.Result D.valid w base params s₀` (inheriting `toDynamicProgram`, -`initFresh_exec`, `initFresh_rename`, `zeroFillCode_exec`, `initClear_exec`, `initClear_rename`, +`initFresh_exec`, `initFresh_rename`, `initClear_exec`, `initClear_rename`, `frame_caller`, and `prepareRegs`), omits `rep_unique` (because concrete potential `potential : Data r_ds → ℕ` depends only on machine data), and existentially quantifies over a permitted outcome `(s', a)` satisfying `D.step s o s' a` in `step_exec`. -/ diff --git a/Complexity/Computability/WordRAM/Subroutine.lean b/Complexity/Computability/WordRAM/Subroutine.lean index feffa199e..b6cd0bd1b 100644 --- a/Complexity/Computability/WordRAM/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Subroutine.lean @@ -45,7 +45,7 @@ A data structure is packaged once as a `SubroutineCode B r_ds k_init` and certif standalone / inline specification and resource certificate for a Word-RAM data structure. * `Cslib.WordRAM.SubroutineCode.initFresh`, `Cslib.WordRAM.SubroutineCode.initClear`: `O(1)` fresh initialization on zeroed memory and generic `O(space)` dirty-memory re-initialization. -* `Cslib.WordRAM.SubroutineBase.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: +* `Cslib.WordRAM.SubroutineCode.toDynamicProgram`, `Cslib.WordRAM.SubroutineDS.toCertificate`: standalone `.standard`-basis `DynamicProgram` and `DynamicCertificate` projections. ## Main results @@ -206,6 +206,169 @@ def mapSimulates (φ : B.Simulates B') (P : SubroutineCode B r_ds k_init) : abbrev toStandard (P : SubroutineCode .exact r_ds k_init) : SubroutineCode .standard r_ds k_init := P.mapSimulates .exactStandard +/-- Project subroutine code and its calling convention to the same operation signature. +The basis is retained, so this projection does not translate instructions or resource claims. -/ +def toDynamicProgram (P : SubroutineCode B r_ds k_init) {O : Type*} {Result : O → Type*} + (encode : O → Fin r_ds → ℕ) (decode : (o : O) → (Fin r_ds → ℕ) → Option (Result o)) : + DynamicProgram ⟨O, Result⟩ r_ds B where + code := P.code + output := P.output + input := P.input + encode := encode + decode := decode + +/-- Generic execution of `P.zeroFillCode` clearing `[base, base + space)` in `4 * space + 6` +instructions over any basis `B`. -/ +theorem zeroFillCode_exec (P : SubroutineCode B r_ds k_init) + {w base space : ℕ} {params : Fin k_init → ℕ} + (hclear : [P.curReg, P.zeroReg, P.oneReg, P.baseReg, P.frontierReg].Nodup) + (hparams : ∀ j, P.initParams j ∉ [P.curReg, P.zeroReg, P.oneReg]) + (hinc : ∀ {x : Data r_ds}, x.regs P.oneReg = 1 → Fits w (x.regs P.curReg + 1) → + P.incCur.eval w x = some (x.set P.curReg (x.regs P.curReg + 1))) + {x₁ : Data r_ds} + (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) + (hfr₁ : x₁.regs P.frontierReg = base + space) + (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : + ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * space + 6) ∧ + x₂.Bounded w ∧ + x₂.regs P.baseReg = base ∧ + x₂.regs P.frontierReg = base + space ∧ + (∀ j, x₂.regs (P.initParams j) = params j) ∧ + (∀ a ∈ (Block.mk base space).footprint, x₂.mem a = 0) ∧ + Modifies (Block.mk base space).footprint x₁.mem x₂.mem := by + have hfit0 : Fits w 0 := (hx₁.1 P.baseReg).of_le (Nat.zero_le _) + have hfit_stop : Fits w (base + space) := hfr₁ ▸ hx₁.1 P.frontierReg + obtain ⟨hcur_not, hzero_not, hone_not, hbase_ne_fr, _⟩ := + (List.nodup_cons.mp hclear).imp_right fun h => + (List.nodup_cons.mp h).imp_right fun h => + (List.nodup_cons.mp h).imp_right List.nodup_cons.mp + have hcur_ne_zero : P.curReg ≠ P.zeroReg := fun h => hcur_not (by simp [h]) + have hcur_ne_one : P.curReg ≠ P.oneReg := fun h => hcur_not (by simp [h]) + have hcur_ne_base : P.curReg ≠ P.baseReg := fun h => hcur_not (by simp [h]) + have hcur_ne_fr : P.curReg ≠ P.frontierReg := fun h => hcur_not (by simp [h]) + have hzero_ne_one : P.zeroReg ≠ P.oneReg := fun h => hzero_not (by simp [h]) + have hzero_ne_base : P.zeroReg ≠ P.baseReg := fun h => hzero_not (by simp [h]) + have hzero_ne_fr : P.zeroReg ≠ P.frontierReg := fun h => hzero_not (by simp [h]) + have hone_ne_base : P.oneReg ≠ P.baseReg := fun h => hone_not (by simp [h]) + have hone_ne_fr : P.oneReg ≠ P.frontierReg := fun h => hone_not (by simp [h]) + by_cases hsp : 0 < space + · have hc_pos : (Cond.lt P.baseReg P.frontierReg).Holds x₁ := by + simp only [Cond.Holds, hb₁, hfr₁] + omega + have hfit1 : Fits w 1 := hfit_stop.of_le (by omega) + let x_z := x₁.set P.zeroReg 0 + let x_o := x_z.set P.oneReg 1 + let x_c := x_o.set P.curReg (x_o.regs P.baseReg) + have hxc_base : x_c.regs P.baseReg = base := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_base.symm, + Function.update_of_ne hone_ne_base.symm, Function.update_of_ne hzero_ne_base.symm, hb₁] + have hxc_fr : x_c.regs P.frontierReg = base + space := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_fr.symm, + Function.update_of_ne hone_ne_fr.symm, Function.update_of_ne hzero_ne_fr.symm, hfr₁] + have hxc_zero : x_c.regs P.zeroReg = 0 := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_zero.symm, + Function.update_of_ne hzero_ne_one] + have hxc_one : x_c.regs P.oneReg = 1 := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_one.symm] + have hxc_cur : x_c.regs P.curReg = base := by + simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hone_ne_base.symm, + Function.update_of_ne hzero_ne_base.symm, hb₁] + let xs : ℕ → Data r_ds := fun i => + ⟨Function.update x_c.regs P.curReg (base + i), clear x₁.mem base i⟩ + have hxs0 : xs 0 = x_c := by + ext r + · by_cases hr : r = P.curReg + · subst hr + simp [xs, hxc_cur] + · simp [xs, Function.update_of_ne hr, x_c, x_o, x_z, Data.set] + · simp [xs, x_c, x_o, x_z, Data.set] + have hloop_cond : ∀ i < space, (Cond.lt P.curReg P.frontierReg).Holds (xs i) := by + intro i hi + simp only [Cond.Holds, xs, Function.update_self, + Function.update_of_ne hcur_ne_fr.symm, hxc_fr] + omega + have hloop_exit : ¬(Cond.lt P.curReg P.frontierReg).Holds (xs space) := by + simp only [Cond.Holds, xs, Function.update_self, + Function.update_of_ne hcur_ne_fr.symm, hxc_fr] + omega + have hloop_body : ∀ i < space, + ExecWithin w (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur)) + (xs i) (xs (i + 1)) 2 := by + intro i hi + have hsi_zero : (xs i).regs P.zeroReg = 0 := by + simp [xs, Function.update_of_ne hcur_ne_zero.symm, hxc_zero] + have hsi_one : (xs i).regs P.oneReg = 1 := by + simp [xs, Function.update_of_ne hcur_ne_one.symm, hxc_one] + have hsi_cur : (xs i).regs P.curReg = base + i := by simp [xs] + let x_st := (xs i).write ((xs i).regs P.curReg) ((xs i).regs P.zeroReg) + have hst_one : x_st.regs P.oneReg = 1 := hsi_one + have hst_fit : Fits w (x_st.regs P.curReg + 1) := by + change Fits w ((xs i).regs P.curReg + 1) + rw [hsi_cur] + exact hfit_stop.of_le (by omega) + have hinc : ExecWithin w (.op P.incCur) x_st (x_st.set P.curReg (x_st.regs P.curReg + 1)) 1 := + (Exec.op (hinc hst_one hst_fit)).within + have htarget : x_st.set P.curReg (x_st.regs P.curReg + 1) = xs (i + 1) := by + ext r + · simp only [x_st, Data.set, Data.write, xs, hsi_cur] + by_cases hr : r = P.curReg + · subst hr + simp [Nat.add_assoc] + · simp [Function.update_of_ne hr] + · simp only [x_st, Data.set, Data.write, hsi_cur, hsi_zero] + simp [xs, clear_succ] + rw [htarget] at hinc + exact ExecWithin.store.seq hinc + have hloop : ExecWithin w + (.loop (.lt P.curReg P.frontierReg) (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) + x_c (xs space) (space * (2 + 2) + 2) := by + simpa [hxs0] using ExecWithin.loop xs space 2 hloop_cond hloop_exit hloop_body + have hbranch : ExecWithin w P.zeroFillCode x₁ (xs space) (4 * space + 6) := by + have hseq : ExecWithin w + (.seq (.op (.const P.zeroReg 0)) <| + .seq (.op (.const P.oneReg 1)) <| + .seq (.op (.copy P.curReg P.baseReg)) <| + .loop (.lt P.curReg P.frontierReg) + (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) + x₁ (xs space) (3 + (space * 4 + 2)) := by + obtain ⟨n₁, hn₁, he₁⟩ := (ExecWithin.const (x := x₁) (d := P.zeroReg) hfit0) + obtain ⟨n₂, hn₂, he₂⟩ := (ExecWithin.const (x := x_z) (d := P.oneReg) hfit1) + obtain ⟨n₃, hn₃, he₃⟩ := (ExecWithin.copy (B := B) (w := w) (x := x_o) + (d := P.curReg) (s := P.baseReg)) + obtain ⟨n₄, hn₄, he₄⟩ := hloop + exact ⟨n₁ + (n₂ + (n₃ + n₄)), by omega, he₁.seq (he₂.seq (he₃.seq he₄))⟩ + exact (ExecWithin.branch_pos hc_pos hseq).mono (by omega) + refine ⟨xs space, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ + · simp [xs, Function.update_of_ne hcur_ne_base.symm, hxc_base] + · simp [xs, Function.update_of_ne hcur_ne_fr.symm, hxc_fr] + · intro j + have hne := hparams j + have h1 : P.initParams j ≠ P.curReg := fun h => hne (by simp [h]) + have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) + have h3 : P.initParams j ≠ P.oneReg := fun h => hne (by simp [h]) + simp [xs, x_c, x_o, x_z, Data.set, Function.update_of_ne h1, Function.update_of_ne h2, + Function.update_of_ne h3, hpar₁ j] + · intro a ha + exact clear_of_owns (Block.mem_footprint.mp ha) + · intro a ha + exact clear_of_not_owns (mt Block.mem_footprint.mpr ha) + · have hsp0 : space = 0 := by omega + have hc_neg : ¬(Cond.lt P.baseReg P.frontierReg).Holds x₁ := by + simp only [Cond.Holds, hb₁, hfr₁, hsp0] + omega + have hbranch : ExecWithin w P.zeroFillCode x₁ (x₁.set P.zeroReg 0) (4 * space + 6) := + (ExecWithin.branch_neg hc_neg (ExecWithin.const hfit0)).mono (by omega) + refine ⟨x₁.set P.zeroReg 0, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ + · simp [Data.set, Function.update_of_ne hzero_ne_base.symm, hb₁] + · simp [Data.set, Function.update_of_ne hzero_ne_fr.symm, hfr₁] + · intro j + have hne := hparams j + have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) + simp [Data.set, Function.update_of_ne h2, hpar₁ j] + · intro a ha + simp [hsp0, Block.footprint] at ha + · exact Modifies.refl _ _ + end SubroutineCode universe u v z @@ -325,15 +488,10 @@ def mapSimulates (φ : B.Simulates B') (C : SubroutineBase P Result valid w base let ⟨y, he, hfr', hrep, hpot, hmod⟩ := C.setup_exec hx₁ hb hfr hpar hz ⟨y, he.mapSimulates φ, hfr', hrep, hpot, hmod⟩ -/-- Project a `.standard`-basis `SubroutineBase` to a standalone `DynamicProgram`. -/ -def toDynamicProgram {P_std : SubroutineCode .standard r_ds k_init} - (C : SubroutineBase P_std Result valid w base params s₀) : - DynamicProgram ⟨O, Result⟩ r_ds where - code := P_std.code - output := P_std.output - input := P_std.input - encode := C.encode - decode := C.decode +/-- Project a subroutine bundle to its signature-level program, retaining the instruction basis. -/ +def toDynamicProgram (C : SubroutineBase P Result valid w base params s₀) : + DynamicProgram ⟨O, Result⟩ r_ds B := + P.toDynamicProgram C.encode C.decode /-- Execute `P.initFresh` on zeroed `[base, base + C.space)` in `C.initTime` instructions. -/ theorem initFresh_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} @@ -378,152 +536,6 @@ theorem initFresh_rename (C : SubroutineBase P Result valid w base params s₀) · rw [Data.comap_extend hf] exact hpot -/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` -instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : SubroutineBase P Result valid w base params s₀) {x₁ : Data r_ds} - (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) - (hfr₁ : x₁.regs P.frontierReg = base + C.space) - (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : - ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ - x₂.Bounded w ∧ - x₂.regs P.baseReg = base ∧ - x₂.regs P.frontierReg = base + C.space ∧ - (∀ j, x₂.regs (P.initParams j) = params j) ∧ - (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ - Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := by - have hfit0 : Fits w 0 := (hx₁.1 P.baseReg).of_le (Nat.zero_le _) - have hfit_stop : Fits w (base + C.space) := hfr₁ ▸ hx₁.1 P.frontierReg - obtain ⟨hcur_not, hzero_not, hone_not, hbase_ne_fr, _⟩ := - (List.nodup_cons.mp C.clear_nodup).imp_right fun h => - (List.nodup_cons.mp h).imp_right fun h => - (List.nodup_cons.mp h).imp_right List.nodup_cons.mp - have hcur_ne_zero : P.curReg ≠ P.zeroReg := fun h => hcur_not (by simp [h]) - have hcur_ne_one : P.curReg ≠ P.oneReg := fun h => hcur_not (by simp [h]) - have hcur_ne_base : P.curReg ≠ P.baseReg := fun h => hcur_not (by simp [h]) - have hcur_ne_fr : P.curReg ≠ P.frontierReg := fun h => hcur_not (by simp [h]) - have hzero_ne_one : P.zeroReg ≠ P.oneReg := fun h => hzero_not (by simp [h]) - have hzero_ne_base : P.zeroReg ≠ P.baseReg := fun h => hzero_not (by simp [h]) - have hzero_ne_fr : P.zeroReg ≠ P.frontierReg := fun h => hzero_not (by simp [h]) - have hone_ne_base : P.oneReg ≠ P.baseReg := fun h => hone_not (by simp [h]) - have hone_ne_fr : P.oneReg ≠ P.frontierReg := fun h => hone_not (by simp [h]) - by_cases hsp : 0 < C.space - · have hc_pos : (Cond.lt P.baseReg P.frontierReg).Holds x₁ := by - simp only [Cond.Holds, hb₁, hfr₁] - omega - have hfit1 : Fits w 1 := hfit_stop.of_le (by omega) - let x_z := x₁.set P.zeroReg 0 - let x_o := x_z.set P.oneReg 1 - let x_c := x_o.set P.curReg (x_o.regs P.baseReg) - have hxc_base : x_c.regs P.baseReg = base := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_base.symm, - Function.update_of_ne hone_ne_base.symm, Function.update_of_ne hzero_ne_base.symm, hb₁] - have hxc_fr : x_c.regs P.frontierReg = base + C.space := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_fr.symm, - Function.update_of_ne hone_ne_fr.symm, Function.update_of_ne hzero_ne_fr.symm, hfr₁] - have hxc_zero : x_c.regs P.zeroReg = 0 := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_zero.symm, - Function.update_of_ne hzero_ne_one] - have hxc_one : x_c.regs P.oneReg = 1 := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hcur_ne_one.symm] - have hxc_cur : x_c.regs P.curReg = base := by - simp [x_c, x_o, x_z, Data.set, Function.update_of_ne hone_ne_base.symm, - Function.update_of_ne hzero_ne_base.symm, hb₁] - let xs : ℕ → Data r_ds := fun i => - ⟨Function.update x_c.regs P.curReg (base + i), clear x₁.mem base i⟩ - have hxs0 : xs 0 = x_c := by - ext r - · by_cases hr : r = P.curReg - · subst hr - simp [xs, hxc_cur] - · simp [xs, Function.update_of_ne hr, x_c, x_o, x_z, Data.set] - · simp [xs, x_c, x_o, x_z, Data.set] - have hloop_cond : ∀ i < C.space, (Cond.lt P.curReg P.frontierReg).Holds (xs i) := by - intro i hi - simp only [Cond.Holds, xs, Function.update_self, - Function.update_of_ne hcur_ne_fr.symm, hxc_fr] - omega - have hloop_exit : ¬(Cond.lt P.curReg P.frontierReg).Holds (xs C.space) := by - simp only [Cond.Holds, xs, Function.update_self, - Function.update_of_ne hcur_ne_fr.symm, hxc_fr] - omega - have hloop_body : ∀ i < C.space, - ExecWithin w (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur)) - (xs i) (xs (i + 1)) 2 := by - intro i hi - have hsi_zero : (xs i).regs P.zeroReg = 0 := by - simp [xs, Function.update_of_ne hcur_ne_zero.symm, hxc_zero] - have hsi_one : (xs i).regs P.oneReg = 1 := by - simp [xs, Function.update_of_ne hcur_ne_one.symm, hxc_one] - have hsi_cur : (xs i).regs P.curReg = base + i := by simp [xs] - let x_st := (xs i).write ((xs i).regs P.curReg) ((xs i).regs P.zeroReg) - have hst_one : x_st.regs P.oneReg = 1 := hsi_one - have hst_fit : Fits w (x_st.regs P.curReg + 1) := by - change Fits w ((xs i).regs P.curReg + 1) - rw [hsi_cur] - exact hfit_stop.of_le (by omega) - have hinc : ExecWithin w (.op P.incCur) x_st (x_st.set P.curReg (x_st.regs P.curReg + 1)) 1 := - (Exec.op (C.incCur_eval hst_one hst_fit)).within - have htarget : x_st.set P.curReg (x_st.regs P.curReg + 1) = xs (i + 1) := by - ext r - · simp only [x_st, Data.set, Data.write, xs, hsi_cur] - by_cases hr : r = P.curReg - · subst hr - simp [Nat.add_assoc] - · simp [Function.update_of_ne hr] - · simp only [x_st, Data.set, Data.write, hsi_cur, hsi_zero] - simp [xs, clear_succ] - rw [htarget] at hinc - exact ExecWithin.store.seq hinc - have hloop : ExecWithin w - (.loop (.lt P.curReg P.frontierReg) (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) - x_c (xs C.space) (C.space * (2 + 2) + 2) := by - simpa [hxs0] using ExecWithin.loop xs C.space 2 hloop_cond hloop_exit hloop_body - have hbranch : ExecWithin w P.zeroFillCode x₁ (xs C.space) (4 * C.space + 6) := by - have hseq : ExecWithin w - (.seq (.op (.const P.zeroReg 0)) <| - .seq (.op (.const P.oneReg 1)) <| - .seq (.op (.copy P.curReg P.baseReg)) <| - .loop (.lt P.curReg P.frontierReg) - (.seq (.op (.store P.curReg P.zeroReg)) (.op P.incCur))) - x₁ (xs C.space) (3 + (C.space * 4 + 2)) := by - obtain ⟨n₁, hn₁, he₁⟩ := (ExecWithin.const (x := x₁) (d := P.zeroReg) hfit0) - obtain ⟨n₂, hn₂, he₂⟩ := (ExecWithin.const (x := x_z) (d := P.oneReg) hfit1) - obtain ⟨n₃, hn₃, he₃⟩ := (ExecWithin.copy (B := B) (w := w) (x := x_o) - (d := P.curReg) (s := P.baseReg)) - obtain ⟨n₄, hn₄, he₄⟩ := hloop - exact ⟨n₁ + (n₂ + (n₃ + n₄)), by omega, he₁.seq (he₂.seq (he₃.seq he₄))⟩ - exact (ExecWithin.branch_pos hc_pos hseq).mono (by omega) - refine ⟨xs C.space, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ - · simp [xs, Function.update_of_ne hcur_ne_base.symm, hxc_base] - · simp [xs, Function.update_of_ne hcur_ne_fr.symm, hxc_fr] - · intro j - have hne := C.params_not_clear j - have h1 : P.initParams j ≠ P.curReg := fun h => hne (by simp [h]) - have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) - have h3 : P.initParams j ≠ P.oneReg := fun h => hne (by simp [h]) - simp [xs, x_c, x_o, x_z, Data.set, Function.update_of_ne h1, Function.update_of_ne h2, - Function.update_of_ne h3, hpar₁ j] - · intro a ha - exact clear_of_owns (Block.mem_footprint.mp ha) - · intro a ha - exact clear_of_not_owns (mt Block.mem_footprint.mpr ha) - · have hsp0 : C.space = 0 := by omega - have hc_neg : ¬(Cond.lt P.baseReg P.frontierReg).Holds x₁ := by - simp only [Cond.Holds, hb₁, hfr₁, hsp0] - omega - have hbranch : ExecWithin w P.zeroFillCode x₁ (x₁.set P.zeroReg 0) (4 * C.space + 6) := - (ExecWithin.branch_neg hc_neg (ExecWithin.const hfit0)).mono (by omega) - refine ⟨x₁.set P.zeroReg 0, hbranch, hbranch.bounded hx₁, ?_, ?_, ?_, ?_, ?_⟩ - · simp [Data.set, Function.update_of_ne hzero_ne_base.symm, hb₁] - · simp [Data.set, Function.update_of_ne hzero_ne_fr.symm, hfr₁] - · intro j - have hne := C.params_not_clear j - have h2 : P.initParams j ≠ P.zeroReg := fun h => hne (by simp [h]) - simp [Data.set, Function.update_of_ne h2, hpar₁ j] - · intro a ha - simp [hsp0, Block.footprint] at ha - · exact Modifies.refl _ _ - /-- Execute `P.initClear` on **arbitrary dirty memory** in `C.initTime + 4 * C.space + 6` instructions. -/ theorem initClear_exec (C : SubroutineBase P Result valid w base params s₀) {x : Data r_ds} @@ -536,7 +548,8 @@ theorem initClear_exec (C : SubroutineBase P Result valid w base params s₀) {x Modifies (Block.mk base C.space).footprint x.mem y.mem := by obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := - C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + P.zeroFillCode_exec C.clear_nodup C.params_not_clear C.incCur_eval + (he₁.bounded hx) hb₁ hfr₁ hpar₁ obtain ⟨y, he₃, hfr₃, hrep, hpot, hmod₃⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ rw [hm₁] at hmod₂ have hmod : Modifies (Block.mk base C.space).footprint x.mem y.mem := by diff --git a/ComplexityTests/WordRAMDynamic.lean b/ComplexityTests/WordRAMDynamic.lean index 3e98d9e66..d27a344b5 100644 --- a/ComplexityTests/WordRAMDynamic.lean +++ b/ComplexityTests/WordRAMDynamic.lean @@ -147,9 +147,11 @@ example : (fun (_, answer, _) => answer) = some none ∧ ((Stack.implementation 4 2).eval 4 0 .size (Stack.initial 3)).map (fun (_, answer, _) => answer) = some 0 ∧ - ({ exchangeProgram 4 with decode := fun _ _ => none }).eval 4 0 7 + ({ exchangeProgram 4 with decode := fun _ _ => none } : + DynamicProgram (exchangeSpec 4) 3 .standard).eval 4 0 7 ⟨fun _ => 0, fun _ => 0⟩ = none ∧ - ({ exchangeProgram 4 with code := .loop (.eq 0 0) (.op (.copy 0 0)) }).eval 4 2 7 + ({ exchangeProgram 4 with code := .loop (.eq 0 0) (.op (.copy 0 0)) } : + DynamicProgram (exchangeSpec 4) 3 .standard).eval 4 2 7 ⟨fun _ => 0, fun _ => 0⟩ = none := by decide From fb42ef60e8b0a8a96fcf3cb1b6cccc7ea3ef5057 Mon Sep 17 00:00:00 2001 From: Samuel Giuseppe Schlesinger Date: Thu, 29 Oct 2026 18:31:40 -0400 Subject: [PATCH 14/18] refactor(wordram): reuse randomized subroutine plumbing and probability APIs Remove surrogate deterministic zeroing and copied argument delivery in favor of the actual basis-preserving SubroutineCode/DynamicProgram APIs. Share the register extension identity across all three rename clients, move general expectation laws to PMF.Expect and reuse Mathlib measure and finite-sum facts. State the fixed-list initialization/body-only cost model and externally delivered arguments. Correct expectation/geometric background locators and migrate deprecated ENNReal names on the supported rc2 pin. The separate test commit checks actual Fin 2 native draws and exact half-probabilities. Validation: default scripts/check.sh passed all six gates; hygiene, changed-file lints against the shared a115e42b layer, and the tidy-surface audit passed. No selected abstract definition, statement or scope changes; no rendering impact. Original geometric/native mathematics is unchanged. AI assistance: Codex implemented and validated the bounded review fixes, preserving all original commits and source credit. --- .../WordRAM/Randomized/Expected.lean | 145 +++++---------- .../WordRAM/Randomized/Structured.lean | 14 +- .../WordRAM/Randomized/Subroutine.lean | 170 ++++++------------ .../Computability/WordRAM/Structured.lean | 3 +- Complexity/Probability/PMF/Expect.lean | 23 ++- 5 files changed, 126 insertions(+), 229 deletions(-) diff --git a/Complexity/Computability/WordRAM/Randomized/Expected.lean b/Complexity/Computability/WordRAM/Randomized/Expected.lean index b0c4ee3e8..d26d77662 100644 --- a/Complexity/Computability/WordRAM/Randomized/Expected.lean +++ b/Complexity/Computability/WordRAM/Randomized/Expected.lean @@ -12,6 +12,7 @@ public import Complexity.Computability.WordRAM.Randomized.Structured public import Complexity.Languages.Probabilistic.Expect public import Complexity.Probability.PMF.Bind public import Complexity.Probability.PMF.Event +import Complexity.Probability.PMF.Expect /-! # Expected running time for randomized word RAMs @@ -72,14 +73,14 @@ additivity (`expectedCodeTime_seq`), branch laws (`expectedCodeTime_branch_pos`, geometric Las Vegas `while` loops and structured programs. Discrete and conditional expectation principles are classical background; see -[MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. The particular native tail-sum/Bellman +[MitzenmacherUpfal2017], Sections 2.1, 2.3, and 2.4. The particular native tail-sum/Bellman definitions, finite structured-execution-tree bridge and ranked-loop expected-cost formalization are original library constructions in this module. ## References * [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017], - Sections 1.5, 2.1, and 2.3. + Sections 2.1, 2.3, and 2.4. -/ @[expose] public section @@ -673,44 +674,17 @@ theorem evalRand_halted_mono (w : ℕ) (p : Program B r) (s : State r) (v : ℕ) = evalRand w p T s (.halted v d) * (stepRand w p (.halted v d)).eval (.halted v d) := by simp [stepRand_halted] _ ≤ ∑' cfg, evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d) := - ENNReal.le_tsum + le_tsum (f := fun cfg => evalRand w p T s cfg * (stepRand w p cfg).eval (.halted v d)) (RandCfg.halted v d) -private theorem finsetSum_iSup_of_monotone {α : Type*} (s : Finset α) {f : ℕ → α → ℝ≥0∞} - (hf : ∀ a, Monotone fun n => f n a) : - (∑ a ∈ s, ⨆ n, f n a) = ⨆ n, ∑ a ∈ s, f n a := by - classical - induction s using Finset.induction_on with - | empty => simp - | @insert a₀ s ha₀ ih => - rw [Finset.sum_insert ha₀, ih] - refine le_antisymm ?_ (iSup_le fun n => ?_) - · rw [ENNReal.add_iSup] - simp_rw [ENNReal.iSup_add] - refine iSup_le fun j => iSup_le fun i => le_iSup_of_le (max i j) ?_ - rw [Finset.sum_insert ha₀] - exact add_le_add (hf a₀ (le_max_left i j)) - (Finset.sum_le_sum fun a _ => hf a (le_max_right i j)) - · rw [Finset.sum_insert ha₀] - exact add_le_add (le_iSup (fun k => f k a₀) n) - (le_iSup (fun k => ∑ a ∈ s, f k a) n) - private theorem tsum_iSup_of_monotone {α : Type*} {f : ℕ → α → ℝ≥0∞} (hf : ∀ a, Monotone fun n => f n a) : (∑' a, ⨆ n, f n a) = ⨆ n, ∑' a, f n a := by - rw [ENNReal.tsum_eq_iSup_sum] - simp_rw [finsetSum_iSup_of_monotone _ hf] + rw [tsum_eq_iSup_sum] + simp_rw [ENNReal.finsetSum_iSup_of_monotone hf] rw [iSup_comm] - simp_rw [← ENNReal.tsum_eq_iSup_sum] - -private theorem tsum_finsetSum {α β : Type*} (s : Finset β) (f : β → α → ℝ≥0∞) : - (∑' a, ∑ b ∈ s, f b a) = ∑ b ∈ s, ∑' a, f b a := by - classical - induction s using Finset.induction_on with - | empty => simp - | @insert b₀ s hb₀ ih => - simp only [Finset.sum_insert hb₀, ENNReal.tsum_add, ih] + simp_rw [← tsum_eq_iSup_sum] /-- Finite-horizon approximation of `expectedStepsFrom`: expected number of active (`.running`) steps within the first `N` steps starting from configuration `cfg`. -/ @@ -766,7 +740,7 @@ theorem expectedStepsWithin_running_succ (w : ℕ) (p : Program B r) (N : ℕ) ( intro t rw [runRand_succ, ProbComp.eval_bind, PMF.toOuterMeasure_bind_apply] simp only [hsucc] - rw [← tsum_finsetSum] + rw [← Summable.tsum_finsetSum (fun _ _ => CompleteLattice.summable)] simp only [ProbComp.expect, expectedStepsWithin, Finset.mul_sum] theorem expectedStepsWithin_running_le (w : ℕ) (p : Program B r) (N : ℕ) (s : State r) : @@ -910,7 +884,7 @@ theorem ExecRand.evalRand_faulted_eq_zero {w : ℕ} {c : Code B r} {x : Data r} | zero => simp [evalRand] | succ k => rw [evalRand, runRand_succ, stepRand_rand_of_fits (s := ⟨pc, x⟩) hp.head hfit, bind_assoc, - ProbComp.eval_bind, PMF.bind_apply, ENNReal.tsum_eq_zero] + ProbComp.eval_bind, PMF.bind_apply, tsum_eq_zero_iff] intro sample have hmem : (x.set dst sample.val, 1) ∈ (ProbComp.unifFin (x.regs src) >>= fun s => pure (x.set dst s.val, 1)).support := @@ -1162,7 +1136,7 @@ theorem loopIterPMF_mono_of_not_holds (cond : Cond r) (μ_body : Data r → PMF = loopIterPMF cond μ_body K x y * loopStepPMF cond μ_body y y := by rw [loopStepPMF_of_not_holds hy, PMF.pure_apply_self, mul_one] _ ≤ ∑' z, loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y := - ENNReal.le_tsum + le_tsum (f := fun z => loopIterPMF cond μ_body K x z * loopStepPMF cond μ_body z y) y theorem loopIterPMF_indicator_mono (cond : Cond r) (μ_body : Data r → PMF (Data r)) @@ -1203,7 +1177,7 @@ theorem loopIterPMF_support_inv {cond : Cond r} {I Q : Data r → Prop} private theorem pmf_toOuterMeasure_compl_add (μ : PMF (Data r)) (cond : Cond r) : μ.toOuterMeasure {y | ¬cond.Holds y} + μ.toOuterMeasure {y | cond.Holds y} = 1 := by - rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← ENNReal.tsum_add] + rw [PMF.toOuterMeasure_apply, PMF.toOuterMeasure_apply, ← tsum_add] convert μ.tsum_coe using 1 congr 1 funext y @@ -1257,7 +1231,7 @@ private theorem loopActiveSteps_succ_of_holds {cond : Cond r} {μ_body : Data r intro K rw [loopIterPMF_succ, loopStepPMF_of_holds hc, PMF.toOuterMeasure_bind_apply] simp only [hsucc] - rw [← tsum_finsetSum] + rw [← Summable.tsum_finsetSum (fun _ _ => CompleteLattice.summable)] simp only [loopActiveSteps, Finset.mul_sum] private theorem loopActiveSteps_le {cond : Cond r} {I Q : Data r → Prop} @@ -1273,7 +1247,7 @@ private theorem loopActiveSteps_le {cond : Cond r} {I Q : Data r → Prop} rcases hx with ⟨hcx, hIx⟩ | ⟨hcx, _⟩ · simp only [hcx, ↓reduceIte] rw [loopActiveSteps_succ_of_holds hcx] - refine (add_le_add le_rfl (ENNReal.tsum_le_tsum fun z => ?_)).trans + refine (add_le_add le_rfl (tsum_le_tsum fun z => ?_)).trans (pmf_step_drift_bound (μ_body x) cond hp_pos hp_top (hp_exit x hIx hcx)) by_cases hz : z ∈ (μ_body x).support · exact mul_le_mul_right (ih (hinv x hIx hcx z hz)) _ @@ -1288,7 +1262,8 @@ theorem hasSum_loopExitDensity {cond : Cond r} {I Q : Data r → Prop} {x : Data r} (hx : LoopStateInv cond I Q x) : HasSum (fun y => ⨆ K : ℕ, Set.indicator {z | ¬cond.Holds z} (loopIterPMF cond μ_body K x) y) 1 := by - rw [ENNReal.summable.hasSum_iff, tsum_iSup_of_monotone (loopIterPMF_indicator_mono cond μ_body x)] + rw [CompleteLattice.summable.hasSum_iff, + tsum_iSup_of_monotone (loopIterPMF_indicator_mono cond μ_body x)] simp_rw [← PMF.toOuterMeasure_apply] set b := ⨆ K : ℕ, (loopIterPMF cond μ_body K x).toOuterMeasure {z | ¬cond.Holds z} have hb_le : b ≤ 1 := iSup_le fun K => Cslib.Probability.PMF.toOuterMeasure_le_one _ _ @@ -1431,23 +1406,6 @@ theorem mem_support_loopExitPMF {cond : Cond r} {I Q : Data r → Prop} · exact absurd hcy hnc · exact ⟨⟨hnc, hQy⟩, K, hK⟩ -/-- Expectation of `k : β → ℝ≥0∞` under `p.map g` equals expectation of `k ∘ g` under `p`. -/ -theorem tsum_pmf_map {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : - (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by - simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] - rw [ENNReal.tsum_comm] - refine tsum_congr fun a => ?_ - rw [tsum_eq_single (g a)] - · simp - · intro b hb - simp [hb] - -/-- Expectation of `k : β → ℝ≥0∞` under `p.bind f` decomposes as an iterated expectation. -/ -theorem tsum_pmf_bind {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : - (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by - simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] - exact ENNReal.tsum_comm - /-! ### Compositional distributional execution (`ExecCodeDist`) -/ /-- `ExecCodeDist w c x μ T`: at word size `w`, running structured program `c` from data `x` @@ -1543,7 +1501,7 @@ theorem seq_const {μ₂ : Data r → PMF (Data r)} {T₂ : Data r → ℝ≥0 calc (∑' y, μ₁ y * T₂ y) ≤ ∑' y, μ₁ y * B₂ := by - refine ENNReal.tsum_le_tsum fun y => ?_ + refine tsum_le_tsum fun y => ?_ by_cases hy : y ∈ μ₁.support · exact mul_le_mul_right (hB₂ y hy) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] @@ -1663,29 +1621,12 @@ theorem mapSimulates {B' : Basis} (ψ : B.Simulates B') (h : ExecCodeDist w c x | loop_geometric I Q hp_pos hp_top _ hB hinv hp_exit hx ih_body => exact loop_geometric I Q hp_pos hp_top ih_body hB hinv hp_exit hx -private theorem extend_comap_eq {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (x : Data r') : - (x.comap f).extend f x = x := by - ext j - · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j - by_cases hj : ∃ i, f i = j - · obtain ⟨i, rfl⟩ := hj - exact hf.extend_apply _ _ i - · exact Function.extend_apply' _ _ _ hj - · rfl - private theorem extend_left_injective {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) (x : Data r') : Function.Injective (fun y : Data r => y.extend f x) := by intro y₁ y₂ h have h' := congrArg (Data.comap f) h simpa [Data.comap_extend hf] using h' -private theorem pmf_map_toOuterMeasure {α β : Type*} (p : PMF α) (g : α → β) (S : Set β) : - (p.map g).toOuterMeasure S = p.toOuterMeasure (g ⁻¹' S) := by - rw [PMF.map, PMF.toOuterMeasure_bind_apply, PMF.toOuterMeasure_apply] - refine tsum_congr fun a => ?_ - rw [Function.comp_apply, PMF.toOuterMeasure_pure_apply] - by_cases ha : g a ∈ S <;> simp [ha] - private theorem pmf_map_apply_injective {α β : Type*} (p : PMF α) {g : α → β} (hg : g.Injective) (a : α) : p.map g (g a) = p a := by @@ -1698,7 +1639,7 @@ private theorem pmf_map_apply_injective {α β : Type*} (p : PMF α) {g : α → private theorem pmf_map_apply_not_mem_range {α β : Type*} (p : PMF α) (g : α → β) {b : β} (hb : b ∉ Set.range g) : p.map g b = 0 := by - rw [PMF.map_apply, ENNReal.tsum_eq_zero] + rw [PMF.map_apply, tsum_eq_zero_iff] intro a have hne : b ≠ g a := fun h => hb ⟨a, h.symm⟩ simp [hne] @@ -1757,7 +1698,7 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' refine congrArg (μ₁.bind ·) (funext fun y₁ => ?_) simp [Data.comap_extend hf] · refine add_le_add le_rfl ?_ - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] simp [Data.comap_extend hf] | branch_pos hc _ ih => subst hx₀ @@ -1791,7 +1732,7 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' rintro z' ⟨u, hIu, rfl⟩ hcz' have hcu : cond.Holds u := by simpa [g, Cond.holds_rename, Data.comap_extend hf] using hcz' - simp only [μ_body', g, Data.comap_extend hf, pmf_map_toOuterMeasure] + simp only [μ_body', g, Data.comap_extend hf, PMF.toOuterMeasure_map_apply] have hset : ((fun y : Data r => y.extend f x) ⁻¹' {y' | ¬(cond.rename f).Holds y'}) = {y | ¬cond.Holds y} := by ext y @@ -1800,9 +1741,10 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' exact hp_exit u hIu hcu have hx_inv' : LoopStateInv (cond.rename f) I' Q' x := by rcases hx_inv with ⟨hcx, hIx⟩ | ⟨hcx, hQx⟩ - · exact Or.inl ⟨(Cond.holds_rename _ _).2 hcx, ⟨x.comap f, hIx, (extend_comap_eq hf x).symm⟩⟩ + · exact Or.inl ⟨(Cond.holds_rename _ _).2 hcx, + ⟨x.comap f, hIx, (Data.extend_comap hf x).symm⟩⟩ · exact Or.inr ⟨fun h => hcx ((Cond.holds_rename _ _).1 h), - ⟨x.comap f, hQx, (extend_comap_eq hf x).symm⟩⟩ + ⟨x.comap f, hQx, (Data.extend_comap hf x).symm⟩⟩ have hloop := loop_geometric (cond := cond.rename f) (body := body.rename f) I' Q' hp_pos hp_top (μ_body := μ_body') (T_body := T_body') (B_body := B_body) (x := x) @@ -1821,7 +1763,7 @@ theorem rename {r' : ℕ} {f : Fin r → Fin r'} (hf : f.Injective) {x : Data r' refine hloop.congr_dist ?_ le_rfl ext y' rw [loopExitPMF_apply_of_hasSum (hasSum_loopExitDensity hp_pos hp_top hinv' hp_exit' hx_inv')] - have hx_eq : x = g (x.comap f) := (extend_comap_eq hf x).symm + have hx_eq : x = g (x.comap f) := (Data.extend_comap hf x).symm by_cases hy' : y' ∈ Set.range g · obtain ⟨y, rfl⟩ := hy' rw [pmf_map_apply_injective _ hg, @@ -1879,20 +1821,20 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program ≤ ∑' y, μ₁ y * (T₂ y + ∑' z, (μ₂ y) z * expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩)) := by - refine ENNReal.tsum_le_tsum fun y => ?_ + refine tsum_le_tsum fun y => ?_ by_cases hy : y ∈ μ₁.support · exact mul_le_mul_right (ih₂ y hy hc₂ N) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] _ = (∑' y, μ₁ y * T₂ y) + ∑' y, μ₁ y * ∑' z, (μ₂ y) z * expectedStepsWithin w p N (.running ⟨pc + c₁.size + c₂.size, z⟩) := by - simp_rw [mul_add, ENNReal.tsum_add] + simp_rw [mul_add, tsum_add] _ = (∑' y, μ₁ y * T₂ y) + ∑' z, (μ₁.bind μ₂) z * expectedStepsWithin w p N (.running ⟨pc + (c₁.seq c₂).size, z⟩) := by congr 1 simp only [Code.size, Nat.add_assoc] - exact (tsum_pmf_bind μ₁ μ₂ _).symm + exact (Cslib.Probability.PMF.tsum_bind_mul μ₁ μ₂ _).symm calc T₁ + ∑' y, μ₁ y * expectedStepsWithin w p N (.running ⟨pc + c₁.size, y⟩) ≤ T₁ + ((∑' y, μ₁ y * T₂ y) + @@ -1931,13 +1873,13 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program (∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) ≤ ∑' y, μ y * (1 + expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩)) := by - refine ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun y => mul_le_mul_right ?_ _ refine (expectedStepsWithin_running_le w p N ⟨pc + 1 + neg.size, y⟩).trans (le_of_eq ?_) rw [stepRand_jump (s := ⟨pc + 1 + neg.size, y⟩) hj, ProbComp.expect_pure, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] _ = 1 + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + (neg.size + pos.size + 2), y⟩) := by - simp_rw [mul_add, ENNReal.tsum_add, mul_one, μ.tsum_coe] + simp_rw [mul_add, tsum_add, mul_one, μ.tsum_coe] calc 1 + (T + ∑' y, μ y * expectedStepsWithin w p N (.running ⟨pc + 1 + neg.size, y⟩)) ≤ 1 + (T + (1 + ∑' y, μ y * @@ -1981,17 +1923,18 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program calc (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc + 2 + body.size, u⟩)) ≤ ∑' u, μ_body z u * (1 + expectedStepsWithin w p n (.running ⟨pc, u⟩)) := by - refine ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _ + refine tsum_le_tsum fun u => mul_le_mul_right ?_ _ refine (expectedStepsWithin_running_le w p n ⟨pc + 2 + body.size, u⟩).trans (le_of_eq ?_) rw [stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back, ProbComp.expect_pure] _ = 1 + ∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩) := by - simp_rw [mul_add, ENNReal.tsum_add, mul_one, (μ_body z).tsum_coe] + simp_rw [mul_add, tsum_add, mul_one, (μ_body z).tsum_coe] have hbind_k : (∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y) = ∑' y, loopExitPMF cond μ_body z y * K_cont y := by rw [loopExitPMF_of_holds hp_pos hp_top hinv hp_exit hIz hcz] - exact (tsum_pmf_bind (μ_body z) (loopExitPMF cond μ_body) K_cont).symm + exact (Cslib.Probability.PMF.tsum_bind_mul (μ_body z) + (loopExitPMF cond μ_body) K_cont).symm have hih_step : (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + @@ -2000,14 +1943,14 @@ theorem expectedStepsWithin_assemble (h : ExecCodeDist w c x μ T) {p : Program (∑' u, μ_body z u * expectedStepsWithin w p n (.running ⟨pc, u⟩)) ≤ ∑' u, μ_body z u * ((if cond.Holds u then M else 0) + 2 + ∑' y, loopExitPMF cond μ_body u y * K_cont y) := by - refine ENNReal.tsum_le_tsum fun u => ?_ + refine tsum_le_tsum fun u => ?_ by_cases hu : u ∈ (μ_body z).support · exact mul_le_mul_right (ihn (Nat.le_of_succ_le hn) u (hinv z hIz hcz u hu)) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + ∑' u, μ_body z u * ∑' y, loopExitPMF cond μ_body u y * K_cont y := by - simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, (μ_body z).tsum_coe, + simp_rw [mul_add, tsum_add, ENNReal.tsum_mul_right, (μ_body z).tsum_coe, one_mul] _ = (∑' u, μ_body z u * (if cond.Holds u then M else 0)) + 2 + ∑' y, loopExitPMF cond μ_body z y * K_cont y := by @@ -2054,7 +1997,7 @@ theorem expectedStepsFrom_assemble (h : ExecCodeDist w c x μ T) {p : Program B T + ∑' y, μ y * expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) := by rw [← iSup_expectedStepsWithin w p (.running ⟨pc, x⟩)] refine iSup_le fun N => (h.expectedStepsWithin_assemble hp N).trans ?_ - refine add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _) + refine add_le_add le_rfl (tsum_le_tsum fun y => mul_le_mul_right ?_ _) exact expectedStepsWithin_le_expectedStepsFrom w p N (.running ⟨pc + c.size, y⟩) /-- Continuation upper-bound form of `expectedStepsFrom_assemble`. -/ @@ -2063,7 +2006,7 @@ theorem expectedStepsFrom_le (h : ExecCodeDist w c x μ T) {p : Program B r} {pc (hk : ∀ y ∈ μ.support, expectedStepsFrom w p (.running ⟨pc + c.size, y⟩) ≤ k y) : expectedStepsFrom w p (.running ⟨pc, x⟩) ≤ T + ∑' y, μ y * k y := by refine (h.expectedStepsFrom_assemble hp).trans - (add_le_add le_rfl (ENNReal.tsum_le_tsum fun y => ?_)) + (add_le_add le_rfl (tsum_le_tsum fun y => ?_)) by_cases hy : y ∈ μ.support · exact mul_le_mul_right (hk y hy) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy, zero_mul, zero_mul] @@ -2201,9 +2144,9 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} ∑' y₁, μ₁ y₁ * ∑' y, (μ₂ y₁) y * ⨆ T_hor : ℕ, evalRand w p T_hor ⟨pc + c₁.size + c₂.size, y⟩ (.halted v d) := by simp only [Code.size, Nat.add_assoc] - exact tsum_pmf_bind μ₁ μ₂ _ + exact Cslib.Probability.PMF.tsum_bind_mul μ₁ μ₂ _ rw [hbind] - refine (ENNReal.tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) + refine (tsum_le_tsum fun y₁ => ?_).trans (ih₁ hp.append_left) by_cases hy₁ : y₁ ∈ μ₁.support · exact mul_le_mul_right (ih₂ y₁ hy₁ hc₂) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hy₁, zero_mul, zero_mul] @@ -2226,7 +2169,7 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} have hstep : stepRand w p (.running ⟨pc, x⟩) = pure (.running ⟨pc + 1, x⟩) := by rw [stepRand_branch (s := ⟨pc, x⟩) hbr] simp [hc] - refine (ENNReal.tsum_le_tsum fun y => mul_le_mul_right ?_ _).trans + refine (tsum_le_tsum fun y => mul_le_mul_right ?_ _).trans ((ih hp.cons_tail.append_left).trans (iSup_evalRand_halted_le_of_step hstep v d)) simpa [Code.size, show pc + neg.size + pos.size + 2 = pc + (neg.size + pos.size + 2) by omega] using iSup_evalRand_halted_le_of_step @@ -2256,7 +2199,7 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} have hbr : stepRand w p (.running ⟨pc, z⟩) = pure (.running ⟨pc + 2, z⟩) := by rw [stepRand_branch (s := ⟨pc, z⟩) hp.head] simp [hcz] - refine (ENNReal.tsum_le_tsum fun u => mul_le_mul_right ?_ _).trans + refine (tsum_le_tsum fun u => mul_le_mul_right ?_ _).trans ((ih_body z hIz hcz hbody_contains).trans (iSup_evalRand_halted_le_of_step hbr v d)) exact iSup_evalRand_halted_le_of_step (stepRand_jump (s := ⟨pc + 2 + body.size, u⟩) hj_back) v d @@ -2274,9 +2217,9 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} · rw [loopIterPMF_succ, loopStepPMF_of_holds hcz] have hbind : (∑' y, ((μ_body z).bind (loopIterPMF cond μ_body K)) y * H_loop y) = ∑' u, μ_body z u * ∑' y, loopIterPMF cond μ_body K u y * H_loop y := - tsum_pmf_bind (μ_body z) (loopIterPMF cond μ_body K) H_loop + Cslib.Probability.PMF.tsum_bind_mul (μ_body z) (loopIterPMF cond μ_body K) H_loop rw [hbind] - refine (ENNReal.tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) + refine (tsum_le_tsum fun u => ?_).trans (hstep_le z hIz hcz) by_cases hu : u ∈ (μ_body z).support · exact mul_le_mul_right (ihK u (hinv z hIz hcz u hu)) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hu, zero_mul, zero_mul] @@ -2292,7 +2235,7 @@ theorem le_iSup_evalRand_halted (h : ExecCodeDist w c x μ T) {p : Program B r} exact tsum_iSup_of_monotone fun y K₁ K₂ hK => mul_le_mul_left (loopIterPMF_indicator_mono cond μ_body x y hK) (H_exit y) rw [hrewrite_sum] - refine iSup_le fun K => (ENNReal.tsum_le_tsum fun y => ?_).trans (hiter_le K x hx_inv) + refine iSup_le fun K => (tsum_le_tsum fun y => ?_).trans (hiter_le K x hx_inv) by_cases hcy : ¬cond.Holds y · rw [Set.indicator_of_mem hcy] exact mul_le_mul_right (hexit_le y hcy) _ @@ -2334,7 +2277,7 @@ theorem program_halts (h : ExecCodeDist w c x μ T) (out : Fin r) : exact ne_top_of_le_ne_top (ENNReal.add_ne_top.mpr ⟨hT, ENNReal.one_ne_top⟩) hexp · have hle := h.le_iSup_evalRand_halted hp (y.regs out) y refine (le_trans ?_ - (ENNReal.le_tsum + (le_tsum (f := fun z => μ z * ⨆ T_hor : ℕ, evalRand w (c.program out) T_hor ⟨0 + c.size, z⟩ (.halted (y.regs out) y)) y)).trans hle diff --git a/Complexity/Computability/WordRAM/Randomized/Structured.lean b/Complexity/Computability/WordRAM/Randomized/Structured.lean index 1c63438b9..e037de1bd 100644 --- a/Complexity/Computability/WordRAM/Randomized/Structured.lean +++ b/Complexity/Computability/WordRAM/Randomized/Structured.lean @@ -305,16 +305,6 @@ section Rename variable {f : Fin r → Fin r'} -private theorem extend_comap_eq (hf : f.Injective) (x : Data r') : - (x.comap f).extend f x = x := by - ext j - · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j - by_cases hj : ∃ i, f i = j - · obtain ⟨i, rfl⟩ := hj - exact hf.extend_apply _ _ i - · exact Function.extend_apply' _ _ _ hj - · rfl - private theorem extend_set_eq (hf : f.Injective) (y : Data r) (x : Data r') (i : Fin r) (v : ℕ) : (y.set i v).extend f x = (y.extend f x).set (f i) v := Data.ext (hf.extend_update _ _ _ _) rfl @@ -342,7 +332,7 @@ theorem ExecRand.rename (hf : f.Injective) {w : ℕ} {c : Code B r} {x : Data r' pure (x.set (f dst) k.val, 1)) := by rw [← bind_pure_comp, bind_assoc] refine congrArg (ProbComp.unifFin (x.regs (f src)) >>= ·) (funext fun k => ?_) - simp only [pure_bind, extend_set_eq hf, extend_comap_eq hf] + simp only [pure_bind, extend_set_eq hf, Data.extend_comap hf] simp only [Code.rename, Data.comap_regs] rw [heq] exact ExecRand.rand hfit' @@ -386,7 +376,7 @@ theorem ExecRand.rename (hf : f.Injective) {w : ℕ} {c : Code B r} {x : Data r' simpa only [Code.rename, Functor.map_map] using h' | @loop_exit cond body _ hc => subst hx - simpa only [Code.rename, map_pure, extend_comap_eq hf] using + simpa only [Code.rename, map_pure, Data.extend_comap hf] using ExecRand.loop_exit (body := body.rename f) (fun h => hc ((Cond.holds_rename _ _).1 h)) | @loop_iter cond body _ mb ml hc _ hl ihb ihl => subst hx diff --git a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean index acfd953f7..6ec488459 100644 --- a/Complexity/Computability/WordRAM/Randomized/Subroutine.lean +++ b/Complexity/Computability/WordRAM/Randomized/Subroutine.lean @@ -9,6 +9,7 @@ module public import Complexity.Computability.WordRAM.Subroutine public import Complexity.Computability.WordRAM.Randomized.Expected +import Complexity.Probability.PMF.Expect /-! # Composable Randomized Word-RAM Data-Structure Subroutines @@ -17,8 +18,8 @@ This file extends the deterministic `Cslib.WordRAM.SubroutineDS` interface to ** structures and Las Vegas subroutines** on the Word-RAM — covering both structures initialized from a random seed (such as universal hash tables, 2-choice hash tables, and cuckoo hash tables) and structures whose operations make random choices via `Instr.rand` (such as treaps, skip lists, -reservoir samplers, or randomized rehashing); see [MitzenmacherUpfal2017], Sections 1.5, 2.1, and -2.3. +reservoir samplers, or randomized rehashing); see [MitzenmacherUpfal2017], Sections 2.1, 2.3, and +2.4. ## Core design @@ -50,6 +51,12 @@ density factor `seedSlack : ℝ≥0∞`: telescopes these one-step bounds without `ℝ≥0∞` subtraction: total expected step cost is at most `initTime + initPotential + D.totalCost charge s₀ ops`. +The sequence theorems charge initialization and subroutine bodies for a fixed operation list. +`prepare` delivers each operation's arguments by a semantic register replacement; its encoding +and delivery, caller dispatch and a whole-client driver are not charged. Callers requiring native +argument preparation can use `call_rename`, whose `B_prep` explicitly charges the preparation +code. Compiling a standalone body adds the final halt separately via `ExecCodeDist.program_halts`. + ## Main definitions * `Cslib.WordRAM.Randomized.SubroutineDS`: reusable randomized Word-RAM subroutine certificate for @@ -65,7 +72,8 @@ density factor `seedSlack : ℝ≥0∞`: * `Cslib.WordRAM.Randomized.SubroutineDS.initFresh_exec`, `initFresh_rename`: expected-time initialization on fresh zeroed memory. -* `Cslib.WordRAM.Randomized.SubroutineDS.zeroFillCode_exec`, `initClear_exec`, `initClear_rename`: +* `Cslib.WordRAM.SubroutineCode.zeroFillCode_exec`, + `Cslib.WordRAM.Randomized.SubroutineDS.initClear_exec`, `initClear_rename`: expected-time initialization on arbitrary dirty memory. * `Cslib.WordRAM.Randomized.SubroutineDS.code_rename`, `call_rename`: inline subroutine invocation on caller state space `Data r_caller` via register injection `f : Fin r_ds → Fin r_caller`. @@ -74,7 +82,7 @@ density factor `seedSlack : ℝ≥0∞`: or randomized caller steps that do not modify persistent DS registers (`P.input i = false`) or `[base, base + space)`. * `Cslib.WordRAM.Randomized.SubroutineDS.steps_expected_amortized`, `seq_expected_amortized`: - end-to-end telescope theorems bounding total expected time across any admissible operation + telescope theorems bounding initialization and expected body time across any admissible operation sequence `ops : List D.Op`. ## References @@ -82,7 +90,7 @@ density factor `seedSlack : ℝ≥0∞`: * [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, Chapter 17][CLRS2009] * [M. Mitzenmacher and E. Upfal, *Probability and Computing: Randomization and Probabilistic - Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017], Sections 1.5, 2.1, and 2.3. + Techniques in Algorithms and Data Analysis*][MitzenmacherUpfal2017], Sections 2.1, 2.3, and 2.4. -/ @[expose] public section @@ -281,7 +289,7 @@ noncomputable def toRandomized (C : SubroutineDS P D w base params s₀) : · calc (∑' p, ν p * ((n p : ℝ≥0∞) + ∑' q, PMF.pure ((), y p) q * (C.potential q.2 : ℝ≥0∞))) ≤ ∑' p, ν p * ((C.charge s o : ℝ≥0∞) + (C.potential p.2 : ℝ≥0∞)) := by - refine ENNReal.tsum_le_tsum fun p => ?_ + refine tsum_le_tsum fun p => ?_ by_cases hp_mem : p ∈ ν.support · refine mul_le_mul_right ?_ _ have hp : C.rep s p.2 ∧ C.argMatch o p.2.regs := hsupp p hp_mem @@ -297,7 +305,7 @@ noncomputable def toRandomized (C : SubroutineDS P D w base params s₀) : exact_mod_cast hpot · rw [(PMF.apply_eq_zero_iff _ _).mpr hp_mem, zero_mul, zero_mul] _ = (C.charge s o : ℝ≥0∞) + ∑' p, ν p * (C.potential p.2 : ℝ≥0∞) := by - simp_rw [mul_add, ENNReal.tsum_add, ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + simp_rw [mul_add, tsum_add, ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] end Cslib.WordRAM.SubroutineDS @@ -451,57 +459,6 @@ theorem initFresh_rename (C : SubroutineDS P D w base params s₀) /-! ### Projection 3: Generic `O(space)` dirty-memory initialization (`zeroFillCode_exec`, `initClear_exec`, `initClear_rename`) -/ -private def zeroFillSpec : Stateful.Spec.{0, 0, 0} where - State := PUnit - Op := PEmpty - Result o := PEmpty.elim o - step _ o := PEmpty.elim o - -private noncomputable def toZeroFillDet (C : SubroutineDS P D w base params s₀) : - Cslib.WordRAM.SubroutineDS - { P with initSetup := .op (.copy P.zeroReg P.zeroReg) } - zeroFillSpec w base params PUnit.unit where - space := C.space - spanTime := C.spanTime - setupTime := 1 - initPotential := 0 - charge _ o := PEmpty.elim o - rep _ x := x.Bounded w - potential _ := 0 - encode o := PEmpty.elim o - argMatch o := PEmpty.elim o - decode o := PEmpty.elim o - bounded h := h - rep_unique _ _ := rfl - arguments_fit o := PEmpty.elim o - encode_matches o := PEmpty.elim o - rep_supported _ _ hb _ := hb - potential_supported _ _ _ := rfl - clear_nodup := C.clear_nodup - params_not_clear := C.params_not_clear - incCur_eval := C.incCur_eval - span_exec := C.span_exec - setup_exec {x₁} hx₁ _ hfr _ _ := by - refine ⟨x₁.set P.zeroReg (x₁.regs P.zeroReg), ExecWithin.copy, ?_, - hx₁.set (hx₁.1 P.zeroReg), Nat.le_refl 0, Modifies.refl _ _⟩ - simp [Data.set, hfr] - step_exec o := PEmpty.elim o - -/-- Generic execution of `P.zeroFillCode` clearing `[base, base + C.space)` in `4 * C.space + 6` -instructions over any basis `B`. -/ -theorem zeroFillCode_exec (C : SubroutineDS P D w base params s₀) {x₁ : Data r_ds} - (hx₁ : x₁.Bounded w) (hb₁ : x₁.regs P.baseReg = base) - (hfr₁ : x₁.regs P.frontierReg = base + C.space) - (hpar₁ : ∀ j, x₁.regs (P.initParams j) = params j) : - ∃ x₂, ExecWithin w P.zeroFillCode x₁ x₂ (4 * C.space + 6) ∧ - x₂.Bounded w ∧ - x₂.regs P.baseReg = base ∧ - x₂.regs P.frontierReg = base + C.space ∧ - (∀ j, x₂.regs (P.initParams j) = params j) ∧ - (∀ a ∈ (Block.mk base C.space).footprint, x₂.mem a = 0) ∧ - Modifies (Block.mk base C.space).footprint x₁.mem x₂.mem := - C.toZeroFillDet.zeroFillCode_exec hx₁ hb₁ hfr₁ hpar₁ - /-- Execute `P.initClear` on **arbitrary dirty memory** in `C.initTime + (4 * C.space + 6 : ℕ)` expected instructions. -/ theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} @@ -518,7 +475,8 @@ theorem initClear_exec (C : SubroutineDS P D w base params s₀) {x : Data r_ds} Modifies (Block.mk base C.space).footprint x.mem p.2.mem := by obtain ⟨x₁, he₁, hm₁, hb₁, hfr₁, hpar₁⟩ := C.span_exec hx hfr hpar obtain ⟨x₂, he₂, hx₂, hb₂, hfr₂, hpar₂, hz₂, hmod₂⟩ := - C.zeroFillCode_exec (he₁.bounded hx) hb₁ hfr₁ hpar₁ + P.zeroFillCode_exec C.clear_nodup C.params_not_clear C.incCur_eval + (he₁.bounded hx) hb₁ hfr₁ hpar₁ obtain ⟨ν₀, he₃, hseed, hsupp⟩ := C.setup_exec hx₂ hb₂ hfr₂ hpar₂ hz₂ rw [hm₁] at hmod₂ have he_clear : ExecCodeDist w P.initClear x (ν₀.map Prod.snd) @@ -618,10 +576,10 @@ theorem frame_caller_dist (C : SubroutineDS P D w base params s₀) obtain ⟨x', hx', rfl⟩ := hq_map obtain ⟨hb', hreg, hmem⟩ := hframe p hp x' hx' exact (C.frame_caller (hrep p hp) hb' hreg hdisj hmem).1 - · rw [frameDist, tsum_pmf_bind] + · rw [frameDist, Cslib.Probability.PMF.tsum_bind_mul] refine tsum_congr fun p => ?_ by_cases hp : p ∈ ν.support - · rw [tsum_pmf_map] + · rw [Cslib.Probability.PMF.tsum_map_mul] have hpot_eq : (∑' x', μ_caller p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = ∑' x', μ_caller p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by refine tsum_congr fun x' => ?_ @@ -700,7 +658,7 @@ theorem code_rename (C : SubroutineDS P D w base params s₀) (∑' q, μ_caller p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞) := by intro p - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] refine tsum_congr fun q_ds => ?_ rw [Data.comap_extend hf] simp_rw [hPot_inner] @@ -709,12 +667,13 @@ theorem code_rename (C : SubroutineDS P D w base params s₀) ∑' q_ds, μ_ds (proj p) q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞))) = ∑' p_ds, ν_ds p_ds * (T_ds p_ds + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)) := - (tsum_pmf_map ν proj fun p_ds => + (Cslib.Probability.PMF.tsum_map_mul ν proj fun p_ds => T_ds p_ds + ∑' q_ds, μ_ds p_ds q_ds * (C.potential q_ds.1 q_ds.2 : ℝ≥0∞)).symm have hRHS : (∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞)) = ∑' p_ds, ν_ds p_ds * (C.potential p_ds.1 p_ds.2 : ℝ≥0∞) := - (tsum_pmf_map ν proj fun p_ds => (C.potential p_ds.1 p_ds.2 : ℝ≥0∞)).symm + (Cslib.Probability.PMF.tsum_map_mul ν proj + fun p_ds => (C.potential p_ds.1 p_ds.2 : ℝ≥0∞)).symm rw [hLHS, hRHS] exact hamort_ds @@ -802,10 +761,10 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) · have hpot_prep : (∑' p', ν_prep p' * (C.potential p'.1 (p'.2.comap f) : ℝ≥0∞)) = ∑' p, ν p * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by - simp only [ν_prep, frameDist, tsum_pmf_bind] + simp only [ν_prep, frameDist, Cslib.Probability.PMF.tsum_bind_mul] refine tsum_congr fun p => ?_ by_cases hp : p ∈ ν.support - · rw [tsum_pmf_map] + · rw [Cslib.Probability.PMF.tsum_map_mul] have hpot_eq : (∑' x', μ_prep p x' * (C.potential p.1 (x'.comap f) : ℝ≥0∞)) = ∑' x', μ_prep p x' * (C.potential p.1 (p.2.comap f) : ℝ≥0∞) := by refine tsum_congr fun x' => ?_ @@ -827,9 +786,9 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) (T_code (p.1, x') + ∑' q, μ_code (p.1, x') q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) := by intro p - simp only [T_call, μ_call, tsum_pmf_bind, add_assoc] + simp only [T_call, μ_call, Cslib.Probability.PMF.tsum_bind_mul, add_assoc] congr 1 - rw [← ENNReal.tsum_add] + rw [← tsum_add] simp_rw [mul_add] have hmul : ∀ p : C.Seed × Data r_caller, ν p * (T_call p + ∑' q, μ_call p q * (C.potential q.1 (q.2.comap f) : ℝ≥0∞)) = @@ -839,18 +798,18 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) intro p rw [hinner p, mul_add] simp_rw [hmul] - rw [ENNReal.tsum_add] + rw [tsum_add] congr 1 symm dsimp only [ν_prep, frameDist] - rw [tsum_pmf_bind] + rw [Cslib.Probability.PMF.tsum_bind_mul] refine tsum_congr fun p => ?_ - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] have hprep_le : (∑' p, ν p * T_prep p) ≤ B_prep := by calc (∑' p, ν p * T_prep p) ≤ ∑' p, ν p * B_prep := by - refine ENNReal.tsum_le_tsum fun p => ?_ + refine tsum_le_tsum fun p => ?_ by_cases hp : p ∈ ν.support · exact mul_le_mul_right (hprep p hp).2.1 _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] @@ -870,60 +829,43 @@ theorem call_rename (C : SubroutineDS P D w base params s₀) /-! ### Projection 6: End-to-end expected-amortized sequence telescope (`steps_expected_amortized`, `seq_expected_amortized`) -/ -/-- Deliver standalone operation arguments for `o` without changing memory or persistent -registers (`P.input i = false`). -/ -def prepare (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : Data r_ds where - regs i := if P.input i then C.encode o i else x.regs i - mem := x.mem - -@[simp] -theorem prepare_mem (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : - (C.prepare o x).mem = x.mem := rfl - -theorem prepare_regs_of_not_input (C : SubroutineDS P D w base params s₀) - {i : Fin r_ds} (hi : P.input i = false) (o : D.Op) (x : Data r_ds) : - (C.prepare o x).regs i = x.regs i := by - simp [prepare, hi] - -theorem prepare_bounded (C : SubroutineDS P D w base params s₀) - {s : D.State} {o : D.Op} {x : Data r_ds} (hx : x.Bounded w) (hv : D.valid s o) : - (C.prepare o x).Bounded w := by - refine ⟨fun i => ?_, hx.2⟩ - change Fits w (if P.input i then C.encode o i else x.regs i) - split - · exact C.arguments_fit o hv i (by assumption) - · exact hx.1 i +/-- Reuse the signature-level program and argument-delivery API with the actual instruction basis. +This projection carries code and the calling convention, without a deterministic certificate. -/ +def toDynamicProgram (C : SubroutineDS P D w base params s₀) : DynamicProgram D r_ds B := + P.toDynamicProgram C.encode C.decode theorem prepare_rep (C : SubroutineDS P D w base params s₀) {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} (hr : C.rep θ s x) (hv : D.valid s o) : - C.rep θ s (C.prepare o x) := + C.rep θ s (C.toDynamicProgram.prepare o x) := C.rep_supported - (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) - (C.prepare_bounded (C.bounded hr) hv) + (C.toDynamicProgram.prepare_bounded (C.bounded hr) (C.arguments_fit _ hv)) hr theorem prepare_potential (C : SubroutineDS P D w base params s₀) {θ : C.Seed} {s : D.State} {o : D.Op} {x : Data r_ds} (hr : C.rep θ s x) : - C.potential θ (C.prepare o x) = C.potential θ x := + C.potential θ (C.toDynamicProgram.prepare o x) = C.potential θ x := C.potential_supported - (fun _ hi => C.prepare_regs_of_not_input hi o x) + (fun _ hi => C.toDynamicProgram.prepare_regs_of_not_input hi o x) (fun _ _ => rfl) hr theorem prepare_argMatch (C : SubroutineDS P D w base params s₀) (o : D.Op) (x : Data r_ds) : - C.argMatch o (C.prepare o x).regs := + C.argMatch o (C.toDynamicProgram.prepare o x).regs := C.encode_matches o x.regs /-- Inductive multi-operation distributional execution relation for a randomized subroutine `C`. `C.ExecSteps s ops ν ν_final T_ops` witnesses that starting from abstract state `s` and coupled state distribution `ν : PMF (C.Seed × Data r_ds)`, executing each operation `o` in `ops` by -delivering arguments via `C.prepare o` and running `P.code` under `ExecCodeDist` produces the +delivering arguments via `C.toDynamicProgram.prepare o` and running `P.code` under `ExecCodeDist` +produces the final coupled distribution `ν_final`, decodes the exact abstract return value `(D.step s o).2` at every step, modifies memory only within `[base, base + C.space)`, and incurs total expected -operation step cost `T_ops`. -/ +operation body cost `T_ops`. Argument delivery is an external register replacement and +contributes no native instructions to this relation. -/ inductive ExecSteps (C : SubroutineDS P D w base params s₀) : D.State → List D.Op → PMF (C.Seed × Data r_ds) → PMF (C.Seed × Data r_ds) → ℝ≥0∞ → Prop where | nil (s : D.State) (ν : PMF (C.Seed × Data r_ds)) : @@ -933,7 +875,8 @@ inductive ExecSteps (C : SubroutineDS P D w base params s₀) : {μ_step : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds)} {T_step : C.Seed × Data r_ds → ℝ≥0∞} {T_rest : ℝ≥0∞} : (∀ p ∈ ν.support, - ExecCodeDist w P.code (C.prepare o p.2) ((μ_step p).map Prod.snd) (T_step p) ∧ + ExecCodeDist w P.code (C.toDynamicProgram.prepare o p.2) + ((μ_step p).map Prod.snd) (T_step p) ∧ ∀ q ∈ (μ_step p).support, C.decode o q.2.regs = some (D.step s o).2 ∧ C.rep q.1 (D.step s o).1 q.2 ∧ @@ -965,7 +908,7 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) exact ⟨ν, 0, ExecSteps.nil s ν, hseed, hrep, by simp⟩ | cons o os ih => obtain ⟨hvo, hvos⟩ := hv - let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.prepare o p.2) + let ν_prep : PMF (C.Seed × Data r_ds) := ν.map fun p => (p.1, C.toDynamicProgram.prepare o p.2) have hseed_prep : ∀ θ, (ν_prep.map Prod.fst) θ ≤ C.seedSlack * C.seedLaw θ := by intro θ rw [PMF.map_comp] @@ -978,7 +921,7 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) have hpot_prep : (∑' p', ν_prep p' * (C.potential p'.1 p'.2 : ℝ≥0∞)) = ∑' p, ν p * (C.potential p.1 p.2 : ℝ≥0∞) := by - rw [tsum_pmf_map] + rw [Cslib.Probability.PMF.tsum_map_mul] refine tsum_congr fun p => ?_ by_cases hp : p ∈ ν.support · rw [C.prepare_potential (hrep p hp)] @@ -986,9 +929,9 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) obtain ⟨μ_step, T_step, hstep, hseed₁, hamort₁⟩ := C.step_exec o ν_prep hvo hseed_prep hsupp_prep let μ_op : C.Seed × Data r_ds → PMF (C.Seed × Data r_ds) := - fun p => μ_step (p.1, C.prepare o p.2) + fun p => μ_step (p.1, C.toDynamicProgram.prepare o p.2) let T_op : C.Seed × Data r_ds → ℝ≥0∞ := - fun p => T_step (p.1, C.prepare o p.2) + fun p => T_step (p.1, C.toDynamicProgram.prepare o p.2) let ν₁ := ν.bind μ_op have hν₁_eq : ν₁ = ν_prep.bind μ_step := by simp only [ν₁, ν_prep, μ_op, PMF.bind_map] @@ -1006,17 +949,18 @@ theorem steps_expected_amortized (C : SubroutineDS P D w base params s₀) let T₁ : ℝ≥0∞ := ∑' p, ν p * T_op p have hT₁_eq : T₁ = ∑' p', ν_prep p' * T_step p' := by symm - exact tsum_pmf_map ν (fun p => (p.1, C.prepare o p.2)) T_step + exact Cslib.Probability.PMF.tsum_map_mul ν + (fun p => (p.1, C.toDynamicProgram.prepare o p.2)) T_step have hex_cons : C.ExecSteps s (o :: os) ν ν_final (T₁ + T_rest) := by refine ExecSteps.cons (μ_step := μ_op) (T_step := T_op) (fun p hp => ?_) hex_rest - have hp' : (p.1, C.prepare o p.2) ∈ ν_prep.support := + have hp' : (p.1, C.toDynamicProgram.prepare o p.2) ∈ ν_prep.support := (PMF.mem_support_map_iff _ _ _).mpr ⟨p, hp, rfl⟩ - exact hstep (p.1, C.prepare o p.2) hp' + exact hstep (p.1, C.toDynamicProgram.prepare o p.2) hp' have hstep_split : T₁ + (∑' q, ν₁ q * (C.potential q.1 q.2 : ℝ≥0∞)) = ∑' p', ν_prep p' * (T_step p' + ∑' q, μ_step p' q * (C.potential q.1 q.2 : ℝ≥0∞)) := by rw [hT₁_eq, hν₁_eq] - simp only [tsum_pmf_bind, ← ENNReal.tsum_add, mul_add] + simp only [Cslib.Probability.PMF.tsum_bind_mul, ← tsum_add, mul_add] refine ⟨ν_final, T₁ + T_rest, hex_cons, hseed_final, hrep_final, ?_⟩ calc (T₁ + T_rest) + ∑' q, ν_final q * (C.potential q.1 q.2 : ℝ≥0∞) @@ -1061,7 +1005,7 @@ theorem seq_expected_amortized (C : SubroutineDS P D w base params s₀) calc (∑' p, ν₀ p * (C.potential p.1 p.2 : ℝ≥0∞)) ≤ ∑' p, ν₀ p * (C.initPotential : ℝ≥0∞) := by - refine ENNReal.tsum_le_tsum fun p => ?_ + refine tsum_le_tsum fun p => ?_ by_cases hp : p ∈ ν₀.support · exact mul_le_mul_right (Nat.cast_le.mpr (hsupp₀ p hp).2.2.1) _ · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] diff --git a/Complexity/Computability/WordRAM/Structured.lean b/Complexity/Computability/WordRAM/Structured.lean index a83d3abfb..82425958f 100644 --- a/Complexity/Computability/WordRAM/Structured.lean +++ b/Complexity/Computability/WordRAM/Structured.lean @@ -686,7 +686,8 @@ theorem Data.extend_extend (hf : f.Injective) (y z : Data r) (x : Data r') : · simp [Function.extend_apply' _ _ _ hj] · rfl -private theorem Data.extend_comap (hf : f.Injective) (x : Data r') : +/-- Writing a register view back into its source data changes nothing. -/ +theorem Data.extend_comap (hf : f.Injective) (x : Data r') : (x.comap f).extend f x = x := by ext j · change Function.extend f (x.regs ∘ f) x.regs j = x.regs j diff --git a/Complexity/Probability/PMF/Expect.lean b/Complexity/Probability/PMF/Expect.lean index 4a00e266b..6606eb95d 100644 --- a/Complexity/Probability/PMF/Expect.lean +++ b/Complexity/Probability/PMF/Expect.lean @@ -1,7 +1,7 @@ /- -Copyright (c) 2026 Samuel Schlesinger. All rights reserved. +Copyright (c) 2026 Samuel Schlesinger, Renfei Zhou. All rights reserved. Released under Apache 2.0 license as described in the file LICENSE. -Authors: Samuel Schlesinger +Authors: Samuel Schlesinger, Renfei Zhou -/ module @@ -29,6 +29,8 @@ into the vocabulary of `Finset.expect` (`𝔼`) and `Finset.dens`. * `Cslib.Probability.PMF.sum_le_affine`: average pointwise affine bounds. * `Cslib.Probability.PMF.sum_map_mul`, `.sum_bind_mul`: average deterministic and random scores. +* `Cslib.Probability.PMF.tsum_map_mul`, `.tsum_bind_mul`: the same laws for extended + nonnegative scores over arbitrary types. * `Cslib.Probability.PMF.toReal_uniformOfFintype_bind_apply`: a uniform first stage averages the second stages. @@ -44,6 +46,23 @@ namespace Cslib.Probability.PMF open Finset open scoped BigOperators ENNReal +/-- Expectation of `k : β → ℝ≥0∞` under `p.map g` equals expectation of `k ∘ g` under `p`. -/ +theorem tsum_map_mul {α β : Type*} (p : PMF α) (g : α → β) (k : β → ℝ≥0∞) : + (∑' b, p.map g b * k b) = ∑' a, p a * k (g a) := by + simp only [PMF.map_apply, ← ENNReal.tsum_mul_right] + rw [tsum_comm] + refine tsum_congr fun a => ?_ + rw [tsum_eq_single (g a)] + · simp + · intro b hb + simp [hb] + +/-- Expectation of `k : β → ℝ≥0∞` under `p.bind f` decomposes as an iterated expectation. -/ +theorem tsum_bind_mul {α β : Type*} (p : PMF α) (f : α → PMF β) (k : β → ℝ≥0∞) : + (∑' b, (p.bind f) b * k b) = ∑' a, p a * ∑' b, f a b * k b := by + simp only [PMF.bind_apply, ← ENNReal.tsum_mul_right, ← ENNReal.tsum_mul_left, mul_assoc] + exact tsum_comm + /-- Averaging a score after a deterministic map averages its composite with that map. Adapted from the cslib `crypto` branch at `5ff69ca`. -/ theorem sum_map_mul {α β : Type*} [Fintype α] [Fintype β] From 78981acccfe01c9eb996e96b66d00b24d1f40006 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Thu, 29 Oct 2026 18:37:07 +0000 Subject: [PATCH 15/18] feat(WordRAM/Hashing): add self-contained k-wise independent and tabulation hashing programs --- Complexity.lean | 2 + .../Algorithms/WordRAM/Hashing/KWise.lean | 2730 +++++++++++++++++ .../WordRAM/Hashing/Tabulation.lean | 1565 ++++++++++ ComplexityTests/WordRAMKWiseHash.lean | 431 +++ 4 files changed, 4728 insertions(+) create mode 100644 Complexity/Algorithms/WordRAM/Hashing/KWise.lean create mode 100644 Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean create mode 100644 ComplexityTests/WordRAMKWiseHash.lean diff --git a/Complexity.lean b/Complexity.lean index f59b96f22..34622b270 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -148,7 +148,9 @@ public import Complexity.Algorithms.WordRAM.FusionTree.Query.Traversal public import Complexity.Algorithms.WordRAM.FusionTree.Rank public import Complexity.Algorithms.WordRAM.FusionTree.ReadHeader public import Complexity.Algorithms.WordRAM.FusionTree.Static +public import Complexity.Algorithms.WordRAM.Hashing.KWise public import Complexity.Algorithms.WordRAM.Hashing.MultiplyShift +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation public import Complexity.Algorithms.WordRAM.Histogram public import Complexity.Algorithms.WordRAM.Karger.Contract public import Complexity.Algorithms.WordRAM.Karger.FastCut diff --git a/Complexity/Algorithms/WordRAM/Hashing/KWise.lean b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean new file mode 100644 index 000000000..3697a633c --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean @@ -0,0 +1,2730 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation +public import Complexity.Foundations.FieldTheory.Finite.Trinomial +public import Complexity.Probability.KWiseIndependent.Trinomial + +/-! +# Self-contained Word-RAM `k`-wise independent hashing + +This file implements self-contained Word-RAM `k`-wise independent hashing over +`Cslib.WordRAM.Basis.exact` by evaluating degree-`(k - 1)` polynomials over the characteristic-`2` +finite field $\mathbb{F}_{2^{2 \cdot 3^t}} = \mathbb{F}_2[X] / (X^{2 \cdot 3^t} + X^{3^t} + 1)$ +(`Cslib.TrinomialField t`, where `t = trinomialStep U M` is chosen at runtime so that +$q = 2 \cdot 3^t \ge \max(\lceil \log_2 \max(U, 2) \rceil, \max(M, 1))$) and projecting to the low +`M` bits (`Cslib.Probability.trinomialPolyHash`) [Carter and Wegman][CarterWegman1979]; +[Wegman and Carter][WegmanCarter1981]; [Lidl and Niederreiter][LidlNiederreiter1997]. + +To achieve `O(1)` Word-RAM multiplication in $\mathbb{F}_{2^q}$ without dedicated carryless +multiplication hardware when $U \le n_{\max}^c$, we precompute the Four-Russians carryless +multiplication lookup table `clmulTab[0 .. 2 ^ (2 * b) - 1]` for $b$-bit chunks +(`b = chunkBits n_max = max 1 (⌊log₂ (max n_max 2)⌋ / 3)`, so $2^{2b} \le \max(4, n_{\max}) = +O(n_{\max}^{2/3}) \le O(n_{\max})$ words), decompose each $q$-bit field element into +$L = \lceil q / b \rceil$ limbs of $b$ bits, accumulate the $L \times L$ chunk carryless products, +and fold the resulting $< 2^{2q}$ polynomial mod $X^{2r} + X^r + 1$ in 8 straight-line bit +operations (`reduceTrinomialCode`). + +The initialization is factored into two reusable stages: +1. **`initTableCode k`**: deterministically computes $r = 3^t$, $q = 2r$, $b$, masks, and populates + `clmulTab[0 .. 2 ^ (2 * b) - 1]` in $O(n_{\max})$ deterministic instructions from + `(base, n_max, U, M)` alone (zero external primes or lookup hints). +2. **`resampleCode k`**: samples the $k$ polynomial coefficients `seed[0 .. k - 1]` uniformly and + independently in `Fin (2 ^ q)` using $k$ `Code.rand` calls in $O(k)$ instructions while + preserving `clmulTab`, so Cuckoo Hashing can rehash in $O(k) = O(1)$ instructions without + rebuilding `clmulTab`. +3. **`initCode k := .seq (initTableCode k) (resampleCode k)`**: full one-shot initialization. +4. **`evalCode k`**: evaluates Horner's rule for `trinomialPolyHash` in $O(k \cdot L^2 + 1)$ + deterministic instructions, preserving all memory and writing + `(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `regOut`. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.Hashing.KWise.chunkBits`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.limbCount`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.space`: chunk bit-width, limb count, and memory footprint. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.TableRep`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.SeedRep`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.Rep`: memory representation predicates separating the + deterministic `clmulTab` / header from the $k$ random seed words. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.initCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode`: structured Word-RAM programs over + `Basis.exact`. + +## Main results + +* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode_execWithin`: deterministic $O(n_{\max})$ + table construction establishing `TableRep`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRand`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.map_extractSeeds_eval_resampleComp`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRandWithin`: $O(k)$-step, $k$-query seed + resampling preserving `TableRep` and inducing `PMF.uniformOfFintype (Fin k → Fin (2 ^ q))`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode_execWithin`: deterministic $O(k \cdot L^2 + 1)$ + Horner evaluation matching `trinomialPolyHash`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.isKWiseIndependentHash_eval`: end-to-end Word-RAM + $k$-wise independence theorem combining `resampleComp`, `evalCode_execWithin`, and + `isKWiseIndependentHash_trinomialPolyHash`. + +## References + +* [J. L. Carter and M. N. Wegman, *Universal Classes of Hash Functions*][CarterWegman1979] +* [M. N. Wegman and J. L. Carter, *New Hash Functions and Their Use in Authentication and Set + Equality*][WegmanCarter1981] +* [R. Lidl and H. Niederreiter, *Finite Fields*][LidlNiederreiter1997] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.Hashing.KWise + +open Cslib.WordRAM Cslib.Probability +open Cslib.Algorithms.WordRAM.Hashing.Tabulation +open scoped ENNReal + +/-! ## Section 1: Parameters, bounds, and memory layout -/ + +/-- Chunk bit-width `b = max 1 (⌊log₂ (max n_max 2)⌋ / 3)` for Four-Russians carryless +multiplication, ensuring `1 ≤ b` and `2 ^ (2 * b) ≤ max 4 n_max`. -/ +@[expose] def chunkBits (n_max : ℕ) : ℕ := + max 1 (Nat.log 2 (max n_max 2) / 3) + +/-- `chunkBits n_max` is always at least `1`. -/ +theorem one_le_chunkBits (n_max : ℕ) : 1 ≤ chunkBits n_max := + Nat.le_max_left 1 _ + +/-- `chunkBits n_max` is always positive. -/ +theorem chunkBits_pos (n_max : ℕ) : 0 < chunkBits n_max := + one_le_chunkBits n_max + +/-- The Four-Russians carryless multiplication table size `2 ^ (2 * chunkBits n_max)` is at most +`max 4 n_max` (`= O(n_max^(2/3)) ≤ O(n_max)` words). -/ +theorem two_pow_two_mul_chunkBits_le (n_max : ℕ) : + 2 ^ (2 * chunkBits n_max) ≤ max 4 n_max := by + by_cases hdiv : Nat.log 2 (max n_max 2) / 3 ≤ 1 + · have hcb : chunkBits n_max = 1 := by simp [chunkBits]; omega + rw [hcb] + exact Nat.le_max_left 4 n_max + · have hcb : chunkBits n_max = Nat.log 2 (max n_max 2) / 3 := by simp [chunkBits]; omega + have h2cb : 2 * chunkBits n_max ≤ Nat.log 2 (max n_max 2) := by omega + calc + 2 ^ (2 * chunkBits n_max) ≤ 2 ^ Nat.log 2 (max n_max 2) := + Nat.pow_le_pow_right (by decide) h2cb + _ ≤ max n_max 2 := Nat.pow_log_le_self 2 (by omega) + _ ≤ max 4 n_max := by omega + +/-- Number of `b`-bit limbs `L = ⌈q / b⌉ = (q + b - 1) / b` needed to cover a `q`-bit field +element. -/ +@[expose] def limbCountFromB (b q : ℕ) : ℕ := + (q + b - 1) / b + +/-- Number of `chunkBits n_max`-bit limbs covering `trinomialDeg U M` bits. -/ +@[expose] def limbCount (n_max U M : ℕ) : ℕ := + limbCountFromB (chunkBits n_max) (trinomialDeg U M) + +/-- When `0 < b`, `limbCountFromB b q * b` is at least `q`. -/ +theorem le_limbCountFromB_mul {b : ℕ} (hb : 0 < b) (q : ℕ) : + q ≤ limbCountFromB b q * b := by + dsimp only [limbCountFromB] + have hmod := Nat.div_add_mod (q + b - 1) b + have hlt : (q + b - 1) % b < b := Nat.mod_lt _ hb + rw [Nat.mul_comm] + omega + +/-- `limbCount n_max U M * chunkBits n_max` covers all `trinomialDeg U M` bits. -/ +theorem trinomialDeg_le_limbCount_mul_chunkBits (n_max U M : ℕ) : + trinomialDeg U M ≤ limbCount n_max U M * chunkBits n_max := + le_limbCountFromB_mul (chunkBits_pos n_max) (trinomialDeg U M) + +/-- Memory footprint `8 + k + 2 ^ (2 * b)` words for `k`-wise independent hashing with chunk width +`b`: 8 header words, `k` seed words, and `2 ^ (2 * b)` carryless multiplication table entries. -/ +@[expose] def spaceFromB (b k : ℕ) : ℕ := + 8 + k + 2 ^ (2 * b) + +/-- Memory footprint `8 + k + 2 ^ (2 * chunkBits n_max)` words at capacity parameter `n_max`. -/ +@[expose] def space (n_max k : ℕ) : ℕ := + spaceFromB (chunkBits n_max) k + +/-- `space n_max k` is at most `8 + k + max 4 n_max = O(n_max + k)`. -/ +theorem space_le (n_max k : ℕ) : space n_max k ≤ 8 + k + max 4 n_max := by + dsimp only [space, spaceFromB] + have := two_pow_two_mul_chunkBits_le n_max + omega + +/-- Deterministic table representation predicate `TableRep base b U M k mem`: +stores the 8 header parameters `(r, q, 2 ^ r - 1, 2 ^ q - 1, 2 ^ M - 1, b, 2 ^ b - 1, L)` at +`base + 0 .. base + 7` and the Four-Russians carryless multiplication table +`clmul (idx >>> b) (idx &&& (2 ^ b - 1))` at `base + 8 + k + idx` for `idx < 2 ^ (2 * b)`. -/ +@[expose] def TableRep (base b U M k : ℕ) (mem : ℕ → ℕ) : Prop := + let r := 3 ^ trinomialStep U M + let q := trinomialDeg U M + mem base = r ∧ + mem (base + 1) = q ∧ + mem (base + 2) = 2 ^ r - 1 ∧ + mem (base + 3) = 2 ^ q - 1 ∧ + mem (base + 4) = 2 ^ M - 1 ∧ + mem (base + 5) = b ∧ + mem (base + 6) = 2 ^ b - 1 ∧ + mem (base + 7) = limbCountFromB b q ∧ + ∀ idx < 2 ^ (2 * b), + mem (base + 8 + k + idx) = clmul (idx >>> b) (idx &&& (2 ^ b - 1)) + +/-- Seed representation predicate `SeedRep base U M k c mem`: stores the `k` polynomial +coefficients `(c j).val < 2 ^ (trinomialDeg U M)` at `base + 8 + j.val`. -/ +@[expose] def SeedRep (base U M k : ℕ) + (c : Fin k → Fin (2 ^ trinomialDeg U M)) (mem : ℕ → ℕ) : Prop := + ∀ j : Fin k, mem (base + 8 + j.val) = (c j).val + +/-- Full representation predicate combining `TableRep` and `SeedRep`. -/ +@[expose] def Rep (base b U M k : ℕ) + (c : Fin k → Fin (2 ^ trinomialDeg U M)) (mem : ℕ → ℕ) : Prop := + TableRep base b U M k mem ∧ SeedRep base U M k c mem + +/-- Extract the `k`-tuple of seeds `Fin k → Fin (2 ^ trinomialDeg U M)` from +`[base + 8, base + 8 + k)`. -/ +@[expose] def extractSeeds (base U M k : ℕ) (mem : ℕ → ℕ) : + Fin k → Fin (2 ^ trinomialDeg U M) := + extractSlice (base + 8) (trinomialDeg U M) k mem + +/-! ## Section 2: Helper loops for `initTableCode` -/ + +/-- Body of `clog2Loop` on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, `reg 2 = 1`): updates +`rem ← (rem + 1) >>> 1` (`= (rem + 1) / 2`) and `acc ← acc + 1`. -/ +@[expose] def clog2Body : Code .exact 3 := + .seq (.op (.exactBin .add 0 0 2)) + (.seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2))) + +/-- Ceiling-`log₂` loop `while 1 < rem do [ rem ← (rem + 1) >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def clog2Loop : Code .exact 3 := + .loop (.lt 2 0) clog2Body + +/-- Closed-form final state of `clog2Loop`. -/ +@[expose] def clog2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.clog 2 rem) + +/-- `clog2Loop` executes deterministically in `5 * Nat.clog 2 rem + 2` instructions, accumulating +`acc + Nat.clog 2 rem` into register `1` and preserving all memory. -/ +theorem clog2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem1 : Fits w (rem + 1)) (hfit_acc : Fits w (acc + Nat.clog 2 rem)) : + Exec w clog2Loop x (clog2LoopData rem acc x) (5 * Nat.clog 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : (rem + 1) >>> 1 = (rem + 1) / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hclog : Nat.clog 2 rem = Nat.clog 2 ((rem + 1) / 2) + 1 := + Nat.clog_of_one_lt (by decide) hlt + have hdiv_lt : (rem + 1) / 2 < rem := by omega + have hfit_div1 : Fits w ((rem + 1) / 2 + 1) := hfit_rem1.of_le (by omega) + have hfit_div : Fits w ((rem + 1) / 2) := hfit_div1.of_le (by omega) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem + 1) + have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone]) hfit_rem1 + set d2 := d1.set 0 ((rem + 1) / 2) + have s2 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hone, hshr]) hfit_div + set d3 := d2.set 1 (acc + 1) + have s3 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d2 d3 1 := + exec_exactBin (by simp +decide [d2, d1, hacc, hone]) hfit_acc1 + have hrec := ih ((rem + 1) / 2) hdiv_lt (acc := acc + 1) (x := d3) + (by simp +decide [d3, d2]) + (by simp +decide [d3]) + (by simp +decide [d3, d2, d1, hone]) + hfit_div1 + (by rwa [show acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem by omega]) + have hd : clog2LoopData ((rem + 1) / 2) (acc + 1) d3 = clog2LoopData rem acc x := by + have hdiv_ne : (rem + 1) / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, d3, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem + omega + · simp +decide [clog2LoopData, d3, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec + simpa only [clog2Loop, clog2Body, hclog, + show 1 + (1 + 1) + (5 * Nat.clog 2 ((rem + 1) / 2) + 2) + 2 = + 5 * (Nat.clog 2 ((rem + 1) / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hclog0 : Nat.clog 2 rem = 0 := Nat.clog_of_right_le_one (by omega) 2 + have hd : clog2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, hrem_eq, hrem] + · simp +decide [clog2LoopData, hclog0, hacc] + · simp +decide [clog2LoopData] + · rfl + rw [hd, hclog0] + exact Exec.loop_exit hc + +/-- Final remainder reached by `trinomialStepAux` when repeatedly mapping `B ↦ (B + 2) / 3` until +`≤ 2`. -/ +@[expose] def trinomialStepRem (B : ℕ) : ℕ := + if B ≤ 2 then B else trinomialStepRem ((B + 2) / 3) +decreasing_by omega + +/-- `trinomialStepRem B` is always at most `2`. -/ +theorem trinomialStepRem_le_two (B : ℕ) : trinomialStepRem B ≤ 2 := by + induction B using Nat.strong_induction_on with + | h B ih => + rw [trinomialStepRem] + split_ifs with hB + · exact hB + · exact ih ((B + 2) / 3) (by omega) + +/-- `trinomialStepAux B` is at most `B`. -/ +theorem trinomialStepAux_le (B : ℕ) : trinomialStepAux B ≤ B := by + induction B using Nat.strong_induction_on with + | h B ih => + rw [trinomialStepAux] + split_ifs with hB + · exact Nat.zero_le B + · have := ih ((B + 2) / 3) (by omega) + omega + +/-- Body of `trinomialStepLoop` on `Data 4` (`reg 0 = rem`, `reg 1 = r`, `reg 2 = 2`, +`reg 3 = 3`): updates `rem ← (rem + 2) / 3` and `r ← r * 3`. -/ +@[expose] def trinomialStepBody : Code .exact 4 := + .seq (.op (.exactBin .add 0 0 2)) + (.seq (.op (.exactBin .div 0 0 3)) + (.op (.exactBin .mul 1 1 3))) + +/-- Loop `while 2 < rem do [ rem ← (rem + 2) / 3; r ← r * 3 ]` computing +`3 ^ trinomialStepAux B`. -/ +@[expose] def trinomialStepLoop : Code .exact 4 := + .loop (.lt 2 0) trinomialStepBody + +/-- Closed-form final state of `trinomialStepLoop`. -/ +@[expose] def trinomialStepLoopData (B acc : ℕ) (x : Data 4) : Data 4 := + (x.set 0 (trinomialStepRem B)).set 1 (acc * 3 ^ trinomialStepAux B) + +/-- `trinomialStepLoop` executes deterministically in `5 * trinomialStepAux B + 2` instructions, +accumulating `acc * 3 ^ trinomialStepAux B` into register `1` and preserving all memory. -/ +theorem trinomialStepLoop_exec {w : ℕ} (B : ℕ) {acc : ℕ} {x : Data 4} + (hrem : x.regs 0 = B) (hacc : x.regs 1 = acc) (htwo : x.regs 2 = 2) (hthree : x.regs 3 = 3) + (hfit_B2 : Fits w (B + 2)) (hfit_res : Fits w (acc * 3 ^ trinomialStepAux B)) : + Exec w trinomialStepLoop x (trinomialStepLoopData B acc x) + (5 * trinomialStepAux B + 2) := by + induction B using Nat.strong_induction_on generalizing acc x with + | h B ih => + by_cases hlt : 2 < B + · have hc : (Cond.lt (2 : Fin 4) 0).Holds x := by + simp [Cond.Holds, hrem, htwo, hlt] + have hstep : trinomialStepAux B = trinomialStepAux ((B + 2) / 3) + 1 := by + rw [trinomialStepAux, ite_eq_right (by omega)] + have hrem_step : trinomialStepRem B = trinomialStepRem ((B + 2) / 3) := by + rw [trinomialStepRem, ite_eq_right (by omega)] + have hmul_eq : (acc * 3) * 3 ^ trinomialStepAux ((B + 2) / 3) = + acc * 3 ^ trinomialStepAux B := by + rw [hstep, Nat.pow_succ'] + ring + have hfit_div2 : Fits w ((B + 2) / 3 + 2) := hfit_B2.of_le (by omega) + have hfit_div : Fits w ((B + 2) / 3) := hfit_div2.of_le (by omega) + have hfit_acc3 : Fits w (acc * 3) := by + have hpos : 1 ≤ 3 ^ trinomialStepAux ((B + 2) / 3) := Nat.one_le_pow _ 3 (by decide) + have hle : acc * 3 ≤ (acc * 3) * 3 ^ trinomialStepAux ((B + 2) / 3) := + Nat.le_mul_of_pos_right _ hpos + exact hfit_res.of_le (hmul_eq ▸ hle) + set d1 := x.set 0 (B + 2) + have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 4) x d1 1 := + exec_exactBin (by simp [hrem, htwo]) hfit_B2 + set d2 := d1.set 0 ((B + 2) / 3) + have s2 : Exec w (.op (.exactBin .div 0 0 3) : Code .exact 4) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hthree]) hfit_div + set d3 := d2.set 1 (acc * 3) + have s3 : Exec w (.op (.exactBin .mul 1 1 3) : Code .exact 4) d2 d3 1 := + exec_exactBin (by simp +decide [d2, d1, hacc, hthree]) hfit_acc3 + have hrec := ih ((B + 2) / 3) (by omega) (acc := acc * 3) (x := d3) + (by simp +decide [d3, d2]) + (by simp +decide [d3]) + (by simp +decide [d3, d2, d1, htwo]) + (by simp +decide [d3, d2, d1, hthree]) + hfit_div2 + (by rwa [hmul_eq]) + have hd : trinomialStepLoopData ((B + 2) / 3) (acc * 3) d3 = + trinomialStepLoopData B acc x := by + ext i : 2 + · fin_cases i + · simp +decide [trinomialStepLoopData, d3, d2, d1, hrem_step] + · simp +decide [trinomialStepLoopData, d3, d2, d1, hmul_eq] + · simp +decide [trinomialStepLoopData, d3, d2, d1] + · simp +decide [trinomialStepLoopData, d3, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec + simpa only [trinomialStepLoop, trinomialStepBody, hstep, + show 1 + (1 + 1) + (5 * trinomialStepAux ((B + 2) / 3) + 2) + 2 = + 5 * (trinomialStepAux ((B + 2) / 3) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 4) 0).Holds x := by + simp [Cond.Holds, hrem, htwo, hlt] + have hstep0 : trinomialStepAux B = 0 := by + rw [trinomialStepAux, ite_eq_left (by omega)] + have hrem0 : trinomialStepRem B = B := by + rw [trinomialStepRem, ite_eq_left (by omega)] + have hd : trinomialStepLoopData B acc x = x := by + ext i : 2 + · fin_cases i + · simp +decide [trinomialStepLoopData, hrem0, hrem] + · simp +decide [trinomialStepLoopData, hstep0, hacc] + · simp +decide [trinomialStepLoopData] + · simp +decide [trinomialStepLoopData] + · rfl + rw [hd, hstep0] + exact Exec.loop_exit hc + +/-- Bitwise decomposition of `idx < 2 ^ (2 * b)` into high `b`-bit chunk `u = idx >>> b` and low +`b`-bit chunk `v = idx &&& (2 ^ b - 1)`, together with properties of +`prevIdx = (u <<< b) + (v >>> 1)`. -/ +private theorem clmulTab_index_props {b idx : ℕ} (hidx : idx < 2 ^ (2 * b)) : + let u := idx >>> b + let v := idx &&& (2 ^ b - 1) + let prevIdx := (u <<< b) + (v >>> 1) + u < 2 ^ b ∧ + v < 2 ^ b ∧ + (u <<< b) + v = idx ∧ + prevIdx ≤ idx ∧ + (v ≠ 0 → prevIdx < idx) ∧ + (v = 0 → prevIdx = idx) ∧ + prevIdx >>> b = u ∧ + prevIdx &&& (2 ^ b - 1) = v >>> 1 := by + intro u v prevIdx + have hpow_pos : 0 < 2 ^ b := Nat.two_pow_pos b + have hu_eq : u = idx / 2 ^ b := Nat.shiftRight_eq_div_pow idx b + have hv_eq : v = idx % 2 ^ b := Nat.and_two_pow_sub_one_eq_mod idx b + have hu_lt : u < 2 ^ b := by + rw [hu_eq] + apply Nat.div_lt_of_lt_mul + rwa [← pow_add, ← two_mul] + have hv_lt : v < 2 ^ b := by + rw [hv_eq] + exact Nat.mod_lt idx hpow_pos + have huv_eq : (u <<< b) + v = idx := by + rw [Nat.shiftLeft_eq, hu_eq, hv_eq, Nat.div_add_mod' idx (2 ^ b)] + have hv_shr : v >>> 1 = v / 2 := Nat.shiftRight_eq_div_pow v 1 + have hv_shr_lt : v >>> 1 < 2 ^ b := by + rw [hv_shr] + exact Nat.lt_of_le_of_lt (Nat.div_le_self v 2) hv_lt + have hprev_eq : prevIdx = u * 2 ^ b + (v >>> 1) := by + dsimp only [prevIdx] + rw [Nat.shiftLeft_eq] + have hprev_le : prevIdx ≤ idx := by + have : v >>> 1 ≤ v := by rw [hv_shr]; exact Nat.div_le_self v 2 + dsimp only [prevIdx] + omega + have hprev_lt : v ≠ 0 → prevIdx < idx := by + intro hv0 + have : v >>> 1 < v := by rw [hv_shr]; omega + dsimp only [prevIdx] + omega + have hprev_zero : v = 0 → prevIdx = idx := by + intro hv0 + have : v >>> 1 = 0 := by rw [hv_shr, hv0] + dsimp only [prevIdx] + omega + have hprev_u : prevIdx >>> b = u := by + rw [Nat.shiftRight_eq_div_pow, hprev_eq, Nat.add_comm, + Nat.add_mul_div_right _ _ hpow_pos, Nat.div_eq_of_lt hv_shr_lt, zero_add] + have hprev_v : prevIdx &&& (2 ^ b - 1) = v >>> 1 := by + rw [Nat.and_two_pow_sub_one_eq_mod, hprev_eq, Nat.add_comm, + Nat.add_mul_mod_self_right, Nat.mod_eq_of_lt hv_shr_lt] + exact ⟨hu_lt, hv_lt, huv_eq, hprev_le, hprev_lt, hprev_zero, hprev_u, hprev_v⟩ + +/-- Step recurrence for `clmul u v` in `clmulTabBody`: +`((v &&& 1) * u) ^^^ (clmul u (v >>> 1) <<< 1)` equals `clmul u v`. -/ +private theorem clmul_eq_bit_mul_xor (u v : ℕ) : + ((v &&& 1) * u) ^^^ (clmul u (v >>> 1) <<< 1) = clmul u v := by + rw [clmul_step u v] + congr 1 + have hand : v &&& 1 = v % 2 := Nat.and_two_pow_sub_one_eq_mod v 1 + rw [hand] + rcases Nat.mod_two_eq_zero_or_one v with h0 | h1 + · simp [h0] + · simp [h1] + +/-- Body of `clmulTabLoop` on `Data 18` (`reg 0 = tabBase`, `reg 1 = idx`, `reg 2 = tabSize`, +`reg 3 = b`, `reg 4 = 2 ^ b - 1`, `reg 5 = 1`, `reg 6..9` scratch): populates +`mem[tabBase + idx] ← clmul (idx >>> b) (idx &&& (2 ^ b - 1))` using previously filled entry +`prevIdx = ((idx >>> b) <<< b) + ((idx &&& (2 ^ b - 1)) >>> 1)` and increments `idx`. -/ +@[expose] def clmulTabBody : Code .exact 18 := + .seq (.op (.exactBin .add 6 0 1)) + (.seq (.op (.const 9 0)) + (.seq (.op (.store 6 9)) + (.seq (.op (.exactBin .shiftRight 7 1 3)) + (.seq (.op (.exactBin .and 8 1 4)) + (.seq (.op (.exactBin .shiftRight 9 8 5)) + (.seq (.op (.exactBin .shiftLeft 6 7 3)) + (.seq (.op (.exactBin .add 9 6 9)) + (.seq (.op (.exactBin .add 9 0 9)) + (.seq (.op (.load 9 9)) + (.seq (.op (.exactBin .shiftLeft 9 9 5)) + (.seq (.op (.exactBin .and 8 8 5)) + (.seq (.op (.exactBin .mul 8 8 7)) + (.seq (.op (.exactBin .xor 9 8 9)) + (.seq (.op (.exactBin .add 6 0 1)) + (.seq (.op (.store 6 9)) + (.op (.exactBin .add 1 1 5))))))))))))))))) + +/-- Loop `while idx < tabSize do clmulTabBody` populating the Four-Russians carryless +multiplication table `clmulTab[0 .. 2 ^ (2 * b) - 1]`. -/ +@[expose] def clmulTabLoop : Code .exact 18 := + .loop (.lt 1 2) clmulTabBody + +/-- `clmulTabLoop` executes deterministically in `19 * (2 ^ (2 * b) - idx) + 2` instructions, +filling `mem[tabBase + i] = clmul (i >>> b) (i &&& (2 ^ b - 1))` for all `i < 2 ^ (2 * b)` and +preserving `reg 0 = tabBase` and memory outside `[tabBase + idx, tabBase + 2 ^ (2 * b))`. -/ +theorem clmulTabLoop_exec {w tabBase b : ℕ} + (hfit_end : Fits w (tabBase + 2 ^ (2 * b))) : + ∀ rem idx : ℕ, ∀ x : Data 18, + idx + rem = 2 ^ (2 * b) → + x.regs 0 = tabBase → + x.regs 1 = idx → + x.regs 2 = 2 ^ (2 * b) → + x.regs 3 = b → + x.regs 4 = 2 ^ b - 1 → + x.regs 5 = 1 → + (∀ i < idx, x.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) → + ∃ y : Data 18, + Exec w clmulTabLoop x y (19 * rem + 2) ∧ + y.regs 0 = tabBase ∧ + (∀ i < 2 ^ (2 * b), y.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) ∧ + (∀ a, (a < tabBase + idx ∨ tabBase + 2 ^ (2 * b) ≤ a) → y.mem a = x.mem a) := by + intro rem + induction rem with + | zero => + intro idx x hsum h0 h1 h2 _ _ _ hinv + have hidx_eq : idx = 2 ^ (2 * b) := by omega + have hc : ¬(Cond.lt (1 : Fin 18) 2).Holds x := by + simp [Cond.Holds, h1, h2, hidx_eq] + refine ⟨x, Exec.loop_exit hc, h0, ?_, fun _ _ => rfl⟩ + intro i hi + exact hinv i (hidx_eq ▸ hi) + | succ rem ih => + intro idx x hsum h0 h1 h2 h3 h4 h5 hinv + have hidx_lt : idx < 2 ^ (2 * b) := by omega + have hc : (Cond.lt (1 : Fin 18) 2).Holds x := by + simp [Cond.Holds, h1, h2, hidx_lt] + obtain ⟨hu_lt, hv_lt, huv_eq, hprev_le, hprev_lt, hprev_zero, hprev_u, hprev_v⟩ := + clmulTab_index_props hidx_lt + set u := idx >>> b + set v := idx &&& (2 ^ b - 1) + set prevIdx := (u <<< b) + (v >>> 1) + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hfit_idx : Fits w idx := hfit_end.of_le (by omega) + have hfit_addr : Fits w (tabBase + idx) := hfit_end.of_le (by omega) + have hfit_0 : Fits w 0 := hfit_end.of_le (Nat.zero_le _) + have hfit_u : Fits w u := hfit_end.of_le (by omega) + have hfit_v : Fits w v := hfit_end.of_le (by omega) + have hfit_vshr : Fits w (v >>> 1) := hfit_v.of_le (Nat.shiftRight_le v 1) + have hfit_ushl : Fits w (u <<< b) := hfit_idx.of_le (by omega) + have hfit_prev : Fits w prevIdx := hfit_idx.of_le hprev_le + have hfit_prevAddr : Fits w (tabBase + prevIdx) := hfit_addr.of_le (by omega) + have hprev_val_lt : clmul u (v >>> 1) < 2 ^ (2 * b - 1) := by + by_cases hb0 : b = 0 + · subst hb0 + have hu0 : u = 0 := by omega + simp [hu0] + · have hv_shr_lt : v >>> 1 < 2 ^ (b - 1) := by + rw [Nat.shiftRight_eq_div_pow, pow_one] + apply Nat.div_lt_of_lt_mul + rwa [← pow_succ', Nat.sub_add_cancel (by omega)] + have hlt := clmul_lt_two_pow hu_lt hv_shr_lt + rwa [show b + (b - 1) = 2 * b - 1 by omega] at hlt + have hsh_lt : clmul u (v >>> 1) <<< 1 < 2 ^ (2 * b) := by + by_cases hb0 : b = 0 + · subst hb0 + have hu0 : u = 0 := by omega + simp [hu0] + · rw [Nat.shiftLeft_eq, pow_one] + have hpow_split : 2 ^ (2 * b) = 2 ^ (2 * b - 1) * 2 := by + conv_lhs => rw [← Nat.sub_add_cancel (show 1 ≤ 2 * b by omega), pow_succ] + omega + have hfit_sh : Fits w (clmul u (v >>> 1) <<< 1) := hfit_end.of_le (by omega) + have hfit_vbit : Fits w (v &&& 1) := hfit_v.of_le Nat.and_le_left + have hbit_le : (v &&& 1) * u ≤ u := by + calc (v &&& 1) * u ≤ 1 * u := Nat.mul_le_mul_right u Nat.and_le_right + _ = u := Nat.one_mul u + have hfit_term : Fits w ((v &&& 1) * u) := hfit_u.of_le hbit_le + have hval_lt : clmul u v < 2 ^ (2 * b) := clmul_lt_two_pow_two_mul hu_lt hv_lt + have hfit_val : Fits w (clmul u v) := hfit_end.of_le (by omega) + have hfit_idx1 : Fits w (idx + 1) := hfit_end.of_le (by omega) + set s1 := x.set 6 (tabBase + idx) + have e1 : Exec w (.op (.exactBin .add 6 0 1) : Code .exact 18) x s1 1 := + exec_exactBin (by simp [h0, h1]) hfit_addr + set s2 := s1.set 9 0 + have e2 : Exec w (.op (.const 9 0) : Code .exact 18) s1 s2 1 := Exec.const hfit_0 + set s3 := s2.write (tabBase + idx) 0 + have e3 : Exec w (.op (.store 6 9) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1] using + (Exec.store (B := .exact) (w := w) (a := 6) (s := 9) (x := s2)) + have hprev_mem : s3.mem (tabBase + prevIdx) = clmul u (v >>> 1) := by + dsimp only [s3, s2, s1, Data.write, Data.set] + by_cases hv0 : v = 0 + · have hpeq : prevIdx = idx := hprev_zero hv0 + have hv_shr0 : v >>> 1 = 0 := by rw [hv0]; rfl + simp [hpeq, hv_shr0] + · have hplt : prevIdx < idx := hprev_lt hv0 + have hne : tabBase + prevIdx ≠ tabBase + idx := by omega + rw [Function.update_of_ne hne, hinv prevIdx hplt, hprev_u, hprev_v] + set s4 := s3.set 7 u + have e4 : Exec w (.op (.exactBin .shiftRight 7 1 3) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, h1, h3, u]) hfit_u + set s5 := s4.set 8 v + have e5 : Exec w (.op (.exactBin .and 8 1 4) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h1, h4, v]) hfit_v + set s6 := s5.set 9 (v >>> 1) + have e6 : Exec w (.op (.exactBin .shiftRight 9 8 5) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h5]) hfit_vshr + set s7 := s6.set 6 (u <<< b) + have e7 : Exec w (.op (.exactBin .shiftLeft 6 7 3) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, s4, s3, s2, s1, h3]) hfit_ushl + set s8 := s7.set 9 prevIdx + have e8 : Exec w (.op (.exactBin .add 9 6 9) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, prevIdx]) hfit_prev + set s9 := s8.set 9 (tabBase + prevIdx) + have e9 : Exec w (.op (.exactBin .add 9 0 9) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h0]) hfit_prevAddr + set s10 := s9.set 9 (clmul u (v >>> 1)) + have e10 : Exec w (.op (.load 9 9) : Code .exact 18) s9 s10 1 := by + simpa +decide [s10, s9, s8, s7, s6, s5, s4, hprev_mem] using + (Exec.load (B := .exact) (w := w) (d := 9) (a := 9) (x := s9)) + set s11 := s10.set 9 (clmul u (v >>> 1) <<< 1) + have e11 : Exec w (.op (.exactBin .shiftLeft 9 9 5) : Code .exact 18) s10 s11 1 := + exec_exactBin (by simp +decide [s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5]) hfit_sh + set s12 := s11.set 8 (v &&& 1) + have e12 : Exec w (.op (.exactBin .and 8 8 5) : Code .exact 18) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5]) hfit_vbit + set s13 := s12.set 8 ((v &&& 1) * u) + have e13 : Exec w (.op (.exactBin .mul 8 8 7) : Code .exact 18) s12 s13 1 := + exec_exactBin (by simp +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4]) hfit_term + have hxor_eq : s13.regs 8 ^^^ s13.regs 9 = clmul u v := by + simpa +decide [s13, s12, s11] using clmul_eq_bit_mul_xor u v + set s14 := s13.set 9 (clmul u v) + have e14 : Exec w (.op (.exactBin .xor 9 8 9) : Code .exact 18) s13 s14 1 := + exec_exactBin (by simp [hxor_eq]) hfit_val + set s15 := s14.set 6 (tabBase + idx) + have e15 : Exec w (.op (.exactBin .add 6 0 1) : Code .exact 18) s14 s15 1 := + exec_exactBin + (by simp +decide [s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0, h1]) + hfit_addr + set s16 := s15.write (tabBase + idx) (clmul u v) + have e16 : Exec w (.op (.store 6 9) : Code .exact 18) s15 s16 1 := by + simpa +decide [s16, s15, s14] using + (Exec.store (B := .exact) (w := w) (a := 6) (s := 9) (x := s15)) + set s17 := s16.set 1 (idx + 1) + have e17 : Exec w (.op (.exactBin .add 1 1 5) : Code .exact 18) s16 s17 1 := + exec_exactBin + (by simp +decide [s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + h1, h5]) + hfit_idx1 + have hbody := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq (e16.seq e17))))))))))))))) + have hs17_mem : s17.mem = Function.update x.mem (tabBase + idx) (clmul u v) := by + dsimp only [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + Data.write, Data.set] + rw [Function.update_idem] + have hinv' : ∀ i < idx + 1, s17.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1)) := by + intro i hi + rw [hs17_mem] + rcases eq_or_ne i idx with rfl | hne + · simp [u, v] + · have hilt : i < idx := by omega + rw [Function.update_of_ne (by omega)] + exact hinv i hilt + have hs17_0 : s17.regs 0 = tabBase := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0] + have hs17_1 : s17.regs 1 = idx + 1 := by simp +decide [s17] + have hs17_2 : s17.regs 2 = 2 ^ (2 * b) := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h2] + have hs17_3 : s17.regs 3 = b := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h3] + have hs17_4 : s17.regs 4 = 2 ^ b - 1 := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h4] + have hs17_5 : s17.regs 5 = 1 := by + simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] + obtain ⟨y, hrec, hy0, hytab, hyout⟩ := + ih (idx + 1) s17 (by omega) hs17_0 hs17_1 hs17_2 hs17_3 hs17_4 hs17_5 hinv' + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, hy0, hytab, ?_⟩ + · simpa only [clmulTabLoop, clmulTabBody, + show 1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + + (1 + (1 + (1 + (1 + (1 + 1))))))))))))))) + (19 * rem + 2) + 2 = + 19 * (rem + 1) + 2 by omega] using hiter + · intro a ha + have ha' : a < tabBase + (idx + 1) ∨ tabBase + 2 ^ (2 * b) ≤ a := by omega + have hne : a ≠ tabBase + idx := by omega + rw [hyout a ha', hs17_mem, Function.update_of_ne hne] + +/-! ## Section 3: Table construction (`initTableCode`) and seed resampling (`resampleCode`) -/ + +/-- Register `0`: holds the base memory address `base`. -/ +abbrev regBase : Fin 18 := 0 + +/-- Register `1`: holds `n_max` on entry to `initTableCode` (and `key` on entry to `evalCode`). -/ +abbrev regNMax : Fin 18 := 1 + +/-- Register `2`: holds `U` on entry to `initTableCode` (and output in `evalCode`). -/ +abbrev regU : Fin 18 := 2 + +/-- Register `3`: holds `M` on entry to `initTableCode`. -/ +abbrev regM : Fin 18 := 3 + +/-- Subprogram on `Data 18` computing `chunkBits n_max` into register `5` and `1` into register `6` +from `n_max` in register `1`, while preserving registers `0, 1, 2, 3` and memory. -/ +@[expose] def chunkBitsCode : Code .exact 18 := + .seq (.op (.const 4 2)) + (.seq (maxCode 4 1 4) + (.seq (.op (.const 5 0)) + (.seq (.op (.const 6 1)) + (.seq (log2Loop.rename ![4, 5, 6]) + (.seq (.op (.const 4 3)) + (.seq (.op (.exactBin .div 5 5 4)) + (maxCode 5 6 5))))))) + +/-- `chunkBitsCode` executes deterministically in `4 * Nat.log 2 (max n_max 2) + 13` steps. -/ +theorem chunkBitsCode_exec {w base n_max U M : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = n_max) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (hfit_n : Fits w (max n_max 2)) (hfit_3 : Fits w 3) : + ∃ y : Data 18, + Exec w chunkBitsCode x y (4 * Nat.log 2 (max n_max 2) + 13) ∧ + y.regs 0 = base ∧ + y.regs 2 = U ∧ + y.regs 3 = M ∧ + y.regs 5 = chunkBits n_max ∧ + y.regs 6 = 1 ∧ + y.mem = x.mem := by + have hfit_2 : Fits w 2 := hfit_3.of_le (by decide) + have hfit_1 : Fits w 1 := hfit_3.of_le (by decide) + have hfit_0 : Fits w 0 := hfit_3.of_le (by decide) + have hlog_le : Nat.log 2 (max n_max 2) ≤ max n_max 2 := Nat.log_le_self 2 _ + have hfit_log : Fits w (0 + Nat.log 2 (max n_max 2)) := hfit_n.of_le (by omega) + have hfit_div : Fits w (Nat.log 2 (max n_max 2) / 3) := hfit_n.of_le (by omega) + set s1 := x.set 4 2 + have e1 : Exec w (.op (.const 4 2) : Code .exact 18) x s1 1 := Exec.const hfit_2 + set s2 := s1.set 4 (max n_max 2) + have e2 : Exec w (maxCode 4 1 4) s1 s2 3 := by + simpa +decide [s2, s1, h1] using maxCode_exec (w := w) 4 1 4 s1 + set s3 := s2.set 5 0 + have e3 : Exec w (.op (.const 5 0) : Code .exact 18) s2 s3 1 := Exec.const hfit_0 + set s4 := s3.set 6 1 + have e4 : Exec w (.op (.const 6 1) : Code .exact 18) s3 s4 1 := Exec.const hfit_1 + set f : Fin 3 → Fin 18 := ![4, 5, 6] + have hmap_inj : Function.Injective f := by decide + have hloop_inner := log2Loop_exec (w := w) (max n_max 2) (acc := 0) + (x := s4.comap f) + (by simp +decide [s4, s3, s2, f, Data.comap]) + (by simp +decide [s4, s3, f, Data.comap]) + (by simp +decide [s4, f, Data.comap]) + hfit_n hfit_log + have e5 := hloop_inner.rename hmap_inj + set s5 := (log2LoopData (max n_max 2) 0 (s4.comap f)).extend f s4 + have hs5_5 : s5.regs 5 = Nat.log 2 (max n_max 2) := by + change (s5.comap f).regs 1 = Nat.log 2 (max n_max 2) + rw [Data.comap_extend hmap_inj] + simp +decide [log2LoopData, s4, s3, s2, s1, f, Data.comap] + have hs5_6 : s5.regs 6 = 1 := by + change (s5.comap f).regs 2 = 1 + rw [Data.comap_extend hmap_inj] + simp +decide [log2LoopData, s4, s3, s2, s1, f, Data.comap] + have hs5_0 : s5.regs 0 = base := by + simp +decide [s5, Data.extend, f, s4, s3, s2, s1, h0] + have hs5_2 : s5.regs 2 = U := by + simp +decide [s5, Data.extend, f, s4, s3, s2, s1, h2] + have hs5_3 : s5.regs 3 = M := by + simp +decide [s5, Data.extend, f, s4, s3, s2, s1, h3] + set s6 := s5.set 4 3 + have e6 : Exec w (.op (.const 4 3) : Code .exact 18) s5 s6 1 := Exec.const hfit_3 + set s7 := s6.set 5 (Nat.log 2 (max n_max 2) / 3) + have e7 : Exec w (.op (.exactBin .div 5 5 4) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, hs5_5]) hfit_div + set s8 := s7.set 5 (chunkBits n_max) + have e8 : Exec w (maxCode 5 6 5) s7 s8 3 := by + simpa +decide [s8, s7, s6, hs5_6, chunkBits] using maxCode_exec (w := w) 5 6 5 s7 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq e8)))))) + refine ⟨s8, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simpa only [chunkBitsCode, f, + show 1 + (3 + (1 + (1 + (4 * Nat.log 2 (max n_max 2) + 2 + (1 + (1 + 3)))))) = + 4 * Nat.log 2 (max n_max 2) + 13 by omega] using hall + · simp +decide [s8, s7, s6, hs5_0] + · simp +decide [s8, s7, s6, hs5_2] + · simp +decide [s8, s7, s6, hs5_3] + · simp +decide [s8] + · simp +decide [s8, s7, s6, hs5_6] + · rfl + +private theorem add_two_le_two_pow {q : ℕ} (hq : 2 ≤ q) : q + 2 ≤ 2 ^ q := by + induction q with + | zero => omega + | succ n ih => + rcases (by omega : n < 2 ∨ 2 ≤ n) with hlt | hge + · have hn1 : n = 1 := by omega + subst hn1 + decide + · have hrec := ih hge + have hpos : 2 ≤ 2 ^ n := + calc 2 = 2 ^ 1 := by decide + _ ≤ 2 ^ n := Nat.pow_le_pow_right (by decide) (by omega) + rw [pow_succ] + omega + +/-- Subprogram on `Data 18` computing `r = 3 ^ trinomialStep U M` into register `1`, +`q = trinomialDeg U M` into register `2`, and `2` into register `4` from `U` in register `2`, +`M` in register `3`, and `1` in register `6`, while preserving `reg 0 = base`, `reg 3 = M`, +`reg 5 = b`, `reg 6 = 1`, and memory. -/ +@[expose] def trinomialParamsCode : Code .exact 18 := + .seq (.op (.const 4 2)) + (.seq (maxCode 7 2 4) + (.seq (.op (.const 8 0)) + (.seq (clog2Loop.rename ![7, 8, 6]) + (.seq (maxCode 7 3 6) + (.seq (maxCode 7 8 7) + (.seq (.op (.const 1 1)) + (.seq (.op (.const 8 3)) + (.seq (trinomialStepLoop.rename ![7, 1, 4, 8]) + (.op (.exactBin .mul 2 4 1)))))))))) + +/-- `trinomialParamsCode` executes deterministically in +`5 * Nat.clog 2 (max U 2) + 5 * trinomialStep U M + 18` steps. -/ +theorem trinomialParamsCode_exec {w base U M b : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (h5 : x.regs 5 = b) (h6 : x.regs 6 = 1) + (hfit_U : Fits w (max U 2 + 1)) (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ∃ y : Data 18, + Exec w trinomialParamsCode x y + (5 * Nat.clog 2 (max U 2) + 5 * trinomialStep U M + 18) ∧ + y.regs 0 = base ∧ + y.regs 1 = 3 ^ trinomialStep U M ∧ + y.regs 2 = trinomialDeg U M ∧ + y.regs 3 = M ∧ + y.regs 4 = 2 ∧ + y.regs 5 = b ∧ + y.regs 6 = 1 ∧ + y.mem = x.mem := by + set B := trinomialTargetBits U M + have hB_le_q : B ≤ trinomialDeg U M := trinomialTargetBits_le_trinomialDeg U M + have hq_ge_2 : 2 ≤ trinomialDeg U M := by + dsimp only [trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + omega + have hpow_q : trinomialDeg U M + 2 ≤ 2 ^ trinomialDeg U M := add_two_le_two_pow hq_ge_2 + have hfit_q2 : Fits w (trinomialDeg U M + 2) := hfit_q.of_le hpow_q + have hfit_B2 : Fits w (B + 2) := hfit_q2.of_le (by omega) + have hclog_le_B : Nat.clog 2 (max U 2) ≤ B := le_max_left _ _ + have hfit_clog : Fits w (0 + Nat.clog 2 (max U 2)) := hfit_B2.of_le (by omega) + have hr_le_q : 3 ^ trinomialStepAux B ≤ trinomialDeg U M := by + change 3 ^ trinomialStep U M ≤ 2 * 3 ^ trinomialStep U M + omega + have hfit_r : Fits w (1 * 3 ^ trinomialStepAux B) := by + rw [Nat.one_mul] + exact hfit_q2.of_le (by omega) + have hfit_q_val : Fits w (trinomialDeg U M) := hfit_q2.of_le (by omega) + have hfit_3 : Fits w 3 := hfit_q2.of_le (by omega) + have hfit_2 : Fits w 2 := hfit_q2.of_le (by omega) + have hfit_1 : Fits w 1 := hfit_q2.of_le (by omega) + have hfit_0 : Fits w 0 := hfit_q2.of_le (by omega) + set s1 := x.set 4 2 + have e1 : Exec w (.op (.const 4 2) : Code .exact 18) x s1 1 := Exec.const hfit_2 + set s2 := s1.set 7 (max U 2) + have e2 : Exec w (maxCode 7 2 4) s1 s2 3 := by + simpa +decide [s2, s1, h2] using maxCode_exec (w := w) 7 2 4 s1 + set s3 := s2.set 8 0 + have e3 : Exec w (.op (.const 8 0) : Code .exact 18) s2 s3 1 := Exec.const hfit_0 + set f1 : Fin 3 → Fin 18 := ![7, 8, 6] + have hmap1 : Function.Injective f1 := by decide + have hclog_inner := clog2Loop_exec (w := w) (max U 2) (acc := 0) + (x := s3.comap f1) + (by simp +decide [s3, s2, f1, Data.comap]) + (by simp +decide [s3, f1, Data.comap]) + (by simp +decide [s3, s2, s1, h6, f1, Data.comap]) + hfit_U hfit_clog + have e4 := hclog_inner.rename hmap1 + set s4 := (clog2LoopData (max U 2) 0 (s3.comap f1)).extend f1 s3 + have hs4_8 : s4.regs 8 = Nat.clog 2 (max U 2) := by + change (s4.comap f1).regs 1 = Nat.clog 2 (max U 2) + rw [Data.comap_extend hmap1] + simp +decide [clog2LoopData, s3, s2, s1, f1, Data.comap] + have hs4_6 : s4.regs 6 = 1 := by + change (s4.comap f1).regs 2 = 1 + rw [Data.comap_extend hmap1] + simp +decide [clog2LoopData, s3, s2, s1, h6, f1, Data.comap] + have hs4_0 : s4.regs 0 = base := by + simp +decide [s4, Data.extend, f1, s3, s2, s1, h0] + have hs4_3 : s4.regs 3 = M := by + simp +decide [s4, Data.extend, f1, s3, s2, s1, h3] + have hs4_4 : s4.regs 4 = 2 := by + simp +decide [s4, Data.extend, f1, s3, s2, s1] + have hs4_5 : s4.regs 5 = b := by + simp +decide [s4, Data.extend, f1, s3, s2, s1, h5] + set s5 := s4.set 7 (max M 1) + have e5 : Exec w (maxCode 7 3 6) s4 s5 3 := by + simpa +decide [s5, hs4_3, hs4_6] using maxCode_exec (w := w) 7 3 6 s4 + set s6 := s5.set 7 B + have e6 : Exec w (maxCode 7 8 7) s5 s6 3 := by + simpa +decide [s6, s5, hs4_8, B, trinomialTargetBits] using maxCode_exec (w := w) 7 8 7 s5 + set s7 := s6.set 1 1 + have e7 : Exec w (.op (.const 1 1) : Code .exact 18) s6 s7 1 := Exec.const hfit_1 + set s8 := s7.set 8 3 + have e8 : Exec w (.op (.const 8 3) : Code .exact 18) s7 s8 1 := Exec.const hfit_3 + set f2 : Fin 4 → Fin 18 := ![7, 1, 4, 8] + have hmap2 : Function.Injective f2 := by decide + have hstep_inner := trinomialStepLoop_exec (w := w) B (acc := 1) + (x := s8.comap f2) + (by simp +decide [s8, s7, s6, f2, Data.comap]) + (by simp +decide [s8, s7, f2, Data.comap]) + (by simp +decide [s8, s7, s6, s5, hs4_4, f2, Data.comap]) + (by simp +decide [s8, f2, Data.comap]) + hfit_B2 hfit_r + have e9 := hstep_inner.rename hmap2 + set s9 := (trinomialStepLoopData B 1 (s8.comap f2)).extend f2 s8 + have hs9_1 : s9.regs 1 = 3 ^ trinomialStep U M := by + change (s9.comap f2).regs 1 = 3 ^ trinomialStep U M + rw [Data.comap_extend hmap2] + simp +decide [trinomialStepLoopData, trinomialStep, B, s8, s7, f2, Data.comap] + have hs9_4 : s9.regs 4 = 2 := by + change (s9.comap f2).regs 2 = 2 + rw [Data.comap_extend hmap2] + simp +decide [trinomialStepLoopData, s8, s7, s6, s5, hs4_4, f2, Data.comap] + have hs9_0 : s9.regs 0 = base := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_0] + have hs9_3 : s9.regs 3 = M := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_3] + have hs9_5 : s9.regs 5 = b := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_5] + have hs9_6 : s9.regs 6 = 1 := by + simp +decide [s9, Data.extend, f2, s8, s7, s6, s5, hs4_6] + set s10 := s9.set 2 (trinomialDeg U M) + have e10 : Exec w (.op (.exactBin .mul 2 4 1) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [hs9_4, hs9_1, trinomialDeg]) hfit_q_val + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq e10)))))))) + refine ⟨s10, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simpa only [trinomialParamsCode, trinomialStep, B, f1, f2, + show 1 + (3 + (1 + (5 * Nat.clog 2 (max U 2) + 2 + (3 + (3 + (1 + (1 + + (5 * trinomialStepAux (trinomialTargetBits U M) + 2 + 1)))))))) = + 5 * Nat.clog 2 (max U 2) + + 5 * trinomialStepAux (trinomialTargetBits U M) + 18 by omega] using hall + · simp +decide [s10, hs9_0] + · simp +decide [s10, hs9_1] + · simp +decide [s10] + · simp +decide [s10, hs9_3] + · simp +decide [s10, hs9_4] + · simp +decide [s10, hs9_5] + · simp +decide [s10, hs9_6] + · rfl + +/-- First 17-instruction header-store block: writes `r, q, 2 ^ r - 1, 2 ^ q - 1, 2 ^ M - 1, b` to +`base + 0 .. base + 5`. -/ +@[expose] def storeHeaderPart1 : Code .exact 18 := + .seq (.op (.store 0 1)) + (.seq (.op (.exactBin .add 7 0 6)) + (.seq (.op (.store 7 2)) + (.seq (.op (.exactBin .shiftLeft 8 6 1)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .shiftLeft 8 6 2)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .shiftLeft 8 6 3)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .add 7 7 6)) + (.op (.store 7 5))))))))))))))))) + +/-- Second 17-instruction header-store and loop-prep block: writes `2 ^ b - 1` and +`limbCountFromB b q` to `base + 6, base + 7` and sets registers `0..5` for `clmulTabLoop`. -/ +@[expose] def storeHeaderPart2 (k : ℕ) : Code .exact 18 := + .seq (.op (.exactBin .shiftLeft 8 6 5)) + (.seq (.op (.exactBin .sub 8 8 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 8)) + (.seq (.op (.exactBin .sub 9 2 6)) + (.seq (.op (.exactBin .div 9 9 5)) + (.seq (.op (.exactBin .add 9 9 6)) + (.seq (.op (.exactBin .add 7 7 6)) + (.seq (.op (.store 7 9)) + (.seq (.op (.exactBin .mul 2 4 5)) + (.seq (.op (.exactBin .shiftLeft 2 6 2)) + (.seq (.op (.copy 3 5)) + (.seq (.op (.copy 4 8)) + (.seq (.op (.copy 5 6)) + (.seq (.op (.const 1 0)) + (.seq (.op (.const 9 (8 + k))) + (.op (.exactBin .add 0 0 9))))))))))))))))) + +/-- When `0 < b` and `0 < q`, `(q - 1) / b + 1 = limbCountFromB b q`. -/ +private theorem div_pred_add_one_eq_limbCountFromB {b q : ℕ} (hb : 0 < b) (hq : 0 < q) : + (q - 1) / b + 1 = limbCountFromB b q := by + dsimp only [limbCountFromB] + have h : q + b - 1 = (q - 1) + 1 * b := by omega + rw [h, Nat.add_mul_div_right _ _ hb] + +/-- `storeHeaderPart1` executes deterministically in `17` steps. -/ +theorem storeHeaderPart1_exec {w base r q M b : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = r) (h2 : x.regs 2 = q) + (h3 : x.regs 3 = M) (h4 : x.regs 4 = 2) (h5 : x.regs 5 = b) (h6 : x.regs 6 = 1) + (hr_le : r ≤ q) (hM_le : M ≤ q) + (hfit_end : Fits w (base + 8)) (hfit_q : Fits w (2 ^ q)) : + ∃ y : Data 18, + Exec w storeHeaderPart1 x y 17 ∧ + y.regs 0 = base ∧ + y.regs 2 = q ∧ + y.regs 4 = 2 ∧ + y.regs 5 = b ∧ + y.regs 6 = 1 ∧ + y.regs 7 = base + 5 ∧ + y.mem base = r ∧ + y.mem (base + 1) = q ∧ + y.mem (base + 2) = 2 ^ r - 1 ∧ + y.mem (base + 3) = 2 ^ q - 1 ∧ + y.mem (base + 4) = 2 ^ M - 1 ∧ + y.mem (base + 5) = b ∧ + (∀ a, (a < base ∨ base + 6 ≤ a) → y.mem a = x.mem a) := by + have hpow_r : 2 ^ r ≤ 2 ^ q := Nat.pow_le_pow_right (by decide) hr_le + have hpow_M : 2 ^ M ≤ 2 ^ q := Nat.pow_le_pow_right (by decide) hM_le + have hpos_r : 1 ≤ 2 ^ r := Nat.one_le_two_pow + have hpos_q : 1 ≤ 2 ^ q := Nat.one_le_two_pow + have hpos_M : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hfit_pow_r : Fits w (2 ^ r) := hfit_q.of_le hpow_r + have hfit_mask_r : Fits w (2 ^ r - 1) := hfit_pow_r.of_le (by omega) + have hfit_mask_q : Fits w (2 ^ q - 1) := hfit_q.of_le (by omega) + have hfit_pow_M : Fits w (2 ^ M) := hfit_q.of_le hpow_M + have hfit_mask_M : Fits w (2 ^ M - 1) := hfit_pow_M.of_le (by omega) + have hfit_b1 : Fits w (base + 1) := hfit_end.of_le (by omega) + have hfit_b2 : Fits w (base + 2) := hfit_end.of_le (by omega) + have hfit_b3 : Fits w (base + 3) := hfit_end.of_le (by omega) + have hfit_b4 : Fits w (base + 4) := hfit_end.of_le (by omega) + have hfit_b5 : Fits w (base + 5) := hfit_end.of_le (by omega) + set s1 := x.write base r + have e1 : Exec w (.op (.store 0 1) : Code .exact 18) x s1 1 := by + simpa +decide [s1, h0, h1] using (Exec.store (B := .exact) (w := w) (a := 0) (s := 1) (x := x)) + set s2 := s1.set 7 (base + 1) + have e2 : Exec w (.op (.exactBin .add 7 0 6) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h0, h6]) hfit_b1 + set s3 := s2.write (base + 1) q + have e3 : Exec w (.op (.store 7 2) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, h2] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 2) (x := s2)) + set s4 := s3.set 8 (2 ^ r) + have e4 : Exec w (.op (.exactBin .shiftLeft 8 6 1) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, h6, h1, Nat.shiftLeft_eq]) hfit_pow_r + set s5 := s4.set 8 (2 ^ r - 1) + have e5 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h6, hpos_r]) hfit_mask_r + set s6 := s5.set 7 (base + 2) + have e6 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h6, Nat.add_assoc]) hfit_b2 + set s7 := s6.write (base + 2) (2 ^ r - 1) + have e7 : Exec w (.op (.store 7 8) : Code .exact 18) s6 s7 1 := by + simpa +decide [s7, s6, s5] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s6)) + set s8 := s7.set 8 (2 ^ q) + have e8 : Exec w (.op (.exactBin .shiftLeft 8 6 2) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, h6, h2, Nat.shiftLeft_eq]) hfit_q + set s9 := s8.set 8 (2 ^ q - 1) + have e9 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h6, hpos_q]) hfit_mask_q + set s10 := s9.set 7 (base + 3) + have e10 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, Nat.add_assoc]) hfit_b3 + set s11 := s10.write (base + 3) (2 ^ q - 1) + have e11 : Exec w (.op (.store 7 8) : Code .exact 18) s10 s11 1 := by + simpa +decide [s11, s10, s9] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s10)) + set s12 := s11.set 8 (2 ^ M) + have e12 : Exec w (.op (.exactBin .shiftLeft 8 6 3) : Code .exact 18) s11 s12 1 := + exec_exactBin + (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, h3, Nat.shiftLeft_eq]) + hfit_pow_M + set s13 := s12.set 8 (2 ^ M - 1) + have e13 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s12 s13 1 := + exec_exactBin (by simp +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, hpos_M]) + hfit_mask_M + set s14 := s13.set 7 (base + 4) + have e14 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s13 s14 1 := + exec_exactBin + (by simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, Nat.add_assoc]) + hfit_b4 + set s15 := s14.write (base + 4) (2 ^ M - 1) + have e15 : Exec w (.op (.store 7 8) : Code .exact 18) s14 s15 1 := by + simpa +decide [s15, s14, s13] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s14)) + set s16 := s15.set 7 (base + 5) + have e16 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s15 s16 1 := + exec_exactBin + (by simp +decide [s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, + Nat.add_assoc]) + hfit_b5 + set s17 := s16.write (base + 5) b + have e17 : Exec w (.op (.store 7 5) : Code .exact 18) s16 s17 1 := by + simpa +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] + using (Exec.store (B := .exact) (w := w) (a := 7) (s := 5) (x := s16)) + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq (e16.seq e17))))))))))))))) + refine ⟨s17, hall, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h2] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h4] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6] + · simp +decide [s17, s16] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · intro a ha + dsimp only [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + Data.write, Data.set] + rw [Function.update_of_ne (by omega), Function.update_of_ne (by omega), + Function.update_of_ne (by omega), Function.update_of_ne (by omega), + Function.update_of_ne (by omega), Function.update_of_ne (by omega)] + +/-- `storeHeaderPart2 k` executes deterministically in `17` steps. -/ +theorem storeHeaderPart2_exec {w base q b k : ℕ} {x : Data 18} + (hb : 0 < b) (hq : 0 < q) + (h0 : x.regs 0 = base) (h2 : x.regs 2 = q) (h4 : x.regs 4 = 2) + (h5 : x.regs 5 = b) (h6 : x.regs 6 = 1) (h7 : x.regs 7 = base + 5) + (hfit_space : Fits w (base + spaceFromB b k)) (hfit_q : Fits w q) : + ∃ y : Data 18, + Exec w (storeHeaderPart2 k) x y 17 ∧ + y.regs 0 = base + 8 + k ∧ + y.regs 1 = 0 ∧ + y.regs 2 = 2 ^ (2 * b) ∧ + y.regs 3 = b ∧ + y.regs 4 = 2 ^ b - 1 ∧ + y.regs 5 = 1 ∧ + y.mem (base + 6) = 2 ^ b - 1 ∧ + y.mem (base + 7) = limbCountFromB b q ∧ + (∀ a, (a < base + 6 ∨ base + 8 ≤ a) → y.mem a = x.mem a) := by + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hpos_b : 1 ≤ 2 ^ b := Nat.one_le_two_pow + have h2b_lt : 2 * b < 2 ^ (2 * b) := Nat.lt_two_pow_self + have hfit_pow2b : Fits w (2 ^ (2 * b)) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_powb : Fits w (2 ^ b) := hfit_pow2b.of_le hpow_le + have hfit_maskb : Fits w (2 ^ b - 1) := hfit_powb.of_le (by omega) + have hfit_b6 : Fits w (base + 6) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b7 : Fits w (base + 7) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_qm1 : Fits w (q - 1) := hfit_q.of_le (by omega) + have hfit_div : Fits w ((q - 1) / b) := hfit_qm1.of_le (Nat.div_le_self _ _) + have hdiv1_le : (q - 1) / b + 1 ≤ q := by + have := Nat.div_le_self (q - 1) b + omega + have hfit_L : Fits w (limbCountFromB b q) := by + rw [← div_pred_add_one_eq_limbCountFromB hb hq] + exact hfit_q.of_le hdiv1_le + have hfit_2b : Fits w (2 * b) := hfit_pow2b.of_le (le_of_lt h2b_lt) + have hfit_0 : Fits w 0 := hfit_space.of_le (Nat.zero_le _) + have hfit_8k : Fits w (8 + k) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_tabBase : Fits w (base + 8 + k) := hfit_space.of_le (by simp [spaceFromB]; omega) + set s1 := x.set 8 (2 ^ b) + have e1 : Exec w (.op (.exactBin .shiftLeft 8 6 5) : Code .exact 18) x s1 1 := + exec_exactBin (by simp +decide [h6, h5, Nat.shiftLeft_eq]) hfit_powb + set s2 := s1.set 8 (2 ^ b - 1) + have e2 : Exec w (.op (.exactBin .sub 8 8 6) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h6, hpos_b]) hfit_maskb + set s3 := s2.set 7 (base + 6) + have e3 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h7, h6, Nat.add_assoc]) hfit_b6 + set s4 := s3.write (base + 6) (2 ^ b - 1) + have e4 : Exec w (.op (.store 7 8) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, s2] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 8) (x := s3)) + set s5 := s4.set 9 (q - 1) + have e5 : Exec w (.op (.exactBin .sub 9 2 6) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h2, h6]; omega) hfit_qm1 + set s6 := s5.set 9 ((q - 1) / b) + have e6 : Exec w (.op (.exactBin .div 9 9 5) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h5, hb.ne']) hfit_div + set s7 := s6.set 9 (limbCountFromB b q) + have e7 : Exec w (.op (.exactBin .add 9 9 6) : Code .exact 18) s6 s7 1 := + exec_exactBin + (by simp +decide [s6, s5, s4, s3, s2, s1, h6, div_pred_add_one_eq_limbCountFromB hb hq]) + hfit_L + set s8 := s7.set 7 (base + 7) + have e8 : Exec w (.op (.exactBin .add 7 7 6) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, h6, Nat.add_assoc]) hfit_b7 + set s9 := s8.write (base + 7) (limbCountFromB b q) + have e9 : Exec w (.op (.store 7 9) : Code .exact 18) s8 s9 1 := by + simpa +decide [s9, s8, s7] using + (Exec.store (B := .exact) (w := w) (a := 7) (s := 9) (x := s8)) + set s10 := s9.set 2 (2 * b) + have e10 : Exec w (.op (.exactBin .mul 2 4 5) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h4, h5]) hfit_2b + set s11 := s10.set 2 (2 ^ (2 * b)) + have e11 : Exec w (.op (.exactBin .shiftLeft 2 6 2) : Code .exact 18) s10 s11 1 := + exec_exactBin + (by simp +decide [s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6, Nat.shiftLeft_eq]) + hfit_pow2b + set s12 := s11.set 3 b + have e12 : Exec w (.op (.copy 3 5) : Code .exact 18) s11 s12 1 := by + simpa +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h5] using + (Exec.copy (B := .exact) (w := w) (d := 3) (s := 5) (x := s11)) + set s13 := s12.set 4 (2 ^ b - 1) + have e13 : Exec w (.op (.copy 4 8) : Code .exact 18) s12 s13 1 := by + simpa +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2] using + (Exec.copy (B := .exact) (w := w) (d := 4) (s := 8) (x := s12)) + set s14 := s13.set 5 1 + have e14 : Exec w (.op (.copy 5 6) : Code .exact 18) s13 s14 1 := by + simpa +decide [s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h6] using + (Exec.copy (B := .exact) (w := w) (d := 5) (s := 6) (x := s13)) + set s15 := s14.set 1 0 + have e15 : Exec w (.op (.const 1 0) : Code .exact 18) s14 s15 1 := Exec.const hfit_0 + set s16 := s15.set 9 (8 + k) + have e16 : Exec w (.op (.const 9 (8 + k)) : Code .exact 18) s15 s16 1 := Exec.const hfit_8k + set s17 := s16.set 0 (base + 8 + k) + have e17 : Exec w (.op (.exactBin .add 0 0 9) : Code .exact 18) s16 s17 1 := + exec_exactBin + (by simp +decide [s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0, + Nat.add_assoc]) + hfit_tabBase + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq (e16.seq e17))))))))))))))) + refine ⟨s17, hall, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simp +decide [s17] + · simp +decide [s17, s16, s15] + · simp +decide [s17, s16, s15, s14, s13, s12, s11] + · simp +decide [s17, s16, s15, s14, s13, s12] + · simp +decide [s17, s16, s15, s14, s13] + · simp +decide [s17, s16, s15, s14] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4] + · simp +decide [s17, s16, s15, s14, s13, s12, s11, s10, s9] + · intro a ha + dsimp only [s17, s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, + Data.write, Data.set] + rw [Function.update_of_ne (by omega), Function.update_of_ne (by omega)] + +/-- 2-instruction suffix restoring `regBase = base` from `base + 8 + k`. -/ +@[expose] def restoreBaseCode (k : ℕ) : Code .exact 18 := + .seq (.op (.const 9 (8 + k))) + (.op (.exactBin .sub 0 0 9)) + +/-- Complete deterministic table initialization program `initTableCode k`: computes +`(r, q, b, L)` and masks from `(base, n_max, U, M)` in registers `0..3` and populates +`clmulTab[0 .. 2 ^ (2 * b) - 1]`. -/ +@[expose] def initTableCode (k : ℕ) : Code .exact 18 := + .seq chunkBitsCode + (.seq trinomialParamsCode + (.seq storeHeaderPart1 + (.seq (storeHeaderPart2 k) + (.seq clmulTabLoop + (restoreBaseCode k))))) + +/-- Exact step count of `initTableCode k`. -/ +@[expose] def initTableSteps (n_max U M : ℕ) : ℕ := + 19 * 2 ^ (2 * chunkBits n_max) + + 4 * Nat.log 2 (max n_max 2) + + 5 * Nat.clog 2 (max U 2) + + 5 * trinomialStep U M + 69 + +/-- **`KWise.initTableCode k` correctness and complexity**: from `reg 0 = base`, `reg 1 = n_max`, +`reg 2 = U`, `reg 3 = M`, `initTableCode k` halts deterministically in +`initTableSteps n_max U M = O(n_max + log U + log M)` instructions, establishing +`TableRep base (chunkBits n_max) U M k y.mem`, preserving `reg 0 = base`, and modifying memory +only within `[base, base + space n_max k)`. -/ +theorem initTableCode_execWithin {w base n_max U M k : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = n_max) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (hfit_space : Fits w (base + space n_max k)) + (hfit_n : Fits w (max n_max 2)) (hfit_U : Fits w (max U 2 + 1)) + (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ∃ y : Data 18, + ExecWithin w (initTableCode k) x y (initTableSteps n_max U M) ∧ + TableRep base (chunkBits n_max) U M k y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base ∨ base + space n_max k ≤ a) → y.mem a = x.mem a) := by + set b := chunkBits n_max + set r := 3 ^ trinomialStep U M + set q := trinomialDeg U M + have hb_pos : 0 < b := chunkBits_pos n_max + have hq_pos : 0 < q := by + dsimp only [q, trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + omega + have hr_le : r ≤ q := by dsimp only [q, trinomialDeg, r]; omega + have hM_le : M ≤ q := le_trinomialDeg U M + have hsp : space n_max k = 8 + k + 2 ^ (2 * b) := rfl + have hpow_pos : 0 < 2 ^ (2 * b) := by positivity + have hfit_3 : Fits w 3 := hfit_space.of_le (by omega) + have hfit_end8 : Fits w (base + 8) := hfit_space.of_le (by omega) + have hfit_qval : Fits w q := hfit_q.of_le (le_of_lt Nat.lt_two_pow_self) + have hfit_tabEnd : Fits w (base + 8 + k + 2 ^ (2 * b)) := hfit_space.of_le (by omega) + have hfit_8k : Fits w (8 + k) := hfit_space.of_le (by omega) + have hfit_base : Fits w base := hfit_space.of_le (by omega) + obtain ⟨d1, e1, hd1_0, hd1_2, hd1_3, hd1_5, hd1_6, hd1_mem⟩ := + chunkBitsCode_exec h0 h1 h2 h3 hfit_n hfit_3 + obtain ⟨d2, e2, hd2_0, hd2_1, hd2_2, hd2_3, hd2_4, hd2_5, hd2_6, hd2_mem⟩ := + trinomialParamsCode_exec hd1_0 hd1_2 hd1_3 hd1_5 hd1_6 hfit_U hfit_q + obtain ⟨d3, e3, hd3_0, hd3_2, hd3_4, hd3_5, hd3_6, hd3_7, + hm0, hm1, hm2, hm3, hm4, hm5, hd3_out⟩ := + storeHeaderPart1_exec hd2_0 hd2_1 hd2_2 hd2_3 hd2_4 hd2_5 hd2_6 hr_le hM_le + hfit_end8 hfit_q + obtain ⟨d4, e4, hd4_0, hd4_1, hd4_2, hd4_3, hd4_4, hd4_5, hm6, hm7, hd4_out⟩ := + storeHeaderPart2_exec hb_pos hq_pos hd3_0 hd3_2 hd3_4 hd3_5 hd3_6 hd3_7 hfit_space hfit_qval + obtain ⟨d5, e5, hd5_0, htab, hd5_out⟩ := + clmulTabLoop_exec hfit_tabEnd (2 ^ (2 * b)) 0 d4 (by omega) + hd4_0 hd4_1 hd4_2 hd4_3 hd4_4 hd4_5 (by intro i hi; omega) + set d6 := d5.set 9 (8 + k) + have er1 : Exec w (.op (.const 9 (8 + k)) : Code .exact 18) d5 d6 1 := Exec.const hfit_8k + set d7 := d6.set 0 base + have er2 : Exec w (.op (.exactBin .sub 0 0 9) : Code .exact 18) d6 d7 1 := + exec_exactBin (by simp +decide [d6, hd5_0]; omega) hfit_base + have e6 := er1.seq er2 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq e6)))) + have hmem_hdr : ∀ j < 8, d7.mem (base + j) = d4.mem (base + j) := by + intro j hj + exact hd5_out (base + j) (Or.inl (by omega)) + refine ⟨d7, ⟨_, ?_, hall⟩, ?_, by simp +decide [d7], ?_⟩ + · dsimp only [initTableSteps, b] + omega + · refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · have := hmem_hdr 0 (by decide) + rw [Nat.add_zero] at this + rw [this, hd4_out base (Or.inl (by omega)), hm0] + · rw [hmem_hdr 1 (by decide), hd4_out (base + 1) (Or.inl (by omega)), hm1] + · rw [hmem_hdr 2 (by decide), hd4_out (base + 2) (Or.inl (by omega)), hm2] + · rw [hmem_hdr 3 (by decide), hd4_out (base + 3) (Or.inl (by omega)), hm3] + · rw [hmem_hdr 4 (by decide), hd4_out (base + 4) (Or.inl (by omega)), hm4] + · rw [hmem_hdr 5 (by decide), hd4_out (base + 5) (Or.inl (by omega)), hm5] + · rw [hmem_hdr 6 (by decide), hm6] + · rw [hmem_hdr 7 (by decide), hm7] + · intro idx hidx + exact htab idx hidx + · intro a ha + have ha_eq : a < base ∨ base + (8 + k + 2 ^ (2 * b)) ≤ a := by + simpa only [space, spaceFromB, b] using ha + have ha_d5 : a < base + 8 + k + 0 ∨ base + 8 + k + 2 ^ (2 * b) ≤ a := by omega + have ha_d4 : a < base + 6 ∨ base + 8 ≤ a := by omega + have ha_d3 : a < base ∨ base + 6 ≤ a := by omega + calc d7.mem a = d5.mem a := rfl + _ = d4.mem a := hd5_out a ha_d5 + _ = d3.mem a := hd4_out a ha_d4 + _ = d2.mem a := hd3_out a ha_d3 + _ = x.mem a := by rw [hd2_mem, hd1_mem] + +/-- Setup prefix for `resampleCode k`: loads `2 ^ q - 1` from `base + 3` into register `1` and +initializes `ptr = base + 8` (`reg 2`), `end = base + 8 + k` (`reg 3`), and `one = 1` (`reg 4`). -/ +@[expose] def resampleSetupCode (k : ℕ) : Code .exact 18 := + .seq (.op (.const 1 3)) + (.seq (.op (.exactBin .add 1 0 1)) + (.seq (.op (.load 1 1)) + (.seq (.op (.const 2 8)) + (.seq (.op (.exactBin .add 2 0 2)) + (.seq (.op (.const 3 k)) + (.seq (.op (.exactBin .add 3 2 3)) + (.op (.const 4 1)))))))) + +/-- Closed-form state after `resampleSetupCode k`. -/ +@[expose] def resampleSetupData (base U M k : ℕ) (x : Data 18) : Data 18 := + ((((x.set 1 (2 ^ trinomialDeg U M - 1)).set 2 (base + 8)).set 3 (base + 8 + k)).set 4 1) + +/-- Injective register map embedding `randFillLoop` into `Data 18` so `regExtra` (`5`) maps to +`regBase` (`0`). -/ +@[expose] def resampleMap : Fin 6 → Fin 18 := + ![2, 3, 1, 4, 5, 0] + +theorem injective_resampleMap : Function.Injective resampleMap := by + decide + +/-- Seed-resampling program `resampleCode k`: samples the `k` polynomial coefficients +`seed[0 .. k - 1]` uniformly in `Fin (2 ^ trinomialDeg U M)` using `k` `Code.rand` calls in +`5 * k + 10` instructions while preserving `TableRep` and `regBase = base`. -/ +@[expose] def resampleCode (k : ℕ) : Code .exact 18 := + .seq (resampleSetupCode k) (randFillLoop.rename resampleMap) + +/-- Exact `ProbComp` semantics of `resampleCode k`. -/ +@[expose] noncomputable def resampleComp (base U M k : ℕ) (x : Data 18) : ProbComp (Data 18 × ℕ) := + let d0 := resampleSetupData base U M k x + (fun z : Data 6 × ℕ => (z.1.extend resampleMap d0, 8 + z.2)) <$> + randFillComp (trinomialDeg U M) (base + 8) k (d0.comap resampleMap) + +/-- `resampleSetupCode k` executes deterministically in `8` steps. -/ +theorem resampleSetupCode_exec {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) : + Exec w (resampleSetupCode k) x (resampleSetupData base U M k x) 8 := by + have hfit_3 : Fits w 3 := hfit_end.of_le (by omega) + have hfit_b3 : Fits w (base + 3) := hfit_end.of_le (by omega) + have hfit_8 : Fits w 8 := hfit_end.of_le (by omega) + have hfit_b8 : Fits w (base + 8) := hfit_end.of_le (by omega) + have hfit_k : Fits w k := hfit_end.of_le (by omega) + have hfit_1 : Fits w 1 := hfit_end.of_le (by omega) + have hm3 : x.mem (base + 3) = 2 ^ trinomialDeg U M - 1 := htab.2.2.2.1 + set s1 := x.set 1 3 + have e1 : Exec w (.op (.const 1 3) : Code .exact 18) x s1 1 := Exec.const hfit_3 + set s2 := s1.set 1 (base + 3) + have e2 : Exec w (.op (.exactBin .add 1 0 1) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h0]) hfit_b3 + set s3 := s2.set 1 (2 ^ trinomialDeg U M - 1) + have e3 : Exec w (.op (.load 1 1) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, hm3] using + (Exec.load (B := .exact) (w := w) (d := 1) (a := 1) (x := s2)) + set s4 := s3.set 2 8 + have e4 : Exec w (.op (.const 2 8) : Code .exact 18) s3 s4 1 := Exec.const hfit_8 + set s5 := s4.set 2 (base + 8) + have e5 : Exec w (.op (.exactBin .add 2 0 2) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h0]) hfit_b8 + set s6 := s5.set 3 k + have e6 : Exec w (.op (.const 3 k) : Code .exact 18) s5 s6 1 := Exec.const hfit_k + set s7 := s6.set 3 (base + 8 + k) + have e7 : Exec w (.op (.exactBin .add 3 2 3) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, Nat.add_assoc]) hfit_end + set s8 := s7.set 4 1 + have e8 : Exec w (.op (.const 4 1) : Code .exact 18) s7 s8 1 := Exec.const hfit_1 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq e8)))))) + have hd : s8 = resampleSetupData base U M k x := by + ext i : 2 + · fin_cases i <;> rfl + · rfl + rwa [hd] at hall + +/-- **`KWise.resampleCode k` exact randomized execution**: executes with exact `ProbComp` +semantics `resampleComp base U M k x`. -/ +theorem resampleCode_execRand {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ExecRand w (resampleCode k) x (resampleComp base U M k x) := by + have hsetup := resampleSetupCode_exec htab h0 hfit_end + set d0 := resampleSetupData base U M k x + have hfit_mask : Fits w (2 ^ trinomialDeg U M - 1) := hfit_q.of_le (Nat.sub_le _ 1) + have hloop0 := randFillLoop_execRand (w := w) (bits := trinomialDeg U M) (arrBase := base + 8) + k (x := d0.comap resampleMap) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regPtr]) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regEnd]) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regMask]) + (by simp +decide [d0, resampleSetupData, resampleMap, Data.comap, regOne]) + hfit_end hfit_mask + have hloop := hloop0.rename injective_resampleMap + have hseq := hsetup.seqRand hloop + simpa only [resampleCode, resampleComp, d0, Functor.map_map] using hseq + +/-- **Uniformity of resampled seeds**: under `(resampleComp base U M k x).eval`, the extracted seed +tuple `extractSeeds base U M k z.1.mem` is uniformly distributed over +`Fin k → Fin (2 ^ trinomialDeg U M)`. -/ +theorem map_extractSeeds_eval_resampleComp (base U M k : ℕ) (x : Data 18) : + ((resampleComp base U M k x).eval).map (fun z => extractSeeds base U M k z.1.mem) = + PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M)) := by + dsimp only [resampleComp, extractSeeds] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractSlice_eval_randFillComp (trinomialDeg U M) (base + 8) k _ + +/-- **`KWise.resampleCode k` Hoare-style specification**: halts in `5 * k + 10 = O(k)` steps and `k` +`rand` queries, preserving `TableRep base b U M k`, establishing +`Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem`, preserving `reg 0 = base`, and +modifying memory only within `[base + 8, base + 8 + k)`. -/ +theorem resampleCode_execRandWithin {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ExecRandWithin w (resampleCode k) x + (fun y => + Rep base b U M k (extractSeeds base U M k y.mem) y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → y.mem a = x.mem a)) + (5 * k + 10) k := by + refine ⟨resampleComp base U M k x, resampleCode_execRand htab h0 hfit_end hfit_q, ?_, ?_⟩ + · dsimp only [resampleComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp (trinomialDeg U M) (base + 8) k _) + · intro z hz + dsimp only [resampleComp] at hz + obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hout⟩ := mem_support_randFillComp hz0 + have htab' : TableRep base b U M k z0.1.mem := by + refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · rw [hout base (Or.inl (by omega))]; exact htab.1 + · rw [hout (base + 1) (Or.inl (by omega))]; exact htab.2.1 + · rw [hout (base + 2) (Or.inl (by omega))]; exact htab.2.2.1 + · rw [hout (base + 3) (Or.inl (by omega))]; exact htab.2.2.2.1 + · rw [hout (base + 4) (Or.inl (by omega))]; exact htab.2.2.2.2.1 + · rw [hout (base + 5) (Or.inl (by omega))]; exact htab.2.2.2.2.2.1 + · rw [hout (base + 6) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.1 + · rw [hout (base + 7) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.2.1 + · intro idx hidx + rw [hout (base + 8 + k + idx) (Or.inr (by omega))] + exact htab.2.2.2.2.2.2.2.2 idx hidx + have hseed : SeedRep base U M k (extractSeeds base U M k z0.1.mem) z0.1.mem := by + intro j + simp [extractSeeds, extractSlice, hlt j.val j.isLt] + have hbase' : (z0.1.extend resampleMap (resampleSetupData base U M k x)).regs 0 = base := by + change ((z0.1.extend resampleMap (resampleSetupData base U M k x)).comap + resampleMap).regs regExtra = base + rw [Data.comap_extend injective_resampleMap, hextra] + simpa +decide [resampleSetupData, resampleMap, Data.comap, regExtra] using h0 + exact ⟨⟨⟨htab', hseed⟩, hbase', hout⟩, by omega⟩ + +/-- Full one-shot initialization program `initCode k := .seq (initTableCode k) (resampleCode k)`. -/ +@[expose] def initCode (k : ℕ) : Code .exact 18 := + .seq (initTableCode k) (resampleCode k) + +/-- **`KWise.initCode k` Hoare-style specification**: runs `initTableCode k` followed by +`resampleCode k` in `initTableSteps n_max U M + (5 * k + 10)` steps and `k` `rand` calls. -/ +theorem initCode_execRandWithin {w base n_max U M k : ℕ} {x : Data 18} + (h0 : x.regs 0 = base) (h1 : x.regs 1 = n_max) (h2 : x.regs 2 = U) (h3 : x.regs 3 = M) + (hfit_space : Fits w (base + space n_max k)) + (hfit_n : Fits w (max n_max 2)) (hfit_U : Fits w (max U 2 + 1)) + (hfit_q : Fits w (2 ^ trinomialDeg U M)) : + ExecRandWithin w (initCode k) x + (fun y => + Rep base (chunkBits n_max) U M k (extractSeeds base U M k y.mem) y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base ∨ base + space n_max k ≤ a) → y.mem a = x.mem a)) + (initTableSteps n_max U M + (5 * k + 10)) k := by + obtain ⟨dTab, ⟨s1, hs1, he1⟩, htab, hdTab_0, hdTab_out⟩ := + initTableCode_execWithin h0 h1 h2 h3 hfit_space hfit_n hfit_U hfit_q + have hsp : space n_max k = 8 + k + 2 ^ (2 * chunkBits n_max) := rfl + have hpow_pos : 0 < 2 ^ (2 * chunkBits n_max) := by positivity + have hfit_end : Fits w (base + 8 + k) := hfit_space.of_le (by omega) + obtain ⟨m2, he2, hqb, hsupp⟩ := resampleCode_execRandWithin htab hdTab_0 hfit_end hfit_q + refine ⟨(fun z : Data 18 × ℕ => (z.1, s1 + z.2)) <$> m2, he1.seqRand he2, + (OracleComp.isCostBounded_map_iff _ _ _).mpr hqb, ?_⟩ + intro z hz + obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨⟨hrep, hz0_0, hz0_out⟩, hz0_steps⟩ := hsupp z0 hz0 + refine ⟨⟨hrep, hz0_0, ?_⟩, by omega⟩ + intro a ha + have ha' : a < base + 8 ∨ base + 8 + k ≤ a := by omega + rw [hz0_out a ha', hdTab_out a ha] + +/-! ## Section 4: Deterministic Horner evaluation (`evalCode`) and `k`-wise independence -/ + +/-- Register `1`: holds the input key `u < U` on entry to `evalCode`. -/ +abbrev regKey : Fin 18 := 1 + +/-- Register `2`: holds the output hash value `< 2 ^ M` on exit from `evalCode`. -/ +abbrev regOut : Fin 18 := 2 + +private theorem and_xor_shiftRight_shiftLeft (a b : ℕ) : + (a &&& (2 ^ b - 1)) ^^^ ((a >>> b) <<< b) = a := by + apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_xor, Nat.testBit_and, Nat.testBit_two_pow_sub_one, + Nat.testBit_shiftLeft, Nat.testBit_shiftRight] + by_cases h : i < b + · have hge : ¬ b ≤ i := not_le.mpr h + simp [h, hge] + · have hge : b ≤ i := le_of_not_gt h + simp [h, hge, Nat.add_sub_of_le hge] + +private theorem clmul_split_right (x y b : ℕ) : + clmul x y = clmul x (y &&& (2 ^ b - 1)) ^^^ (clmul x (y >>> b) <<< b) := by + conv_lhs => rw [← and_xor_shiftRight_shiftLeft y b] + rw [clmul_xor_right, clmul_shiftLeft_right] + +private theorem clmul_split_left (x y b : ℕ) : + clmul x y = clmul (x &&& (2 ^ b - 1)) y ^^^ (clmul (x >>> b) y <<< b) := by + conv_lhs => rw [← and_xor_shiftRight_shiftLeft x b] + rw [clmul_xor_left, clmul_shiftLeft_left] + +private theorem shiftLeft_add_eq (a m n : ℕ) : + a <<< (m + n) = (a <<< m) <<< n := by + rw [Nat.shiftLeft_eq, Nat.shiftLeft_eq, Nat.shiftLeft_eq, pow_add, ← mul_assoc] + +private theorem xor_shiftLeft_eq (a b k : ℕ) : + (a ^^^ b) <<< k = (a <<< k) ^^^ (b <<< k) := by + apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_shiftLeft, Nat.testBit_xor, Nat.testBit_xor, + Nat.testBit_shiftLeft, Nat.testBit_shiftLeft] + by_cases h : i ≥ k <;> simp [h] + +private theorem shiftLeft_xor_parts {u v b : ℕ} (hu : u < 2 ^ b) (hv : v < 2 ^ b) : + (u <<< b) ^^^ v < 2 ^ (2 * b) ∧ + ((u <<< b) ^^^ v) >>> b = u ∧ + ((u <<< b) ^^^ v) &&& (2 ^ b - 1) = v := by + have hush : u <<< b < 2 ^ (2 * b) := by + rw [Nat.shiftLeft_eq, two_mul, pow_add] + exact Nat.mul_lt_mul_of_pos_right hu (by positivity) + have hv2 : v < 2 ^ (2 * b) := + lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) (by omega)) + refine ⟨Nat.xor_lt_two_pow hush hv2, ?_, ?_⟩ + · apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_shiftRight, Nat.testBit_xor, Nat.testBit_shiftLeft] + have hge : b + i ≥ b := Nat.le_add_right b i + have hsub : b + i - b = i := Nat.add_sub_cancel_left b i + have hv_false : v.testBit (b + i) = false := + Nat.testBit_lt_two_pow (lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) hge)) + simp [hge, hsub, hv_false] + · apply Nat.eq_of_testBit_eq + intro i + rw [Nat.testBit_and, Nat.testBit_two_pow_sub_one, Nat.testBit_xor, Nat.testBit_shiftLeft] + by_cases hi : i < b + · have hge : ¬ i ≥ b := not_le.mpr hi + simp [hi, hge] + · have hge : b ≤ i := le_of_not_gt hi + have hv_false : v.testBit i = false := + Nat.testBit_lt_two_pow (lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) hge)) + simp [hi, hv_false] + +private theorem shiftRight_mul_two_pow_add_le (a b s : ℕ) : + (a >>> b) * 2 ^ (s + b) ≤ a * 2 ^ s := by + rw [Nat.shiftRight_eq_div_pow, pow_add, ← mul_assoc, mul_comm (a / 2 ^ b) (2 ^ s), mul_assoc, + mul_comm a (2 ^ s)] + exact Nat.mul_le_mul_left (2 ^ s) (Nat.div_mul_le_self a (2 ^ b)) + +private theorem shiftRight_lt_two_pow_mul {a rem b : ℕ} (ha : a < 2 ^ ((rem + 1) * b)) : + a >>> b < 2 ^ (rem * b) := by + rw [Nat.shiftRight_eq_div_pow] + apply Nat.div_lt_of_lt_mul + rwa [← pow_add, show b + rem * b = (rem + 1) * b by ring] + +private theorem lt_two_pow_sub_of_mul_two_pow_lt {c s q : ℕ} (hc : 0 < c) + (h : c * 2 ^ s < 2 ^ q) : s ≤ q ∧ c < 2 ^ (q - s) := by + have hs : s ≤ q := by + by_contra hlt + have hle : 2 ^ q ≤ c * 2 ^ s := + calc 2 ^ q ≤ 2 ^ s := Nat.pow_le_pow_right (by decide) (by omega) + _ ≤ c * 2 ^ s := Nat.le_mul_of_pos_left _ hc + omega + refine ⟨hs, ?_⟩ + apply Nat.lt_of_mul_lt_mul_right (a := 2 ^ s) + rwa [← pow_add, Nat.sub_add_cancel hs] + +private theorem clmul_and_shiftLeft_lt {x_rem y_rem sh_x sh_y b q : ℕ} + (hx : x_rem * 2 ^ sh_x < 2 ^ q) (hy : y_rem * 2 ^ sh_y < 2 ^ q) : + clmul (x_rem &&& (2 ^ b - 1)) (y_rem &&& (2 ^ b - 1)) <<< (sh_x + sh_y) < 2 ^ (2 * q) := by + set xc := x_rem &&& (2 ^ b - 1) + set yc := y_rem &&& (2 ^ b - 1) + rcases eq_or_ne xc 0 with hxc | hxc + · simp [hxc] + · rcases eq_or_ne yc 0 with hyc | hyc + · simp [hyc] + · have hxc_lt : xc * 2 ^ sh_x < 2 ^ q := + (Nat.mul_le_mul_right (2 ^ sh_x) Nat.and_le_left).trans_lt hx + have hyc_lt : yc * 2 ^ sh_y < 2 ^ q := + (Nat.mul_le_mul_right (2 ^ sh_y) Nat.and_le_left).trans_lt hy + obtain ⟨hsx, hxc'⟩ := lt_two_pow_sub_of_mul_two_pow_lt (Nat.pos_of_ne_zero hxc) hxc_lt + obtain ⟨hsy, hyc'⟩ := lt_two_pow_sub_of_mul_two_pow_lt (Nat.pos_of_ne_zero hyc) hyc_lt + have hcl := clmul_lt_two_pow hxc' hyc' + rw [Nat.shiftLeft_eq] + calc clmul xc yc * 2 ^ (sh_x + sh_y) + < 2 ^ (q - sh_x + (q - sh_y)) * 2 ^ (sh_x + sh_y) := + Nat.mul_lt_mul_of_pos_right hcl (by positivity) + _ = 2 ^ (2 * q) := by + rw [← pow_add] + congr 1 + omega + +private theorem fits_of_lt_two_pow_two_mul {w q v : ℕ} (hw : 2 * q ≤ w) + (hv : v < 2 ^ (2 * q)) : Fits w v := + lt_of_lt_of_le hv (Nat.pow_le_pow_right (by decide) hw) + +/-- Body of `innerClmulLoop`: looks up `clmul (x_rem &&& mask_b) (y_rem &&& mask_b)` in `clmulTab`, +shifts left by `shift_j`, XORs into `acc` (`reg 2`), shifts `y_rem` right by `b`, advances +`shift_j` by `b`, and decrements `rem_j`. -/ +@[expose] def innerClmulBody : Code .exact 18 := + .seq (.op (.exactBin .and 17 15 5)) + (.seq (.op (.exactBin .xor 17 13 17)) + (.seq (.op (.exactBin .add 17 7 17)) + (.seq (.op (.load 17 17)) + (.seq (.op (.exactBin .shiftLeft 17 17 16)) + (.seq (.op (.exactBin .xor 2 2 17)) + (.seq (.op (.exactBin .shiftRight 15 15 4)) + (.seq (.op (.exactBin .add 16 16 4)) + (.op (.exactBin .sub 14 14 8))))))))) + +/-- Inner Four-Russians loop `while 0 < rem_j do innerClmulBody`. -/ +@[expose] def innerClmulLoop : Code .exact 18 := + .loop (.lt 9 14) innerClmulBody + +/-- `innerClmulLoop` executes deterministically in `11 * rem_j + 2` steps, XORing +`clmul (x_rem &&& (2 ^ b - 1)) y_rem <<< (sh_x + sh_y)` into `reg 2` (`acc`). -/ +theorem innerClmulLoop_exec {w tabBase b q x_rem sh_x : ℕ} + (hb : 0 < b) (hw : 2 * q ≤ w) + (hfit_tabEnd : Fits w (tabBase + 2 ^ (2 * b))) + (hx_bound : x_rem * 2 ^ sh_x < 2 ^ q) : + ∀ rem_j y_rem sh_y acc : ℕ, ∀ s : Data 18, + y_rem < 2 ^ (rem_j * b) → + y_rem * 2 ^ sh_y < 2 ^ q → + acc < 2 ^ (2 * q) → + Fits w (sh_x + sh_y + rem_j * b) → + s.regs 2 = acc → + s.regs 4 = b → + s.regs 5 = 2 ^ b - 1 → + s.regs 7 = tabBase → + s.regs 8 = 1 → + s.regs 9 = 0 → + s.regs 13 = (x_rem &&& (2 ^ b - 1)) <<< b → + s.regs 14 = rem_j → + s.regs 15 = y_rem → + s.regs 16 = sh_x + sh_y → + (∀ i < 2 ^ (2 * b), s.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) → + ∃ y : Data 18, + Exec w innerClmulLoop s y (11 * rem_j + 2) ∧ + y.regs 2 = acc ^^^ (clmul (x_rem &&& (2 ^ b - 1)) y_rem <<< (sh_x + sh_y)) ∧ + (∀ r : Fin 18, r ≠ 2 ∧ r ≠ 14 ∧ r ≠ 15 ∧ r ≠ 16 ∧ r ≠ 17 → y.regs r = s.regs r) ∧ + y.mem = s.mem := by + intro rem_j + induction rem_j with + | zero => + intro y_rem sh_y acc s hy_lt _ _ _ h2 _ _ _ _ h9 _ h14 _ _ _ + have hy0 : y_rem = 0 := by + rw [Nat.zero_mul, pow_zero] at hy_lt + omega + have hc : ¬(Cond.lt (9 : Fin 18) 14).Holds s := by + simp [Cond.Holds, h9, h14] + refine ⟨s, Exec.loop_exit hc, ?_, fun _ _ => rfl, rfl⟩ + simp [h2, hy0] + | succ rem_j ih => + intro y_rem sh_y acc s hy_lt hy_bound hacc_lt hfit_sh h2 h4 h5 h7 h8 h9 h13 h14 h15 h16 htab + have hc : (Cond.lt (9 : Fin 18) 14).Holds s := by + simp [Cond.Holds, h9, h14] + set xc := x_rem &&& (2 ^ b - 1) + set yc := y_rem &&& (2 ^ b - 1) + set idx := (xc <<< b) ^^^ yc + have hxc_lt : xc < 2 ^ b := by + dsimp only [xc] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hyc_lt : yc < 2 ^ b := by + dsimp only [yc] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + obtain ⟨hidx_lt, hidx_u, hidx_v⟩ := shiftLeft_xor_parts hxc_lt hyc_lt + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hyc_lt2 : yc < 2 ^ (2 * b) := lt_of_lt_of_le hyc_lt hpow_le + have hidx_lt' : idx < 2 ^ (2 * b) := hidx_lt + have hfit_yc : Fits w yc := hfit_tabEnd.of_le (by omega) + have hfit_idx : Fits w idx := hfit_tabEnd.of_le (by omega) + have hfit_addr : Fits w (tabBase + idx) := hfit_tabEnd.of_le (by omega) + have hval_sh_lt : clmul xc yc <<< (sh_x + sh_y) < 2 ^ (2 * q) := + clmul_and_shiftLeft_lt hx_bound hy_bound + have hfit_val_sh : Fits w (clmul xc yc <<< (sh_x + sh_y)) := + fits_of_lt_two_pow_two_mul hw hval_sh_lt + have hacc'_lt : acc ^^^ (clmul xc yc <<< (sh_x + sh_y)) < 2 ^ (2 * q) := + Nat.xor_lt_two_pow hacc_lt hval_sh_lt + have hfit_acc' : Fits w (acc ^^^ (clmul xc yc <<< (sh_x + sh_y))) := + fits_of_lt_two_pow_two_mul hw hacc'_lt + have hy_shr_lt : y_rem >>> b < 2 ^ (rem_j * b) := shiftRight_lt_two_pow_mul hy_lt + have hy_shr_bound : (y_rem >>> b) * 2 ^ (sh_y + b) < 2 ^ q := + (shiftRight_mul_two_pow_add_le y_rem b sh_y).trans_lt hy_bound + have hy_shr_lt_2q : y_rem >>> b < 2 ^ (2 * q) := by + have hy_lt_q : y_rem < 2 ^ q := + (Nat.le_mul_of_pos_right _ (by positivity)).trans_lt hy_bound + exact (Nat.shiftRight_le y_rem b).trans_lt + (lt_of_lt_of_le hy_lt_q (Nat.pow_le_pow_right (by decide) (by omega))) + have hfit_y_shr : Fits w (y_rem >>> b) := fits_of_lt_two_pow_two_mul hw hy_shr_lt_2q + have hsh_le : sh_x + sh_y + b ≤ sh_x + sh_y + (rem_j + 1) * b := by + rw [Nat.succ_mul] + omega + have hfit_sh_next : Fits w (sh_x + sh_y + b) := hfit_sh.of_le hsh_le + have hfit_rem : Fits w rem_j := by + have : rem_j * 1 ≤ (rem_j + 1) * b := Nat.mul_le_mul (by omega) hb + exact hfit_sh.of_le (by omega) + set s1 := s.set 17 yc + have e1 : Exec w (.op (.exactBin .and 17 15 5) : Code .exact 18) s s1 1 := + exec_exactBin (by simp [h15, h5, yc]) hfit_yc + set s2 := s1.set 17 idx + have e2 : Exec w (.op (.exactBin .xor 17 13 17) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h13, xc, idx]) hfit_idx + set s3 := s2.set 17 (tabBase + idx) + have e3 : Exec w (.op (.exactBin .add 17 7 17) : Code .exact 18) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h7]) hfit_addr + set s4 := s3.set 17 (clmul xc yc) + have hmem_idx : s3.mem (tabBase + idx) = clmul xc yc := by + change s.mem (tabBase + idx) = clmul xc yc + rw [htab idx hidx_lt, hidx_u, hidx_v] + have e4 : Exec w (.op (.load 17 17) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, hmem_idx] using + (Exec.load (B := .exact) (w := w) (d := 17) (a := 17) (x := s3)) + set s5 := s4.set 17 (clmul xc yc <<< (sh_x + sh_y)) + have e5 : Exec w (.op (.exactBin .shiftLeft 17 17 16) : Code .exact 18) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h16]) hfit_val_sh + set s6 := s5.set 2 (acc ^^^ (clmul xc yc <<< (sh_x + sh_y))) + have e6 : Exec w (.op (.exactBin .xor 2 2 17) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h2]) hfit_acc' + set s7 := s6.set 15 (y_rem >>> b) + have e7 : Exec w (.op (.exactBin .shiftRight 15 15 4) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, s4, s3, s2, s1, h15, h4]) hfit_y_shr + set s8 := s7.set 16 (sh_x + sh_y + b) + have e8 : Exec w (.op (.exactBin .add 16 16 4) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, h16, h4]) hfit_sh_next + set s9 := s8.set 14 rem_j + have e9 : Exec w (.op (.exactBin .sub 14 14 8) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h14, h8]) hfit_rem + have hbody := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq e9))))))) + have hfit_sh' : Fits w (sh_x + (sh_y + b) + rem_j * b) := by + rwa [show sh_x + (sh_y + b) + rem_j * b = sh_x + sh_y + (rem_j + 1) * b by ring] + obtain ⟨y, hrec, hy2, hy_regs, hy_mem⟩ := + ih (y_rem >>> b) (sh_y + b) (acc ^^^ (clmul xc yc <<< (sh_x + sh_y))) s9 + hy_shr_lt hy_shr_bound hacc'_lt hfit_sh' + (by simp +decide [s9, s8, s7, s6]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h4]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h5]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h7]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h8]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h9]) + (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h13, xc]) + (by simp +decide [s9]) + (by simp +decide [s9, s8, s7]) + (by simp +decide [s9, s8, Nat.add_assoc]) + htab + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, ?_, ?_, hy_mem⟩ + · simpa only [innerClmulLoop, innerClmulBody, + show 1 + (1 + (1 + (1 + (1 + (1 + (1 + (1 + 1))))))) + (11 * rem_j + 2) + 2 = + 11 * (rem_j + 1) + 2 by omega] using hiter + · rw [hy2, Nat.xor_assoc] + congr 1 + calc (clmul xc yc <<< (sh_x + sh_y)) ^^^ (clmul xc (y_rem >>> b) <<< (sh_x + (sh_y + b))) + = (clmul xc yc <<< (sh_x + sh_y)) ^^^ + ((clmul xc (y_rem >>> b) <<< b) <<< (sh_x + sh_y)) := by + rw [show sh_x + (sh_y + b) = b + (sh_x + sh_y) by omega, + shiftLeft_add_eq _ b (sh_x + sh_y)] + _ = (clmul xc yc ^^^ (clmul xc (y_rem >>> b) <<< b)) <<< (sh_x + sh_y) := by + rw [← xor_shiftLeft_eq] + _ = clmul xc y_rem <<< (sh_x + sh_y) := by + rw [← clmul_split_right] + · intro r hr + rcases hr with ⟨hr2, hr14, hr15, hr16, hr17⟩ + rw [hy_regs r ⟨hr2, hr14, hr15, hr16, hr17⟩] + simp only [s9, s8, s7, s6, s5, s4, s3, s2, s1, Data.set, + Function.update_of_ne hr2, Function.update_of_ne hr14, + Function.update_of_ne hr15, Function.update_of_ne hr16, + Function.update_of_ne hr17] + +/-- Body of `outerClmulLoop`: computes `x_chunk_sh = (x_rem &&& mask_b) <<< b`, initializes the +inner loop over `L` limbs of `u` starting at shift `shift_i`, runs `innerClmulLoop`, and advances +`x_rem`, `shift_i`, and `rem_i`. -/ +@[expose] def outerClmulBody : Code .exact 18 := + .seq (.op (.exactBin .and 13 11 5)) + (.seq (.op (.exactBin .shiftLeft 13 13 4)) + (.seq (.op (.copy 14 6)) + (.seq (.op (.copy 15 1)) + (.seq (.op (.copy 16 12)) + (.seq innerClmulLoop + (.seq (.op (.exactBin .shiftRight 11 11 4)) + (.seq (.op (.exactBin .add 12 12 4)) + (.op (.exactBin .sub 10 10 8))))))))) + +/-- Outer Four-Russians loop `while 0 < rem_i do outerClmulBody`. -/ +@[expose] def outerClmulLoop : Code .exact 18 := + .loop (.lt 9 10) outerClmulBody + +/-- `outerClmulLoop` executes deterministically in `(11 * L + 12) * rem_i + 2` steps, XORing +`clmul x_rem u <<< sh_x` into `reg 2` (`acc`). -/ +theorem outerClmulLoop_exec {w tabBase b q L u : ℕ} + (hb : 0 < b) (hw : 2 * q ≤ w) + (hfit_tabEnd : Fits w (tabBase + 2 ^ (2 * b))) + (hu_limbs : u < 2 ^ (L * b)) (hu_bound : u < 2 ^ q) : + ∀ rem_i x_rem sh_x acc : ℕ, ∀ s : Data 18, + x_rem < 2 ^ (rem_i * b) → + x_rem * 2 ^ sh_x < 2 ^ q → + acc < 2 ^ (2 * q) → + Fits w (sh_x + rem_i * b + L * b) → + s.regs 1 = u → + s.regs 2 = acc → + s.regs 4 = b → + s.regs 5 = 2 ^ b - 1 → + s.regs 6 = L → + s.regs 7 = tabBase → + s.regs 8 = 1 → + s.regs 9 = 0 → + s.regs 10 = rem_i → + s.regs 11 = x_rem → + s.regs 12 = sh_x → + (∀ i < 2 ^ (2 * b), s.mem (tabBase + i) = clmul (i >>> b) (i &&& (2 ^ b - 1))) → + ∃ y : Data 18, + Exec w outerClmulLoop s y ((11 * L + 12) * rem_i + 2) ∧ + y.regs 2 = acc ^^^ (clmul x_rem u <<< sh_x) ∧ + (∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → y.regs r = s.regs r) ∧ + y.mem = s.mem := by + intro rem_i + induction rem_i with + | zero => + intro x_rem sh_x acc s hx_lt _ _ _ _ h2 _ _ _ _ _ h9 h10 _ _ _ + have hx0 : x_rem = 0 := by + rw [Nat.zero_mul, pow_zero] at hx_lt + omega + have hc : ¬(Cond.lt (9 : Fin 18) 10).Holds s := by + simp [Cond.Holds, h9, h10] + refine ⟨s, Exec.loop_exit hc, ?_, fun _ _ => rfl, rfl⟩ + simp [h2, hx0] + | succ rem_i ih => + intro x_rem sh_x acc s hx_lt hx_bound hacc_lt hfit_sh h1 h2 h4 h5 h6 h7 h8 h9 h10 h11 h12 htab + have hc : (Cond.lt (9 : Fin 18) 10).Holds s := by + simp [Cond.Holds, h9, h10] + set xc := x_rem &&& (2 ^ b - 1) + have hxc_lt : xc < 2 ^ b := by + dsimp only [xc] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hxc_sh_lt : xc <<< b < 2 ^ (2 * b) := by + rw [Nat.shiftLeft_eq, two_mul, pow_add] + exact Nat.mul_lt_mul_of_pos_right hxc_lt (by positivity) + have hpow_le : 2 ^ b ≤ 2 ^ (2 * b) := Nat.pow_le_pow_right (by decide) (by omega) + have hxc_lt2 : xc < 2 ^ (2 * b) := lt_of_lt_of_le hxc_lt hpow_le + have hfit_xc : Fits w xc := hfit_tabEnd.of_le (by omega) + have hfit_xc_sh : Fits w (xc <<< b) := hfit_tabEnd.of_le (by omega) + set s1 := s.set 13 xc + have e1 : Exec w (.op (.exactBin .and 13 11 5) : Code .exact 18) s s1 1 := + exec_exactBin (by simp [h11, h5, xc]) hfit_xc + set s2 := s1.set 13 (xc <<< b) + have e2 : Exec w (.op (.exactBin .shiftLeft 13 13 4) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h4]) hfit_xc_sh + set s3 := s2.set 14 L + have e3 : Exec w (.op (.copy 14 6) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, h6] using + (Exec.copy (B := .exact) (w := w) (d := 14) (s := 6) (x := s2)) + set s4 := s3.set 15 u + have e4 : Exec w (.op (.copy 15 1) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, h1] using + (Exec.copy (B := .exact) (w := w) (d := 15) (s := 1) (x := s3)) + set s5 := s4.set 16 sh_x + have e5 : Exec w (.op (.copy 16 12) : Code .exact 18) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, h12] using + (Exec.copy (B := .exact) (w := w) (d := 16) (s := 12) (x := s4)) + have hfit_inner_sh : Fits w (sh_x + 0 + L * b) := + hfit_sh.of_le (by omega) + obtain ⟨s6, e6, hs6_2, hs6_regs, hs6_mem⟩ := + innerClmulLoop_exec hb hw hfit_tabEnd hx_bound L u 0 acc s5 + hu_limbs (by simpa using hu_bound) hacc_lt hfit_inner_sh + (by simp +decide [s5, s4, s3, s2, s1, h2]) + (by simp +decide [s5, s4, s3, s2, s1, h4]) + (by simp +decide [s5, s4, s3, s2, s1, h5]) + (by simp +decide [s5, s4, s3, s2, s1, h7]) + (by simp +decide [s5, s4, s3, s2, s1, h8]) + (by simp +decide [s5, s4, s3, s2, s1, h9]) + (by simp +decide [s5, s4, s3, s2, xc]) + (by simp +decide [s5, s4, s3]) + (by simp +decide [s5, s4]) + (by simp +decide [s5]) + htab + rw [Nat.add_zero] at hs6_2 + have hs6_4 : s6.regs 4 = b := by + rw [hs6_regs 4 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h4] + have hs6_8 : s6.regs 8 = 1 := by + rw [hs6_regs 8 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h8] + have hs6_10 : s6.regs 10 = rem_i + 1 := by + rw [hs6_regs 10 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h10] + have hs6_11 : s6.regs 11 = x_rem := by + rw [hs6_regs 11 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h11] + have hs6_12 : s6.regs 12 = sh_x := by + rw [hs6_regs 12 (by decide)] + simp +decide [s5, s4, s3, s2, s1, h12] + have hx_shr_lt : x_rem >>> b < 2 ^ (rem_i * b) := shiftRight_lt_two_pow_mul hx_lt + have hx_shr_bound : (x_rem >>> b) * 2 ^ (sh_x + b) < 2 ^ q := + (shiftRight_mul_two_pow_add_le x_rem b sh_x).trans_lt hx_bound + have hx_shr_lt_2q : x_rem >>> b < 2 ^ (2 * q) := by + have hx_lt_q : x_rem < 2 ^ q := + (Nat.le_mul_of_pos_right _ (by positivity)).trans_lt hx_bound + exact (Nat.shiftRight_le x_rem b).trans_lt + (lt_of_lt_of_le hx_lt_q (Nat.pow_le_pow_right (by decide) (by omega))) + have hfit_x_shr : Fits w (x_rem >>> b) := fits_of_lt_two_pow_two_mul hw hx_shr_lt_2q + have hfit_sh_next : Fits w (sh_x + b) := + hfit_sh.of_le (by rw [Nat.succ_mul]; omega) + have hfit_rem_i : Fits w rem_i := by + have : rem_i * 1 ≤ (rem_i + 1) * b := Nat.mul_le_mul (by omega) hb + exact hfit_sh.of_le (by omega) + set s7 := s6.set 11 (x_rem >>> b) + have e7 : Exec w (.op (.exactBin .shiftRight 11 11 4) : Code .exact 18) s6 s7 1 := + exec_exactBin (by simp +decide [hs6_11, hs6_4]) hfit_x_shr + set s8 := s7.set 12 (sh_x + b) + have e8 : Exec w (.op (.exactBin .add 12 12 4) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, hs6_12, hs6_4]) hfit_sh_next + set s9 := s8.set 10 rem_i + have e9 : Exec w (.op (.exactBin .sub 10 10 8) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, hs6_10, hs6_8]) hfit_rem_i + have hbody := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq e9))))))) + have hcl_xc_u_lt : clmul xc u <<< sh_x < 2 ^ (2 * q) := by + rcases eq_or_ne xc 0 with hxc0 | hxc0 + · simp [hxc0] + · have hxc_q : xc * 2 ^ sh_x < 2 ^ q := + (Nat.mul_le_mul_right (2 ^ sh_x) Nat.and_le_left).trans_lt hx_bound + obtain ⟨hsx, hxc'⟩ := lt_two_pow_sub_of_mul_two_pow_lt (Nat.pos_of_ne_zero hxc0) hxc_q + have hcl := clmul_lt_two_pow hxc' hu_bound + rw [Nat.shiftLeft_eq] + calc clmul xc u * 2 ^ sh_x + < 2 ^ (q - sh_x + q) * 2 ^ sh_x := Nat.mul_lt_mul_of_pos_right hcl (by positivity) + _ = 2 ^ (2 * q) := by + rw [← pow_add] + congr 1 + omega + have hacc'_lt : acc ^^^ (clmul xc u <<< sh_x) < 2 ^ (2 * q) := + Nat.xor_lt_two_pow hacc_lt hcl_xc_u_lt + have hfit_sh' : Fits w (sh_x + b + rem_i * b + L * b) := by + rwa [show sh_x + b + rem_i * b + L * b = sh_x + (rem_i + 1) * b + L * b by ring] + have hs9_low : ∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → s9.regs r = s.regs r := by + intro r ⟨hr_lt, hr2⟩ + have hr10 : r ≠ 10 := by omega + have hr11 : r ≠ 11 := by omega + have hr12 : r ≠ 12 := by omega + have hr13 : r ≠ 13 := by omega + have hr14 : r ≠ 14 := by omega + have hr15 : r ≠ 15 := by omega + have hr16 : r ≠ 16 := by omega + have hr17 : r ≠ 17 := by omega + calc s9.regs r = s6.regs r := by + simp only [s9, s8, s7, Data.set, + Function.update_of_ne hr10, Function.update_of_ne hr11, Function.update_of_ne hr12] + _ = s5.regs r := hs6_regs r ⟨hr2, hr14, hr15, hr16, hr17⟩ + _ = s.regs r := by + simp only [s5, s4, s3, s2, s1, Data.set, + Function.update_of_ne hr13, Function.update_of_ne hr14, + Function.update_of_ne hr15, Function.update_of_ne hr16] + have hs9_2 : s9.regs 2 = acc ^^^ (clmul xc u <<< sh_x) := by + change s6.regs 2 = acc ^^^ (clmul xc u <<< sh_x) + exact hs6_2 + obtain ⟨y, hrec, hy2, hy_regs, hy_mem⟩ := + ih (x_rem >>> b) (sh_x + b) (acc ^^^ (clmul xc u <<< sh_x)) s9 + hx_shr_lt hx_shr_bound hacc'_lt hfit_sh' + (by rw [hs9_low 1 ⟨by decide, by decide⟩, h1]) + hs9_2 + (by rw [hs9_low 4 ⟨by decide, by decide⟩, h4]) + (by rw [hs9_low 5 ⟨by decide, by decide⟩, h5]) + (by rw [hs9_low 6 ⟨by decide, by decide⟩, h6]) + (by rw [hs9_low 7 ⟨by decide, by decide⟩, h7]) + (by rw [hs9_low 8 ⟨by decide, by decide⟩, h8]) + (by rw [hs9_low 9 ⟨by decide, by decide⟩, h9]) + (by simp +decide [s9]) + (by simp +decide [s9, s8, s7]) + (by simp +decide [s9, s8]) + (by rw [show s9.mem = s.mem from hs6_mem]; exact htab) + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, ?_, ?_, by rw [hy_mem]; exact hs6_mem⟩ + · simpa only [outerClmulLoop, outerClmulBody, + show 1 + (1 + (1 + (1 + (1 + ((11 * L + 2) + (1 + (1 + 1))))))) + + ((11 * L + 12) * rem_i + 2) + 2 = (11 * L + 12) * (rem_i + 1) + 2 by ring] using hiter + · rw [hy2, Nat.xor_assoc] + congr 1 + calc (clmul xc u <<< sh_x) ^^^ (clmul (x_rem >>> b) u <<< (sh_x + b)) + = (clmul xc u <<< sh_x) ^^^ ((clmul (x_rem >>> b) u <<< b) <<< sh_x) := by + rw [show sh_x + b = b + sh_x by omega, shiftLeft_add_eq _ b sh_x] + _ = (clmul xc u ^^^ (clmul (x_rem >>> b) u <<< b)) <<< sh_x := by + rw [← xor_shiftLeft_eq] + _ = clmul x_rem u <<< sh_x := by + rw [← clmul_split_left] + · intro r hr + rw [hy_regs r hr, hs9_low r hr] + +/-- 16-instruction straight-line reduction block on `Data 18`: loads `r`, `q = 2 * r`, +`2 ^ r - 1`, and `2 ^ q - 1` from `base + 0 .. base + 3` into registers `10..13` and replaces `c` +in `reg 2` with `reduceTrinomialCode (trinomialStep U M) c`, preserving registers `0, 1, 3..9` and +memory. -/ +@[expose] def reduceStepCode : Code .exact 18 := + .seq (.op (.load 10 0)) + (.seq (.op (.exactBin .add 15 0 8)) + (.seq (.op (.load 11 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 12 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 13 15)) + (.seq (.op (.exactBin .add 14 11 10)) + (.seq (.op (.exactBin .shiftRight 15 2 14)) + (.seq (.op (.exactBin .shiftRight 16 2 11)) + (.seq (.op (.exactBin .and 16 16 12)) + (.seq (.op (.exactBin .and 2 2 13)) + (.seq (.op (.exactBin .xor 2 2 15)) + (.seq (.op (.exactBin .xor 2 2 16)) + (.seq (.op (.exactBin .shiftLeft 16 16 10)) + (.op (.exactBin .xor 2 2 16)))))))))))))))) + +/-- `reduceStepCode` executes deterministically in `16` instructions, replacing `c` in `reg 2` +with `reduceTrinomialCode (trinomialStep U M) c`. -/ +theorem reduceStepCode_exec {w base b U M k c : ℕ} {s : Data 18} + (htab : TableRep base b U M k s.mem) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_end8 : Fits w (base + 8)) + (hc : c < 2 ^ (2 * trinomialDeg U M)) + (h0 : s.regs 0 = base) (h2 : s.regs 2 = c) (h8 : s.regs 8 = 1) : + ∃ y : Data 18, + Exec w reduceStepCode s y 16 ∧ + y.regs 2 = reduceTrinomialCode (trinomialStep U M) c ∧ + (∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → y.regs r = s.regs r) ∧ + y.mem = s.mem := by + set t := trinomialStep U M + set r := 3 ^ t + set q := trinomialDeg U M + have hq_eq : q = 2 * r := rfl + have hm0 : s.mem base = r := htab.1 + have hm1 : s.mem (base + 1) = q := htab.2.1 + have hm2 : s.mem (base + 2) = 2 ^ r - 1 := htab.2.2.1 + have hm3 : s.mem (base + 3) = 2 ^ q - 1 := htab.2.2.2.1 + have hfit_b1 : Fits w (base + 1) := hfit_end8.of_le (by omega) + have hfit_b2 : Fits w (base + 2) := hfit_end8.of_le (by omega) + have hfit_b3 : Fits w (base + 3) := hfit_end8.of_le (by omega) + have h3r_lt : q + r < 2 ^ (2 * q) := by + have : 1 ≤ r := Nat.one_le_pow t 3 (by decide) + have h2q : q + r ≤ 2 * q := by omega + exact lt_of_le_of_lt h2q Nat.lt_two_pow_self + have hfit_3r : Fits w (q + r) := fits_of_lt_two_pow_two_mul hw h3r_lt + set c3 := c >>> (q + r) + set c2_raw := c >>> q + set c2 := c2_raw &&& (2 ^ r - 1) + set cLow := c &&& (2 ^ q - 1) + have hfit_c : Fits w c := fits_of_lt_two_pow_two_mul hw hc + have hfit_c3 : Fits w c3 := hfit_c.of_le (Nat.shiftRight_le _ _) + have hfit_c2_raw : Fits w c2_raw := hfit_c.of_le (Nat.shiftRight_le _ _) + have hfit_c2 : Fits w c2 := hfit_c2_raw.of_le Nat.and_le_left + have hfit_cLow : Fits w cLow := hfit_c.of_le Nat.and_le_left + have hcLow_lt : cLow < 2 ^ q := by + dsimp only [cLow] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hc3_lt : c3 < 2 ^ q := by + have hlt : c >>> (q + r) < 2 ^ r := by + rw [Nat.shiftRight_eq_div_pow] + apply Nat.div_lt_of_lt_mul + rw [← pow_add 2 (q + r) r, show q + r + r = 2 * q by omega] + exact hc + exact lt_of_lt_of_le hlt (Nat.pow_le_pow_right (by decide) (by omega)) + have hc2_r_lt : c2 < 2 ^ r := by + dsimp only [c2] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (by positivity) + have hc2_lt : c2 < 2 ^ q := + lt_of_lt_of_le hc2_r_lt (Nat.pow_le_pow_right (by decide) (by omega)) + have hc2_sh_lt : c2 <<< r < 2 ^ q := by + rw [Nat.shiftLeft_eq, hq_eq, two_mul, pow_add] + exact Nat.mul_lt_mul_of_pos_right hc2_r_lt (by positivity) + have hpow_q_le : 2 ^ q ≤ 2 ^ (2 * q) := Nat.pow_le_pow_right (by decide) (by omega) + have hfit_xor1 : Fits w (cLow ^^^ c3) := + fits_of_lt_two_pow_two_mul hw ((Nat.xor_lt_two_pow hcLow_lt hc3_lt).trans_le hpow_q_le) + have hfit_xor2 : Fits w ((cLow ^^^ c3) ^^^ c2) := + fits_of_lt_two_pow_two_mul hw + ((Nat.xor_lt_two_pow (Nat.xor_lt_two_pow hcLow_lt hc3_lt) hc2_lt).trans_le hpow_q_le) + have hfit_c2_sh : Fits w (c2 <<< r) := + fits_of_lt_two_pow_two_mul hw (hc2_sh_lt.trans_le hpow_q_le) + have hfit_xor3 : Fits w (((cLow ^^^ c3) ^^^ c2) ^^^ (c2 <<< r)) := + fits_of_lt_two_pow_two_mul hw + ((Nat.xor_lt_two_pow + (Nat.xor_lt_two_pow (Nat.xor_lt_two_pow hcLow_lt hc3_lt) hc2_lt) hc2_sh_lt).trans_le + hpow_q_le) + set s1 := s.set 10 r + have e1 : Exec w (.op (.load 10 0) : Code .exact 18) s s1 1 := by + simpa +decide [s1, h0, hm0] using (Exec.load (B := .exact) (w := w) (d := 10) (a := 0) (x := s)) + set s2 := s1.set 15 (base + 1) + have e2 : Exec w (.op (.exactBin .add 15 0 8) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h0, h8]) hfit_b1 + set s3 := s2.set 11 q + have e3 : Exec w (.op (.load 11 15) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, hm1] using + (Exec.load (B := .exact) (w := w) (d := 11) (a := 15) (x := s2)) + set s4 := s3.set 15 (base + 2) + have e4 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, h8, Nat.add_assoc]) hfit_b2 + set s5 := s4.set 12 (2 ^ r - 1) + have e5 : Exec w (.op (.load 12 15) : Code .exact 18) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, hm2] using + (Exec.load (B := .exact) (w := w) (d := 12) (a := 15) (x := s4)) + set s6 := s5.set 15 (base + 3) + have e6 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h8, Nat.add_assoc]) hfit_b3 + set s7 := s6.set 13 (2 ^ q - 1) + have e7 : Exec w (.op (.load 13 15) : Code .exact 18) s6 s7 1 := by + simpa +decide [s7, s6, s5, s4, s3, s2, s1, hm3] using + (Exec.load (B := .exact) (w := w) (d := 13) (a := 15) (x := s6)) + set s8 := s7.set 14 (q + r) + have e8 : Exec w (.op (.exactBin .add 14 11 10) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1]) hfit_3r + set s9 := s8.set 15 c3 + have e9 : Exec w (.op (.exactBin .shiftRight 15 2 14) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h2, c3]) hfit_c3 + set s10 := s9.set 16 c2_raw + have e10 : Exec w (.op (.exactBin .shiftRight 16 2 11) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, h2, c2_raw]) hfit_c2_raw + set s11 := s10.set 16 c2 + have e11 : Exec w (.op (.exactBin .and 16 16 12) : Code .exact 18) s10 s11 1 := + exec_exactBin (by simp +decide [s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, c2]) hfit_c2 + set s12 := s11.set 2 cLow + have e12 : Exec w (.op (.exactBin .and 2 2 13) : Code .exact 18) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h2, cLow]) + hfit_cLow + set s13 := s12.set 2 (cLow ^^^ c3) + have e13 : Exec w (.op (.exactBin .xor 2 2 15) : Code .exact 18) s12 s13 1 := + exec_exactBin (by simp +decide [s12, s11, s10, s9]) hfit_xor1 + set s14 := s13.set 2 ((cLow ^^^ c3) ^^^ c2) + have e14 : Exec w (.op (.exactBin .xor 2 2 16) : Code .exact 18) s13 s14 1 := + exec_exactBin (by simp +decide [s13, s12, s11]) hfit_xor2 + set s15 := s14.set 16 (c2 <<< r) + have e15 : Exec w (.op (.exactBin .shiftLeft 16 16 10) : Code .exact 18) s14 s15 1 := + exec_exactBin (by simp +decide [s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1]) + hfit_c2_sh + set s16 := s15.set 2 (((cLow ^^^ c3) ^^^ c2) ^^^ (c2 <<< r)) + have e16 : Exec w (.op (.exactBin .xor 2 2 16) : Code .exact 18) s15 s16 1 := + exec_exactBin (by simp +decide [s15, s14]) hfit_xor3 + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq e16)))))))))))))) + refine ⟨s16, hall, ?_, ?_, rfl⟩ + · simp only [s16, Data.set, Function.update_self, reduceTrinomialCode, + cLow, c3, c2, c2_raw, q, trinomialDeg, r, t, + show 2 * 3 ^ trinomialStep U M + 3 ^ trinomialStep U M = 3 * 3 ^ trinomialStep U M by ring] + · intro reg ⟨hreg_lt, hreg2⟩ + have hr10 : reg ≠ 10 := by omega + have hr11 : reg ≠ 11 := by omega + have hr12 : reg ≠ 12 := by omega + have hr13 : reg ≠ 13 := by omega + have hr14 : reg ≠ 14 := by omega + have hr15 : reg ≠ 15 := by omega + have hr16 : reg ≠ 16 := by omega + simp only [s16, s15, s14, s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, Data.set, + Function.update_of_ne hreg2, Function.update_of_ne hr10, Function.update_of_ne hr11, + Function.update_of_ne hr12, Function.update_of_ne hr13, Function.update_of_ne hr14, + Function.update_of_ne hr15, Function.update_of_ne hr16] + +/-- Body of `hornerLoop` on `Data 18`: decrements `j` (`reg 3`), multiplies `acc` (`reg 2`) by +`u` (`reg 1`) in `TrinomialField (trinomialStep U M)` via `outerClmulLoop` and `reduceStepCode`, +and adds `seed[j]` from `mem[base + 8 + j]`. -/ +@[expose] def hornerBody : Code .exact 18 := + .seq (.op (.exactBin .sub 3 3 8)) + (.seq (.op (.copy 11 2)) + (.seq (.op (.copy 2 9)) + (.seq (.op (.copy 10 6)) + (.seq (.op (.copy 12 9)) + (.seq outerClmulLoop + (.seq reduceStepCode + (.seq (.op (.const 15 8)) + (.seq (.op (.exactBin .add 15 0 15)) + (.seq (.op (.exactBin .add 15 15 3)) + (.seq (.op (.load 15 15)) + (.op (.exactBin .xor 2 2 15)))))))))))) + +/-- Horner evaluation loop `while 0 < j do hornerBody`. -/ +@[expose] def hornerLoop : Code .exact 18 := + .loop (.lt 9 3) hornerBody + +/-- `hornerLoop` executes deterministically in `((11 * L + 12) * L + 30) * m + 2` instructions, +updating `reg 2` from `(equivFin t acc).val` to +`(equivFin t (acc * x ^ m + ∑ i : Fin m, c_field (Fin.castLE hm i) * x ^ (i : ℕ))).val`. -/ +theorem hornerLoop_exec {w base b U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} (u : Fin U) + (hb : 0 < b) (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + spaceFromB b k)) : + let t := trinomialStep U M + let q := trinomialDeg U M + let L := limbCountFromB b q + let c_field : Fin k → TrinomialField t := fun j => (TrinomialField.equivFin t).symm (c j) + let x_field : TrinomialField t := AdjoinRoot.mk (trinomial t) (polyOfCode u.val) + ∀ m : ℕ, ∀ hm : m ≤ k, ∀ acc : TrinomialField t, ∀ s : Data 18, + Rep base b U M k c s.mem → + s.regs 0 = base → + s.regs 1 = u.val → + s.regs 2 = (TrinomialField.equivFin t acc).val → + s.regs 3 = m → + s.regs 4 = b → + s.regs 5 = 2 ^ b - 1 → + s.regs 6 = L → + s.regs 7 = base + 8 + k → + s.regs 8 = 1 → + s.regs 9 = 0 → + ∃ y : Data 18, + Exec w hornerLoop s y (((11 * L + 12) * L + 30) * m + 2) ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = (TrinomialField.equivFin t + (acc * x_field ^ m + ∑ i : Fin m, c_field (Fin.castLE hm i) * x_field ^ (i : ℕ))).val ∧ + y.mem = s.mem := by + intro t q L c_field x_field m + induction m with + | zero => + intro hm acc s _ h0 h1 h2 h3 _ _ _ _ _ h9 + have hc : ¬(Cond.lt (9 : Fin 18) 3).Holds s := by + simp [Cond.Holds, h9, h3] + refine ⟨s, Exec.loop_exit hc, h0, h1, ?_, rfl⟩ + simp [h2] + | succ m ih => + intro hm acc s hrep h0 h1 h2 h3 h4 h5 h6 h7 h8 h9 + have hc : (Cond.lt (9 : Fin 18) 3).Holds s := by + simp [Cond.Holds, h9, h3] + obtain ⟨htab, hseed⟩ := hrep + have hpow_pos : 0 < 2 ^ (2 * b) := by positivity + have hfit_tabEnd : Fits w (base + 8 + k + 2 ^ (2 * b)) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + have hfit_end8 : Fits w (base + 8) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + have hfit_end_k : Fits w (base + 8 + k) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + have hfit_m : Fits w m := hfit_end_k.of_le (by omega) + have hq_le_Lb : q ≤ L * b := le_limbCountFromB_mul hb q + have hpow_q_Lb : 2 ^ q ≤ 2 ^ (L * b) := Nat.pow_le_pow_right (by decide) hq_le_Lb + have hu_bound : u.val < 2 ^ q := + lt_of_lt_of_le u.isLt (le_two_pow_trinomialDeg U M) + have hu_limbs : u.val < 2 ^ (L * b) := lt_of_lt_of_le hu_bound hpow_q_Lb + set acc_val := (TrinomialField.equivFin t acc).val + have hacc_bound : acc_val < 2 ^ q := (TrinomialField.equivFin t acc).isLt + have hacc_limbs : acc_val < 2 ^ (L * b) := lt_of_lt_of_le hacc_bound hpow_q_Lb + have hLb_le : L * b ≤ q + b := by + dsimp only [L, limbCountFromB] + have := Nat.div_mul_le_self (q + b - 1) b + omega + have h2b_lt_w : 2 * b < w := by + apply (Nat.pow_lt_pow_iff_right one_lt_two).mp + exact lt_of_le_of_lt (by dsimp only [spaceFromB]; omega) hfit_space + have hq_ge_2 : 2 ≤ q := by + dsimp only [q, trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + omega + have hw_pos : 1 ≤ w := by omega + have hw_le_pow : w ≤ 2 ^ (w - 1) := by + have hlt : w - 1 < 2 ^ (w - 1) := Nat.lt_two_pow_self + omega + have h2b_lt_pow : 2 * b < 2 ^ (w - 1) := + calc 2 * b < 2 ^ (2 * b) := Nat.lt_two_pow_self + _ ≤ 2 ^ (w - 1) := Nat.pow_le_pow_right (by decide) (by omega) + have hpow_w_eq : 2 ^ (w - 1) + 2 ^ (w - 1) = 2 ^ w := by + conv_rhs => rw [← Nat.sub_add_cancel hw_pos, pow_succ] + omega + have hfit_2Lb : Fits w (0 + L * b + L * b) := by + dsimp only [Fits] + omega + set s1 := s.set 3 m + have e1 : Exec w (.op (.exactBin .sub 3 3 8) : Code .exact 18) s s1 1 := + exec_exactBin (by simp +decide [h3, h8]) hfit_m + set s2 := s1.set 11 acc_val + have e2 : Exec w (.op (.copy 11 2) : Code .exact 18) s1 s2 1 := by + simpa +decide [s2, s1, h2, acc_val] using + (Exec.copy (B := .exact) (w := w) (d := 11) (s := 2) (x := s1)) + set s3 := s2.set 2 0 + have e3 : Exec w (.op (.copy 2 9) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, h9] using + (Exec.copy (B := .exact) (w := w) (d := 2) (s := 9) (x := s2)) + set s4 := s3.set 10 L + have e4 : Exec w (.op (.copy 10 6) : Code .exact 18) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, h6] using + (Exec.copy (B := .exact) (w := w) (d := 10) (s := 6) (x := s3)) + set s5 := s4.set 12 0 + have e5 : Exec w (.op (.copy 12 9) : Code .exact 18) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, h9] using + (Exec.copy (B := .exact) (w := w) (d := 12) (s := 9) (x := s4)) + obtain ⟨s6, e6, hs6_2, hs6_low, hs6_mem⟩ := + outerClmulLoop_exec hb hw hfit_tabEnd hu_limbs hu_bound L acc_val 0 0 s5 + hacc_limbs (by simpa using hacc_bound) (by positivity) hfit_2Lb + (by simp +decide [s5, s4, s3, s2, s1, h1]) + (by simp +decide [s5, s4, s3]) + (by simp +decide [s5, s4, s3, s2, s1, h4]) + (by simp +decide [s5, s4, s3, s2, s1, h5]) + (by simp +decide [s5, s4, s3, s2, s1, h6]) + (by simp +decide [s5, s4, s3, s2, s1, h7]) + (by simp +decide [s5, s4, s3, s2, s1, h8]) + (by simp +decide [s5, s4, s3, s2, s1, h9]) + (by simp +decide [s5, s4]) + (by simp +decide [s5, s4, s3, s2]) + (by simp +decide [s5]) + htab.2.2.2.2.2.2.2.2 + simp only [Nat.shiftLeft_zero, Nat.zero_xor] at hs6_2 + have hcl_lt : clmul acc_val u.val < 2 ^ (2 * q) := + clmul_lt_two_pow_two_mul hacc_bound hu_bound + have hs6_0 : s6.regs 0 = base := by + rw [hs6_low 0 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1, h0] + have hs6_8 : s6.regs 8 = 1 := by + rw [hs6_low 8 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1, h8] + have htab_s6 : TableRep base b U M k s6.mem := by + rw [hs6_mem] + exact htab + obtain ⟨s7, e7, hs7_2, hs7_low, hs7_mem⟩ := + reduceStepCode_exec htab_s6 hw hfit_end8 hcl_lt hs6_0 hs6_2 hs6_8 + have hs7_low_eq : ∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → s7.regs r = s5.regs r := by + intro r hr + rw [hs7_low r hr, hs6_low r hr] + have hs7_0 : s7.regs 0 = base := by + rw [hs7_low_eq 0 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1, h0] + have hs7_3 : s7.regs 3 = m := by + rw [hs7_low_eq 3 ⟨by decide, by decide⟩] + simp +decide [s5, s4, s3, s2, s1] + have hfit_8 : Fits w 8 := hfit_end8.of_le (by omega) + have hfit_b8m : Fits w (base + 8 + m) := hfit_end_k.of_le (by omega) + have hm_lt : m < k := by omega + have hmem_seed : s7.mem (base + 8 + m) = (c ⟨m, hm_lt⟩).val := by + rw [hs7_mem, hs6_mem] + exact hseed ⟨m, hm_lt⟩ + have hx_val : (TrinomialField.equivFin t x_field).val = u.val := + TrinomialField.val_equivFin_mk_polyOfCode t hu_bound + have hmul_val : reduceTrinomialCode t (clmul acc_val u.val) = + (TrinomialField.equivFin t (acc * x_field)).val := by + rw [TrinomialField.val_equivFin_mul, hx_val] + have hseed_val : (c ⟨m, hm_lt⟩).val = + (TrinomialField.equivFin t (c_field ⟨m, hm_lt⟩)).val := by + have h : TrinomialField.equivFin t (c_field ⟨m, hm_lt⟩) = c ⟨m, hm_lt⟩ := + (TrinomialField.equivFin t).apply_symm_apply (c ⟨m, hm_lt⟩) + exact (congrArg Fin.val h).symm + set acc_next := acc * x_field + c_field ⟨m, hm_lt⟩ + have hxor_val : reduceTrinomialCode t (clmul acc_val u.val) ^^^ (c ⟨m, hm_lt⟩).val = + (TrinomialField.equivFin t acc_next).val := by + rw [hmul_val, hseed_val, ← TrinomialField.val_equivFin_add] + have hacc_next_lt_q : (TrinomialField.equivFin t acc_next).val < 2 ^ q := + (TrinomialField.equivFin t acc_next).isLt + have hfit_acc_next : Fits w (TrinomialField.equivFin t acc_next).val := + fits_of_lt_two_pow_two_mul hw + (lt_of_lt_of_le hacc_next_lt_q + (Nat.pow_le_pow_right (by decide) (by omega))) + set s8 := s7.set 15 8 + have e8 : Exec w (.op (.const 15 8) : Code .exact 18) s7 s8 1 := Exec.const hfit_8 + set s9 := s8.set 15 (base + 8) + have e9 : Exec w (.op (.exactBin .add 15 0 15) : Code .exact 18) s8 s9 1 := + exec_exactBin (by simp +decide [s8, hs7_0]) hfit_end8 + set s10 := s9.set 15 (base + 8 + m) + have e10 : Exec w (.op (.exactBin .add 15 15 3) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, hs7_3]) hfit_b8m + set s11 := s10.set 15 (c ⟨m, hm_lt⟩).val + have e11 : Exec w (.op (.load 15 15) : Code .exact 18) s10 s11 1 := by + simpa +decide [s11, s10, s9, s8, hmem_seed] using + (Exec.load (B := .exact) (w := w) (d := 15) (a := 15) (x := s10)) + set s12 := s11.set 2 (TrinomialField.equivFin t acc_next).val + have e12 : Exec w (.op (.exactBin .xor 2 2 15) : Code .exact 18) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, hs7_2, t, hxor_val]) hfit_acc_next + have hbody := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq e12)))))))))) + have hs12_low : ∀ r : Fin 18, r.val < 10 ∧ r ≠ 2 → s12.regs r = s5.regs r := by + intro r ⟨hr_lt, hr2⟩ + have hr15 : r ≠ 15 := by omega + calc s12.regs r = s7.regs r := by + simp only [s12, s11, s10, s9, s8, Data.set, + Function.update_of_ne hr2, Function.update_of_ne hr15] + _ = s5.regs r := hs7_low_eq r ⟨hr_lt, hr2⟩ + have hs12_mem : s12.mem = s.mem := by + change s7.mem = s.mem + rw [hs7_mem, hs6_mem] + rfl + have hm_le : m ≤ k := Nat.le_of_succ_le hm + obtain ⟨y, hrec, hy0, hy1, hy2, hy_mem⟩ := + ih hm_le acc_next s12 + (by rw [hs12_mem]; exact ⟨htab, hseed⟩) + (by rw [hs12_low 0 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h0]) + (by rw [hs12_low 1 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h1]) + (by simp only [s12, Data.set, Function.update_self]) + (by rw [hs12_low 3 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1]) + (by rw [hs12_low 4 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h4]) + (by rw [hs12_low 5 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h5]) + (by rw [hs12_low 6 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h6]) + (by rw [hs12_low 7 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h7]) + (by rw [hs12_low 8 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h8]) + (by rw [hs12_low 9 ⟨by decide, by decide⟩]; simp +decide [s5, s4, s3, s2, s1, h9]) + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, hy0, hy1, ?_, by rw [hy_mem, hs12_mem]⟩ + · simpa only [hornerLoop, hornerBody, + show 1 + (1 + (1 + (1 + (1 + (((11 * L + 12) * L + 2) + + (16 + (1 + (1 + (1 + (1 + 1)))))))))) + (((11 * L + 12) * L + 30) * m + 2) + 2 = + ((11 * L + 12) * L + 30) * (m + 1) + 2 by ring] using hiter + · rw [hy2] + congr 2 + rw [Fin.sum_univ_castSucc] + simp only [acc_next, Fin.val_castSucc, Fin.val_last, pow_succ] + have hcast : ∀ i : Fin m, Fin.castLE hm i.castSucc = Fin.castLE hm_le i := + fun _ => rfl + have hlast : Fin.castLE hm (Fin.last m) = ⟨m, hm_lt⟩ := rfl + simp only [hcast, hlast] + ring + +/-- 13-instruction setup block for `evalCode k`: initializes `reg 8 = 1`, `reg 9 = 0`, +`reg 2 = 0`, `reg 3 = k`, loads `b`, `2 ^ b - 1`, `L` from `base + 5 .. base + 7` into `reg 4..6`, +and sets `reg 7 = base + 8 + k`. -/ +@[expose] def evalSetupCode (k : ℕ) : Code .exact 18 := + .seq (.op (.const 8 1)) + (.seq (.op (.const 9 0)) + (.seq (.op (.copy 2 9)) + (.seq (.op (.const 3 k)) + (.seq (.op (.const 15 5)) + (.seq (.op (.exactBin .add 15 0 15)) + (.seq (.op (.load 4 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 5 15)) + (.seq (.op (.exactBin .add 15 15 8)) + (.seq (.op (.load 6 15)) + (.seq (.op (.const 7 (8 + k))) + (.op (.exactBin .add 7 0 7))))))))))))) + +/-- 4-instruction suffix for `evalCode k`: loads `2 ^ M - 1` from `base + 4` and masks `reg 2`. -/ +@[expose] def evalMaskCode : Code .exact 18 := + .seq (.op (.const 15 4)) + (.seq (.op (.exactBin .add 15 0 15)) + (.seq (.op (.load 15 15)) + (.op (.exactBin .and 2 2 15)))) + +/-- Deterministic evaluation program `evalCode k`: given `reg 0 = base` and `reg 1 = u.val`, +evaluates `trinomialPolyHash (trinomialStep U M) k U M c u` into `reg 2` (`regOut`) in +`evalSteps b U M k` instructions while preserving `reg 0 = base`, `reg 1 = u.val`, and +all memory. -/ +@[expose] def evalCode (k : ℕ) : Code .exact 18 := + .seq (evalSetupCode k) (.seq hornerLoop evalMaskCode) + +/-- Exact instruction count of `evalCode k` for chunk width `b`, universe `U`, output bits `M`, +and independence parameter `k`. -/ +@[expose] def evalStepsFromB (b U M k : ℕ) : ℕ := + let L := limbCountFromB b (trinomialDeg U M) + ((11 * L + 12) * L + 30) * k + 19 + +/-- Exact instruction count of `evalCode k` for capacity `n_max`, universe `U`, output bits `M`, +and independence parameter `k`. -/ +@[expose] def evalSteps (n_max U M k : ℕ) : ℕ := + evalStepsFromB (chunkBits n_max) U M k + +/-- `evalSetupCode k` executes deterministically in `13` steps. -/ +theorem evalSetupCode_exec {w base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + (hfit_end : Fits w (base + 8 + k)) : + ∃ y : Data 18, + Exec w (evalSetupCode k) x y 13 ∧ + y.regs 0 = base ∧ + y.regs 1 = x.regs 1 ∧ + y.regs 2 = 0 ∧ + y.regs 3 = k ∧ + y.regs 4 = b ∧ + y.regs 5 = 2 ^ b - 1 ∧ + y.regs 6 = limbCountFromB b (trinomialDeg U M) ∧ + y.regs 7 = base + 8 + k ∧ + y.regs 8 = 1 ∧ + y.regs 9 = 0 ∧ + y.mem = x.mem := by + have hfit_1 : Fits w 1 := hfit_end.of_le (by omega) + have hfit_0 : Fits w 0 := hfit_end.of_le (by omega) + have hfit_k : Fits w k := hfit_end.of_le (by omega) + have hfit_5 : Fits w 5 := hfit_end.of_le (by omega) + have hfit_b5 : Fits w (base + 5) := hfit_end.of_le (by omega) + have hfit_b6 : Fits w (base + 6) := hfit_end.of_le (by omega) + have hfit_b7 : Fits w (base + 7) := hfit_end.of_le (by omega) + have hfit_8k : Fits w (8 + k) := hfit_end.of_le (by omega) + have hm5 : x.mem (base + 5) = b := htab.2.2.2.2.2.1 + have hm6 : x.mem (base + 6) = 2 ^ b - 1 := htab.2.2.2.2.2.2.1 + have hm7 : x.mem (base + 7) = limbCountFromB b (trinomialDeg U M) := htab.2.2.2.2.2.2.2.1 + set s1 := x.set 8 1 + have e1 : Exec w (.op (.const 8 1) : Code .exact 18) x s1 1 := Exec.const hfit_1 + set s2 := s1.set 9 0 + have e2 : Exec w (.op (.const 9 0) : Code .exact 18) s1 s2 1 := Exec.const hfit_0 + set s3 := s2.set 2 0 + have e3 : Exec w (.op (.copy 2 9) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2] using (Exec.copy (B := .exact) (w := w) (d := 2) (s := 9) (x := s2)) + set s4 := s3.set 3 k + have e4 : Exec w (.op (.const 3 k) : Code .exact 18) s3 s4 1 := Exec.const hfit_k + set s5 := s4.set 15 5 + have e5 : Exec w (.op (.const 15 5) : Code .exact 18) s4 s5 1 := Exec.const hfit_5 + set s6 := s5.set 15 (base + 5) + have e6 : Exec w (.op (.exactBin .add 15 0 15) : Code .exact 18) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, h0]) hfit_b5 + set s7 := s6.set 4 b + have e7 : Exec w (.op (.load 4 15) : Code .exact 18) s6 s7 1 := by + simpa +decide [s7, s6, s5, s4, s3, s2, s1, hm5] using + (Exec.load (B := .exact) (w := w) (d := 4) (a := 15) (x := s6)) + set s8 := s7.set 15 (base + 6) + have e8 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, Nat.add_assoc]) hfit_b6 + set s9 := s8.set 5 (2 ^ b - 1) + have e9 : Exec w (.op (.load 5 15) : Code .exact 18) s8 s9 1 := by + simpa +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, hm6] using + (Exec.load (B := .exact) (w := w) (d := 5) (a := 15) (x := s8)) + set s10 := s9.set 15 (base + 7) + have e10 : Exec w (.op (.exactBin .add 15 15 8) : Code .exact 18) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1, Nat.add_assoc]) hfit_b7 + set s11 := s10.set 6 (limbCountFromB b (trinomialDeg U M)) + have e11 : Exec w (.op (.load 6 15) : Code .exact 18) s10 s11 1 := by + simpa +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, hm7] using + (Exec.load (B := .exact) (w := w) (d := 6) (a := 15) (x := s10)) + set s12 := s11.set 7 (8 + k) + have e12 : Exec w (.op (.const 7 (8 + k)) : Code .exact 18) s11 s12 1 := Exec.const hfit_8k + set s13 := s12.set 7 (base + 8 + k) + have e13 : Exec w (.op (.exactBin .add 7 0 7) : Code .exact 18) s12 s13 1 := + exec_exactBin + (by simp +decide [s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0, Nat.add_assoc]) + hfit_end + have hall := + e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq (e9.seq (e10.seq + (e11.seq (e12.seq e13))))))))))) + refine ⟨s13, hall, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, rfl⟩ + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1, h0] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4] + · simp +decide [s13, s12, s11, s10, s9, s8, s7] + · simp +decide [s13, s12, s11, s10, s9] + · simp +decide [s13, s12, s11] + · simp +decide [s13] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1] + · simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2] + +/-- **`KWise.evalCode k` correctness and complexity (general `b`)**: from +`Rep base b U M k c x.mem`, `x.regs 0 = base`, and `x.regs 1 = u.val` (`u : Fin U`), `evalCode k` +halts deterministically in `evalStepsFromB b U M k = O(k * L ^ 2 + 1)` instructions, writing +`(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `reg 2` (`regOut`) and preserving +`reg 0 = base`, `reg 1 = u.val`, and all memory (`y.mem = x.mem`). -/ +theorem evalCode_execWithin_fromB {w base b U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} (u : Fin U) {x : Data 18} + (hb : 0 < b) (hrep : Rep base b U M k c x.mem) + (h0 : x.regs 0 = base) (h1 : x.regs 1 = u.val) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + spaceFromB b k)) : + ∃ y : Data 18, + ExecWithin w (evalCode k) x y (evalStepsFromB b U M k) ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = (trinomialPolyHash (trinomialStep U M) k U M c u).val ∧ + y.mem = x.mem := by + have hpow_pos : 0 < 2 ^ (2 * b) := by positivity + have hfit_end : Fits w (base + 8 + k) := + hfit_space.of_le (by dsimp only [spaceFromB]; omega) + obtain ⟨d1, e1, hd1_0, hd1_1, hd1_2, hd1_3, hd1_4, hd1_5, hd1_6, hd1_7, + hd1_8, hd1_9, hd1_mem⟩ := + evalSetupCode_exec hrep.1 h0 hfit_end + rw [h1] at hd1_1 + set t := trinomialStep U M + have hzero_val : (TrinomialField.equivFin t 0).val = 0 := by + have h0_eq : (0 : TrinomialField t) = AdjoinRoot.mk (trinomial t) (polyOfCode 0) := by + simp + rw [h0_eq] + exact TrinomialField.val_equivFin_mk_polyOfCode t (by positivity) + have hd1_2' : d1.regs 2 = (TrinomialField.equivFin t 0).val := by + rw [hd1_2, hzero_val] + obtain ⟨d2, e2, hd2_0, hd2_1, hd2_2, hd2_mem⟩ := + hornerLoop_exec u hb hw hfit_space k le_rfl 0 d1 + (by rw [hd1_mem]; exact hrep) + hd1_0 hd1_1 hd1_2' hd1_3 hd1_4 hd1_5 hd1_6 hd1_7 hd1_8 hd1_9 + have hpoly_eq : (0 : TrinomialField t) * + (AdjoinRoot.mk (trinomial t) (polyOfCode u.val)) ^ k + + ∑ i : Fin k, (TrinomialField.equivFin t).symm (c (Fin.castLE le_rfl i)) * + (AdjoinRoot.mk (trinomial t) (polyOfCode u.val)) ^ (i : ℕ) = + polynomialHash (fun j => (TrinomialField.equivFin t).symm (c j)) + (AdjoinRoot.mk (trinomial t) (polyOfCode u.val)) := by + rw [zero_mul, zero_add] + rfl + have hd2_eval : d2.regs 2 = (trinomialPolyEval t k U c u).val := by + rw [hd2_2, hpoly_eq] + rfl + have hfit_4 : Fits w 4 := hfit_end.of_le (by omega) + have hfit_b4 : Fits w (base + 4) := hfit_end.of_le (by omega) + have hm4 : d2.mem (base + 4) = 2 ^ M - 1 := by + rw [hd2_mem, hd1_mem] + exact hrep.1.2.2.2.2.1 + have heval_lt : + (trinomialPolyEval t k U c u).val &&& (2 ^ M - 1) < 2 ^ (2 * trinomialDeg U M) := + lt_of_le_of_lt Nat.and_le_left + (lt_of_lt_of_le (trinomialPolyEval t k U c u).isLt + (Nat.pow_le_pow_right (by decide) (by dsimp only [trinomialDeg, t]; omega))) + have hfit_out : Fits w ((trinomialPolyEval t k U c u).val &&& (2 ^ M - 1)) := + fits_of_lt_two_pow_two_mul hw heval_lt + set s1 := d2.set 15 4 + have em1 : Exec w (.op (.const 15 4) : Code .exact 18) d2 s1 1 := Exec.const hfit_4 + set s2 := s1.set 15 (base + 4) + have em2 : Exec w (.op (.exactBin .add 15 0 15) : Code .exact 18) s1 s2 1 := + exec_exactBin (by simp +decide [s1, hd2_0]) hfit_b4 + set s3 := s2.set 15 (2 ^ M - 1) + have em3 : Exec w (.op (.load 15 15) : Code .exact 18) s2 s3 1 := by + simpa +decide [s3, s2, s1, hm4] using + (Exec.load (B := .exact) (w := w) (d := 15) (a := 15) (x := s2)) + set s4 := s3.set 2 ((trinomialPolyEval t k U c u).val &&& (2 ^ M - 1)) + have em4 : Exec w (.op (.exactBin .and 2 2 15) : Code .exact 18) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, hd2_eval]) hfit_out + have e3 := em1.seq (em2.seq (em3.seq em4)) + have hall := e1.seq (e2.seq e3) + refine ⟨s4, ⟨_, ?_, hall⟩, ?_, ?_, ?_, ?_⟩ + · dsimp only [evalStepsFromB] + omega + · simp +decide [s4, s3, s2, s1, hd2_0] + · simp +decide [s4, s3, s2, s1, hd2_1] + · simp only [s4, Data.set, Function.update_self, t] + exact (val_trinomialPolyHash (trinomialStep U M) k U M c u).symm + · change d2.mem = x.mem + rw [hd2_mem, hd1_mem] + +/-- **`KWise.evalCode k` correctness and complexity**: from +`Rep base (chunkBits n_max) U M k c x.mem`, `x.regs 0 = base`, and `x.regs 1 = u.val` +(`u : Fin U`), `evalCode k` halts deterministically in `evalSteps n_max U M k = O(k * L ^ 2 + 1)` +instructions, writing `(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `reg 2` +(`regOut`) and preserving `reg 0 = base`, `reg 1 = u.val`, and all memory (`y.mem = x.mem`). -/ +theorem evalCode_execWithin {w base n_max U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} (u : Fin U) {x : Data 18} + (hrep : Rep base (chunkBits n_max) U M k c x.mem) + (h0 : x.regs 0 = base) (h1 : x.regs 1 = u.val) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) : + ∃ y : Data 18, + ExecWithin w (evalCode k) x y (evalSteps n_max U M k) ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = (trinomialPolyHash (trinomialStep U M) k U M c u).val ∧ + y.mem = x.mem := + evalCode_execWithin_fromB u (chunkBits_pos n_max) hrep h0 h1 hw hfit_space + +/-- **End-to-end Word-RAM `k`-wise independent hash theorem (`isKWiseIndependentHash_eval`)**: +1. The hash family `trinomialPolyHash (trinomialStep U M) k U M` evaluated by `evalCode k` is + `k`-wise independent onto `Fin (2 ^ M)` (`IsKWiseIndependentHash k`). +2. From any machine state `x` satisfying `TableRep base (chunkBits n_max) U M k x.mem` and + `x.regs 0 = base`, `resampleCode k` runs in `5 * k + 10` instructions and `k` `Code.rand` + queries with distribution `resampleComp base U M k x`, whose extracted seed + `extractSeeds base U M k z.1.mem` is uniformly distributed over + `Fin k → Fin (2 ^ trinomialDeg U M)` (`PMF.uniformOfFintype`). +3. Every resampled state `y` produced by `resampleCode k` satisfies `Rep`, so for every key + `u : Fin U`, running `evalCode k` from any state `x_eval` with `x_eval.mem = y.mem`, + `x_eval.regs 0 = base`, and `x_eval.regs 1 = u.val` halts deterministically in + `evalSteps n_max U M k` instructions, preserving all memory and writing + `(trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val` to + `reg 2`. -/ +theorem isKWiseIndependentHash_eval {w base n_max U M k : ℕ} {x : Data 18} + (htab : TableRep base (chunkBits n_max) U M k x.mem) + (h0 : x.regs 0 = base) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) : + IsKWiseIndependentHash k (trinomialPolyHash (trinomialStep U M) k U M) ∧ + ExecRand w (resampleCode k) x (resampleComp base U M k x) ∧ + ((resampleComp base U M k x).eval).map (fun z => extractSeeds base U M k z.1.mem) = + PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M)) ∧ + ExecRandWithin w (resampleCode k) x + (fun y => + Rep base (chunkBits n_max) U M k (extractSeeds base U M k y.mem) y.mem ∧ + y.regs 0 = base ∧ + (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → y.mem a = x.mem a) ∧ + (∀ u : Fin U, ∀ x_eval : Data 18, + x_eval.mem = y.mem → + x_eval.regs 0 = base → + x_eval.regs 1 = u.val → + ∃ y_eval : Data 18, + ExecWithin w (evalCode k) x_eval y_eval (evalSteps n_max U M k) ∧ + y_eval.regs 0 = base ∧ + y_eval.regs 1 = u.val ∧ + y_eval.regs 2 = + (trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val ∧ + y_eval.mem = y.mem)) + (5 * k + 10) k := by + have hpow_pos : 0 < 2 ^ (2 * chunkBits n_max) := by positivity + have hsp : space n_max k = 8 + k + 2 ^ (2 * chunkBits n_max) := rfl + have hfit_end : Fits w (base + 8 + k) := hfit_space.of_le (by omega) + have hfit_q : Fits w (2 ^ trinomialDeg U M) := by + have hlt : 2 ^ trinomialDeg U M < 2 ^ (2 * trinomialDeg U M) := by + apply (Nat.pow_lt_pow_iff_right one_lt_two).mpr + have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) + dsimp only [trinomialDeg] + omega + exact fits_of_lt_two_pow_two_mul hw hlt + obtain ⟨m, he_rand, hqb, hsupp⟩ := resampleCode_execRandWithin htab h0 hfit_end hfit_q + refine ⟨isKWiseIndependentHash_trinomialPolyHash + (le_two_pow_trinomialDeg U M) (le_trinomialDeg U M), + resampleCode_execRand htab h0 hfit_end hfit_q, + map_extractSeeds_eval_resampleComp base U M k x, + ⟨m, he_rand, hqb, ?_⟩⟩ + intro z hz + obtain ⟨⟨hrep, hz0, hzout⟩, hsteps⟩ := hsupp z hz + refine ⟨⟨hrep, hz0, hzout, ?_⟩, hsteps⟩ + intro u x_eval hmem he0 he1 + have hrep_eval : Rep base (chunkBits n_max) U M k (extractSeeds base U M k z.1.mem) x_eval.mem := + hmem ▸ hrep + obtain ⟨y_eval, heval, hy0, hy1, hy2, hymem⟩ := + evalCode_execWithin u hrep_eval he0 he1 hw hfit_space + exact ⟨y_eval, heval, hy0, hy1, hy2, hymem.trans hmem⟩ + +end Cslib.Algorithms.WordRAM.Hashing.KWise diff --git a/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean new file mode 100644 index 000000000..d6aba6086 --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean @@ -0,0 +1,1565 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Computability.WordRAM.Randomized.Structured +public import Complexity.Probability.PairwiseIndependentHash.Shift +public import Complexity.Probability.Product +public import Complexity.Probability.Uniform +public import Mathlib.Data.Nat.Log + +/-! +# Self-contained Word-RAM random tables and simple tabulation hashing + +This file provides two self-contained Word-RAM hashing primitives over `Cslib.WordRAM.Basis.exact`, +built on a shared uniform word-slice sampler (`randFillLoop`): + +1. **`RandomTable`**: initializes or resamples a table `Z[0 .. 2 ^ M - 1]` of `m = 2 ^ M` + independent uniform `M`-bit words in `O(2 ^ M)` instructions and `2 ^ M` calls to `Code.rand` + (`PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`), and supports `O(1)` deterministic table + lookup `Z[idx]` as well as XOR-shifted evaluation `f(u) ^^^ Z[g(u)]` + [Carter and Wegman][CarterWegman1979]; [Mitzenmacher and Upfal][MitzenmacherUpfal2017]. +2. **`TabulatedOddStride`**: `d`-character simple tabulation with character bit-width + `b = max 1 (Nat.log 2 (max n_max 2) / 2)` (`2 ^ b ≤ max 2 n_max` words per character table), + initialized via `Code.rand` in `O(d * 2 ^ b + log n_max) = O(n_max)` steps and evaluated in + `O(d)` deterministic instructions, producing an odd stride + `(2 * simpleTab(u) + 1) &&& (2 ^ M - 1)` coprime to `2 ^ M` whose linear probe sequence + `(h₁(u) + i * h₂(u)) % 2 ^ M` is a full-cycle permutation of `Fin (2 ^ M)` + [Cormen, Leiserson, Rivest, and Stein, Section 11.4][CLRS2009]. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.randFillLoop`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.randFillComp`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.extractSlice`: generic loop filling a contiguous + memory slice with independent uniform `bits`-bit words. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.resampleCode`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.lookupCode`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.xorLookupCode`: 1-D random table + initialization/resampling and `O(1)` deterministic lookup. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.charBits`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.simpleTab`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.oddStride`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.probeSeq`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.initCode`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.evalCode`: `d`-character simple + tabulation and odd-stride permutation generator on the Word RAM. + +## Main results + +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.randFillLoop_execRand`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.map_extractSlice_eval_randFillComp`: exact execution + and product uniformity of `randFillLoop`. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.resampleCode_execRand`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.map_extractTable_eval_resampleComp`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.lookupCode_exec`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable.xorLookupCode_exec`: exact execution and + uniformity of `RandomTable`. +* `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.bijective_probeSeq`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.initCode_execRandWithin`, + `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.evalCode_execWithin`: full-cycle + permutation property, `O(d * 2 ^ b)` initialization, and `O(d)` deterministic evaluation of + `TabulatedOddStride`. + +## References + +* [J. L. Carter and M. N. Wegman, *Universal Classes of Hash Functions*][CarterWegman1979] +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, + *Introduction to Algorithms*][CLRS2009] +* [M. Mitzenmacher and E. Upfal, *Probability and Computing*][MitzenmacherUpfal2017] +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.Hashing.Tabulation + +open Cslib.WordRAM Cslib.Probability +open scoped ENNReal + +/-! ## Exact-basis instruction helpers -/ + +/-- Execute a single `.exactBin` operation when `evalExact` returns `some v` and `Fits w v`. -/ +theorem exec_exactBin {w r : ℕ} {f : BinOp} {dst i j : Fin r} {d : Data r} {v : ℕ} + (hev : f.evalExact (d.regs i) (d.regs j) = some v) (hfit : Fits w v) : + Exec w (.op (.exactBin f dst i j) : Code .exact r) d (d.set dst v) 1 := by + apply Exec.op + rw [Op.eval_exactBin, hev] + exact Data.put_of_fits hfit + +/-- 3-instruction equal-cost branch setting register `dst` to `max (x.regs a) (x.regs b)`. -/ +@[expose] def maxCode {n : ℕ} (dst a b : Fin n) : Code .exact n := + .branch (.lt a b) (.seq (.op (.copy dst b)) (.op (.copy dst dst))) (.op (.copy dst a)) + +/-- `maxCode dst a b` executes deterministically in 3 instructions and sets `dst` to +`max (x.regs a) (x.regs b)`. -/ +theorem maxCode_exec {w n : ℕ} (dst a b : Fin n) (x : Data n) : + Exec w (maxCode dst a b) x (x.set dst (max (x.regs a) (x.regs b))) 3 := by + by_cases hlt : x.regs a < x.regs b + · have hc : (Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs b := Nat.max_eq_right (le_of_lt hlt) + rw [hmax] + set d1 := x.set dst (x.regs b) + have s1 : Exec w (.op (.copy dst b) : Code .exact n) x d1 1 := Exec.copy + have hd1 : d1.set dst (d1.regs dst) = d1 := by + ext i : 2 <;> simp [d1, Data.set] + have s2 : Exec w (.op (.copy dst dst) : Code .exact n) d1 d1 1 := by + simpa only [hd1] using (Exec.copy (B := .exact) (w := w) (d := dst) (s := dst) (x := d1)) + exact Exec.branch_pos hc (s1.seq s2) + · have hc : ¬(Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs a := Nat.max_eq_left (not_lt.mp hlt) + rw [hmax] + exact Exec.branch_neg hc (Exec.copy (B := .exact)) + +/-! ## Section 1: Uniform random word-slice filler (`randFillLoop`) -/ + +/-- Register `0` of `randFillLoop`: write pointer `ptr`. -/ +abbrev regPtr : Fin 6 := 0 + +/-- Register `1` of `randFillLoop`: end pointer `arrBase + len`. -/ +abbrev regEnd : Fin 6 := 1 + +/-- Register `2` of `randFillLoop`: random bitmask `2 ^ bits - 1`. -/ +abbrev regMask : Fin 6 := 2 + +/-- Register `3` of `randFillLoop`: constant `1`. -/ +abbrev regOne : Fin 6 := 3 + +/-- Register `4` of `randFillLoop`: sampled value scratch register. -/ +abbrev regVal : Fin 6 := 4 + +/-- Register `5` of `randFillLoop`: caller-preserved register. -/ +abbrev regExtra : Fin 6 := 5 + +/-- Body of `randFillLoop`: samples `regVal ← rand regMask`, stores `*regPtr ← regVal`, and +advances `regPtr ← regPtr + regOne`. -/ +@[expose] def randFillBody : Code .exact 6 := + .seq (.rand regVal regMask) + (.seq (.op (.store regPtr regVal)) + (.op (.exactBin .add regPtr regPtr regOne))) + +/-- Structured Word-RAM loop that fills `[regPtr, regEnd)` with independent uniform draws in +`[0, regMask]`. -/ +@[expose] def randFillLoop : Code .exact 6 := + .loop (.lt regPtr regEnd) randFillBody + +/-- Single-iteration state update of `randFillBody` when `Code.rand` draws `v`. -/ +@[expose] def randFillStepData (arrBase v : ℕ) (x : Data 6) : Data 6 := + ((x.set regVal v).write arrBase v).set regPtr (arrBase + 1) + +/-- Explicit `ProbComp (Data 6 × ℕ)` induced by `randFillLoop` over `len` words starting at +`arrBase`. -/ +@[expose] def randFillComp (bits arrBase : ℕ) : ℕ → Data 6 → ProbComp (Data 6 × ℕ) + | 0, x => pure (x, 2) + | len + 1, x => do + let k ← ProbComp.unifFin (2 ^ bits - 1) + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) + pure (z.1, 3 + z.2 + 2) + +/-- Extract a `< 2 ^ bits` word as an element of `Fin (2 ^ bits)`, defaulting to `0` out of +range. -/ +@[expose] def extractWord (bits v : ℕ) : Fin (2 ^ bits) := + if h : v < 2 ^ bits then ⟨v, h⟩ else ⟨0, by positivity⟩ + +/-- Extract a contiguous slice of `len` words at `arrBase` as a tuple `Fin len → Fin (2 ^ bits)`. -/ +@[expose] def extractSlice (arrBase bits len : ℕ) (mem : ℕ → ℕ) : Fin len → Fin (2 ^ bits) := + fun i => extractWord bits (mem (arrBase + i.val)) + +@[simp] +theorem extractWord_val_of_lt {bits v : ℕ} (hv : v < 2 ^ bits) : + (extractWord bits v).val = v := by + simp [extractWord, hv] + +@[simp] +theorem extractWord_unifFin (bits : ℕ) (k : Fin (2 ^ bits - 1 + 1)) : + extractWord bits k.val = finCongr (Nat.sub_add_cancel Nat.one_le_two_pow) k := by + have hlt : k.val < 2 ^ bits := by + have hpos : 1 ≤ 2 ^ bits := Nat.one_le_two_pow + have hk := k.isLt + omega + apply Fin.ext + simp [extractWord, hlt] + +/-- One iteration of `randFillBody` executes in `3` instructions and `1` `rand` call. -/ +theorem randFillBody_execRand {w bits arrBase : ℕ} {x : Data 6} + (hptr : x.regs regPtr = arrBase) (hmask : x.regs regMask = 2 ^ bits - 1) + (hone : x.regs regOne = 1) (hfit_ptr : Fits w (arrBase + 1)) + (hfit_mask : Fits w (2 ^ bits - 1)) : + ExecRand w randFillBody x (do + let k ← ProbComp.unifFin (2 ^ bits - 1) + pure (randFillStepData arrBase k.val x, 3)) := by + have hfit_src : Fits w (x.regs regMask) := by rwa [hmask] + have h1 := ExecRand.rand (B := .exact) (w := w) (dst := regVal) (src := regMask) hfit_src + rw [hmask] at h1 + have h23 : ∀ z ∈ (do + let k ← ProbComp.unifFin (2 ^ bits - 1) + pure (x.set regVal k.val, 1) : ProbComp (Data 6 × ℕ)).support, + ExecRand w (.seq (.op (.store regPtr regVal)) + (.op (.exactBin .add regPtr regPtr regOne))) z.1 + (pure (randFillStepData arrBase (z.1.regs regVal) x, 2)) := by + intro z hz + obtain ⟨k, -, hk⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + cases OracleComp.eq_of_mem_supportWith_pure hk + set d1 := x.set regVal k.val + have hd1_ptr : d1.regs regPtr = arrBase := by simp [d1, hptr, regPtr, regVal] + have hd1_val : d1.regs regVal = k.val := by simp [d1] + have s2 : Exec w (.op (.store regPtr regVal) : Code .exact 6) + d1 (d1.write arrBase k.val) 1 := by + simpa [hd1_ptr, hd1_val] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regVal) (x := d1)) + set d2 := d1.write arrBase k.val + have s3 : Exec w (.op (.exactBin .add regPtr regPtr regOne) : Code .exact 6) + d2 (d2.set regPtr (arrBase + 1)) 1 := + exec_exactBin (by simp [d2, d1, hptr, hone, regPtr, regOne, regVal]) hfit_ptr + simpa [randFillStepData, d2, d1, hd1_val] using (s2.seq s3).execRand + have hseq := ExecRand.seq + (m₂ := fun d => pure (randFillStepData arrBase (d.regs regVal) x, 2)) h1 h23 + simpa only [randFillBody, bind_assoc, pure_bind, Data.set_regs, Function.update_self] using hseq + +/-- **Exact `ExecRand` semantics of `randFillLoop`** over `len` words starting at `arrBase`. -/ +theorem randFillLoop_execRand {w bits arrBase : ℕ} (len : ℕ) {x : Data 6} + (hptr : x.regs regPtr = arrBase) (hend : x.regs regEnd = arrBase + len) + (hmask : x.regs regMask = 2 ^ bits - 1) (hone : x.regs regOne = 1) + (hfit_end : Fits w (arrBase + len)) (hfit_mask : Fits w (2 ^ bits - 1)) : + ExecRand w randFillLoop x (randFillComp bits arrBase len x) := by + induction len generalizing arrBase x with + | zero => + have hc : ¬(Cond.lt regPtr regEnd).Holds x := by + simp [Cond.Holds, hptr, hend] + exact ExecRand.loop_exit hc + | succ len ih => + have hc : (Cond.lt regPtr regEnd).Holds x := by + simp [Cond.Holds, hptr, hend] + have hfit_ptr1 : Fits w (arrBase + 1) := hfit_end.of_le (by omega) + have hb := randFillBody_execRand hptr hmask hone hfit_ptr1 hfit_mask + have hcont : ∀ z ∈ (do + let k ← ProbComp.unifFin (2 ^ bits - 1) + pure (randFillStepData arrBase k.val x, 3) : ProbComp (Data 6 × ℕ)).support, + ExecRand w randFillLoop z.1 (randFillComp bits (arrBase + 1) len z.1) := by + intro z hz + obtain ⟨k, -, hk⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + cases OracleComp.eq_of_mem_supportWith_pure hk + refine ih (arrBase := arrBase + 1) + (by simp +decide [randFillStepData, regPtr, regVal]) + (by simp +decide [randFillStepData, regEnd, regPtr, regVal, hend]; omega) + (by simp +decide [randFillStepData, regMask, regPtr, regVal, hmask]) + (by simp +decide [randFillStepData, regOne, regPtr, regVal, hone]) + (by rwa [show arrBase + 1 + len = arrBase + (len + 1) by omega]) + have hiter := ExecRand.loop_iter hc hb hcont + simpa only [randFillLoop, randFillComp, bind_assoc, pure_bind] using hiter + +/-- `randFillComp bits arrBase len x` makes at most `len` `rand` calls. -/ +theorem isQueryBounded_randFillComp (bits arrBase len : ℕ) (x : Data 6) : + (randFillComp bits arrBase len x).IsQueryBounded len := by + induction len generalizing arrBase x with + | zero => + exact .pure _ 0 + | succ len ih => + dsimp only [randFillComp] + rw [Nat.add_comm len 1] + refine (ProbComp.isQueryBounded_unifFin (2 ^ bits - 1)).bind fun k => ?_ + simpa only [Nat.add_zero] using + (ih (arrBase + 1) (randFillStepData arrBase k.val x)).bind fun _ => .pure _ 0 + +/-- Pathwise postcondition of `randFillComp`: takes `5 * len + 2` instructions, preserves registers +`regEnd`, `regMask`, `regOne`, `regExtra`, fills `[arrBase, arrBase + len)` with `< 2 ^ bits` +words, and preserves all memory outside `[arrBase, arrBase + len)`. -/ +theorem mem_support_randFillComp {bits arrBase len : ℕ} {x : Data 6} {z : Data 6 × ℕ} + (hz : z ∈ (randFillComp bits arrBase len x).support) : + z.2 = 5 * len + 2 ∧ + z.1.regs regPtr = (if len = 0 then x.regs regPtr else arrBase + len) ∧ + z.1.regs regEnd = x.regs regEnd ∧ + z.1.regs regMask = x.regs regMask ∧ + z.1.regs regOne = x.regs regOne ∧ + z.1.regs regExtra = x.regs regExtra ∧ + (∀ i < len, z.1.mem (arrBase + i) < 2 ^ bits) ∧ + (∀ a, (a < arrBase ∨ arrBase + len ≤ a) → z.1.mem a = x.mem a) := by + induction len generalizing arrBase x z with + | zero => + dsimp only [randFillComp] at hz + cases OracleComp.eq_of_mem_supportWith_pure hz + refine ⟨rfl, by simp, rfl, rfl, rfl, rfl, fun _ hi => by omega, fun _ _ => rfl⟩ + | succ len ih => + dsimp only [randFillComp] at hz + obtain ⟨k, -, hrest⟩ := OracleComp.mem_supportWith_bind_iff.mp hz + obtain ⟨z', hz', hpure⟩ := OracleComp.mem_supportWith_bind_iff.mp hrest + cases OracleComp.eq_of_mem_supportWith_pure hpure + obtain ⟨hsteps, hptr, hend, hmask, hone, hextra, hlt, hframe⟩ := ih hz' + have hk_lt : k.val < 2 ^ bits := by + have hpos : 1 ≤ 2 ^ bits := Nat.one_le_two_pow + have hk := k.isLt + omega + refine ⟨by omega, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · simp only [Nat.succ_ne_zero, ↓reduceIte] + rcases eq_or_ne len 0 with rfl | hlen + · simp [hptr, randFillStepData] + · rw [hptr, ite_eq_right hlen] + omega + · simpa [randFillStepData, regEnd, regPtr, regVal] using hend + · simpa [randFillStepData, regMask, regPtr, regVal] using hmask + · simpa [randFillStepData, regOne, regPtr, regVal] using hone + · simpa [randFillStepData, regExtra, regPtr, regVal] using hextra + · intro i hi + rcases eq_or_ne i 0 with rfl | hi0 + · have hmem0 := hframe arrBase (Or.inl (by omega)) + simpa [randFillStepData, hmem0] using hk_lt + · have hlt' := hlt (i - 1) (by omega) + rwa [show arrBase + 1 + (i - 1) = arrBase + i by omega] at hlt' + · intro a ha + have h1 := hframe a (by omega) + have hne : a ≠ arrBase := by omega + simpa [randFillStepData, Function.update_of_ne hne] using h1 + +/-- Extracting `len + 1` words at `arrBase` after one `randFillStepData` step and the remaining +`len` steps equals `Fin.cons` of the first draw and the remaining `len`-word slice. -/ +private theorem extractSlice_succ_of_support {bits arrBase len : ℕ} {x : Data 6} + (k : Fin (2 ^ bits - 1 + 1)) {z : Data 6 × ℕ} + (hz : z ∈ (randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x)).support) : + extractSlice arrBase bits (len + 1) z.1.mem = + Fin.cons (finCongr (Nat.sub_add_cancel Nat.one_le_two_pow) k) + (extractSlice (arrBase + 1) bits len z.1.mem) := by + obtain ⟨-, -, -, -, -, -, -, hframe⟩ := mem_support_randFillComp hz + funext j + refine Fin.cases ?_ ?_ j + · have h0 := hframe arrBase (Or.inl (by omega)) + simp [extractSlice, randFillStepData, h0] + · intro i + simp only [extractSlice, Fin.cons_succ, Fin.val_succ] + congr 2 + omega + +/-- **Uniformity of `randFillComp`**: the slice `extractSlice arrBase bits len z.1.mem` produced by +`randFillComp bits arrBase len x` has the exact product uniform distribution +`PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))`. -/ +theorem map_extractSlice_eval_randFillComp (bits arrBase len : ℕ) (x : Data 6) : + ((randFillComp bits arrBase len x).eval).map (fun z => extractSlice arrBase bits len z.1.mem) = + PMF.uniformOfFintype (Fin len → Fin (2 ^ bits)) := by + induction len generalizing arrBase x with + | zero => + ext f + have heq : f = extractSlice arrBase bits 0 x.mem := Subsingleton.elim _ _ + simp [randFillComp, ProbComp.eval_pure, PMF.map, PMF.pure_apply, heq, + PMF.uniformOfFintype_apply] + | succ len ih => + let e := finCongr (Nat.sub_add_cancel (Nat.one_le_two_pow (n := bits))) + have hstep : ((randFillComp bits arrBase (len + 1) x).eval).map + (fun z => extractSlice arrBase bits (len + 1) z.1.mem) = + (PMF.uniformOfFintype (Fin (2 ^ bits))).bind (fun first => + (PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))).map (Fin.cons first)) := by + dsimp only [randFillComp] + rw [ProbComp.eval_bind, ProbComp.eval_unifFin, PMF.map_bind] + have hbody : ∀ k : Fin (2 ^ bits - 1 + 1), + ((do + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) + pure (z.1, 3 + z.2 + 2)).eval).map + (fun z : Data 6 × ℕ => extractSlice arrBase bits (len + 1) z.1.mem) = + (PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))).map (Fin.cons (e k)) := by + intro k + have hd : (do + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) + pure (z.1, 3 + z.2 + 2)) = + (fun z : Data 6 × ℕ => (z.1, 3 + z.2 + 2)) <$> + randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x) := + bind_pure_comp _ _ + rw [hd, ProbComp.eval_map, PMF.map_comp] + have hcongr : + ((randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x)).eval).map + ((fun z : Data 6 × ℕ => extractSlice arrBase bits (len + 1) z.1.mem) ∘ + fun z : Data 6 × ℕ => (z.1, 3 + z.2 + 2)) = + ((randFillComp bits (arrBase + 1) len (randFillStepData arrBase k.val x)).eval).map + (Fin.cons (e k) ∘ + fun z : Data 6 × ℕ => extractSlice (arrBase + 1) bits len z.1.mem) := by + apply PMF.map_congr_on_support + intro z hz + rw [ProbComp.support_eval] at hz + exact extractSlice_succ_of_support (z := z) k hz + rw [hcongr, ← PMF.map_comp, ih (arrBase + 1) (randFillStepData arrBase k.val x)] + have hbind : (PMF.uniformOfFintype (Fin (2 ^ bits - 1 + 1))).bind + (fun a => + PMF.map (fun z : Data 6 × ℕ => extractSlice arrBase bits (len + 1) z.1.mem) + (do + let z ← randFillComp bits (arrBase + 1) len (randFillStepData arrBase a.val x) + pure (z.1, 3 + z.2 + 2)).eval) = + (PMF.uniformOfFintype (Fin (2 ^ bits - 1 + 1))).bind + (fun a => (PMF.uniformOfFintype (Fin len → Fin (2 ^ bits))).map (Fin.cons (e a))) := by + congr 1 + funext k + exact hbody k + rw [hbind] + have heq := PMF.uniformOfFintype_map_equiv e + rw [← heq, PMF.bind_map] + rfl + rw [hstep, ← PMF.pi_uniformOfFintype, ← PMF.pi_fin_succ, PMF.pi_uniformOfFintype] + +/-! ## Section 2: `RandomTable` — 1-D random table for Cuckoo Hashing -/ + +namespace RandomTable + +/-- Footprint size of `RandomTable M`: `2 ^ M` words at `[base, base + 2 ^ M)`. -/ +@[expose] def space (M : ℕ) : ℕ := 2 ^ M + +/-- Register `0`: holds the base memory address `base` of the table. -/ +abbrev regBase : Fin 6 := 0 + +/-- Register `1`: holds `M` during `resampleCode`, or index `idx < 2 ^ M` during `lookupCode`. -/ +abbrev regM : Fin 6 := 1 + +/-- Alias for register `1` when used as the lookup index `idx < 2 ^ M`. -/ +abbrev regIdx : Fin 6 := 1 + +/-- Register `2`: receives `Z[idx]` in `lookupCode` or `fx ^^^ Z[idx]` in `xorLookupCode`. -/ +abbrev regOut : Fin 6 := 2 + +/-- Register `3`: scratch register 1. -/ +abbrev regT1 : Fin 6 := 3 + +/-- Register `4`: scratch register 2. -/ +abbrev regT2 : Fin 6 := 4 + +/-- Register `5`: scratch register 3. -/ +abbrev regT3 : Fin 6 := 5 + +/-- Memory representation invariant: `[base, base + 2 ^ M)` holds the entries of +`Z : Fin (2 ^ M) → Fin (2 ^ M)`. -/ +def Rep (base M : ℕ) (Z : Fin (2 ^ M) → Fin (2 ^ M)) (mem : ℕ → ℕ) : Prop := + ∀ i : Fin (2 ^ M), mem (base + i.val) = (Z i).val + +/-- Extract the random table `Z : Fin (2 ^ M) → Fin (2 ^ M)` from memory at `base`. -/ +@[expose] def extractTable (base M : ℕ) (mem : ℕ → ℕ) : Fin (2 ^ M) → Fin (2 ^ M) := + extractSlice base M (2 ^ M) mem + +/-- If every word in `[base, base + 2 ^ M)` is `< 2 ^ M`, then +`Rep base M (extractTable base M mem) mem` holds. -/ +theorem rep_extractTable {base M : ℕ} {mem : ℕ → ℕ} + (h : ∀ i < 2 ^ M, mem (base + i) < 2 ^ M) : + Rep base M (extractTable base M mem) mem := by + intro i + simp [extractTable, extractSlice, h i.val i.isLt] + +/-- Injective register permutation mapping `randFillLoop`'s registers into `RandomTable` so that +`regExtra` maps to `regBase` (preserving `regBase`). -/ +@[expose] def fillMap : Fin 6 → Fin 6 + | 0 => regT1 + | 1 => regT2 + | 2 => regT3 + | 3 => regOut + | 4 => regM + | 5 => regBase + +theorem injective_fillMap : Function.Injective fillMap := by + decide + +/-- Deterministic setup before `randFillLoop` in `resampleCode`: computes `2 ^ M`, `2 ^ M - 1`, +`base + 2 ^ M`, and initializes the loop registers. -/ +@[expose] def setupCode : Code .exact 6 := + .seq (.op (.const regOut 1)) + (.seq (.op (.exactBin .shiftLeft regT2 regOut regM)) + (.seq (.op (.exactBin .sub regT3 regT2 regOut)) + (.seq (.op (.copy regT1 regBase)) + (.op (.exactBin .add regT2 regBase regT2))))) + +/-- Closed-form machine data after `setupCode`. -/ +@[expose] def setupData (base M : ℕ) (x : Data 6) : Data 6 := + ((((x.set regOut 1).set regT2 (2 ^ M)).set regT3 (2 ^ M - 1)).set regT1 base).set + regT2 (base + 2 ^ M) + +/-- Structured Word-RAM program that initializes or resamples `Z[0 .. 2 ^ M - 1]` at `base` using +`2 ^ M` calls to `Code.rand` on `2 ^ M - 1`. -/ +@[expose] def resampleCode : Code .exact 6 := + .seq setupCode (randFillLoop.rename fillMap) + +/-- Alias for `resampleCode` when used for initial table construction. -/ +abbrev initCode : Code .exact 6 := resampleCode + +/-- Explicit `ProbComp (Data 6 × ℕ)` induced by `resampleCode` at `(base, M)` from `x`. -/ +@[expose] noncomputable def resampleComp (base M : ℕ) (x : Data 6) : ProbComp (Data 6 × ℕ) := + let s := setupData base M x + (fun z => (z.1.extend fillMap s, 5 + z.2)) <$> + randFillComp M base (2 ^ M) (s.comap fillMap) + +/-- `setupCode` executes deterministically in `5` instructions when `Fits w (base + 2 ^ M)` and +`Fits w (2 ^ M)`. -/ +theorem setupCode_exec {w base M : ℕ} {x : Data 6} + (hbase : x.regs regBase = base) (hM : x.regs regM = M) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + Exec w setupCode x (setupData base M x) 5 := by + have hpos : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hfit_1 : Fits w 1 := hfit_pow.of_le hpos + have hfit_mask : Fits w (2 ^ M - 1) := hfit_pow.of_le (Nat.sub_le _ _) + set d1 := x.set regOut 1 + have s1 : Exec w (.op (.const regOut 1) : Code .exact 6) x d1 1 := Exec.const hfit_1 + set d2 := d1.set regT2 (2 ^ M) + have s2 : Exec w (.op (.exactBin .shiftLeft regT2 regOut regM) : Code .exact 6) d1 d2 1 := + exec_exactBin (by simp [d1, hM, regOut, regM, Nat.shiftLeft_eq]) hfit_pow + set d3 := d2.set regT3 (2 ^ M - 1) + have s3 : Exec w (.op (.exactBin .sub regT3 regT2 regOut) : Code .exact 6) d2 d3 1 := + exec_exactBin (by simp [d2, d1, hpos, regT2, regOut]) hfit_mask + set d4 := d3.set regT1 base + have s4 : Exec w (.op (.copy regT1 regBase) : Code .exact 6) d3 d4 1 := by + simpa [d3, d2, d1, hbase, regBase, regT1, regT2, regT3, regOut] using + (Exec.copy (B := .exact) (w := w) (d := regT1) (s := regBase) (x := d3)) + set d5 := d4.set regT2 (base + 2 ^ M) + have s5 : Exec w (.op (.exactBin .add regT2 regBase regT2) : Code .exact 6) d4 d5 1 := + exec_exactBin (by simp [d4, d3, d2, d1, hbase, regBase, regT1, regT2, regT3, regOut]) hfit_end + exact s1.seq (s2.seq (s3.seq (s4.seq s5))) + +/-- **Exact `ExecRand` execution of `RandomTable.resampleCode`**: runs in `5 * 2 ^ M + 7` +instructions and `2 ^ M` `rand` calls. -/ +theorem resampleCode_execRand {w base M : ℕ} {x : Data 6} + (hbase : x.regs regBase = base) (hM : x.regs regM = M) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + ExecRand w resampleCode x (resampleComp base M x) := by + have h1 := setupCode_exec hbase hM hfit_end hfit_pow + set s := setupData base M x + have hfit_mask : Fits w (2 ^ M - 1) := hfit_pow.of_le (Nat.sub_le _ _) + have hloop := randFillLoop_execRand (w := w) (bits := M) (arrBase := base) (len := 2 ^ M) + (x := s.comap fillMap) + (by simp +decide [s, setupData, fillMap, Data.comap]) + (by simp +decide [s, setupData, fillMap, Data.comap]) + (by simp +decide [s, setupData, fillMap, Data.comap]) + (by simp +decide [s, setupData, fillMap, Data.comap]) + hfit_end hfit_mask + have hren := hloop.rename injective_fillMap + have hseq := h1.seqRand hren + rw [resampleCode] + simpa only [resampleComp, s, Functor.map_map] using hseq + +/-- **Uniformity of `RandomTable.resampleComp`**: the extracted table `extractTable base M z.1.mem` +is distributed according to `PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`. -/ +theorem map_extractTable_eval_resampleComp (base M : ℕ) (x : Data 6) : + ((resampleComp base M x).eval).map (fun z => extractTable base M z.1.mem) = + PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M)) := by + dsimp only [resampleComp, extractTable] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractSlice_eval_randFillComp M base (2 ^ M) ((setupData base M x).comap fillMap) + +/-- Hoare-style `ExecRandWithin` specification for `RandomTable.resampleCode`: runs in +`5 * 2 ^ M + 7` steps and `2 ^ M` `rand` calls, establishing `Rep`, preserving `regBase`, and +modifying only `[base, base + 2 ^ M)`. -/ +theorem resampleCode_execRandWithin {w base M : ℕ} {x : Data 6} + (hbase : x.regs regBase = base) (hM : x.regs regM = M) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + ExecRandWithin w resampleCode x + (fun y => Rep base M (extractTable base M y.mem) y.mem ∧ + y.regs regBase = base ∧ + (∀ a, (a < base ∨ base + 2 ^ M ≤ a) → y.mem a = x.mem a)) + (5 * 2 ^ M + 7) (2 ^ M) := by + refine ⟨resampleComp base M x, resampleCode_execRand hbase hM hfit_end hfit_pow, ?_, ?_⟩ + · dsimp only [resampleComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp M base (2 ^ M) _) + · intro z hz + dsimp only [resampleComp] at hz + obtain ⟨z₀, hz₀, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hframe⟩ := mem_support_randFillComp hz₀ + have hbase' : (z₀.1.extend fillMap (setupData base M x)).regs regBase = base := by + change ((z₀.1.extend fillMap (setupData base M x)).comap fillMap).regs regExtra = base + rw [Data.comap_extend injective_fillMap, hextra] + simpa +decide [setupData, fillMap, Data.comap] using hbase + exact ⟨⟨rep_extractTable hlt, hbase', hframe⟩, by omega⟩ + +/-- Structured Word-RAM program that reads `Z[regIdx]` at `regBase + regIdx` into `regOut` in `2` +deterministic instructions. -/ +@[expose] def lookupCode : Code .exact 6 := + .seq (.op (.exactBin .add regT1 regBase regIdx)) + (.op (.load regOut regT1)) + +/-- Final machine data after `lookupCode`. -/ +@[expose] def lookupData (x : Data 6) : Data 6 := + let addr := x.regs regBase + x.regs regIdx + (x.set regT1 addr).set regOut (x.mem addr) + +/-- `lookupCode` executes in `2` deterministic instructions, preserving memory and `regBase` and +writing `(Z idx).val` into `regOut`. -/ +theorem lookupCode_exec {w base M : ℕ} {Z : Fin (2 ^ M) → Fin (2 ^ M)} {x : Data 6} + (hrep : Rep base M Z x.mem) (idx : Fin (2 ^ M)) + (hbase : x.regs regBase = base) (hidx : x.regs regIdx = idx.val) + (hfit_end : Fits w (base + 2 ^ M)) : + Exec w lookupCode x (lookupData x) 2 ∧ + (lookupData x).regs regOut = (Z idx).val ∧ + (lookupData x).regs regBase = base ∧ + (lookupData x).mem = x.mem := by + have hfit_addr : Fits w (base + idx.val) := hfit_end.of_le (by have := idx.isLt; omega) + set d1 := x.set regT1 (base + idx.val) + have s1 : Exec w (.op (.exactBin .add regT1 regBase regIdx) : Code .exact 6) x d1 1 := + exec_exactBin (by simp [hbase, hidx]) hfit_addr + have s2 : Exec w (.op (.load regOut regT1) : Code .exact 6) d1 + (d1.set regOut (x.mem (base + idx.val))) 1 := by + simpa [d1] using (Exec.load (B := .exact) (w := w) (d := regOut) (a := regT1) (x := d1)) + have hd : d1.set regOut (x.mem (base + idx.val)) = lookupData x := by + simp [d1, lookupData, hbase, hidx] + rw [hd] at s2 + refine ⟨s1.seq s2, ?_, by simp [lookupData, hbase, regBase, regT1, regOut], rfl⟩ + simp [lookupData, hbase, hidx, hrep idx] + +/-- Structured Word-RAM program that computes `regOut ← regOut ^^^ Z[regIdx]` in `3` deterministic +instructions (computing `f(u) ^^^ Z[g(u)]` when `regOut` holds `f(u)` and `regIdx` holds +`g(u)`). -/ +@[expose] def xorLookupCode : Code .exact 6 := + .seq (.op (.exactBin .add regT1 regBase regIdx)) + (.seq (.op (.load regT2 regT1)) + (.op (.exactBin .xor regOut regOut regT2))) + +/-- Final machine data after `xorLookupCode`. -/ +@[expose] def xorLookupData (x : Data 6) : Data 6 := + let addr := x.regs regBase + x.regs regIdx + let zval := x.mem addr + ((x.set regT1 addr).set regT2 zval).set regOut (x.regs regOut ^^^ zval) + +/-- `xorLookupCode` executes in `3` deterministic instructions, preserving memory and `regBase` and +writing `fx.val ^^^ (Z idx).val < 2 ^ M` into `regOut`. -/ +theorem xorLookupCode_exec {w base M : ℕ} {Z : Fin (2 ^ M) → Fin (2 ^ M)} {x : Data 6} + (hrep : Rep base M Z x.mem) (idx fx : Fin (2 ^ M)) + (hbase : x.regs regBase = base) (hidx : x.regs regIdx = idx.val) + (hfx : x.regs regOut = fx.val) + (hfit_end : Fits w (base + 2 ^ M)) (hfit_pow : Fits w (2 ^ M)) : + Exec w xorLookupCode x (xorLookupData x) 3 ∧ + (xorLookupData x).regs regOut = fx.val ^^^ (Z idx).val ∧ + (xorLookupData x).regs regOut < 2 ^ M ∧ + (xorLookupData x).regs regBase = base ∧ + (xorLookupData x).mem = x.mem := by + have hfit_addr : Fits w (base + idx.val) := hfit_end.of_le (by have := idx.isLt; omega) + have hxor_lt : fx.val ^^^ (Z idx).val < 2 ^ M := Nat.xor_lt_two_pow fx.isLt (Z idx).isLt + have hfit_xor : Fits w (fx.val ^^^ (Z idx).val) := hfit_pow.of_le (le_of_lt hxor_lt) + set d1 := x.set regT1 (base + idx.val) + have s1 : Exec w (.op (.exactBin .add regT1 regBase regIdx) : Code .exact 6) x d1 1 := + exec_exactBin (by simp [hbase, hidx]) hfit_addr + set d2 := d1.set regT2 (Z idx).val + have s2 : Exec w (.op (.load regT2 regT1) : Code .exact 6) d1 d2 1 := by + simpa [d1, d2, hrep idx] using + (Exec.load (B := .exact) (w := w) (d := regT2) (a := regT1) (x := d1)) + set d3 := d2.set regOut (fx.val ^^^ (Z idx).val) + have s3 : Exec w (.op (.exactBin .xor regOut regOut regT2) : Code .exact 6) d2 d3 1 := + exec_exactBin (by simp [d2, d1, hfx, regOut, regT1, regT2]) hfit_xor + have hd3 : d3 = xorLookupData x := by + simp [d3, d2, d1, xorLookupData, hbase, hidx, hfx, hrep idx] + rw [hd3] at s3 + refine ⟨s1.seq (s2.seq s3), ?_, ?_, + by simp [xorLookupData, hbase, regBase, regT1, regT2, regOut], rfl⟩ + · simp [xorLookupData, hbase, hidx, hfx, hrep idx] + · simpa [xorLookupData, hbase, hidx, hfx, hrep idx] using hxor_lt + +end RandomTable + +/-! ## Section 3: Shared `Nat.log 2` loop and `TabulatedOddStride` -/ + +/-- One iteration of the floor-`log₂` loop on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, +`reg 2 = 1`): shifts `rem` right by `1` (`rem / 2`) and increments `acc`. -/ +@[expose] def log2Body : Code .exact 3 := + .seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2)) + +/-- Floor-`log₂` loop `while 1 < rem do [ rem ← rem >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def log2Loop : Code .exact 3 := + .loop (.lt 2 0) log2Body + +/-- Closed-form final state of `log2Loop` starting from `rem` in register `0` and `acc` in +register `1`. -/ +@[expose] def log2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.log 2 rem) + +/-- `log2Loop` executes deterministically in `4 * Nat.log 2 rem + 2` instructions, accumulating +`acc + Nat.log 2 rem` into register `1` and preserving all memory. -/ +theorem log2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem : Fits w rem) (hfit_acc : Fits w (acc + Nat.log 2 rem)) : + Exec w log2Loop x (log2LoopData rem acc x) (4 * Nat.log 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : rem >>> 1 = rem / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hlog : Nat.log 2 rem = Nat.log 2 (rem / 2) + 1 := + Nat.log_of_one_lt_of_le (by decide) (by omega) + have hfit_div : Fits w (rem / 2) := hfit_rem.of_le (Nat.div_le_self rem 2) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem / 2) + have s1 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone, hshr]) hfit_div + set d2 := d1.set 1 (acc + 1) + have s2 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hacc, hone]) hfit_acc1 + have hdiv_lt : rem / 2 < rem := Nat.div_lt_self (by omega) (by decide) + have hrec := ih (rem / 2) hdiv_lt (acc := acc + 1) (x := d2) + (by simp +decide [d2, d1]) + (by simp +decide [d2]) + (by simp +decide [d2, d1, hone]) + hfit_div + (by rwa [show acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem by omega]) + have hd : log2LoopData (rem / 2) (acc + 1) d2 = log2LoopData rem acc x := by + have hdiv_ne : rem / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem + omega + · simp +decide [log2LoopData, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq s2) hrec + simpa only [log2Loop, log2Body, hlog, show 1 + 1 + (4 * Nat.log 2 (rem / 2) + 2) + 2 = + 4 * (Nat.log 2 (rem / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hlog0 : Nat.log 2 rem = 0 := Nat.log_of_lt (by omega) + have hd : log2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by + split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, hrem_eq, hrem] + · simp +decide [log2LoopData, hlog0, hacc] + · simp +decide [log2LoopData] + · rfl + rw [hd, hlog0] + exact Exec.loop_exit hc + +namespace TabulatedOddStride + +/-- Character bit-width `b = max 1 (⌊log₂ (max n_max 2)⌋ / 2)` for `d`-character simple tabulation, +ensuring `1 ≤ b` and `2 ^ b ≤ max 2 n_max` (`2 ^ (2 * b) ≤ max 4 n_max`). -/ +@[expose] def charBits (n_max : ℕ) : ℕ := + max 1 (Nat.log 2 (max n_max 2) / 2) + +/-- `charBits n_max` is always at least `1`. -/ +theorem one_le_charBits (n_max : ℕ) : 1 ≤ charBits n_max := + Nat.le_max_left 1 _ + +/-- `charBits n_max` is always positive. -/ +theorem charBits_pos (n_max : ℕ) : 0 < charBits n_max := + one_le_charBits n_max + +/-- The character alphabet size `2 ^ charBits n_max` is at most `max 2 n_max`. -/ +theorem two_pow_charBits_le (n_max : ℕ) : 2 ^ charBits n_max ≤ max 2 n_max := by + have hlog_ge : 1 ≤ Nat.log 2 (max n_max 2) := + Nat.le_log_of_pow_le (by decide) (by omega) + have hle_log : charBits n_max ≤ Nat.log 2 (max n_max 2) := by + simp only [charBits] + omega + calc + 2 ^ charBits n_max ≤ 2 ^ Nat.log 2 (max n_max 2) := + Nat.pow_le_pow_right (by decide) hle_log + _ ≤ max n_max 2 := Nat.pow_log_le_self 2 (by omega) + _ = max 2 n_max := Nat.max_comm n_max 2 + +/-- Odd stride `(2 * tab + 1) &&& (2 ^ M - 1) = (2 * tab + 1) % 2 ^ M` in `Fin (2 ^ M)`. -/ +@[expose] def oddStride (M tab : ℕ) : Fin (2 ^ M) := + ⟨(2 * tab + 1) &&& (2 ^ M - 1), by + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt _ (Nat.two_pow_pos M)⟩ + +@[simp] +theorem val_oddStride (M tab : ℕ) : (oddStride M tab).val = (2 * tab + 1) % 2 ^ M := + Nat.and_two_pow_sub_one_eq_mod (2 * tab + 1) M + +/-- When `0 < M`, `(oddStride M tab).val` is odd (`% 2 = 1`). -/ +theorem odd_oddStride {M : ℕ} (hM : 0 < M) (tab : ℕ) : (oddStride M tab).val % 2 = 1 := by + rw [val_oddStride] + have hdvd : 2 ∣ 2 ^ M := by + cases M with + | zero => omega + | succ m => exact ⟨2 ^ m, by ring⟩ + rw [Nat.mod_mod_of_dvd _ hdvd] + omega + +/-- `(oddStride M tab).val` is always coprime to `2 ^ M`. -/ +theorem coprime_oddStride_two_pow (M tab : ℕ) : Nat.Coprime (oddStride M tab).val (2 ^ M) := by + rw [val_oddStride, Nat.Coprime, ← Nat.gcd_rec] + have hcop2 : Nat.Coprime 2 (2 * tab + 1) := by + rw [Nat.coprime_two_left, Nat.odd_iff] + omega + exact hcop2.pow_left M + +/-- Double-hashing probe sequence `probeSeq M h₁ tab i = (h₁ + i * oddStride M tab) mod 2 ^ M` +on `Fin (2 ^ M)`. -/ +@[expose] def probeSeq (M h₁ tab : ℕ) (i : Fin (2 ^ M)) : Fin (2 ^ M) := + ⟨(h₁ + i.val * (oddStride M tab).val) % 2 ^ M, Nat.mod_lt _ (Nat.two_pow_pos M)⟩ + +/-- Because `oddStride M tab` is coprime to `2 ^ M`, `probeSeq M h₁ tab` is a full-cycle +bijection (permutation) of `Fin (2 ^ M)` for every `M`, initial probe `h₁`, and raw table sum +`tab`. -/ +theorem bijective_probeSeq (M h₁ tab : ℕ) : Function.Bijective (probeSeq M h₁ tab) := by + rw [Fintype.bijective_iff_injective_and_card] + refine ⟨?_, rfl⟩ + have hinj : ∀ i j : Fin (2 ^ M), j.val ≤ i.val → + probeSeq M h₁ tab i = probeSeq M h₁ tab j → i = j := by + intro i j hij heq + have hmod : (h₁ + j.val * (oddStride M tab).val) ≡ + (h₁ + i.val * (oddStride M tab).val) [MOD 2 ^ M] := by + simpa [probeSeq, Fin.ext_iff, Nat.ModEq] using heq.symm + have hmul : j.val * (oddStride M tab).val ≡ + i.val * (oddStride M tab).val [MOD 2 ^ M] := + Nat.ModEq.add_left_cancel' h₁ hmod + have hsub : 0 + j.val * (oddStride M tab).val ≡ + (i.val - j.val) * (oddStride M tab).val + j.val * (oddStride M tab).val [MOD 2 ^ M] := by + simpa [Nat.zero_add, ← Nat.add_mul, Nat.sub_add_cancel hij] using hmul + have hzero : 0 ≡ (i.val - j.val) * (oddStride M tab).val [MOD 2 ^ M] := + Nat.ModEq.add_right_cancel' (j.val * (oddStride M tab).val) hsub + have hdvd_mul : 2 ^ M ∣ (i.val - j.val) * (oddStride M tab).val := + (Nat.modEq_zero_iff_dvd.mp hzero.symm) + have hdvd : 2 ^ M ∣ i.val - j.val := + (coprime_oddStride_two_pow M tab).symm.dvd_of_dvd_mul_right hdvd_mul + have hlt : i.val - j.val < 2 ^ M := by + have := i.isLt + omega + have hdiff0 : i.val - j.val = 0 := Nat.eq_zero_of_dvd_of_lt hdvd hlt + exact Fin.ext (by omega) + intro i j heq + rcases le_total j.val i.val with hij | hij + · exact hinj i j hij heq + · exact (hinj j i hij heq.symm).symm + +/-- Tail-recursive `d`-character simple tabulation accumulator from character offset `j` across +`rem` remaining characters: XORs `T[(j + k) * 2 ^ b + ((u >>> (k * b)) &&& (2 ^ b - 1))]` into +`acc`. -/ +@[expose] def simpleTabAux (b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) : + ℕ → ℕ → ℕ → Fin (2 ^ M) → Fin (2 ^ M) + | 0, _, _, acc => acc + | rem + 1, j, u, acc => + let ch := u &&& (2 ^ b - 1) + let idx := j * 2 ^ b + ch + let entry : Fin (2 ^ M) := + if h : idx < d * 2 ^ b then T ⟨idx, h⟩ else ⟨0, Nat.two_pow_pos M⟩ + simpleTabAux b d M T rem (j + 1) (u >>> b) + ⟨acc.val ^^^ entry.val, Nat.xor_lt_two_pow acc.isLt entry.isLt⟩ + +/-- `d`-character simple tabulation hash `simpleTab b d M T u : Fin (2 ^ M)`, XORing the table +entries for the first `d` base-`2 ^ b` characters of `u`. -/ +@[expose] def simpleTab (b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) (u : ℕ) : Fin (2 ^ M) := + simpleTabAux b d M T d 0 u ⟨0, Nat.two_pow_pos M⟩ + +/-- Tabulated odd stride `tabulatedOddStride b d M T u : Fin (2 ^ M)` obtained by mapping the +`d`-character simple tabulation hash through `oddStride M`. -/ +@[expose] def tabulatedOddStride (b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) (u : ℕ) : + Fin (2 ^ M) := + oddStride M (simpleTab b d M T u).val + +/-- Memory footprint `4 + d * 2 ^ b` words for a `d`-character simple tabulation table with +character bit-width `b` (4 header words `(b, 2 ^ b, 2 ^ b - 1, 2 ^ M - 1)` followed by +`d * 2 ^ b` random `< 2 ^ M` words). -/ +@[expose] def spaceFromB (b d : ℕ) : ℕ := 4 + d * 2 ^ b + +/-- Memory footprint `4 + d * 2 ^ (charBits n_max)` words at capacity parameter `n_max`. -/ +@[expose] def space (n_max d : ℕ) : ℕ := spaceFromB (charBits n_max) d + +/-- Memory representation predicate for `TabulatedOddStride` at `[base, base + 4 + d * 2 ^ b)`: +header words `b`, `2 ^ b`, `2 ^ b - 1`, `2 ^ M - 1` at `base + 0 .. base + 3`, and table entries +`(T i).val` at `base + 4 + i.val` for `i : Fin (d * 2 ^ b)`. -/ +@[expose] def Rep (base b d M : ℕ) (T : Fin (d * 2 ^ b) → Fin (2 ^ M)) (mem : ℕ → ℕ) : Prop := + mem base = b ∧ + mem (base + 1) = 2 ^ b ∧ + mem (base + 2) = 2 ^ b - 1 ∧ + mem (base + 3) = 2 ^ M - 1 ∧ + ∀ i : Fin (d * 2 ^ b), mem (base + 4 + i.val) = (T i).val + +/-- Extract the flat character table `Fin (d * 2 ^ b) → Fin (2 ^ M)` from `[base + 4, ...)`. -/ +@[expose] def extractTable (base b d M : ℕ) (mem : ℕ → ℕ) : Fin (d * 2 ^ b) → Fin (2 ^ M) := + extractSlice (base + 4) M (d * 2 ^ b) mem + +/-- Register `0` of `TabulatedOddStride`: base memory address `base`. -/ +abbrev regBase : Fin 9 := 0 + +/-- Register `1` of `TabulatedOddStride`: holds `n_max` in `initCode` or input key `u` in +`evalCode`. -/ +abbrev regKey : Fin 9 := 1 + +/-- Register `2` of `TabulatedOddStride`: holds `M` in `initCode` or receives the odd stride in +`evalCode`. -/ +abbrev regOut : Fin 9 := 2 + +/-- Register `3` of `TabulatedOddStride`: table pointer scratch register. -/ +abbrev regPtr : Fin 9 := 3 + +/-- Register `4` of `TabulatedOddStride`: end pointer / mask scratch register. -/ +abbrev regEnd : Fin 9 := 4 + +/-- Register `5` of `TabulatedOddStride`: character bit-width `b` / loop bound register. -/ +abbrev regB : Fin 9 := 5 + +/-- Register `6` of `TabulatedOddStride`: character alphabet size `2 ^ b` / sampled value +register. -/ +abbrev regPowB : Fin 9 := 6 + +/-- Register `7` of `TabulatedOddStride`: character mask `2 ^ b - 1` register. -/ +abbrev regMaskB : Fin 9 := 7 + +/-- Register `8` of `TabulatedOddStride`: general scratch register. -/ +abbrev regTmp : Fin 9 := 8 + +/-- Register injection `Fin 3 → Fin 9` mapping `log2Loop`'s `(rem, acc, one)` to +`(regPtr, regB, regTmp)`. -/ +@[expose] def logMap9 : Fin 3 → Fin 9 + | ⟨0, _⟩ => regPtr + | ⟨1, _⟩ => regB + | ⟨2, _⟩ => regTmp + +theorem injective_logMap9 : Function.Injective logMap9 := by + decide + +/-- Register injection `Fin 6 → Fin 9` mapping `randFillLoop`'s `(ptr, end, mask, one, val, extra)` +to `(regPtr, regB, regEnd, regTmp, regPowB, regBase)`. -/ +@[expose] def fillMap9 : Fin 6 → Fin 9 + | ⟨0, _⟩ => regPtr + | ⟨1, _⟩ => regB + | ⟨2, _⟩ => regEnd + | ⟨3, _⟩ => regTmp + | ⟨4, _⟩ => regPowB + | ⟨5, _⟩ => regBase + +theorem injective_fillMap9 : Function.Injective fillMap9 := by + decide + +/-- Structured Word-RAM program that computes `regB ← charBits n_max` +(`= max 1 (⌊log₂ (max n_max 2)⌋ / 2)`) at runtime from `n_max` in `regKey`, using `regPtr`, +`regB`, `regTmp` as scratch registers and preserving `regBase`, `regOut`, and all memory. -/ +@[expose] def charBitsCode : Code .exact 9 := + .seq (.op (.const regTmp 2)) + (.seq (maxCode regPtr regKey regTmp) + (.seq (.op (.const regB 0)) + (.seq (.op (.const regTmp 1)) + (.seq (log2Loop.rename logMap9) + (.seq (.op (.exactBin .shiftRight regB regB regTmp)) + (maxCode regB regB regTmp)))))) + +/-- State after `charBitsCode` starting from `n_max` in `regKey`. -/ +noncomputable def charBitsData (n_max : ℕ) (x : Data 9) : Data 9 := + let dPre := (((x.set regTmp 2).set regPtr (max n_max 2)).set regB 0).set regTmp 1 + let dLog := (log2LoopData (max n_max 2) 0 (dPre.comap logMap9)).extend logMap9 dPre + (dLog.set regB (Nat.log 2 (max n_max 2) / 2)).set regB (charBits n_max) + +/-- `charBitsCode` executes deterministically in `4 * Nat.log 2 (max n_max 2) + 12` instructions, +setting `regB = charBits n_max` while preserving `regBase`, `regOut`, and all memory. -/ +theorem charBitsCode_exec {w n_max : ℕ} {x : Data 9} + (hn : x.regs regKey = n_max) (hfit_n : Fits w (max n_max 2)) : + Exec w charBitsCode x (charBitsData n_max x) (4 * Nat.log 2 (max n_max 2) + 12) ∧ + (charBitsData n_max x).regs regB = charBits n_max ∧ + (charBitsData n_max x).regs regBase = x.regs regBase ∧ + (charBitsData n_max x).regs regOut = x.regs regOut ∧ + (charBitsData n_max x).mem = x.mem := by + have hfit_2 : Fits w 2 := hfit_n.of_le (Nat.le_max_right n_max 2) + have hfit_1 : Fits w 1 := hfit_2.of_le (by decide) + have hfit_0 : Fits w 0 := hfit_1.of_le (by decide) + have hlog_le : Nat.log 2 (max n_max 2) ≤ max n_max 2 := Nat.log_le_self 2 (max n_max 2) + have hfit_log : Fits w (Nat.log 2 (max n_max 2)) := hfit_n.of_le hlog_le + set d1 := x.set regTmp 2 + have s1 : Exec w (.op (.const regTmp 2) : Code .exact 9) x d1 1 := Exec.const hfit_2 + set d2 := d1.set regPtr (max n_max 2) + have s2 : Exec w (maxCode regPtr regKey regTmp) d1 d2 3 := by + simpa +decide [d2, d1, hn] using maxCode_exec (w := w) regPtr regKey regTmp d1 + set d3 := d2.set regB 0 + have s3 : Exec w (.op (.const regB 0) : Code .exact 9) d2 d3 1 := Exec.const hfit_0 + set d4 := d3.set regTmp 1 + have s4 : Exec w (.op (.const regTmp 1) : Code .exact 9) d3 d4 1 := Exec.const hfit_1 + have slog0 := log2Loop_exec (w := w) (max n_max 2) (acc := 0) (x := d4.comap logMap9) + (by simp +decide [d4, d3, d2, logMap9, Data.comap]) + (by simp +decide [d4, d3, logMap9, Data.comap]) + (by simp +decide [d4, logMap9, Data.comap]) + hfit_n (by simpa using hfit_log) + have slog := slog0.rename injective_logMap9 + set d5 := (log2LoopData (max n_max 2) 0 (d4.comap logMap9)).extend logMap9 d4 + have hd5_b : d5.regs regB = Nat.log 2 (max n_max 2) := by + change (d5.comap logMap9).regs 1 = Nat.log 2 (max n_max 2) + rw [Data.comap_extend injective_logMap9] + simp [log2LoopData] + have hd5_tmp : d5.regs regTmp = 1 := by + change (d5.comap logMap9).regs 2 = 1 + rw [Data.comap_extend injective_logMap9] + simp +decide [log2LoopData, d4, logMap9, Data.comap] + have hshr : Nat.log 2 (max n_max 2) >>> 1 = Nat.log 2 (max n_max 2) / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hfit_div : Fits w (Nat.log 2 (max n_max 2) / 2) := + hfit_log.of_le (Nat.div_le_self _ 2) + set d6 := d5.set regB (Nat.log 2 (max n_max 2) / 2) + have s6 : Exec w (.op (.exactBin .shiftRight regB regB regTmp) : Code .exact 9) d5 d6 1 := + exec_exactBin (by simp [hd5_b, hd5_tmp, hshr]) hfit_div + set d7 := d6.set regB (charBits n_max) + have hcb : max (Nat.log 2 (max n_max 2) / 2) 1 = charBits n_max := Nat.max_comm _ _ + have s7 : Exec w (maxCode regB regB regTmp) d6 d7 3 := by + simpa +decide [d7, d6, hd5_tmp, hcb] using maxCode_exec (w := w) regB regB regTmp d6 + have hall := s1.seq (s2.seq (s3.seq (s4.seq (slog.seq (s6.seq s7))))) + have hsteps : 1 + (3 + (1 + (1 + (4 * Nat.log 2 (max n_max 2) + 2 + (1 + 3))))) = + 4 * Nat.log 2 (max n_max 2) + 12 := by omega + rw [hsteps] at hall + have hbase_eq : d7.regs regBase = x.regs regBase := by + simp +decide [d7, d6, d5, Data.extend, d4, d3, d2, d1] + have hout_eq : d7.regs regOut = x.regs regOut := by + simp +decide [d7, d6, d5, Data.extend, d4, d3, d2, d1] + refine ⟨hall, by simp [charBitsData], hbase_eq, hout_eq, rfl⟩ + +/-- Deterministic 16-instruction setup prefix of `initFromBCode d`: writes the 4 header words +`(b, 2 ^ b, 2 ^ b - 1, 2 ^ M - 1)` to `base + 0 .. base + 3` and sets registers for +`randFillLoop.rename fillMap9` on `[base + 4, base + 4 + d * 2 ^ b)`. -/ +@[expose] def setupFromBCode (d : ℕ) : Code .exact 9 := + .seq (.op (.store regBase regB)) + (.seq (.op (.const regTmp 1)) + (.seq (.op (.exactBin .shiftLeft regPowB regTmp regB)) + (.seq (.op (.exactBin .add regPtr regBase regTmp)) + (.seq (.op (.store regPtr regPowB)) + (.seq (.op (.exactBin .sub regMaskB regPowB regTmp)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.store regPtr regMaskB)) + (.seq (.op (.exactBin .shiftLeft regEnd regTmp regOut)) + (.seq (.op (.exactBin .sub regEnd regEnd regTmp)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.store regPtr regEnd)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.const regB d)) + (.seq (.op (.exactBin .mul regB regB regPowB)) + (.op (.exactBin .add regB regPtr regB)))))))))))))))) + +/-- State after `setupFromBCode d`. -/ +@[expose] def setupFromBData (base b d M : ℕ) (x : Data 9) : Data 9 := + let m0 := (((x.write base b).write (base + 1) (2 ^ b)).write + (base + 2) (2 ^ b - 1)).write (base + 3) (2 ^ M - 1) + ((((((m0.set regTmp 1).set regPowB (2 ^ b)).set regPtr (base + 1)).set + regMaskB (2 ^ b - 1)).set regPtr (base + 2)).set regEnd (2 ^ M)).set + regEnd (2 ^ M - 1) |>.set regPtr (base + 3) |>.set regPtr (base + 4) |>.set + regB d |>.set regB (d * 2 ^ b) |>.set regB (base + 4 + d * 2 ^ b) + +/-- `setupFromBCode d` executes in `16` deterministic steps. -/ +theorem setupFromBCode_exec {w base b d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hb : x.regs regB = b) (hM : x.regs regOut = M) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powB : Fits w (2 ^ b)) (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + Exec w (setupFromBCode d) x (setupFromBData base b d M x) 16 := by + have h1_le_b : 1 ≤ 2 ^ b := Nat.one_le_two_pow + have h1_le_M : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hfit_1 : Fits w 1 := hfit_powB.of_le h1_le_b + have hfit_maskB : Fits w (2 ^ b - 1) := hfit_powB.of_le (Nat.sub_le _ 1) + have hfit_maskM : Fits w (2 ^ M - 1) := hfit_powM.of_le (Nat.sub_le _ 1) + have hfit_b1 : Fits w (base + 1) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b2 : Fits w (base + 2) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b3 : Fits w (base + 3) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b4 : Fits w (base + 4) := hfit_space.of_le (by simp [spaceFromB]) + have hfit_dmul : Fits w (d * 2 ^ b) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_end : Fits w (base + 4 + d * 2 ^ b) := + hfit_space.of_le (by simp [spaceFromB]; omega) + set s0 := x.write base b + have e1 : Exec w (.op (.store regBase regB) : Code .exact 9) x s0 1 := by + simpa [s0, hbase, hb] using + (Exec.store (B := .exact) (w := w) (a := regBase) (s := regB) (x := x)) + set s1 := s0.set regTmp 1 + have e2 : Exec w (.op (.const regTmp 1) : Code .exact 9) s0 s1 1 := Exec.const hfit_1 + set s2 := s1.set regPowB (2 ^ b) + have e3 : Exec w (.op (.exactBin .shiftLeft regPowB regTmp regB) : Code .exact 9) s1 s2 1 := + exec_exactBin (by simp +decide [s1, s0, hb, Nat.shiftLeft_eq]) hfit_powB + set s3 := s2.set regPtr (base + 1) + have e4 : Exec w (.op (.exactBin .add regPtr regBase regTmp) : Code .exact 9) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, s0, hbase]) hfit_b1 + set s4 := s3.write (base + 1) (2 ^ b) + have e5 : Exec w (.op (.store regPtr regPowB) : Code .exact 9) s3 s4 1 := by + simpa +decide [s4, s3, s2] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regPowB) (x := s3)) + set s5 := s4.set regMaskB (2 ^ b - 1) + have e6 : Exec w (.op (.exactBin .sub regMaskB regPowB regTmp) : Code .exact 9) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h1_le_b]) hfit_maskB + set s6 := s5.set regPtr (base + 2) + have e7 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1]) hfit_b2 + set s7 := s6.write (base + 2) (2 ^ b - 1) + have e8 : Exec w (.op (.store regPtr regMaskB) : Code .exact 9) s6 s7 1 := by + simpa +decide [s7, s6, s5] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regMaskB) (x := s6)) + set s8 := s7.set regEnd (2 ^ M) + have e9 : Exec w (.op (.exactBin .shiftLeft regEnd regTmp regOut) : Code .exact 9) s7 s8 1 := + exec_exactBin (by simp +decide [s7, s6, s5, s4, s3, s2, s1, s0, hM, Nat.shiftLeft_eq]) hfit_powM + set s9 := s8.set regEnd (2 ^ M - 1) + have e10 : Exec w (.op (.exactBin .sub regEnd regEnd regTmp) : Code .exact 9) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h1_le_M]) hfit_maskM + set s10 := s9.set regPtr (base + 3) + have e11 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7, s6, s5, s4, s3, s2, s1]) hfit_b3 + set s11 := s10.write (base + 3) (2 ^ M - 1) + have e12 : Exec w (.op (.store regPtr regEnd) : Code .exact 9) s10 s11 1 := by + simpa +decide [s11, s10, s9] using + (Exec.store (B := .exact) (w := w) (a := regPtr) (s := regEnd) (x := s10)) + set s12 := s11.set regPtr (base + 4) + have e13 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s11 s12 1 := + exec_exactBin (by simp +decide [s11, s10, s9, s8, s7, s6, s5, s4, s3, s2, s1]) hfit_b4 + set s13 := s12.set regB d + have e14 : Exec w (.op (.const regB d) : Code .exact 9) s12 s13 1 := Exec.const hfit_d + set s14 := s13.set regB (d * 2 ^ b) + have e15 : Exec w (.op (.exactBin .mul regB regB regPowB) : Code .exact 9) s13 s14 1 := + exec_exactBin (by simp +decide [s13, s12, s11, s10, s9, s8, s7, s6, s5, s4, s3, s2]) hfit_dmul + set s15 := s14.set regB (base + 4 + d * 2 ^ b) + have e16 : Exec w (.op (.exactBin .add regB regPtr regB) : Code .exact 9) s14 s15 1 := + exec_exactBin (by simp +decide [s14, s13, s12]) hfit_end + have hs15 : s15 = setupFromBData base b d M x := by + ext i : 2 + · fin_cases i <;> rfl + · rfl + rw [hs15] at e16 + exact e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq + (e9.seq (e10.seq (e11.seq (e12.seq (e13.seq (e14.seq (e15.seq e16)))))))))))))) + +/-- Structured Word-RAM program that initializes a `d`-character simple tabulation table from +`(base, M, b)` in `(regBase, regOut, regB)`. -/ +@[expose] def initFromBCode (d : ℕ) : Code .exact 9 := + .seq (setupFromBCode d) (randFillLoop.rename fillMap9) + +/-- Explicit `ProbComp (Data 9 × ℕ)` produced by `initFromBCode d`. -/ +@[expose] noncomputable def initFromBComp (base b d M : ℕ) (x : Data 9) : ProbComp (Data 9 × ℕ) := + let s := setupFromBData base b d M x + (fun z => (z.1.extend fillMap9 s, 16 + z.2)) <$> + randFillComp M (base + 4) (d * 2 ^ b) (s.comap fillMap9) + +/-- Exact `ExecRand` semantics of `initFromBCode d`. -/ +theorem initFromBCode_execRand {w base b d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hb : x.regs regB = b) (hM : x.regs regOut = M) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powB : Fits w (2 ^ b)) (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRand w (initFromBCode d) x (initFromBComp base b d M x) := by + have hsetup := setupFromBCode_exec hbase hb hM hfit_space hfit_powB hfit_powM hfit_d + set s := setupFromBData base b d M x + have hloop0 := randFillLoop_execRand (w := w) (bits := M) (arrBase := base + 4) + (d * 2 ^ b) (x := s.comap fillMap9) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (by simp +decide [s, setupFromBData, fillMap9, Data.comap]) + (hfit_space.of_le (by simp [spaceFromB]; omega)) + (hfit_powM.of_le (Nat.sub_le _ 1)) + have hloop := hloop0.rename injective_fillMap9 + have hseq := hsetup.seqRand hloop + have heq : ((fun z => (z.1, 16 + z.2)) <$> + ((fun z => (z.1.extend fillMap9 s, z.2)) <$> + randFillComp M (base + 4) (d * 2 ^ b) (s.comap fillMap9))) = + initFromBComp base b d M x := by + simp [initFromBComp, s, Functor.map_map] + rwa [heq] at hseq + +/-- The character table `extractTable base b d M z.1.mem` produced by `initFromBComp` is uniformly +distributed over `Fin (d * 2 ^ b) → Fin (2 ^ M)`. -/ +theorem map_extractTable_eval_initFromBComp (base b d M : ℕ) (x : Data 9) : + ((initFromBComp base b d M x).eval).map (fun z => extractTable base b d M z.1.mem) = + PMF.uniformOfFintype (Fin (d * 2 ^ b) → Fin (2 ^ M)) := by + dsimp only [initFromBComp, extractTable] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractSlice_eval_randFillComp M (base + 4) (d * 2 ^ b) + ((setupFromBData base b d M x).comap fillMap9) + +/-- Self-contained `initCode d` that computes `b = charBits n_max` from `n_max` in `regKey` and +initializes the `d`-character simple tabulation table at `base` (`regBase`) with `M`-bit entries +(`regOut = M`). -/ +@[expose] def initCode (d : ℕ) : Code .exact 9 := + .seq charBitsCode (initFromBCode d) + +/-- Pathwise postcondition of `initFromBComp`: runs in `5 * (d * 2 ^ b) + 18` instructions and +establishes `Rep base b d M (extractTable base b d M z.1.mem) z.1.mem`, `z.1.regs regBase = base`, +and frame preservation outside `[base, base + spaceFromB b d)`. -/ +theorem mem_support_initFromBComp {base b d M : ℕ} {x : Data 9} {z : Data 9 × ℕ} + (hbase : x.regs regBase = base) + (hz : z ∈ (initFromBComp base b d M x).support) : + z.2 = 5 * (d * 2 ^ b) + 18 ∧ + Rep base b d M (extractTable base b d M z.1.mem) z.1.mem ∧ + z.1.regs regBase = base ∧ + (∀ a, (a < base ∨ base + spaceFromB b d ≤ a) → z.1.mem a = x.mem a) := by + dsimp only [initFromBComp] at hz + obtain ⟨z₀, hz₀, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hframe⟩ := mem_support_randFillComp hz₀ + set s := setupFromBData base b d M x + have hmem0 : z₀.1.mem base = b := by + rw [hframe base (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write, Function.update_of_ne] + have hmem1 : z₀.1.mem (base + 1) = 2 ^ b := by + rw [hframe (base + 1) (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write, Function.update_of_ne] + have hmem2 : z₀.1.mem (base + 2) = 2 ^ b - 1 := by + rw [hframe (base + 2) (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write, Function.update_of_ne] + have hmem3 : z₀.1.mem (base + 3) = 2 ^ M - 1 := by + rw [hframe (base + 3) (Or.inl (by omega))] + simp +decide [s, setupFromBData, Data.comap, Data.write] + have hrep : Rep base b d M (extractTable base b d M z₀.1.mem) z₀.1.mem := by + refine ⟨hmem0, hmem1, hmem2, hmem3, fun i => ?_⟩ + simp [extractTable, extractSlice, hlt i.val i.isLt] + have hbase' : (z₀.1.extend fillMap9 s).regs regBase = base := by + change ((z₀.1.extend fillMap9 s).comap fillMap9).regs regExtra = base + rw [Data.comap_extend injective_fillMap9, hextra] + simpa +decide [s, setupFromBData, fillMap9, Data.comap] using hbase + have hframe' : ∀ a, (a < base ∨ base + spaceFromB b d ≤ a) → z₀.1.mem a = x.mem a := by + intro a ha + have ha4 : a < base + 4 ∨ base + 4 + d * 2 ^ b ≤ a := by + simp only [spaceFromB] at ha + omega + rw [hframe a ha4] + have hne0 : a ≠ base := by simp only [spaceFromB] at ha; omega + have hne1 : a ≠ base + 1 := by simp only [spaceFromB] at ha; omega + have hne2 : a ≠ base + 2 := by simp only [spaceFromB] at ha; omega + have hne3 : a ≠ base + 3 := by simp only [spaceFromB] at ha; omega + simp [s, setupFromBData, Data.comap, Data.write, + Function.update_of_ne hne0, Function.update_of_ne hne1, + Function.update_of_ne hne2, Function.update_of_ne hne3] + exact ⟨by omega, hrep, hbase', hframe'⟩ + +/-- Hoare-style `ExecRandWithin` specification for `initFromBCode d`. -/ +theorem initFromBCode_execRandWithin {w base b d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hb : x.regs regB = b) (hM : x.regs regOut = M) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powB : Fits w (2 ^ b)) (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRandWithin w (initFromBCode d) x + (fun y => Rep base b d M (extractTable base b d M y.mem) y.mem ∧ + y.regs regBase = base ∧ + (∀ a, (a < base ∨ base + spaceFromB b d ≤ a) → y.mem a = x.mem a)) + (5 * (d * 2 ^ b) + 18) (d * 2 ^ b) := by + refine ⟨initFromBComp base b d M x, + initFromBCode_execRand hbase hb hM hfit_space hfit_powB hfit_powM hfit_d, ?_, ?_⟩ + · dsimp only [initFromBComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp M (base + 4) (d * 2 ^ b) _) + · intro z hz + obtain ⟨hsteps, hrep, hbase', hframe'⟩ := mem_support_initFromBComp hbase hz + exact ⟨⟨hrep, hbase', hframe'⟩, by omega⟩ + +/-- Explicit `ProbComp (Data 9 × ℕ)` produced by `initCode d` from `(base, n_max, M)`. -/ +@[expose] noncomputable def initComp (base n_max d M : ℕ) (x : Data 9) : ProbComp (Data 9 × ℕ) := + (fun z => (z.1, (4 * Nat.log 2 (max n_max 2) + 12) + z.2)) <$> + initFromBComp base (charBits n_max) d M (charBitsData n_max x) + +/-- Exact `ExecRand` semantics of `initCode d`. -/ +theorem initCode_execRand {w base n_max d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hn : x.regs regKey = n_max) (hM : x.regs regOut = M) + (hfit_n : Fits w (max n_max 2)) + (hfit_space : Fits w (base + space n_max d)) + (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRand w (initCode d) x (initComp base n_max d M x) := by + obtain ⟨hcb, hb_eq, hbase_eq, hout_eq, _⟩ := charBitsCode_exec (w := w) hn hfit_n + have hfit_powB : Fits w (2 ^ charBits n_max) := by + have hle := two_pow_charBits_le n_max + rw [Nat.max_comm 2 n_max] at hle + exact hfit_n.of_le hle + have hinit := initFromBCode_execRand + (hbase_eq.trans hbase) hb_eq (hout_eq.trans hM) hfit_space hfit_powB hfit_powM hfit_d + exact hcb.seqRand hinit + +/-- **Uniformity of `TabulatedOddStride.initComp`**: the character table extracted from final +memory has the exact uniform distribution over `Fin (d * 2 ^ (charBits n_max)) → Fin (2 ^ M)`. -/ +theorem map_extractTable_eval_initComp (base n_max d M : ℕ) (x : Data 9) : + ((initComp base n_max d M x).eval).map + (fun z => extractTable base (charBits n_max) d M z.1.mem) = + PMF.uniformOfFintype (Fin (d * 2 ^ (charBits n_max)) → Fin (2 ^ M)) := by + dsimp only [initComp] + rw [ProbComp.eval_map, PMF.map_comp] + exact map_extractTable_eval_initFromBComp base (charBits n_max) d M (charBitsData n_max x) + +/-- Hoare-style `ExecRandWithin` specification for `TabulatedOddStride.initCode d`. -/ +theorem initCode_execRandWithin {w base n_max d M : ℕ} {x : Data 9} + (hbase : x.regs regBase = base) (hn : x.regs regKey = n_max) (hM : x.regs regOut = M) + (hfit_n : Fits w (max n_max 2)) + (hfit_space : Fits w (base + space n_max d)) + (hfit_powM : Fits w (2 ^ M)) (hfit_d : Fits w d) : + ExecRandWithin w (initCode d) x + (fun y => + Rep base (charBits n_max) d M (extractTable base (charBits n_max) d M y.mem) y.mem ∧ + y.regs regBase = base ∧ + (∀ a, (a < base ∨ base + space n_max d ≤ a) → y.mem a = x.mem a)) + (4 * Nat.log 2 (max n_max 2) + 5 * (d * 2 ^ (charBits n_max)) + 30) + (d * 2 ^ (charBits n_max)) := by + refine ⟨initComp base n_max d M x, + initCode_execRand hbase hn hM hfit_n hfit_space hfit_powM hfit_d, ?_, ?_⟩ + · dsimp only [initComp, initFromBComp] + exact (OracleComp.isCostBounded_map_iff _ _ _).mpr + ((OracleComp.isCostBounded_map_iff _ _ _).mpr + (isQueryBounded_randFillComp M (base + 4) (d * 2 ^ (charBits n_max)) _)) + · intro z hz + dsimp only [initComp] at hz + obtain ⟨z₁, hz₁, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨-, -, hbase_cb, -, hmem_cb⟩ := charBitsCode_exec (w := w) hn hfit_n + obtain ⟨hsteps, hrep, hbase', hframe'⟩ := + mem_support_initFromBComp (hbase_cb.trans hbase) hz₁ + refine ⟨⟨hrep, hbase', fun a ha => ?_⟩, by omega⟩ + rw [hframe' a ha, hmem_cb] + +/-- 12-instruction deterministic setup before `tabLoop` in `evalCode d`: loads `b`, `2 ^ b`, +`2 ^ b - 1` from `base + 0 .. base + 2`, sets `regPtr = base + 4` and +`regEnd = base + 4 + d * 2 ^ b`, and zeros `regOut`. -/ +@[expose] def evalSetupCode (d : ℕ) : Code .exact 9 := + .seq (.op (.load regB regBase)) + (.seq (.op (.const regTmp 1)) + (.seq (.op (.exactBin .add regPtr regBase regTmp)) + (.seq (.op (.load regPowB regPtr)) + (.seq (.op (.exactBin .add regPtr regPtr regTmp)) + (.seq (.op (.load regMaskB regPtr)) + (.seq (.op (.const regOut 2)) + (.seq (.op (.exactBin .add regPtr regPtr regOut)) + (.seq (.op (.const regOut d)) + (.seq (.op (.exactBin .mul regEnd regOut regPowB)) + (.seq (.op (.exactBin .add regEnd regPtr regEnd)) + (.op (.const regOut 0)))))))))))) + +/-- State after `evalSetupCode d`. -/ +@[expose] def evalSetupData (base b d : ℕ) (x : Data 9) : Data 9 := + (((((((((((x.set regB b).set regTmp 1).set regPtr (base + 1)).set + regPowB (2 ^ b)).set regPtr (base + 2)).set regMaskB (2 ^ b - 1)).set + regOut 2).set regPtr (base + 4)).set regOut d).set regEnd (d * 2 ^ b)).set + regEnd (base + 4 + d * 2 ^ b)).set regOut 0 + +/-- `evalSetupCode d` executes in `12` deterministic steps when `Rep base b d M T x.mem` holds. -/ +theorem evalSetupCode_exec {w base b d M : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 ^ M)} {x : Data 9} + (hrep : Rep base b d M T x.mem) (hbase : x.regs regBase = base) + (hfit_space : Fits w (base + spaceFromB b d)) (hfit_d : Fits w d) : + Exec w (evalSetupCode d) x (evalSetupData base b d x) 12 := by + obtain ⟨hm0, hm1, hm2, -, -⟩ := hrep + have hfit_2 : Fits w 2 := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_1 : Fits w 1 := hfit_2.of_le (by decide) + have hfit_0 : Fits w 0 := hfit_1.of_le (by decide) + have hfit_b1 : Fits w (base + 1) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b2 : Fits w (base + 2) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b4 : Fits w (base + 4) := hfit_space.of_le (by simp [spaceFromB]) + have hfit_dmul : Fits w (d * 2 ^ b) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_end : Fits w (base + 4 + d * 2 ^ b) := hfit_space.of_le (by simp [spaceFromB]; omega) + set s0 := x.set regB b + have e1 : Exec w (.op (.load regB regBase) : Code .exact 9) x s0 1 := by + simpa [s0, hbase, hm0] using + (Exec.load (B := .exact) (w := w) (d := regB) (a := regBase) (x := x)) + set s1 := s0.set regTmp 1 + have e2 : Exec w (.op (.const regTmp 1) : Code .exact 9) s0 s1 1 := Exec.const hfit_1 + set s2 := s1.set regPtr (base + 1) + have e3 : Exec w (.op (.exactBin .add regPtr regBase regTmp) : Code .exact 9) s1 s2 1 := + exec_exactBin (by simp +decide [s1, s0, hbase]) hfit_b1 + set s3 := s2.set regPowB (2 ^ b) + have e4 : Exec w (.op (.load regPowB regPtr) : Code .exact 9) s2 s3 1 := by + simpa +decide [s3, s2, s1, s0, hm1] using + (Exec.load (B := .exact) (w := w) (d := regPowB) (a := regPtr) (x := s2)) + set s4 := s3.set regPtr (base + 2) + have e5 : Exec w (.op (.exactBin .add regPtr regPtr regTmp) : Code .exact 9) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1]) hfit_b2 + set s5 := s4.set regMaskB (2 ^ b - 1) + have e6 : Exec w (.op (.load regMaskB regPtr) : Code .exact 9) s4 s5 1 := by + simpa +decide [s5, s4, s3, s2, s1, s0, hm2] using + (Exec.load (B := .exact) (w := w) (d := regMaskB) (a := regPtr) (x := s4)) + set s6 := s5.set regOut 2 + have e7 : Exec w (.op (.const regOut 2) : Code .exact 9) s5 s6 1 := Exec.const hfit_2 + set s7 := s6.set regPtr (base + 4) + have e8 : Exec w (.op (.exactBin .add regPtr regPtr regOut) : Code .exact 9) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, s4]) hfit_b4 + set s8 := s7.set regOut d + have e9 : Exec w (.op (.const regOut d) : Code .exact 9) s7 s8 1 := Exec.const hfit_d + set s9 := s8.set regEnd (d * 2 ^ b) + have e10 : Exec w (.op (.exactBin .mul regEnd regOut regPowB) : Code .exact 9) s8 s9 1 := + exec_exactBin (by simp +decide [s8, s7, s6, s5, s4, s3]) hfit_dmul + set s10 := s9.set regEnd (base + 4 + d * 2 ^ b) + have e11 : Exec w (.op (.exactBin .add regEnd regPtr regEnd) : Code .exact 9) s9 s10 1 := + exec_exactBin (by simp +decide [s9, s8, s7]) hfit_end + set s11 := s10.set regOut 0 + have e12 : Exec w (.op (.const regOut 0) : Code .exact 9) s10 s11 1 := Exec.const hfit_0 + exact e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq + (e7.seq (e8.seq (e9.seq (e10.seq (e11.seq e12)))))))))) + +/-- Body of `tabLoop` (6 instructions): extracts `ch = regKey &&& regMaskB`, loads +`T[regPtr + ch]`, XORs into `regOut`, shifts `regKey` right by `regB`, and advances `regPtr` by +`regPowB`. -/ +@[expose] def tabBody : Code .exact 9 := + .seq (.op (.exactBin .and regTmp regKey regMaskB)) + (.seq (.op (.exactBin .add regTmp regPtr regTmp)) + (.seq (.op (.load regTmp regTmp)) + (.seq (.op (.exactBin .xor regOut regOut regTmp)) + (.seq (.op (.exactBin .shiftRight regKey regKey regB)) + (.op (.exactBin .add regPtr regPtr regPowB)))))) + +/-- Loop over the `d` character tables `[base + 4, base + 4 + d * 2 ^ b)`. -/ +@[expose] def tabLoop : Code .exact 9 := + .loop (.lt regPtr regEnd) tabBody + +/-- State after one iteration of `tabBody`. -/ +@[expose] def tabStepData (ptr ch entry newAcc newKey nextPtr : ℕ) (x : Data 9) : Data 9 := + (((((x.set regTmp ch).set regTmp (ptr + ch)).set regTmp entry).set + regOut newAcc).set regKey newKey).set regPtr nextPtr + +/-- `tabLoop` executes deterministically in `8 * rem + 2` instructions from character offset `j` +with `j + rem = d`, accumulating `(simpleTabAux b d M T rem j u acc).val` into `regOut` and +preserving `regBase` and all memory. -/ +theorem tabLoop_exec {w base b d M : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 ^ M)} {x : Data 9} + (hrep : Rep base b d M T x.mem) + (hfit_space : Fits w (base + spaceFromB b d)) (hfit_powM : Fits w (2 ^ M)) : + ∀ (rem j u : ℕ) (acc : Fin (2 ^ M)) (d0 : Data 9), + j + rem = d → + d0.mem = x.mem → + d0.regs regBase = base → + d0.regs regPtr = base + 4 + j * 2 ^ b → + d0.regs regEnd = base + 4 + d * 2 ^ b → + d0.regs regB = b → + d0.regs regPowB = 2 ^ b → + d0.regs regMaskB = 2 ^ b - 1 → + d0.regs regKey = u → + d0.regs regOut = acc.val → + Fits w u → + ∃ y : Data 9, + Exec w tabLoop d0 y (8 * rem + 2) ∧ + y.regs regOut = (simpleTabAux b d M T rem j u acc).val ∧ + y.regs regBase = base ∧ + y.mem = x.mem := by + intro rem + induction rem with + | zero => + intro j u acc d0 hj hmem hbase hptr hend _ _ _ _ hout _ + have hj_eq : j = d := by omega + have hc : ¬(Cond.lt regPtr regEnd).Holds d0 := by + simp [Cond.Holds, hptr, hend, hj_eq] + exact ⟨d0, Exec.loop_exit hc, by simp [simpleTabAux, hout], hbase, hmem⟩ + | succ rem ih => + intro j u acc d0 hj hmem hbase hptr hend hb hpowB hmaskB hkey hout hfit_u + have hj_lt : j < d := by omega + have hpow_pos : 0 < 2 ^ b := Nat.two_pow_pos b + have hj_mul_lt : j * 2 ^ b < d * 2 ^ b := Nat.mul_lt_mul_of_pos_right hj_lt hpow_pos + have hc : (Cond.lt regPtr regEnd).Holds d0 := by + simp only [Cond.Holds, hptr, hend] + omega + set ch := u &&& (2 ^ b - 1) + have hch_lt : ch < 2 ^ b := by + dsimp only [ch] + rw [Nat.and_two_pow_sub_one_eq_mod] + exact Nat.mod_lt u hpow_pos + set idx := j * 2 ^ b + ch + have hidx_lt : idx < d * 2 ^ b := by + have : (j + 1) * 2 ^ b ≤ d * 2 ^ b := Nat.mul_le_mul_right (2 ^ b) (by omega) + calc + idx < j * 2 ^ b + 2 ^ b := by omega + _ = (j + 1) * 2 ^ b := by ring + _ ≤ d * 2 ^ b := this + set entry := (T ⟨idx, hidx_lt⟩).val + have hentry_mem : d0.mem (base + 4 + j * 2 ^ b + ch) = entry := by + rw [hmem, show base + 4 + j * 2 ^ b + ch = base + 4 + idx by omega] + exact hrep.2.2.2.2 ⟨idx, hidx_lt⟩ + set newAcc : Fin (2 ^ M) := + ⟨acc.val ^^^ entry, Nat.xor_lt_two_pow acc.isLt (T ⟨idx, hidx_lt⟩).isLt⟩ + have hfit_ch : Fits w ch := hfit_u.of_le Nat.and_le_left + have hfit_addr : Fits w (base + 4 + j * 2 ^ b + ch) := + hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_xor : Fits w newAcc.val := hfit_powM.of_le (le_of_lt newAcc.isLt) + have hfit_shr : Fits w (u >>> b) := hfit_u.of_le (Nat.shiftRight_le u b) + have hnext_eq : base + 4 + j * 2 ^ b + 2 ^ b = base + 4 + (j + 1) * 2 ^ b := by ring + have hfit_next : Fits w (base + 4 + j * 2 ^ b + 2 ^ b) := by + have : (j + 1) * 2 ^ b ≤ d * 2 ^ b := Nat.mul_le_mul_right (2 ^ b) (by omega) + exact hfit_space.of_le (by simp [spaceFromB]; omega) + set s1 := d0.set regTmp ch + have e1 : Exec w (.op (.exactBin .and regTmp regKey regMaskB) : Code .exact 9) d0 s1 1 := + exec_exactBin (by simp [hkey, hmaskB, ch]) hfit_ch + set s2 := s1.set regTmp (base + 4 + j * 2 ^ b + ch) + have e2 : Exec w (.op (.exactBin .add regTmp regPtr regTmp) : Code .exact 9) s1 s2 1 := + exec_exactBin (by simp +decide [s1, hptr]) hfit_addr + set s3 := s2.set regTmp entry + have e3 : Exec w (.op (.load regTmp regTmp) : Code .exact 9) s2 s3 1 := by + simpa +decide [s3, s2, s1, hentry_mem] using + (Exec.load (B := .exact) (w := w) (d := regTmp) (a := regTmp) (x := s2)) + set s4 := s3.set regOut newAcc.val + have e4 : Exec w (.op (.exactBin .xor regOut regOut regTmp) : Code .exact 9) s3 s4 1 := + exec_exactBin (by simp +decide [s3, s2, s1, hout, newAcc]) hfit_xor + set s5 := s4.set regKey (u >>> b) + have e5 : Exec w (.op (.exactBin .shiftRight regKey regKey regB) : Code .exact 9) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, hkey, hb]) hfit_shr + set s6 := s5.set regPtr (base + 4 + j * 2 ^ b + 2 ^ b) + have e6 : Exec w (.op (.exactBin .add regPtr regPtr regPowB) : Code .exact 9) s5 s6 1 := + exec_exactBin (by simp +decide [s5, s4, s3, s2, s1, hptr, hpowB]) hfit_next + have hbody := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq e6)))) + obtain ⟨y, hrec, hy_out, hy_base, hy_mem⟩ := + ih (j + 1) (u >>> b) newAcc s6 (by omega) hmem + (by simp +decide [s6, s5, s4, s3, s2, s1, hbase]) + (by simp +decide [s6, hnext_eq]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hend]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hb]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hpowB]) + (by simp +decide [s6, s5, s4, s3, s2, s1, hmaskB]) + (by simp +decide [s6, s5]) + (by simp +decide [s6, s5, s4]) + hfit_shr + have hiter := Exec.loop_iter hc hbody hrec + refine ⟨y, ?_, ?_, hy_base, hy_mem⟩ + · simpa only [tabLoop, tabBody, + show 1 + (1 + (1 + (1 + (1 + 1)))) + (8 * rem + 2) + 2 = 8 * (rem + 1) + 2 by omega] + using hiter + · rw [hy_out] + simp only [simpleTabAux, ch, idx, hidx_lt, ↓reduceDIte, entry, newAcc] + +/-- 7-instruction finishing suffix of `evalCode d`: loads `2 ^ M - 1` from `base + 3` and maps +`acc` in `regOut` to `(2 * acc + 1) &&& (2 ^ M - 1)`. -/ +@[expose] def evalFinishCode : Code .exact 9 := + .seq (.op (.const regTmp 3)) + (.seq (.op (.exactBin .add regTmp regBase regTmp)) + (.seq (.op (.load regTmp regTmp)) + (.seq (.op (.exactBin .add regOut regOut regOut)) + (.seq (.op (.const regB 1)) + (.seq (.op (.exactBin .add regOut regOut regB)) + (.op (.exactBin .and regOut regOut regTmp))))))) + +/-- Complete `d`-character simple tabulation odd-stride evaluation program `evalCode d`. -/ +@[expose] def evalCode (d : ℕ) : Code .exact 9 := + .seq (evalSetupCode d) (.seq tabLoop evalFinishCode) + +/-- **`TabulatedOddStride.evalCode d` correctness and complexity**: for any character table +`T : Fin (d * 2 ^ b) → Fin (2 ^ M)` represented at `base` and key `u` with `Fits w u`, +`evalCode d` halts deterministically in `8 * d + 21 = O(d)` instructions, preserving `regBase` +and all memory and writing `(tabulatedOddStride b d M T u).val` into `regOut`. -/ +theorem evalCode_execWithin {w base b d M u : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 ^ M)} {x : Data 9} + (hrep : Rep base b d M T x.mem) + (hbase : x.regs regBase = base) (hu : x.regs regKey = u) + (hfit_space : Fits w (base + spaceFromB b d)) + (hfit_powM : Fits w (2 * 2 ^ M)) (hfit_d : Fits w d) (hfit_u : Fits w u) : + ∃ y : Data 9, + ExecWithin w (evalCode d) x y (8 * d + 21) ∧ + y.regs regOut = (tabulatedOddStride b d M T u).val ∧ + y.regs regBase = base ∧ + y.mem = x.mem := by + have hfit_M : Fits w (2 ^ M) := hfit_powM.of_le (by omega) + have hsetup := evalSetupCode_exec hrep hbase hfit_space hfit_d + set d0 := evalSetupData base b d x + obtain ⟨yLoop, hloop, hy_out, hy_base, hy_mem⟩ := + tabLoop_exec hrep hfit_space hfit_M d 0 u ⟨0, Nat.two_pow_pos M⟩ d0 + (by omega) rfl + (by simp +decide [d0, evalSetupData, hbase]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData]) + (by simp +decide [d0, evalSetupData, hu]) + (by simp +decide [d0, evalSetupData]) + hfit_u + set tab := simpleTab b d M T u + have htab_eq : yLoop.regs regOut = tab.val := hy_out + have hfit_3 : Fits w 3 := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_b3 : Fits w (base + 3) := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_1 : Fits w 1 := hfit_space.of_le (by simp [spaceFromB]; omega) + have hfit_2tab : Fits w (tab.val + tab.val) := + hfit_powM.of_le (by have := tab.isLt; omega) + have hfit_2tab1 : Fits w (tab.val + tab.val + 1) := + hfit_powM.of_le (by have := tab.isLt; omega) + have hfit_and : Fits w ((tab.val + tab.val + 1) &&& (2 ^ M - 1)) := + hfit_2tab1.of_le Nat.and_le_left + have hm3 : yLoop.mem (base + 3) = 2 ^ M - 1 := by rw [hy_mem]; exact hrep.2.2.2.1 + set f1 := yLoop.set regTmp 3 + have e1 : Exec w (.op (.const regTmp 3) : Code .exact 9) yLoop f1 1 := Exec.const hfit_3 + set f2 := f1.set regTmp (base + 3) + have e2 : Exec w (.op (.exactBin .add regTmp regBase regTmp) : Code .exact 9) f1 f2 1 := + exec_exactBin (by simp +decide [f1, hy_base]) hfit_b3 + set f3 := f2.set regTmp (2 ^ M - 1) + have e3 : Exec w (.op (.load regTmp regTmp) : Code .exact 9) f2 f3 1 := by + simpa +decide [f3, f2, f1, hm3] using + (Exec.load (B := .exact) (w := w) (d := regTmp) (a := regTmp) (x := f2)) + set f4 := f3.set regOut (tab.val + tab.val) + have e4 : Exec w (.op (.exactBin .add regOut regOut regOut) : Code .exact 9) f3 f4 1 := + exec_exactBin (by simp +decide [f3, f2, f1, htab_eq]) hfit_2tab + set f5 := f4.set regB 1 + have e5 : Exec w (.op (.const regB 1) : Code .exact 9) f4 f5 1 := Exec.const hfit_1 + set f6 := f5.set regOut (tab.val + tab.val + 1) + have e6 : Exec w (.op (.exactBin .add regOut regOut regB) : Code .exact 9) f5 f6 1 := + exec_exactBin (by simp +decide [f5, f4]) hfit_2tab1 + set f7 := f6.set regOut ((tab.val + tab.val + 1) &&& (2 ^ M - 1)) + have e7 : Exec w (.op (.exactBin .and regOut regOut regTmp) : Code .exact 9) f6 f7 1 := + exec_exactBin (by simp +decide [f6, f5, f4, f3]) hfit_and + have hfin := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq e7))))) + have hall := hsetup.seq (hloop.seq hfin) + refine ⟨f7, ⟨12 + (8 * d + 2 + (1 + (1 + (1 + (1 + (1 + (1 + 1))))))), by omega, hall⟩, + ?_, by simp +decide [f7, f6, f5, f4, f3, f2, f1, hy_base], hy_mem⟩ + simp +decide [f7, tabulatedOddStride, oddStride, tab, Nat.two_mul] + +end TabulatedOddStride + +end Cslib.Algorithms.WordRAM.Hashing.Tabulation diff --git a/ComplexityTests/WordRAMKWiseHash.lean b/ComplexityTests/WordRAMKWiseHash.lean new file mode 100644 index 000000000..77b8a398d --- /dev/null +++ b/ComplexityTests/WordRAMKWiseHash.lean @@ -0,0 +1,431 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Algorithms.WordRAM.Hashing.KWise +public import Complexity.Algorithms.WordRAM.Hashing.Tabulation + +/-! +# Concrete interpreter and non-vacuity tests for `Hashing.KWise` and `Hashing.Tabulation` + +Verifies both concrete `Cslib.WordRAM.Code.eval` execution and theorem-level non-vacuity of: +1. `Cslib.Algorithms.WordRAM.Hashing.KWise`: + - `initTableCode` and `evalCode` executed via `Code.eval` at `k = 5` for both `b = 1, L = 2` + (`n_max = 8`, `U = 4`, `M = 2`) and `b = 2, L = 1` (`n_max = 64`, `U = 4`, `M = 2`), and at + `t = 1` (`q = 6`, `b = 2`, `L = 3`, `n_max = 64`, `U = 16`, `M = 3`, `k = 5`), agreeing on + every tested key with direct unchunked `reduceTrinomialCode`/`clmul` Horner evaluation while + preserving boundary sentinels; + - non-vacuous instantiations of `initTableCode_execWithin`, `resampleCode_execRand`, + `initCode_execRandWithin`, `evalCode_execWithin`, and `isKWiseIndependentHash_eval` at `k = 5`, + `U = 1000`, `M = 10`, `n_max = 64`, `w = 64`. +2. `Cslib.Algorithms.WordRAM.Hashing.Tabulation.RandomTable`: + - `lookupCode` and `xorLookupCode` executed via `Code.eval` on a concrete table of size + `2 ^ 3 = 8`; + - non-vacuous instantiations of `resampleCode_execRand`, `map_extractTable_eval_resampleComp`, + `resampleCode_execRandWithin`, `lookupCode_exec`, and `xorLookupCode_exec`. +3. `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride`: + - `evalCode 3` executed via `Code.eval` on concrete character tables (`d = 3`, `b = 1`, `M = 3`), + matching `tabulatedOddStride` and producing odd strides whose linear probe sequence `probeSeq` + visits all `2 ^ M = 8` table slots; + - non-vacuous instantiations of `initCode_execRand`, `map_extractTable_eval_initComp`, + `initCode_execRandWithin`, `evalCode_execWithin`, and `bijective_probeSeq`. +-/ + +@[expose] public section + +namespace ComplexityTests.WordRAMKWiseHash + +open Cslib Cslib.Probability Cslib.WordRAM +open Cslib.Algorithms.WordRAM.Hashing +open Cslib.Algorithms.WordRAM.Hashing.Tabulation + +/-! ## Helper for compacting interpreter state between `Code.eval` calls -/ + +/-- Rebuild a natural number from `0` and `Nat.succ` so that `Function.update` closures in its +originating expression are eliminated during kernel reduction. -/ +def forceNat : ℕ → Option ℕ + | 0 => some 0 + | n + 1 => (forceNat n).map Nat.succ + +/-- Eagerly force every element of a list of natural numbers via `forceNat`. -/ +def forceList : List ℕ → Option (List ℕ) + | [] => some [] + | x :: xs => + match forceNat x, forceList xs with + | some y, some ys => some (y :: ys) + | _, _ => none + +/-- Tabulate registers `0..r - 1` and memory `0..memCap - 1` into flat lists between interpreter +calls so `Function.update` closures do not accumulate across steps. -/ +def compactData {r : ℕ} (memCap : ℕ) (d : Data r) : Option (Data r) := do + let rs ← forceList ((List.finRange r).map d.regs) + let ms ← forceList ((List.range memCap).map d.mem) + let tail ← forceNat (d.mem memCap) + pure + { regs := fun i => rs.getD i.val 0 + mem := fun a => if a < memCap then ms.getD a 0 else tail } + +/-! ## 1. `KWise`: concrete `Code.eval` execution matching unchunked trinomial Horner evaluation -/ + +/-- Structurally recursive fuel-bounded carryless multiplication for kernel `decide` reduction. -/ +def clmulFuel : ℕ → ℕ → ℕ → ℕ + | 0, _, _ => 0 + | fuel + 1, a, b => + if b = 0 then 0 + else (if b % 2 = 1 then a else 0) ^^^ (clmulFuel fuel a (b / 2) <<< 1) + +/-- `clmulFuel fuel a b` agrees with `clmul a b` whenever `b < 2 ^ fuel`. -/ +theorem clmulFuel_eq_clmul : ∀ (fuel a b : ℕ), b < 2 ^ fuel → clmulFuel fuel a b = clmul a b + | 0, a, b, hb => by + have hb0 : b = 0 := by omega + subst hb0 + simp [clmulFuel] + | fuel + 1, a, b, hb => by + rcases eq_or_ne b 0 with rfl | hne + · simp [clmulFuel] + · have hb_div : b / 2 < 2 ^ fuel := by + rw [Nat.pow_succ] at hb + omega + rw [clmulFuel, clmul, ite_eq_right hne, ite_eq_right hne, + clmulFuel_eq_clmul fuel a (b / 2) hb_div] + +/-- Sample `5`-tuple of coefficients in `Fin 4` (`t = 0`, `q = 2`, `2 ^ q = 4`). -/ +def sampleCoeffs5 : Fin 5 → Fin 4 + | ⟨0, _⟩ => ⟨3, by decide⟩ + | ⟨1, _⟩ => ⟨1, by decide⟩ + | ⟨2, _⟩ => ⟨2, by decide⟩ + | ⟨3, _⟩ => ⟨3, by decide⟩ + | ⟨_, _⟩ => ⟨2, by decide⟩ + +/-- Computable unchunked reference Horner evaluator for `k = 5` over `TrinomialField t` using +`clmulFuel 6` and `reduceTrinomialCode` directly (without limb splitting), masked to `M` bits. -/ +def trinomialHornerRef5 (t M : ℕ) (c : Fin 5 → Fin (2 ^ (2 * 3 ^ t))) (u : ℕ) : ℕ := + let a4 := (c ⟨4, by decide⟩).val + let a3 := reduceTrinomialCode t (clmulFuel 6 a4 u) ^^^ (c ⟨3, by decide⟩).val + let a2 := reduceTrinomialCode t (clmulFuel 6 a3 u) ^^^ (c ⟨2, by decide⟩).val + let a1 := reduceTrinomialCode t (clmulFuel 6 a2 u) ^^^ (c ⟨1, by decide⟩).val + let a0 := reduceTrinomialCode t (clmulFuel 6 a1 u) ^^^ (c ⟨0, by decide⟩).val + a0 &&& (2 ^ M - 1) + +/-- Write the `5` coefficients of `sampleCoeffs5` into `[base + 8, base + 13)` of `d.mem`. -/ +def writeSampleCoeffs5 (base : ℕ) (d : Data 18) : Data 18 := + ((((d.write (base + 8) 3).write (base + 9) 1).write (base + 10) 2).write + (base + 11) 3).write (base + 12) 2 + +/-- Run `KWise.initTableCode 5` with `(base, n_max, U, M)` in registers `0..3`, populate +`sampleCoeffs5`, and evaluate `KWise.evalCode 5` via `Code.eval` on a list of keys `us`, returning +the list of output hash values together with the final values at boundary sentinels `base - 1` and +`base + KWise.space n_max 5`. -/ +def runKWiseEval5 (w fuel n_max U M base : ℕ) (us : List ℕ) : + Option (List ℕ × ℕ × ℕ) := do + let sp := KWise.space n_max 5 + let d0 : Data 18 := + ((((((Data.init [] : Data 18).set 0 base).set 1 n_max).set 2 U).set 3 M).write + (base - 1) 77).write (base + sp) 88 + let (d1, _) ← (KWise.initTableCode 5).eval w fuel d0 + let d2 ← compactData (base + sp + 2) (writeSampleCoeffs5 base d1) + let outs ← us.mapM (fun u => do + let dIn := (d2.set 0 base).set KWise.regKey u + let (dOut, _) ← (KWise.evalCode 5).eval w fuel dIn + forceNat (dOut.regs KWise.regOut)) + let sLo ← forceNat (d2.mem (base - 1)) + let sHi ← forceNat (d2.mem (base + sp)) + pure (outs, sLo, sHi) + +/-- Concrete `Code.eval` test at `k = 5, b = 1, L = 2` (`n_max = 8`, `U = 4`, `M = 2`, `base = 10`): +`KWise.initTableCode 5` builds the `4`-entry carryless multiplication table and metadata in +`[10, 27)`, and `KWise.evalCode 5` (performing `2 × 2` multi-limb chunk multiplication at each of +the `5` Horner steps) matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys +`u ∈ [0, 1, 2, 3]` while preserving sentinels at addresses `9` (`77`) and `27` (`88`). -/ +theorem kwise_eval5_b1_concrete : + runKWiseEval5 16 16 8 4 2 10 [0, 1, 2, 3] = + some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by + decide +kernel + +/-- Concrete `Code.eval` test at `k = 5, b = 2, L = 1` (`n_max = 64`, `U = 4`, `M = 2`, `base = 4`): +`KWise.initTableCode 5` builds the `16`-entry carryless multiplication table in `[4, 33)`, and +`KWise.evalCode 5` matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys `u ∈ [0, 1, 2, 3]` +while preserving sentinels at addresses `3` (`77`) and `33` (`88`). -/ +theorem kwise_eval5_b2_concrete : + runKWiseEval5 16 32 64 4 2 4 [0, 1, 2, 3] = + some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by + decide +kernel + +/-- Sample `5`-tuple of coefficients in `Fin 64` (`t = 1`, `q = 6`, `2 ^ q = 64`) for testing +`k = 5` with `3 × 3` multi-limb multiplication over `TrinomialField 1` (`X ^ 6 + X ^ 3 + 1`). -/ +def sampleCoeffs5T1 : Fin 5 → Fin 64 + | ⟨0, _⟩ => ⟨13, by decide⟩ + | ⟨1, _⟩ => ⟨42, by decide⟩ + | ⟨2, _⟩ => ⟨7, by decide⟩ + | ⟨3, _⟩ => ⟨59, by decide⟩ + | ⟨_, _⟩ => ⟨25, by decide⟩ + +/-- Run `KWise.initTableCode 5` and `KWise.evalCode 5` at `t = 1, q = 6, b = 2, L = 3` +(`n_max = 64, U = 16, M = 3, base = 4`). -/ +def runKWiseEval5T1 (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := do + let base := 4 + let sp := KWise.space 64 5 + let d0 : Data 18 := + ((((((Data.init [] : Data 18).set 0 base).set 1 64).set 2 16).set 3 3).write + (base - 1) 77).write (base + sp) 88 + let (d1, _) ← (KWise.initTableCode 5).eval 16 32 d0 + let d2 ← compactData (base + sp + 2) + (((((d1.write (base + 8) 13).write (base + 9) 42).write (base + 10) 7).write + (base + 11) 59).write (base + 12) 25) + let outs ← us.mapM (fun u => do + let dIn := (d2.set 0 base).set KWise.regKey u + let (dOut, _) ← (KWise.evalCode 5).eval 16 32 dIn + forceNat (dOut.regs KWise.regOut)) + let sLo ← forceNat (d2.mem (base - 1)) + let sHi ← forceNat (d2.mem (base + sp)) + pure (outs, sLo, sHi) + +/-- Concrete `Code.eval` test at `k = 5, t = 1, q = 6, b = 2, L = 3` (`3 × 3` limb multiplication +over `X ^ 6 + X ^ 3 + 1`): `KWise.evalCode 5` matches `trinomialHornerRef5 1 3 sampleCoeffs5T1` on +keys `u ∈ [0, 5, 11, 15]`. -/ +theorem kwise_eval5_t1_b2_L3_concrete : + runKWiseEval5T1 [0, 5, 11, 15] = + some ([0, 5, 11, 15].map (trinomialHornerRef5 1 3 sampleCoeffs5T1), 77, 88) := by + decide +kernel + +/-- Non-vacuous theorem-level instantiation of `KWise.initTableCode_execWithin`, +`KWise.resampleCode_execRand`, `KWise.initCode_execRandWithin`, `KWise.evalCode_execWithin`, and +`KWise.isKWiseIndependentHash_eval` at `k = 5`, `U = 1000`, `M = 10`, `n_max = 64`, `w = 64`, +`base = 0`. -/ +theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) + (h0 : x.regs 0 = 0) (h1 : x.regs 1 = 64) (h2 : x.regs 2 = 1000) (h3 : x.regs 3 = 10) : + (∃ y : Data 18, + ExecWithin 64 (KWise.initTableCode 5) x y (KWise.initTableSteps 64 1000 10) ∧ + KWise.TableRep 0 (KWise.chunkBits 64) 1000 10 5 y.mem) ∧ + ExecRandWithin 64 (KWise.initCode 5) x + (fun y => + KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 + (KWise.extractSeeds 0 1000 10 5 y.mem) y.mem ∧ + y.regs 0 = 0 ∧ + (∀ a, (a < 0 ∨ 0 + KWise.space 64 5 ≤ a) → y.mem a = x.mem a)) + (KWise.initTableSteps 64 1000 10 + 35) 5 ∧ + (∀ y : Data 18, + KWise.TableRep 0 (KWise.chunkBits 64) 1000 10 5 y.mem → + y.regs 0 = 0 → + IsKWiseIndependentHash 5 (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10) ∧ + ExecRand 64 (KWise.resampleCode 5) y (KWise.resampleComp 0 1000 10 5 y) ∧ + ((KWise.resampleComp 0 1000 10 5 y).eval).map + (fun z => KWise.extractSeeds 0 1000 10 5 z.1.mem) = + PMF.uniformOfFintype (Fin 5 → Fin (2 ^ trinomialDeg 1000 10)) ∧ + ExecRandWithin 64 (KWise.resampleCode 5) y + (fun z => + KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 + (KWise.extractSeeds 0 1000 10 5 z.mem) z.mem ∧ + z.regs 0 = 0 ∧ + (∀ a, (a < 0 + 8 ∨ 0 + 8 + 5 ≤ a) → z.mem a = y.mem a) ∧ + (∀ u : Fin 1000, ∀ x_eval : Data 18, + x_eval.mem = z.mem → + x_eval.regs 0 = 0 → + x_eval.regs 1 = u.val → + ∃ y_eval : Data 18, + ExecWithin 64 (KWise.evalCode 5) x_eval y_eval (KWise.evalSteps 64 1000 10 5) ∧ + y_eval.regs 0 = 0 ∧ + y_eval.regs 1 = u.val ∧ + y_eval.regs 2 = + (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10 + (KWise.extractSeeds 0 1000 10 5 z.mem) u).val ∧ + y_eval.mem = z.mem)) + 35 5) := by + have htr : trinomialDeg 1000 10 = 18 := by + have hclog : Nat.clog 2 (max 1000 2) ≤ 10 := + (Nat.clog_le_iff_le_pow one_lt_two).mpr (by decide) + have ht : trinomialTargetBits 1000 10 = 10 := by + rw [trinomialTargetBits]; omega + have hs : trinomialStep 1000 10 = 2 := by + rw [trinomialStep, ht, trinomialStepAux, trinomialStepAux, trinomialStepAux]; decide + rw [trinomialDeg, hs]; decide + have hw : 2 * trinomialDeg 1000 10 ≤ 64 := by omega + have hfit_sp : Fits 64 (0 + KWise.space 64 5) := by + rw [fits_iff] + dsimp [KWise.space, KWise.spaceFromB, KWise.chunkBits] + decide + have hfit_n : Fits 64 (max 64 2) := by rw [fits_iff]; decide + have hfit_U : Fits 64 (max 1000 2 + 1) := by rw [fits_iff]; decide + have hfit_q : Fits 64 (2 ^ trinomialDeg 1000 10) := by + rw [htr, fits_iff]; decide + obtain ⟨y, hy_exec, hy_tab, -, -⟩ := + KWise.initTableCode_execWithin h0 h1 h2 h3 hfit_sp hfit_n hfit_U hfit_q + refine ⟨⟨y, hy_exec, hy_tab⟩, + KWise.initCode_execRandWithin h0 h1 h2 h3 hfit_sp hfit_n hfit_U hfit_q, + fun y' htab' h0' => + KWise.isKWiseIndependentHash_eval htab' h0' hw hfit_sp⟩ + +/-! ## 2. `RandomTable`: concrete `Code.eval` execution and theorem instantiations -/ + +/-- Populate `Z = [6, 3, 7, 1, 5, 0, 4, 2]` at `base = 4` (`M = 3`) and run +`RandomTable.lookupCode` and `RandomTable.xorLookupCode` via `Code.eval`. -/ +def runRandomTableLookups : Option (List ℕ × List ℕ) := do + let base := 4 + let d0 : Data 6 := + (((((((((Data.init [] : Data 6).write 4 6).write 5 3).write 6 7).write 7 1).write + 8 5).write 9 0).write 10 4).write 11 2) + let lookups ← ([0, 2, 4, 7] : List ℕ).mapM (fun idx => do + let dIn := (d0.set RandomTable.regBase base).set RandomTable.regIdx idx + let (dOut, _) ← RandomTable.lookupCode.eval 16 8 dIn + forceNat (dOut.regs RandomTable.regOut)) + let xorLookups ← ([(5, 0), (5, 2), (3, 4), (7, 7)] : List (ℕ × ℕ)).mapM (fun (f, g) => do + let dIn := ((d0.set RandomTable.regBase base).set RandomTable.regIdx g).set + RandomTable.regOut f + let (dOut, _) ← RandomTable.xorLookupCode.eval 16 8 dIn + forceNat (dOut.regs RandomTable.regOut)) + pure (lookups, xorLookups) + +/-- Concrete `Code.eval` verification of `RandomTable.lookupCode` (`Z[idx]`) and +`RandomTable.xorLookupCode` (`f ^^^ Z[g]`). -/ +theorem randomTable_lookup_and_xorLookup_concrete : + runRandomTableLookups = + some ([6, 7, 5, 2], [5 ^^^ 6, 5 ^^^ 7, 3 ^^^ 5, 7 ^^^ 2]) := by + decide + +/-- Non-vacuous theorem-level instantiation of `RandomTable.resampleCode_execRand`, +`RandomTable.map_extractTable_eval_resampleComp`, `RandomTable.resampleCode_execRandWithin`, +`RandomTable.lookupCode_exec`, and `RandomTable.xorLookupCode_exec` at `M = 4` (`2 ^ M = 16`), +`base = 10`, `w = 16`. -/ +theorem randomTable_wordram_nonvacuous (x : Data 6) + (h0 : x.regs RandomTable.regBase = 10) (h1 : x.regs RandomTable.regM = 4) : + ExecRand 16 RandomTable.resampleCode x (RandomTable.resampleComp 10 4 x) ∧ + ((RandomTable.resampleComp 10 4 x).eval).map (fun z => RandomTable.extractTable 10 4 z.1.mem) = + PMF.uniformOfFintype (Fin 16 → Fin 16) ∧ + ExecRandWithin 16 RandomTable.resampleCode x + (fun y => + RandomTable.Rep 10 4 (RandomTable.extractTable 10 4 y.mem) y.mem ∧ + y.regs RandomTable.regBase = 10 ∧ + (∀ a, (a < 10 ∨ 10 + 16 ≤ a) → y.mem a = x.mem a)) + 87 16 ∧ + (∀ (Z : Fin 16 → Fin 16) (idx : Fin 16) (y : Data 6), + RandomTable.Rep 10 4 Z y.mem → + y.regs RandomTable.regBase = 10 → y.regs RandomTable.regIdx = idx.val → + Exec 16 RandomTable.lookupCode y (RandomTable.lookupData y) 2 ∧ + (RandomTable.lookupData y).regs RandomTable.regOut = (Z idx).val ∧ + (RandomTable.lookupData y).mem = y.mem) ∧ + (∀ (Z : Fin 16 → Fin 16) (idx fx : Fin 16) (y : Data 6), + RandomTable.Rep 10 4 Z y.mem → + y.regs RandomTable.regBase = 10 → y.regs RandomTable.regIdx = idx.val → + y.regs RandomTable.regOut = fx.val → + Exec 16 RandomTable.xorLookupCode y (RandomTable.xorLookupData y) 3 ∧ + (RandomTable.xorLookupData y).regs RandomTable.regOut = fx.val ^^^ (Z idx).val ∧ + (RandomTable.xorLookupData y).mem = y.mem) := by + have hfit_end : Fits 16 (10 + 2 ^ 4) := by rw [fits_iff]; decide + have hfit_pow : Fits 16 (2 ^ 4) := by rw [fits_iff]; decide + refine ⟨RandomTable.resampleCode_execRand h0 h1 hfit_end hfit_pow, + RandomTable.map_extractTable_eval_resampleComp 10 4 x, + RandomTable.resampleCode_execRandWithin h0 h1 hfit_end hfit_pow, + fun Z idx y hrep hy0 hy1 => ?_, + fun Z idx fx y hrep hy0 hy1 hy2 => ?_⟩ + · obtain ⟨he, ho, -, hm⟩ := RandomTable.lookupCode_exec hrep idx hy0 hy1 hfit_end + exact ⟨he, ho, hm⟩ + · obtain ⟨he, ho, -, -, hm⟩ := + RandomTable.xorLookupCode_exec hrep idx fx hy0 hy1 hy2 hfit_end hfit_pow + exact ⟨he, ho, hm⟩ + +/-! ## 3. `TabulatedOddStride`: concrete `Code.eval` execution and full-cycle permutation test -/ + +/-- Sample flat `3 * 2 ^ 1 = 6`-entry character table `T : Fin 6 → Fin 8` (`d = 3`, `b = 1`, +`M = 3`). -/ +def sampleFlatCharTable : Fin 6 → Fin 8 + | ⟨0, _⟩ => ⟨1, by decide⟩ + | ⟨1, _⟩ => ⟨3, by decide⟩ + | ⟨2, _⟩ => ⟨2, by decide⟩ + | ⟨3, _⟩ => ⟨0, by decide⟩ + | ⟨4, _⟩ => ⟨3, by decide⟩ + | ⟨_, _⟩ => ⟨2, by decide⟩ + +/-- Populate the 4 header words `(b = 1, 2 ^ b = 2, 2 ^ b - 1 = 1, 2 ^ M - 1 = 7)` and the 6 +entries of `sampleFlatCharTable` starting at `base = 2` (addresses `2..11`) and run +`TabulatedOddStride.evalCode 3` via `Code.eval` on all keys `u ∈ 0..7`. -/ +def runTabulatedOddStrideEval : Option (List ℕ) := do + let base := 2 + let d0 : Data 9 := + (((((((((((Data.init [] : Data 9).write 2 1).write 3 2).write 4 1).write 5 7).write + 6 1).write 7 3).write 8 2).write 9 0).write 10 3).write 11 2) + (List.range 8).mapM (fun u => do + let dIn := (d0.set TabulatedOddStride.regBase base).set TabulatedOddStride.regKey u + let (dOut, _) ← (TabulatedOddStride.evalCode 3).eval 16 16 dIn + forceNat (dOut.regs TabulatedOddStride.regOut)) + +/-- Concrete `Code.eval` verification of `TabulatedOddStride.evalCode 3`: on all `8` keys +`u ∈ 0..7`, `evalCode 3` matches +`(TabulatedOddStride.tabulatedOddStride 1 3 3 sampleFlatCharTable u).val` and always produces an odd +stride in `{1, 3, 5, 7}`. -/ +theorem tabulatedOddStride_evalCode_matches_spec : + runTabulatedOddStrideEval = + some ((List.range 8).map (fun u => + (TabulatedOddStride.tabulatedOddStride 1 3 3 sampleFlatCharTable u).val)) ∧ + ∀ u : ℕ, + (TabulatedOddStride.tabulatedOddStride 1 3 3 sampleFlatCharTable u).val % 2 = 1 := by + refine ⟨by decide, fun u => TabulatedOddStride.odd_oddStride (by decide) _⟩ + +/-- Full-cycle permutation property on `Fin 8` (`M = 3`): for every starting slot `h₁ : ℕ` and +every raw tabulation sum `tab : ℕ`, the linear probe sequence `i ↦ probeSeq 3 h₁ tab i` is a +bijection on `Fin 8` (and concretely visits all `8` slots without repetition). -/ +theorem tabulatedOddStride_probeSeq_full_cycle (h₁ tab : ℕ) : + Function.Bijective (TabulatedOddStride.probeSeq 3 h₁ tab) ∧ + ((List.finRange 8).map (fun i => + (TabulatedOddStride.probeSeq 3 3 + (TabulatedOddStride.simpleTab 1 3 3 sampleFlatCharTable 5).val i).val)).Nodup := + ⟨TabulatedOddStride.bijective_probeSeq 3 h₁ tab, by decide⟩ + +/-- Non-vacuous theorem-level instantiation of `TabulatedOddStride.initCode_execRand`, +`TabulatedOddStride.map_extractTable_eval_initComp`, `TabulatedOddStride.initCode_execRandWithin`, +and `TabulatedOddStride.evalCode_execWithin` at `n_max = 16`, `d = 4`, `M = 5` (`2 ^ M = 32`), +`base = 0`, `w = 16`. -/ +theorem tabulatedOddStride_wordram_nonvacuous (x : Data 9) + (h0 : x.regs TabulatedOddStride.regBase = 0) + (h1 : x.regs TabulatedOddStride.regKey = 16) + (h2 : x.regs TabulatedOddStride.regOut = 5) : + ExecRand 16 (TabulatedOddStride.initCode 4) x + (TabulatedOddStride.initComp 0 16 4 5 x) ∧ + ((TabulatedOddStride.initComp 0 16 4 5 x).eval).map + (fun z => TabulatedOddStride.extractTable 0 (TabulatedOddStride.charBits 16) 4 5 z.1.mem) = + PMF.uniformOfFintype (Fin (4 * 2 ^ TabulatedOddStride.charBits 16) → Fin 32) ∧ + ExecRandWithin 16 (TabulatedOddStride.initCode 4) x + (fun y => + TabulatedOddStride.Rep 0 (TabulatedOddStride.charBits 16) 4 5 + (TabulatedOddStride.extractTable 0 (TabulatedOddStride.charBits 16) 4 5 y.mem) y.mem ∧ + y.regs TabulatedOddStride.regBase = 0 ∧ + (∀ a, (a < 0 ∨ 0 + TabulatedOddStride.space 16 4 ≤ a) → y.mem a = x.mem a)) + (4 * Nat.log 2 (max 16 2) + 5 * (4 * 2 ^ TabulatedOddStride.charBits 16) + 30) + (4 * 2 ^ TabulatedOddStride.charBits 16) ∧ + (∀ (T : Fin (4 * 2 ^ TabulatedOddStride.charBits 16) → Fin 32) (u : Fin 256) (y : Data 9), + TabulatedOddStride.Rep 0 (TabulatedOddStride.charBits 16) 4 5 T y.mem → + y.regs TabulatedOddStride.regBase = 0 → + y.regs TabulatedOddStride.regKey = u.val → + ∃ z : Data 9, + ExecWithin 16 (TabulatedOddStride.evalCode 4) y z 53 ∧ + z.regs TabulatedOddStride.regOut = + (TabulatedOddStride.tabulatedOddStride + (TabulatedOddStride.charBits 16) 4 5 T u.val).val ∧ + z.regs TabulatedOddStride.regBase = 0 ∧ + z.mem = y.mem) := by + have hfit_n : Fits 16 (max 16 2) := by rw [fits_iff]; decide + have hfit_sp : Fits 16 (0 + TabulatedOddStride.space 16 4) := by + rw [fits_iff] + dsimp [TabulatedOddStride.space, TabulatedOddStride.spaceFromB, TabulatedOddStride.charBits] + decide + have hfit_powM : Fits 16 (2 ^ 5) := by rw [fits_iff]; decide + have hfit_2powM : Fits 16 (2 * 2 ^ 5) := by rw [fits_iff]; decide + have hfit_d : Fits 16 4 := by rw [fits_iff]; decide + refine ⟨TabulatedOddStride.initCode_execRand h0 h1 h2 hfit_n hfit_sp hfit_powM hfit_d, + TabulatedOddStride.map_extractTable_eval_initComp 0 16 4 5 x, + TabulatedOddStride.initCode_execRandWithin h0 h1 h2 hfit_n hfit_sp hfit_powM hfit_d, + fun T u y hrep hy0 hy1 => ?_⟩ + have hfit_u : Fits 16 u.val := by + rw [fits_iff] + have := u.isLt + omega + exact TabulatedOddStride.evalCode_execWithin hrep hy0 hy1 hfit_sp hfit_2powM hfit_d hfit_u + +end ComplexityTests.WordRAMKWiseHash + +end From 8f23b1138509fb3422afa61008c99d590a3e81cd Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Fri, 30 Oct 2026 00:33:43 +0000 Subject: [PATCH 16/18] fix(WordRAM/Hashing): compose isKWiseIndependentHash_eval and extract ExactLog --- Complexity.lean | 1 + Complexity/Algorithms/WordRAM/ExactLog.lean | 239 +++++++++++ .../Algorithms/WordRAM/Hashing/KWise.lean | 384 +++++++++--------- .../WordRAM/Hashing/Tabulation.lean | 155 ++----- ComplexityTests/WordRAMKWiseHash.lean | 99 +++-- 5 files changed, 503 insertions(+), 375 deletions(-) create mode 100644 Complexity/Algorithms/WordRAM/ExactLog.lean diff --git a/Complexity.lean b/Complexity.lean index 34622b270..5cc6274c0 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -89,6 +89,7 @@ public import Complexity.Algorithms.WordRAM.Dijkstra.Statement public import Complexity.Algorithms.WordRAM.DynamicArray public import Complexity.Algorithms.WordRAM.ElementDistinctness public import Complexity.Algorithms.WordRAM.ElementDistinctnessRAM +public import Complexity.Algorithms.WordRAM.ExactLog public import Complexity.Algorithms.WordRAM.ExactTriangle.Catalogue public import Complexity.Algorithms.WordRAM.ExactTriangle.ChunkCatalogue public import Complexity.Algorithms.WordRAM.ExactTriangle.Chunks diff --git a/Complexity/Algorithms/WordRAM/ExactLog.lean b/Complexity/Algorithms/WordRAM/ExactLog.lean new file mode 100644 index 000000000..adbfb2226 --- /dev/null +++ b/Complexity/Algorithms/WordRAM/ExactLog.lean @@ -0,0 +1,239 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Computability.WordRAM.Structured +public import Mathlib.Data.Nat.Log +import Mathlib.Tactic.FinCases + +/-! +# Exact-basis Word-RAM binary logarithms and arithmetic helpers + +This file provides structured subprograms over `Cslib.WordRAM.Basis.exact` for floor and ceiling +base-`2` logarithms (`log2Loop` and `clog2Loop`), an equal-cost register-maximum gadget (`maxCode`), +and an `.exactBin` execution helper (`exec_exactBin`). + +Whereas `Cslib.Algorithms.WordRAM.NatLog` and `Cslib.Algorithms.WordRAM.NatClog` are +`Code .standard 4` programs with `ExecWithin` upper bounds, the subprograms here operate in the +`.exact` basis using right-shift division by `2` (`>>> 1`) and carry exact closed-form `Exec` step +counts (`4 * Nat.log 2 rem + 2` and `5 * Nat.clog 2 rem + 2`) together with explicit final-state +formulas (`log2LoopData` and `clog2LoopData`) for embedding via `Code.rename`. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.ExactLog.maxCode`: 3-instruction equal-cost branch computing the + maximum of two registers. +* `Cslib.Algorithms.WordRAM.ExactLog.log2Body`, + `Cslib.Algorithms.WordRAM.ExactLog.log2Loop`, + `Cslib.Algorithms.WordRAM.ExactLog.log2LoopData`: floor-`log₂` loop over `Basis.exact`. +* `Cslib.Algorithms.WordRAM.ExactLog.clog2Body`, + `Cslib.Algorithms.WordRAM.ExactLog.clog2Loop`, + `Cslib.Algorithms.WordRAM.ExactLog.clog2LoopData`: ceiling-`log₂` loop over `Basis.exact`. + +## Main results + +* `Cslib.Algorithms.WordRAM.ExactLog.exec_exactBin`: single-instruction execution of `.exactBin`. +* `Cslib.Algorithms.WordRAM.ExactLog.maxCode_exec`: 3-instruction execution of `maxCode`. +* `Cslib.Algorithms.WordRAM.ExactLog.log2Loop_exec`: exact `4 * Nat.log 2 rem + 2`-instruction + execution of `log2Loop`. +* `Cslib.Algorithms.WordRAM.ExactLog.clog2Loop_exec`: exact `5 * Nat.clog 2 rem + 2`-instruction + execution of `clog2Loop`. +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.ExactLog + +open Cslib.WordRAM + +/-- Execute a single `.exactBin` operation when `evalExact` returns `some v` and `Fits w v`. -/ +theorem exec_exactBin {w r : ℕ} {f : BinOp} {dst i j : Fin r} {d : Data r} {v : ℕ} + (hev : f.evalExact (d.regs i) (d.regs j) = some v) (hfit : Fits w v) : + Exec w (.op (.exactBin f dst i j) : Code .exact r) d (d.set dst v) 1 := by + apply Exec.op + rw [Op.eval_exactBin, hev] + exact Data.put_of_fits hfit + +/-- 3-instruction equal-cost branch setting register `dst` to `max (x.regs a) (x.regs b)`. -/ +@[expose] def maxCode {n : ℕ} (dst a b : Fin n) : Code .exact n := + .branch (.lt a b) (.seq (.op (.copy dst b)) (.op (.copy dst dst))) (.op (.copy dst a)) + +/-- `maxCode dst a b` executes deterministically in 3 instructions and sets `dst` to +`max (x.regs a) (x.regs b)`. -/ +theorem maxCode_exec {w n : ℕ} (dst a b : Fin n) (x : Data n) : + Exec w (maxCode dst a b) x (x.set dst (max (x.regs a) (x.regs b))) 3 := by + by_cases hlt : x.regs a < x.regs b + · have hc : (Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs b := Nat.max_eq_right (le_of_lt hlt) + rw [hmax] + set d1 := x.set dst (x.regs b) + have s1 : Exec w (.op (.copy dst b) : Code .exact n) x d1 1 := Exec.copy + have hd1 : d1.set dst (d1.regs dst) = d1 := by + ext i : 2 <;> simp [d1, Data.set] + have s2 : Exec w (.op (.copy dst dst) : Code .exact n) d1 d1 1 := by + simpa only [hd1] using (Exec.copy (B := .exact) (w := w) (d := dst) (s := dst) (x := d1)) + exact Exec.branch_pos hc (s1.seq s2) + · have hc : ¬(Cond.lt a b).Holds x := hlt + have hmax : max (x.regs a) (x.regs b) = x.regs a := Nat.max_eq_left (not_lt.mp hlt) + rw [hmax] + exact Exec.branch_neg hc (Exec.copy (B := .exact)) + +/-- One iteration of the floor-`log₂` loop on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, +`reg 2 = 1`): shifts `rem` right by `1` (`rem / 2`) and increments `acc`. -/ +@[expose] def log2Body : Code .exact 3 := + .seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2)) + +/-- Floor-`log₂` loop `while 1 < rem do [ rem ← rem >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def log2Loop : Code .exact 3 := + .loop (.lt 2 0) log2Body + +/-- Closed-form final state of `log2Loop` starting from `rem` in register `0` and `acc` in +register `1`. -/ +@[expose] def log2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.log 2 rem) + +/-- `log2Loop` executes deterministically in `4 * Nat.log 2 rem + 2` instructions, accumulating +`acc + Nat.log 2 rem` into register `1` and preserving all memory. -/ +theorem log2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem : Fits w rem) (hfit_acc : Fits w (acc + Nat.log 2 rem)) : + Exec w log2Loop x (log2LoopData rem acc x) (4 * Nat.log 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : rem >>> 1 = rem / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hlog : Nat.log 2 rem = Nat.log 2 (rem / 2) + 1 := + Nat.log_of_one_lt_of_le (by decide) (by omega) + have hfit_div : Fits w (rem / 2) := hfit_rem.of_le (Nat.div_le_self rem 2) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem / 2) + have s1 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone, hshr]) hfit_div + set d2 := d1.set 1 (acc + 1) + have s2 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hacc, hone]) hfit_acc1 + have hdiv_lt : rem / 2 < rem := Nat.div_lt_self (by omega) (by decide) + have hrec := ih (rem / 2) hdiv_lt (acc := acc + 1) (x := d2) + (by simp +decide [d2, d1]) + (by simp +decide [d2]) + (by simp +decide [d2, d1, hone]) + hfit_div + (by rwa [show acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem by omega]) + have hd : log2LoopData (rem / 2) (acc + 1) d2 = log2LoopData rem acc x := by + have hdiv_ne : rem / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem + omega + · simp +decide [log2LoopData, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq s2) hrec + simpa only [log2Loop, log2Body, hlog, show 1 + 1 + (4 * Nat.log 2 (rem / 2) + 2) + 2 = + 4 * (Nat.log 2 (rem / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hlog0 : Nat.log 2 rem = 0 := Nat.log_of_lt (by omega) + have hd : log2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by + split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [log2LoopData, hrem_eq, hrem] + · simp +decide [log2LoopData, hlog0, hacc] + · simp +decide [log2LoopData] + · rfl + rw [hd, hlog0] + exact Exec.loop_exit hc + +/-- Body of `clog2Loop` on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, `reg 2 = 1`): updates +`rem ← (rem + 1) >>> 1` (`= (rem + 1) / 2`) and `acc ← acc + 1`. -/ +@[expose] def clog2Body : Code .exact 3 := + .seq (.op (.exactBin .add 0 0 2)) + (.seq (.op (.exactBin .shiftRight 0 0 2)) + (.op (.exactBin .add 1 1 2))) + +/-- Ceiling-`log₂` loop `while 1 < rem do [ rem ← (rem + 1) >>> 1; acc ← acc + 1 ]`. -/ +@[expose] def clog2Loop : Code .exact 3 := + .loop (.lt 2 0) clog2Body + +/-- Closed-form final state of `clog2Loop`. -/ +@[expose] def clog2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := + (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.clog 2 rem) + +/-- `clog2Loop` executes deterministically in `5 * Nat.clog 2 rem + 2` instructions, accumulating +`acc + Nat.clog 2 rem` into register `1` and preserving all memory. -/ +theorem clog2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} + (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) + (hfit_rem1 : Fits w (rem + 1)) (hfit_acc : Fits w (acc + Nat.clog 2 rem)) : + Exec w clog2Loop x (clog2LoopData rem acc x) (5 * Nat.clog 2 rem + 2) := by + induction rem using Nat.strong_induction_on generalizing acc x with + | h rem ih => + by_cases hlt : 1 < rem + · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hshr : (rem + 1) >>> 1 = (rem + 1) / 2 := by + rw [Nat.shiftRight_eq_div_pow] + have hclog : Nat.clog 2 rem = Nat.clog 2 ((rem + 1) / 2) + 1 := + Nat.clog_of_one_lt (by decide) hlt + have hdiv_lt : (rem + 1) / 2 < rem := by omega + have hfit_div1 : Fits w ((rem + 1) / 2 + 1) := hfit_rem1.of_le (by omega) + have hfit_div : Fits w ((rem + 1) / 2) := hfit_div1.of_le (by omega) + have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) + set d1 := x.set 0 (rem + 1) + have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 3) x d1 1 := + exec_exactBin (by simp [hrem, hone]) hfit_rem1 + set d2 := d1.set 0 ((rem + 1) / 2) + have s2 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) d1 d2 1 := + exec_exactBin (by simp +decide [d1, hone, hshr]) hfit_div + set d3 := d2.set 1 (acc + 1) + have s3 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d2 d3 1 := + exec_exactBin (by simp +decide [d2, d1, hacc, hone]) hfit_acc1 + have hrec := ih ((rem + 1) / 2) hdiv_lt (acc := acc + 1) (x := d3) + (by simp +decide [d3, d2]) + (by simp +decide [d3]) + (by simp +decide [d3, d2, d1, hone]) + hfit_div1 + (by rwa [show acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem by omega]) + have hd : clog2LoopData ((rem + 1) / 2) (acc + 1) d3 = clog2LoopData rem acc x := by + have hdiv_ne : (rem + 1) / 2 ≠ 0 := by omega + have hrem_ne : rem ≠ 0 := by omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, d3, d2, d1, hdiv_ne, hrem_ne] + · change acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem + omega + · simp +decide [clog2LoopData, d3, d2, d1] + · rfl + rw [hd] at hrec + have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec + simpa only [clog2Loop, clog2Body, hclog, + show 1 + (1 + 1) + (5 * Nat.clog 2 ((rem + 1) / 2) + 2) + 2 = + 5 * (Nat.clog 2 ((rem + 1) / 2) + 1) + 2 by omega] using hiter + · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by + simp [Cond.Holds, hrem, hone, hlt] + have hclog0 : Nat.clog 2 rem = 0 := Nat.clog_of_right_le_one (by omega) 2 + have hd : clog2LoopData rem acc x = x := by + have hrem_eq : (if rem = 0 then 0 else 1) = rem := by split_ifs <;> omega + ext i : 2 + · fin_cases i + · simp +decide [clog2LoopData, hrem_eq, hrem] + · simp +decide [clog2LoopData, hclog0, hacc] + · simp +decide [clog2LoopData] + · rfl + rw [hd, hclog0] + exact Exec.loop_exit hc + +end Cslib.Algorithms.WordRAM.ExactLog diff --git a/Complexity/Algorithms/WordRAM/Hashing/KWise.lean b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean index 3697a633c..95df6c383 100644 --- a/Complexity/Algorithms/WordRAM/Hashing/KWise.lean +++ b/Complexity/Algorithms/WordRAM/Hashing/KWise.lean @@ -7,9 +7,11 @@ Authors: Renfei Zhou module -- upstream: candidate +public import Complexity.Algorithms.WordRAM.ExactLog public import Complexity.Algorithms.WordRAM.Hashing.Tabulation public import Complexity.Foundations.FieldTheory.Finite.Trinomial public import Complexity.Probability.KWiseIndependent.Trinomial +import Complexity.Probability.PMF.Map /-! # Self-contained Word-RAM `k`-wise independent hashing @@ -19,29 +21,35 @@ This file implements self-contained Word-RAM `k`-wise independent hashing over finite field $\mathbb{F}_{2^{2 \cdot 3^t}} = \mathbb{F}_2[X] / (X^{2 \cdot 3^t} + X^{3^t} + 1)$ (`Cslib.TrinomialField t`, where `t = trinomialStep U M` is chosen at runtime so that $q = 2 \cdot 3^t \ge \max(\lceil \log_2 \max(U, 2) \rceil, \max(M, 1))$) and projecting to the low -`M` bits (`Cslib.Probability.trinomialPolyHash`) [Carter and Wegman][CarterWegman1979]; -[Wegman and Carter][WegmanCarter1981]; [Lidl and Niederreiter][LidlNiederreiter1997]. - -To achieve `O(1)` Word-RAM multiplication in $\mathbb{F}_{2^q}$ without dedicated carryless -multiplication hardware when $U \le n_{\max}^c$, we precompute the Four-Russians carryless -multiplication lookup table `clmulTab[0 .. 2 ^ (2 * b) - 1]` for $b$-bit chunks -(`b = chunkBits n_max = max 1 (⌊log₂ (max n_max 2)⌋ / 3)`, so $2^{2b} \le \max(4, n_{\max}) = -O(n_{\max}^{2/3}) \le O(n_{\max})$ words), decompose each $q$-bit field element into -$L = \lceil q / b \rceil$ limbs of $b$ bits, accumulate the $L \times L$ chunk carryless products, -and fold the resulting $< 2^{2q}$ polynomial mod $X^{2r} + X^r + 1$ in 8 straight-line bit -operations (`reduceTrinomialCode`). +`M` bits (`Cslib.Probability.trinomialPolyHash`). The binary-field polynomial construction and +bit-prefix projection follow [Vadhan, Section 3.5.5, Construction 3.32 / Proposition 3.33 and +Corollary 3.34][Vadhan2012] and [Wegman and Carter][WegmanCarter1981], building on the universal +hashing framework of [Carter and Wegman, pp. 144–145][CarterWegman1979] and finite-field arithmetic +in [Lidl and Niederreiter][LidlNiederreiter1997]. + +Because `Basis.exact` lacks a hardware carryless-multiplication instruction, `initTableCode k` +precomputes a Four-Russians carryless multiplication table `clmulTab[0 .. 2 ^ (2 * b) - 1]` for +$b$-bit chunks (`b = chunkBits n_max = max 1 (⌊log₂ (max n_max 2)⌋ / 3)`). This table uses at most +$\max(4, n_{\max})$ words for all $n_{\max}$ (`two_pow_two_mul_chunkBits_le`) and satisfies the +sublinear cube bound $(2^{2b})^3 \le n_{\max}^2$, i.e., $2^{2b} \le n_{\max}^{2/3}$, whenever +$n_{\max} \ge 8$ (`two_pow_two_mul_chunkBits_pow_three_le`). Each $q$-bit field multiplication +splits its operands into $L = \lceil q / b \rceil$ limbs of $b$ bits, accumulates the $L \times L$ +chunk products from `clmulTab`, and reduces the resulting $< 2^{2q}$ polynomial modulo +$X^{2r} + X^r + 1$ in 8 bit operations (`reduceTrinomialCode`). Under the closed-form bound +`evalSteps n_max U M k = ((11 * L + 12) * L + 30) * k + 19`, each field multiplication takes +$O(L^2)$ instructions, which is $O(1)$ instructions when both $U \le n_{\max}^c$ and +$M \le c \log_2 n_{\max}$ for a fixed constant $c$. The initialization is factored into two reusable stages: 1. **`initTableCode k`**: deterministically computes $r = 3^t$, $q = 2r$, $b$, masks, and populates - `clmulTab[0 .. 2 ^ (2 * b) - 1]` in $O(n_{\max})$ deterministic instructions from - `(base, n_max, U, M)` alone (zero external primes or lookup hints). + `clmulTab[0 .. 2 ^ (2 * b) - 1]` in `initTableSteps n_max U M` + ($= O(n_{\max} + \log U + \log M)$) deterministic instructions from `(base, n_max, U, M)` alone. 2. **`resampleCode k`**: samples the $k$ polynomial coefficients `seed[0 .. k - 1]` uniformly and - independently in `Fin (2 ^ q)` using $k$ `Code.rand` calls in $O(k)$ instructions while - preserving `clmulTab`, so Cuckoo Hashing can rehash in $O(k) = O(1)$ instructions without - rebuilding `clmulTab`. + independently in `Fin (2 ^ q)` using $k$ `Code.rand` calls in $5k + 10 = O(k)$ instructions while + preserving `clmulTab`, so rehashing takes $O(k)$ instructions without rebuilding `clmulTab`. 3. **`initCode k := .seq (initTableCode k) (resampleCode k)`**: full one-shot initialization. -4. **`evalCode k`**: evaluates Horner's rule for `trinomialPolyHash` in $O(k \cdot L^2 + 1)$ - deterministic instructions, preserving all memory and writing +4. **`evalCode k`**: evaluates Horner's rule for `trinomialPolyHash` in `evalSteps n_max U M k` + ($= O(k \cdot L^2 + 1)$) deterministic instructions, preserving all memory and writing `(trinomialPolyHash (trinomialStep U M) k U M c u).val` into `regOut`. ## Main definitions @@ -56,25 +64,33 @@ The initialization is factored into two reusable stages: * `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode`, `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode`, `Cslib.Algorithms.WordRAM.Hashing.KWise.initCode`, - `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode`: structured Word-RAM programs over - `Basis.exact`. + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalHashOut`: structured Word-RAM programs over + `Basis.exact` and the induced deterministic hash output map. ## Main results -* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode_execWithin`: deterministic $O(n_{\max})$ - table construction establishing `TableRep`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.two_pow_two_mul_chunkBits_le`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.two_pow_two_mul_chunkBits_pow_three_le`: coarse + $\max(4, n_{\max})$ and sublinear $n_{\max}^{2/3}$ bounds on the Four-Russians table size. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.initTableCode_execWithin`: deterministic + `initTableSteps n_max U M` ($= O(n_{\max} + \log U + \log M)$) table construction establishing + `TableRep`. * `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRand`, `Cslib.Algorithms.WordRAM.Hashing.KWise.map_extractSeeds_eval_resampleComp`, - `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRandWithin`: $O(k)$-step, $k$-query seed - resampling preserving `TableRep` and inducing `PMF.uniformOfFintype (Fin k → Fin (2 ^ q))`. -* `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode_execWithin`: deterministic $O(k \cdot L^2 + 1)$ - Horner evaluation matching `trinomialPolyHash`. -* `Cslib.Algorithms.WordRAM.Hashing.KWise.isKWiseIndependentHash_eval`: end-to-end Word-RAM - $k$-wise independence theorem combining `resampleComp`, `evalCode_execWithin`, and - `isKWiseIndependentHash_trinomialPolyHash`. + `Cslib.Algorithms.WordRAM.Hashing.KWise.resampleCode_execRandWithin`: $(5k + 10)$-step, $k$-query + seed resampling preserving `TableRep` and inducing `PMF.uniformOfFintype (Fin k → Fin (2 ^ q))`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.evalCode_execWithin`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.evalHashOut_eq_trinomialPolyHash`: deterministic + `evalSteps n_max U M k` ($= O(k \cdot L^2 + 1)$) Horner evaluation matching `trinomialPolyHash`. +* `Cslib.Algorithms.WordRAM.Hashing.KWise.map_evalHashOut_eval_resampleComp`, + `Cslib.Algorithms.WordRAM.Hashing.KWise.isKWiseIndependentHash_eval`: composed Word-RAM $k$-wise + independence theorem proving that for every set $T$ of at most $k$ keys, the restricted tuple of + `evalCode k` outputs after `resampleCode k` is uniformly distributed on $T \to \text{Fin}(2^M)$. ## References +* [S. P. Vadhan, *Pseudorandomness*, Section 3.5.5][Vadhan2012] * [J. L. Carter and M. N. Wegman, *Universal Classes of Hash Functions*][CarterWegman1979] * [M. N. Wegman and J. L. Carter, *New Hash Functions and Their Use in Authentication and Set Equality*][WegmanCarter1981] @@ -87,6 +103,7 @@ public section namespace Cslib.Algorithms.WordRAM.Hashing.KWise open Cslib.WordRAM Cslib.Probability +open Cslib.Algorithms.WordRAM.ExactLog open Cslib.Algorithms.WordRAM.Hashing.Tabulation open scoped ENNReal @@ -106,7 +123,7 @@ theorem chunkBits_pos (n_max : ℕ) : 0 < chunkBits n_max := one_le_chunkBits n_max /-- The Four-Russians carryless multiplication table size `2 ^ (2 * chunkBits n_max)` is at most -`max 4 n_max` (`= O(n_max^(2/3)) ≤ O(n_max)` words). -/ +`max 4 n_max` for all `n_max`. -/ theorem two_pow_two_mul_chunkBits_le (n_max : ℕ) : 2 ^ (2 * chunkBits n_max) ≤ max 4 n_max := by by_cases hdiv : Nat.log 2 (max n_max 2) / 3 ≤ 1 @@ -121,6 +138,24 @@ theorem two_pow_two_mul_chunkBits_le (n_max : ℕ) : _ ≤ max n_max 2 := Nat.pow_log_le_self 2 (by omega) _ ≤ max 4 n_max := by omega +/-- For `8 ≤ n_max`, the Four-Russians table size `2 ^ (2 * chunkBits n_max)` satisfies the +sublinear cube bound `(2 ^ (2 * chunkBits n_max)) ^ 3 ≤ n_max ^ 2` (i.e., +`2 ^ (2 * chunkBits n_max) ≤ n_max ^ (2 / 3)`). -/ +theorem two_pow_two_mul_chunkBits_pow_three_le {n_max : ℕ} (hn : 8 ≤ n_max) : + (2 ^ (2 * chunkBits n_max)) ^ 3 ≤ n_max ^ 2 := by + have hmax : max n_max 2 = n_max := Nat.max_eq_left (by omega) + have hlog3 : 3 ≤ Nat.log 2 n_max := + Nat.le_log_of_pow_le (by decide) (by omega) + have hcb : chunkBits n_max = Nat.log 2 n_max / 3 := by + simp only [chunkBits, hmax] + omega + have hexp : 2 * chunkBits n_max * 3 ≤ Nat.log 2 n_max * 2 := by omega + calc + (2 ^ (2 * chunkBits n_max)) ^ 3 = 2 ^ (2 * chunkBits n_max * 3) := (pow_mul 2 _ 3).symm + _ ≤ 2 ^ (Nat.log 2 n_max * 2) := Nat.pow_le_pow_right (by decide) hexp + _ = (2 ^ Nat.log 2 n_max) ^ 2 := pow_mul 2 _ 2 + _ ≤ n_max ^ 2 := Nat.pow_le_pow_left (Nat.pow_log_le_self 2 (by omega)) 2 + /-- Number of `b`-bit limbs `L = ⌈q / b⌉ = (q + b - 1) / b` needed to cover a `q`-bit field element. -/ @[expose] def limbCountFromB (b q : ℕ) : ℕ := @@ -196,84 +231,6 @@ coefficients `(c j).val < 2 ^ (trinomialDeg U M)` at `base + 8 + j.val`. -/ /-! ## Section 2: Helper loops for `initTableCode` -/ -/-- Body of `clog2Loop` on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, `reg 2 = 1`): updates -`rem ← (rem + 1) >>> 1` (`= (rem + 1) / 2`) and `acc ← acc + 1`. -/ -@[expose] def clog2Body : Code .exact 3 := - .seq (.op (.exactBin .add 0 0 2)) - (.seq (.op (.exactBin .shiftRight 0 0 2)) - (.op (.exactBin .add 1 1 2))) - -/-- Ceiling-`log₂` loop `while 1 < rem do [ rem ← (rem + 1) >>> 1; acc ← acc + 1 ]`. -/ -@[expose] def clog2Loop : Code .exact 3 := - .loop (.lt 2 0) clog2Body - -/-- Closed-form final state of `clog2Loop`. -/ -@[expose] def clog2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := - (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.clog 2 rem) - -/-- `clog2Loop` executes deterministically in `5 * Nat.clog 2 rem + 2` instructions, accumulating -`acc + Nat.clog 2 rem` into register `1` and preserving all memory. -/ -theorem clog2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} - (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) - (hfit_rem1 : Fits w (rem + 1)) (hfit_acc : Fits w (acc + Nat.clog 2 rem)) : - Exec w clog2Loop x (clog2LoopData rem acc x) (5 * Nat.clog 2 rem + 2) := by - induction rem using Nat.strong_induction_on generalizing acc x with - | h rem ih => - by_cases hlt : 1 < rem - · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hshr : (rem + 1) >>> 1 = (rem + 1) / 2 := by - rw [Nat.shiftRight_eq_div_pow] - have hclog : Nat.clog 2 rem = Nat.clog 2 ((rem + 1) / 2) + 1 := - Nat.clog_of_one_lt (by decide) hlt - have hdiv_lt : (rem + 1) / 2 < rem := by omega - have hfit_div1 : Fits w ((rem + 1) / 2 + 1) := hfit_rem1.of_le (by omega) - have hfit_div : Fits w ((rem + 1) / 2) := hfit_div1.of_le (by omega) - have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) - set d1 := x.set 0 (rem + 1) - have s1 : Exec w (.op (.exactBin .add 0 0 2) : Code .exact 3) x d1 1 := - exec_exactBin (by simp [hrem, hone]) hfit_rem1 - set d2 := d1.set 0 ((rem + 1) / 2) - have s2 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) d1 d2 1 := - exec_exactBin (by simp +decide [d1, hone, hshr]) hfit_div - set d3 := d2.set 1 (acc + 1) - have s3 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d2 d3 1 := - exec_exactBin (by simp +decide [d2, d1, hacc, hone]) hfit_acc1 - have hrec := ih ((rem + 1) / 2) hdiv_lt (acc := acc + 1) (x := d3) - (by simp +decide [d3, d2]) - (by simp +decide [d3]) - (by simp +decide [d3, d2, d1, hone]) - hfit_div1 - (by rwa [show acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem by omega]) - have hd : clog2LoopData ((rem + 1) / 2) (acc + 1) d3 = clog2LoopData rem acc x := by - have hdiv_ne : (rem + 1) / 2 ≠ 0 := by omega - have hrem_ne : rem ≠ 0 := by omega - ext i : 2 - · fin_cases i - · simp +decide [clog2LoopData, d3, d2, d1, hdiv_ne, hrem_ne] - · change acc + 1 + Nat.clog 2 ((rem + 1) / 2) = acc + Nat.clog 2 rem - omega - · simp +decide [clog2LoopData, d3, d2, d1] - · rfl - rw [hd] at hrec - have hiter := Exec.loop_iter hc (s1.seq (s2.seq s3)) hrec - simpa only [clog2Loop, clog2Body, hclog, - show 1 + (1 + 1) + (5 * Nat.clog 2 ((rem + 1) / 2) + 2) + 2 = - 5 * (Nat.clog 2 ((rem + 1) / 2) + 1) + 2 by omega] using hiter - · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hclog0 : Nat.clog 2 rem = 0 := Nat.clog_of_right_le_one (by omega) 2 - have hd : clog2LoopData rem acc x = x := by - have hrem_eq : (if rem = 0 then 0 else 1) = rem := by split_ifs <;> omega - ext i : 2 - · fin_cases i - · simp +decide [clog2LoopData, hrem_eq, hrem] - · simp +decide [clog2LoopData, hclog0, hacc] - · simp +decide [clog2LoopData] - · rfl - rw [hd, hclog0] - exact Exec.loop_exit hc - /-- Final remainder reached by `trinomialStepAux` when repeatedly mapping `B ↦ (B + 2) / 3` until `≤ 2`. -/ @[expose] def trinomialStepRem (B : ℕ) : ℕ := @@ -1428,6 +1385,42 @@ theorem map_extractSeeds_eval_resampleComp (base U M k : ℕ) (x : Data 18) : rw [ProbComp.eval_map, PMF.map_comp] exact map_extractSlice_eval_randFillComp (trinomialDeg U M) (base + 8) k _ +/-- Every outcome `z ∈ (resampleComp base U M k x).support` preserves `TableRep base b U M k`, +establishes `Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem`, sets `z.1.regs 0 = base`, +modifies memory only within `[base + 8, base + 8 + k)`, and takes at most `5 * k + 10` steps. -/ +theorem mem_support_resampleComp {base b U M k : ℕ} {x : Data 18} + (htab : TableRep base b U M k x.mem) (h0 : x.regs 0 = base) + {z : Data 18 × ℕ} (hz : z ∈ (resampleComp base U M k x).support) : + (Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem ∧ + z.1.regs 0 = base ∧ + (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → z.1.mem a = x.mem a)) ∧ + z.2 ≤ 5 * k + 10 := by + dsimp only [resampleComp] at hz + obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz + obtain ⟨hsteps, -, -, -, -, hextra, hlt, hout⟩ := mem_support_randFillComp hz0 + have htab' : TableRep base b U M k z0.1.mem := by + refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ + · rw [hout base (Or.inl (by omega))]; exact htab.1 + · rw [hout (base + 1) (Or.inl (by omega))]; exact htab.2.1 + · rw [hout (base + 2) (Or.inl (by omega))]; exact htab.2.2.1 + · rw [hout (base + 3) (Or.inl (by omega))]; exact htab.2.2.2.1 + · rw [hout (base + 4) (Or.inl (by omega))]; exact htab.2.2.2.2.1 + · rw [hout (base + 5) (Or.inl (by omega))]; exact htab.2.2.2.2.2.1 + · rw [hout (base + 6) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.1 + · rw [hout (base + 7) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.2.1 + · intro idx hidx + rw [hout (base + 8 + k + idx) (Or.inr (by omega))] + exact htab.2.2.2.2.2.2.2.2 idx hidx + have hseed : SeedRep base U M k (extractSeeds base U M k z0.1.mem) z0.1.mem := by + intro j + simp [extractSeeds, extractSlice, hlt j.val j.isLt] + have hbase' : (z0.1.extend resampleMap (resampleSetupData base U M k x)).regs 0 = base := by + change ((z0.1.extend resampleMap (resampleSetupData base U M k x)).comap + resampleMap).regs regExtra = base + rw [Data.comap_extend injective_resampleMap, hextra] + simpa +decide [resampleSetupData, resampleMap, Data.comap, regExtra] using h0 + exact ⟨⟨⟨htab', hseed⟩, hbase', hout⟩, by omega⟩ + /-- **`KWise.resampleCode k` Hoare-style specification**: halts in `5 * k + 10 = O(k)` steps and `k` `rand` queries, preserving `TableRep base b U M k`, establishing `Rep base b U M k (extractSeeds base U M k z.1.mem) z.1.mem`, preserving `reg 0 = base`, and @@ -1446,31 +1439,7 @@ theorem resampleCode_execRandWithin {w base b U M k : ℕ} {x : Data 18} exact (OracleComp.isCostBounded_map_iff _ _ _).mpr (isQueryBounded_randFillComp (trinomialDeg U M) (base + 8) k _) · intro z hz - dsimp only [resampleComp] at hz - obtain ⟨z0, hz0, rfl⟩ := OracleComp.mem_supportWith_map_iff.mp hz - obtain ⟨hsteps, -, -, -, -, hextra, hlt, hout⟩ := mem_support_randFillComp hz0 - have htab' : TableRep base b U M k z0.1.mem := by - refine ⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩ - · rw [hout base (Or.inl (by omega))]; exact htab.1 - · rw [hout (base + 1) (Or.inl (by omega))]; exact htab.2.1 - · rw [hout (base + 2) (Or.inl (by omega))]; exact htab.2.2.1 - · rw [hout (base + 3) (Or.inl (by omega))]; exact htab.2.2.2.1 - · rw [hout (base + 4) (Or.inl (by omega))]; exact htab.2.2.2.2.1 - · rw [hout (base + 5) (Or.inl (by omega))]; exact htab.2.2.2.2.2.1 - · rw [hout (base + 6) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.1 - · rw [hout (base + 7) (Or.inl (by omega))]; exact htab.2.2.2.2.2.2.2.1 - · intro idx hidx - rw [hout (base + 8 + k + idx) (Or.inr (by omega))] - exact htab.2.2.2.2.2.2.2.2 idx hidx - have hseed : SeedRep base U M k (extractSeeds base U M k z0.1.mem) z0.1.mem := by - intro j - simp [extractSeeds, extractSlice, hlt j.val j.isLt] - have hbase' : (z0.1.extend resampleMap (resampleSetupData base U M k x)).regs 0 = base := by - change ((z0.1.extend resampleMap (resampleSetupData base U M k x)).comap - resampleMap).regs regExtra = base - rw [Data.comap_extend injective_resampleMap, hextra] - simpa +decide [resampleSetupData, resampleMap, Data.comap, regExtra] using h0 - exact ⟨⟨⟨htab', hseed⟩, hbase', hout⟩, by omega⟩ + exact mem_support_resampleComp htab h0 hz /-- Full one-shot initialization program `initCode k := .seq (initTableCode k) (resampleCode k)`. -/ @[expose] def initCode (k : ℕ) : Code .exact 18 := @@ -2461,7 +2430,7 @@ and sets `reg 7 = base + 8 + k`. -/ /-- Deterministic evaluation program `evalCode k`: given `reg 0 = base` and `reg 1 = u.val`, evaluates `trinomialPolyHash (trinomialStep U M) k U M c u` into `reg 2` (`regOut`) in -`evalSteps b U M k` instructions while preserving `reg 0 = base`, `reg 1 = u.val`, and +`evalStepsFromB b U M k` instructions while preserving `reg 0 = base`, `reg 1 = u.val`, and all memory. -/ @[expose] def evalCode (k : ℕ) : Code .exact 18 := .seq (evalSetupCode k) (.seq hornerLoop evalMaskCode) @@ -2661,70 +2630,95 @@ theorem evalCode_execWithin {w base n_max U M k : ℕ} y.mem = x.mem := evalCode_execWithin_fromB u (chunkBits_pos n_max) hrep h0 h1 hw hfit_space -/-- **End-to-end Word-RAM `k`-wise independent hash theorem (`isKWiseIndependentHash_eval`)**: -1. The hash family `trinomialPolyHash (trinomialStep U M) k U M` evaluated by `evalCode k` is - `k`-wise independent onto `Fin (2 ^ M)` (`IsKWiseIndependentHash k`). -2. From any machine state `x` satisfying `TableRep base (chunkBits n_max) U M k x.mem` and - `x.regs 0 = base`, `resampleCode k` runs in `5 * k + 10` instructions and `k` `Code.rand` - queries with distribution `resampleComp base U M k x`, whose extracted seed - `extractSeeds base U M k z.1.mem` is uniformly distributed over - `Fin k → Fin (2 ^ trinomialDeg U M)` (`PMF.uniformOfFintype`). -3. Every resampled state `y` produced by `resampleCode k` satisfies `Rep`, so for every key - `u : Fin U`, running `evalCode k` from any state `x_eval` with `x_eval.mem = y.mem`, - `x_eval.regs 0 = base`, and `x_eval.regs 1 = u.val` halts deterministically in - `evalSteps n_max U M k` instructions, preserving all memory and writing - `(trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val` to - `reg 2`. -/ -theorem isKWiseIndependentHash_eval {w base n_max U M k : ℕ} {x : Data 18} +/-- Output of deterministic `evalCode k` execution at word size `w` from state `y` with +`reg 0 ← base` and `reg 1 ← u.val`, reduced modulo `2 ^ M` into `Fin (2 ^ M)`. -/ +noncomputable def evalHashOut (w base k U M : ℕ) (y : Data 18) (u : Fin U) : Fin (2 ^ M) := + open Classical in + let x_u := (y.set 0 base).set 1 u.val + if h : ∃ z n, Exec w (evalCode k) x_u z n then + ⟨(Classical.choose h).regs 2 % 2 ^ M, Nat.mod_lt _ (Nat.two_pow_pos M)⟩ + else + ⟨0, Nat.two_pow_pos M⟩ + +/-- On any machine state `y` satisfying `Rep base (chunkBits n_max) U M k c y.mem`, the +deterministic `evalCode k` output `evalHashOut w base k U M y u` equals +`trinomialPolyHash (trinomialStep U M) k U M c u`. -/ +theorem evalHashOut_eq_trinomialPolyHash {w base n_max U M k : ℕ} + {c : Fin k → Fin (2 ^ trinomialDeg U M)} {y : Data 18} + (hrep : Rep base (chunkBits n_max) U M k c y.mem) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) + (u : Fin U) : + evalHashOut w base k U M y u = trinomialPolyHash (trinomialStep U M) k U M c u := by + let x_u := (y.set 0 base).set 1 u.val + have hrep_u : Rep base (chunkBits n_max) U M k c x_u.mem := hrep + have h0_u : x_u.regs 0 = base := by simp +decide [x_u] + have h1_u : x_u.regs 1 = u.val := by simp +decide [x_u] + obtain ⟨z, ⟨n, _, hexec⟩, -, -, hz2, -⟩ := + evalCode_execWithin u hrep_u h0_u h1_u hw hfit_space + have hex : ∃ z n, Exec w (evalCode k) x_u z n := ⟨z, n, hexec⟩ + obtain ⟨n', hexec'⟩ := Classical.choose_spec hex + have huniq : Classical.choose hex = z := (hexec'.unique hexec).1 + ext + simp only [evalHashOut, x_u, hex, ↓reduceDIte, huniq, hz2, + Nat.mod_eq_of_lt (trinomialPolyHash _ k U M c u).isLt] + +/-- Under `resampleComp base U M k x`, the Word-RAM hash function `evalHashOut w base k U M z.1` +has the exact distribution of `trinomialPolyHash (trinomialStep U M) k U M c` for a uniform seed +`c : Fin k → Fin (2 ^ trinomialDeg U M)`. -/ +theorem map_evalHashOut_eval_resampleComp {w base n_max U M k : ℕ} {x : Data 18} (htab : TableRep base (chunkBits n_max) U M k x.mem) (h0 : x.regs 0 = base) (hw : 2 * trinomialDeg U M ≤ w) (hfit_space : Fits w (base + space n_max k)) : - IsKWiseIndependentHash k (trinomialPolyHash (trinomialStep U M) k U M) ∧ - ExecRand w (resampleCode k) x (resampleComp base U M k x) ∧ - ((resampleComp base U M k x).eval).map (fun z => extractSeeds base U M k z.1.mem) = - PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M)) ∧ - ExecRandWithin w (resampleCode k) x - (fun y => - Rep base (chunkBits n_max) U M k (extractSeeds base U M k y.mem) y.mem ∧ - y.regs 0 = base ∧ - (∀ a, (a < base + 8 ∨ base + 8 + k ≤ a) → y.mem a = x.mem a) ∧ - (∀ u : Fin U, ∀ x_eval : Data 18, - x_eval.mem = y.mem → - x_eval.regs 0 = base → - x_eval.regs 1 = u.val → - ∃ y_eval : Data 18, - ExecWithin w (evalCode k) x_eval y_eval (evalSteps n_max U M k) ∧ - y_eval.regs 0 = base ∧ - y_eval.regs 1 = u.val ∧ - y_eval.regs 2 = - (trinomialPolyHash (trinomialStep U M) k U M (extractSeeds base U M k y.mem) u).val ∧ - y_eval.mem = y.mem)) - (5 * k + 10) k := by - have hpow_pos : 0 < 2 ^ (2 * chunkBits n_max) := by positivity - have hsp : space n_max k = 8 + k + 2 ^ (2 * chunkBits n_max) := rfl - have hfit_end : Fits w (base + 8 + k) := hfit_space.of_le (by omega) - have hfit_q : Fits w (2 ^ trinomialDeg U M) := by - have hlt : 2 ^ trinomialDeg U M < 2 ^ (2 * trinomialDeg U M) := by - apply (Nat.pow_lt_pow_iff_right one_lt_two).mpr - have : 1 ≤ 3 ^ trinomialStep U M := Nat.one_le_pow _ 3 (by decide) - dsimp only [trinomialDeg] - omega - exact fits_of_lt_two_pow_two_mul hw hlt - obtain ⟨m, he_rand, hqb, hsupp⟩ := resampleCode_execRandWithin htab h0 hfit_end hfit_q - refine ⟨isKWiseIndependentHash_trinomialPolyHash - (le_two_pow_trinomialDeg U M) (le_trinomialDeg U M), - resampleCode_execRand htab h0 hfit_end hfit_q, - map_extractSeeds_eval_resampleComp base U M k x, - ⟨m, he_rand, hqb, ?_⟩⟩ - intro z hz - obtain ⟨⟨hrep, hz0, hzout⟩, hsteps⟩ := hsupp z hz - refine ⟨⟨hrep, hz0, hzout, ?_⟩, hsteps⟩ - intro u x_eval hmem he0 he1 - have hrep_eval : Rep base (chunkBits n_max) U M k (extractSeeds base U M k z.1.mem) x_eval.mem := - hmem ▸ hrep - obtain ⟨y_eval, heval, hy0, hy1, hy2, hymem⟩ := - evalCode_execWithin u hrep_eval he0 he1 hw hfit_space - exact ⟨y_eval, heval, hy0, hy1, hy2, hymem.trans hmem⟩ + ((resampleComp base U M k x).eval).map (fun z => evalHashOut w base k U M z.1) = + (PMF.uniformOfFintype (Fin k → Fin (2 ^ trinomialDeg U M))).map + (trinomialPolyHash (trinomialStep U M) k U M) := by + rw [← map_extractSeeds_eval_resampleComp base U M k x] + dsimp only [trinomialDeg, extractSeeds, extractSlice] + rw [PMF.map_comp] + refine Cslib.Probability.PMF.map_congr_on_support _ _ _ fun z hz => ?_ + rw [ProbComp.support_eval] at hz + have hrep := (mem_support_resampleComp htab h0 hz).1.1 + funext u + exact evalHashOut_eq_trinomialPolyHash hrep hw hfit_space u + +/-- For any `k`-wise independent hash family `h : ι → X → Y` and any set `T : Finset X` of at most +`k` points, restricting `h i` to `T` under a uniform seed `i ← PMF.uniformOfFintype ι` yields the +uniform distribution `PMF.uniformOfFintype (T → Y)`. -/ +theorem uniformOfFintype_map_restrict_of_isKWiseIndependentHash + {ι X Y : Type*} [Fintype ι] [Nonempty ι] [DecidableEq X] + [Fintype Y] [DecidableEq Y] [Nonempty Y] + {k : ℕ} {h : ι → X → Y} (hh : IsKWiseIndependentHash k h) + {T : Finset X} (hT : Finset.card T ≤ k) : + (PMF.uniformOfFintype ι).map (fun i => T.restrict (h i)) = + PMF.uniformOfFintype (T → Y) := by + ext y + apply (ENNReal.toReal_eq_toReal_iff' (PMF.apply_ne_top _ _) (PMF.apply_ne_top _ _)).mp + rw [Cslib.Probability.PMF.toReal_uniformOfFintype_map_apply, hh.dens_filter_restrict_eq hT y] + simp + +/-- **Composed Word-RAM `k`-wise independence of `resampleCode` and `evalCode` +(`isKWiseIndependentHash_eval`)**: +from any machine state `x` satisfying `TableRep base (chunkBits n_max) U M k x.mem` and +`x.regs 0 = base`, under the resampling distribution `(resampleComp base U M k x).eval` of +`resampleCode k`, for every set `T : Finset (Fin U)` of at most `k` keys (`T.card ≤ k`), the +restricted tuple of deterministic `evalCode k` outputs `T.restrict (evalHashOut w base k U M z.1)` +is uniformly distributed on `T → Fin (2 ^ M)` (`PMF.uniformOfFintype (T → Fin (2 ^ M))`). -/ +theorem isKWiseIndependentHash_eval {w base n_max U M k : ℕ} {x : Data 18} + (htab : TableRep base (chunkBits n_max) U M k x.mem) + (h0 : x.regs 0 = base) + (hw : 2 * trinomialDeg U M ≤ w) + (hfit_space : Fits w (base + space n_max k)) + {T : Finset (Fin U)} (hT : Finset.card T ≤ k) : + ((resampleComp base U M k x).eval).map (fun z => T.restrict (evalHashOut w base k U M z.1)) = + PMF.uniformOfFintype (T → Fin (2 ^ M)) := by + have hmap := congrArg (PMF.map T.restrict) + (map_evalHashOut_eval_resampleComp htab h0 hw hfit_space) + dsimp only [trinomialDeg] at hmap + rw [PMF.map_comp, PMF.map_comp] at hmap + exact hmap.trans (uniformOfFintype_map_restrict_of_isKWiseIndependentHash + (isKWiseIndependentHash_trinomialPolyHash (le_two_pow_trinomialDeg U M) (le_trinomialDeg U M)) + hT) end Cslib.Algorithms.WordRAM.Hashing.KWise diff --git a/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean index d6aba6086..cabfafe4d 100644 --- a/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean +++ b/Complexity/Algorithms/WordRAM/Hashing/Tabulation.lean @@ -7,6 +7,7 @@ Authors: Renfei Zhou module -- upstream: candidate +public import Complexity.Algorithms.WordRAM.ExactLog public import Complexity.Computability.WordRAM.Randomized.Structured public import Complexity.Probability.PairwiseIndependentHash.Shift public import Complexity.Probability.Product @@ -14,23 +15,31 @@ public import Complexity.Probability.Uniform public import Mathlib.Data.Nat.Log /-! -# Self-contained Word-RAM random tables and simple tabulation hashing +# Self-contained Word-RAM random tables and tabulated odd-stride generation This file provides two self-contained Word-RAM hashing primitives over `Cslib.WordRAM.Basis.exact`, built on a shared uniform word-slice sampler (`randFillLoop`): -1. **`RandomTable`**: initializes or resamples a table `Z[0 .. 2 ^ M - 1]` of `m = 2 ^ M` - independent uniform `M`-bit words in `O(2 ^ M)` instructions and `2 ^ M` calls to `Code.rand` - (`PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`), and supports `O(1)` deterministic table - lookup `Z[idx]` as well as XOR-shifted evaluation `f(u) ^^^ Z[g(u)]` - [Carter and Wegman][CarterWegman1979]; [Mitzenmacher and Upfal][MitzenmacherUpfal2017]. -2. **`TabulatedOddStride`**: `d`-character simple tabulation with character bit-width - `b = max 1 (Nat.log 2 (max n_max 2) / 2)` (`2 ^ b ≤ max 2 n_max` words per character table), - initialized via `Code.rand` in `O(d * 2 ^ b + log n_max) = O(n_max)` steps and evaluated in - `O(d)` deterministic instructions, producing an odd stride - `(2 * simpleTab(u) + 1) &&& (2 ^ M - 1)` coprime to `2 ^ M` whose linear probe sequence - `(h₁(u) + i * h₂(u)) % 2 ^ M` is a full-cycle permutation of `Fin (2 ^ M)` - [Cormen, Leiserson, Rivest, and Stein, Section 11.4][CLRS2009]. +1. **`RandomTable`**: `resampleCode` populates an array `Z[0 .. 2 ^ M - 1]` of `m = 2 ^ M` + independent uniform `M`-bit words in `5 * 2 ^ M + 7` instructions and `2 ^ M` calls to + `Code.rand`, inducing `PMF.uniformOfFintype (Fin (2 ^ M) → Fin (2 ^ M))`. `lookupCode` reads + `Z[idx]` in `2` deterministic instructions, and `xorLookupCode` evaluates the XOR-shifted hash + `f(u) ^^^ Z[g(u)]` in `3` deterministic instructions [Carter and Wegman][CarterWegman1979]; + [Mitzenmacher and Upfal][MitzenmacherUpfal2017]. +2. **`TabulatedOddStride`**: `simpleTab` splits the low `d * b` bits of an input key `u` into `d` + characters of `b = max 1 (Nat.log 2 (max n_max 2) / 2)` bits (`2 ^ b ≤ max 2 n_max` words per + character table) and XORs their table entries (following the simple tabulation scheme of Zobrist + 1970 and Pătraşcu and Thorup 2012). `initCode d` samples the `d` character tables via + `Code.rand` in `4 * Nat.log 2 (max n_max 2) + 5 * (d * 2 ^ b) + 30` instructions + (`O(d * 2 ^ b + log n_max)`, which is `O(n_max)` for fixed `d`), and `evalCode d` evaluates + `oddStride M (simpleTab b d M T u).val = (2 * (simpleTab b d M T u).val + 1) &&& (2 ^ M - 1)` in + `8 * d + 21` deterministic instructions (`O(d)`). When `M > 0`, this stride `h₂(u)` is odd (at + `M = 0` it is `0` on `Fin 1`), and for every `M` it is coprime to `2 ^ M`, so for any initial + slot `h₁(u) : Fin (2 ^ M)` the linear probe sequence `probeSeq M h₁(u) tab i = + (h₁(u) + i * h₂(u)) % 2 ^ M` is a bijection on `Fin (2 ^ M)` (`bijective_probeSeq`) + [Cormen, Leiserson, Rivest, and Stein, Section 11.4][CLRS2009]. Only this full-cycle permutation + property is proved here: no `k`-wise or 3-wise independence theorem for simple tabulation is + claimed in this module, and `simpleTab` inspects only the low `d * b` bits of `u`. ## Main definitions @@ -63,8 +72,8 @@ built on a shared uniform word-slice sampler (`randFillLoop`): * `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.bijective_probeSeq`, `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.initCode_execRandWithin`, `Cslib.Algorithms.WordRAM.Hashing.Tabulation.TabulatedOddStride.evalCode_execWithin`: full-cycle - permutation property, `O(d * 2 ^ b)` initialization, and `O(d)` deterministic evaluation of - `TabulatedOddStride`. + permutation property, `O(d * 2 ^ b + log n_max)` initialization, and `O(d)` deterministic + evaluation of `TabulatedOddStride`. ## References @@ -79,42 +88,9 @@ public section namespace Cslib.Algorithms.WordRAM.Hashing.Tabulation open Cslib.WordRAM Cslib.Probability +open Cslib.Algorithms.WordRAM.ExactLog open scoped ENNReal -/-! ## Exact-basis instruction helpers -/ - -/-- Execute a single `.exactBin` operation when `evalExact` returns `some v` and `Fits w v`. -/ -theorem exec_exactBin {w r : ℕ} {f : BinOp} {dst i j : Fin r} {d : Data r} {v : ℕ} - (hev : f.evalExact (d.regs i) (d.regs j) = some v) (hfit : Fits w v) : - Exec w (.op (.exactBin f dst i j) : Code .exact r) d (d.set dst v) 1 := by - apply Exec.op - rw [Op.eval_exactBin, hev] - exact Data.put_of_fits hfit - -/-- 3-instruction equal-cost branch setting register `dst` to `max (x.regs a) (x.regs b)`. -/ -@[expose] def maxCode {n : ℕ} (dst a b : Fin n) : Code .exact n := - .branch (.lt a b) (.seq (.op (.copy dst b)) (.op (.copy dst dst))) (.op (.copy dst a)) - -/-- `maxCode dst a b` executes deterministically in 3 instructions and sets `dst` to -`max (x.regs a) (x.regs b)`. -/ -theorem maxCode_exec {w n : ℕ} (dst a b : Fin n) (x : Data n) : - Exec w (maxCode dst a b) x (x.set dst (max (x.regs a) (x.regs b))) 3 := by - by_cases hlt : x.regs a < x.regs b - · have hc : (Cond.lt a b).Holds x := hlt - have hmax : max (x.regs a) (x.regs b) = x.regs b := Nat.max_eq_right (le_of_lt hlt) - rw [hmax] - set d1 := x.set dst (x.regs b) - have s1 : Exec w (.op (.copy dst b) : Code .exact n) x d1 1 := Exec.copy - have hd1 : d1.set dst (d1.regs dst) = d1 := by - ext i : 2 <;> simp [d1, Data.set] - have s2 : Exec w (.op (.copy dst dst) : Code .exact n) d1 d1 1 := by - simpa only [hd1] using (Exec.copy (B := .exact) (w := w) (d := dst) (s := dst) (x := d1)) - exact Exec.branch_pos hc (s1.seq s2) - · have hc : ¬(Cond.lt a b).Holds x := hlt - have hmax : max (x.regs a) (x.regs b) = x.regs a := Nat.max_eq_left (not_lt.mp hlt) - rw [hmax] - exact Exec.branch_neg hc (Exec.copy (B := .exact)) - /-! ## Section 1: Uniform random word-slice filler (`randFillLoop`) -/ /-- Register `0` of `randFillLoop`: write pointer `ptr`. -/ @@ -645,81 +621,7 @@ theorem xorLookupCode_exec {w base M : ℕ} {Z : Fin (2 ^ M) → Fin (2 ^ M)} {x end RandomTable -/-! ## Section 3: Shared `Nat.log 2` loop and `TabulatedOddStride` -/ - -/-- One iteration of the floor-`log₂` loop on `Data 3` (`reg 0 = rem`, `reg 1 = acc`, -`reg 2 = 1`): shifts `rem` right by `1` (`rem / 2`) and increments `acc`. -/ -@[expose] def log2Body : Code .exact 3 := - .seq (.op (.exactBin .shiftRight 0 0 2)) - (.op (.exactBin .add 1 1 2)) - -/-- Floor-`log₂` loop `while 1 < rem do [ rem ← rem >>> 1; acc ← acc + 1 ]`. -/ -@[expose] def log2Loop : Code .exact 3 := - .loop (.lt 2 0) log2Body - -/-- Closed-form final state of `log2Loop` starting from `rem` in register `0` and `acc` in -register `1`. -/ -@[expose] def log2LoopData (rem acc : ℕ) (x : Data 3) : Data 3 := - (x.set 0 (if rem = 0 then 0 else 1)).set 1 (acc + Nat.log 2 rem) - -/-- `log2Loop` executes deterministically in `4 * Nat.log 2 rem + 2` instructions, accumulating -`acc + Nat.log 2 rem` into register `1` and preserving all memory. -/ -theorem log2Loop_exec {w : ℕ} (rem : ℕ) {acc : ℕ} {x : Data 3} - (hrem : x.regs 0 = rem) (hacc : x.regs 1 = acc) (hone : x.regs 2 = 1) - (hfit_rem : Fits w rem) (hfit_acc : Fits w (acc + Nat.log 2 rem)) : - Exec w log2Loop x (log2LoopData rem acc x) (4 * Nat.log 2 rem + 2) := by - induction rem using Nat.strong_induction_on generalizing acc x with - | h rem ih => - by_cases hlt : 1 < rem - · have hc : (Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hshr : rem >>> 1 = rem / 2 := by - rw [Nat.shiftRight_eq_div_pow] - have hlog : Nat.log 2 rem = Nat.log 2 (rem / 2) + 1 := - Nat.log_of_one_lt_of_le (by decide) (by omega) - have hfit_div : Fits w (rem / 2) := hfit_rem.of_le (Nat.div_le_self rem 2) - have hfit_acc1 : Fits w (acc + 1) := hfit_acc.of_le (by omega) - set d1 := x.set 0 (rem / 2) - have s1 : Exec w (.op (.exactBin .shiftRight 0 0 2) : Code .exact 3) x d1 1 := - exec_exactBin (by simp [hrem, hone, hshr]) hfit_div - set d2 := d1.set 1 (acc + 1) - have s2 : Exec w (.op (.exactBin .add 1 1 2) : Code .exact 3) d1 d2 1 := - exec_exactBin (by simp +decide [d1, hacc, hone]) hfit_acc1 - have hdiv_lt : rem / 2 < rem := Nat.div_lt_self (by omega) (by decide) - have hrec := ih (rem / 2) hdiv_lt (acc := acc + 1) (x := d2) - (by simp +decide [d2, d1]) - (by simp +decide [d2]) - (by simp +decide [d2, d1, hone]) - hfit_div - (by rwa [show acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem by omega]) - have hd : log2LoopData (rem / 2) (acc + 1) d2 = log2LoopData rem acc x := by - have hdiv_ne : rem / 2 ≠ 0 := by omega - have hrem_ne : rem ≠ 0 := by omega - ext i : 2 - · fin_cases i - · simp +decide [log2LoopData, d2, d1, hdiv_ne, hrem_ne] - · change acc + 1 + Nat.log 2 (rem / 2) = acc + Nat.log 2 rem - omega - · simp +decide [log2LoopData, d2, d1] - · rfl - rw [hd] at hrec - have hiter := Exec.loop_iter hc (s1.seq s2) hrec - simpa only [log2Loop, log2Body, hlog, show 1 + 1 + (4 * Nat.log 2 (rem / 2) + 2) + 2 = - 4 * (Nat.log 2 (rem / 2) + 1) + 2 by omega] using hiter - · have hc : ¬(Cond.lt (2 : Fin 3) 0).Holds x := by - simp [Cond.Holds, hrem, hone, hlt] - have hlog0 : Nat.log 2 rem = 0 := Nat.log_of_lt (by omega) - have hd : log2LoopData rem acc x = x := by - have hrem_eq : (if rem = 0 then 0 else 1) = rem := by - split_ifs <;> omega - ext i : 2 - · fin_cases i - · simp +decide [log2LoopData, hrem_eq, hrem] - · simp +decide [log2LoopData, hlog0, hacc] - · simp +decide [log2LoopData] - · rfl - rw [hd, hlog0] - exact Exec.loop_exit hc +/-! ## Section 3: `TabulatedOddStride` -/ namespace TabulatedOddStride @@ -1368,11 +1270,6 @@ theorem evalSetupCode_exec {w base b d M : ℕ} {T : Fin (d * 2 ^ b) → Fin (2 @[expose] def tabLoop : Code .exact 9 := .loop (.lt regPtr regEnd) tabBody -/-- State after one iteration of `tabBody`. -/ -@[expose] def tabStepData (ptr ch entry newAcc newKey nextPtr : ℕ) (x : Data 9) : Data 9 := - (((((x.set regTmp ch).set regTmp (ptr + ch)).set regTmp entry).set - regOut newAcc).set regKey newKey).set regPtr nextPtr - /-- `tabLoop` executes deterministically in `8 * rem + 2` instructions from character offset `j` with `j + rem = d`, accumulating `(simpleTabAux b d M T rem j u acc).val` into `regOut` and preserving `regBase` and all memory. -/ diff --git a/ComplexityTests/WordRAMKWiseHash.lean b/ComplexityTests/WordRAMKWiseHash.lean index 77b8a398d..ec6055db0 100644 --- a/ComplexityTests/WordRAMKWiseHash.lean +++ b/ComplexityTests/WordRAMKWiseHash.lean @@ -113,29 +113,33 @@ def trinomialHornerRef5 (t M : ℕ) (c : Fin 5 → Fin (2 ^ (2 * 3 ^ t))) (u : let a0 := reduceTrinomialCode t (clmulFuel 6 a1 u) ^^^ (c ⟨0, by decide⟩).val a0 &&& (2 ^ M - 1) -/-- Write the `5` coefficients of `sampleCoeffs5` into `[base + 8, base + 13)` of `d.mem`. -/ -def writeSampleCoeffs5 (base : ℕ) (d : Data 18) : Data 18 := - ((((d.write (base + 8) 3).write (base + 9) 1).write (base + 10) 2).write - (base + 11) 3).write (base + 12) 2 - -/-- Run `KWise.initTableCode 5` with `(base, n_max, U, M)` in registers `0..3`, populate -`sampleCoeffs5`, and evaluate `KWise.evalCode 5` via `Code.eval` on a list of keys `us`, returning -the list of output hash values together with the final values at boundary sentinels `base - 1` and -`base + KWise.space n_max 5`. -/ -def runKWiseEval5 (w fuel n_max U M base : ℕ) (us : List ℕ) : +/-- Write `5` coefficients `c : Fin 5 → Fin n` into `[base + 8, base + 13)` of `d.mem`. -/ +def writeSampleCoeffs5 {n : ℕ} (base : ℕ) (c : Fin 5 → Fin n) (d : Data 18) : Data 18 := + ((((d.write (base + 8) (c ⟨0, by decide⟩).val).write (base + 9) (c ⟨1, by decide⟩).val).write + (base + 10) (c ⟨2, by decide⟩).val).write (base + 11) (c ⟨3, by decide⟩).val).write + (base + 12) (c ⟨4, by decide⟩).val + +/-- Run `KWise.initTableCode 5` with `(base, n_max, U, M)` in registers `0..3`, populate `c` via +`writeSampleCoeffs5`, and evaluate `KWise.evalCode 5` via `Code.eval` on a list of keys `us`, +verifying that boundary sentinels at `base - 1` and `base + KWise.space n_max 5` are preserved both +after `initTableCode 5` and after each `evalCode 5` invocation. -/ +def runKWiseEval5 {n : ℕ} (w fuel n_max U M base : ℕ) (c : Fin 5 → Fin n) (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := do let sp := KWise.space n_max 5 let d0 : Data 18 := ((((((Data.init [] : Data 18).set 0 base).set 1 n_max).set 2 U).set 3 M).write (base - 1) 77).write (base + sp) 88 let (d1, _) ← (KWise.initTableCode 5).eval w fuel d0 - let d2 ← compactData (base + sp + 2) (writeSampleCoeffs5 base d1) + let d2 ← compactData (base + sp + 2) (writeSampleCoeffs5 base c d1) + let sLo ← forceNat (d2.mem (base - 1)) + let sHi ← forceNat (d2.mem (base + sp)) let outs ← us.mapM (fun u => do let dIn := (d2.set 0 base).set KWise.regKey u let (dOut, _) ← (KWise.evalCode 5).eval w fuel dIn - forceNat (dOut.regs KWise.regOut)) - let sLo ← forceNat (d2.mem (base - 1)) - let sHi ← forceNat (d2.mem (base + sp)) + if dOut.mem (base - 1) = sLo ∧ dOut.mem (base + sp) = sHi then + forceNat (dOut.regs KWise.regOut) + else + none) pure (outs, sLo, sHi) /-- Concrete `Code.eval` test at `k = 5, b = 1, L = 2` (`n_max = 8`, `U = 4`, `M = 2`, `base = 10`): @@ -144,7 +148,7 @@ def runKWiseEval5 (w fuel n_max U M base : ℕ) (us : List ℕ) : the `5` Horner steps) matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys `u ∈ [0, 1, 2, 3]` while preserving sentinels at addresses `9` (`77`) and `27` (`88`). -/ theorem kwise_eval5_b1_concrete : - runKWiseEval5 16 16 8 4 2 10 [0, 1, 2, 3] = + runKWiseEval5 16 16 8 4 2 10 sampleCoeffs5 [0, 1, 2, 3] = some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by decide +kernel @@ -153,7 +157,7 @@ theorem kwise_eval5_b1_concrete : `KWise.evalCode 5` matches `trinomialHornerRef5 0 2 sampleCoeffs5 u` on all keys `u ∈ [0, 1, 2, 3]` while preserving sentinels at addresses `3` (`77`) and `33` (`88`). -/ theorem kwise_eval5_b2_concrete : - runKWiseEval5 16 32 64 4 2 4 [0, 1, 2, 3] = + runKWiseEval5 16 32 64 4 2 4 sampleCoeffs5 [0, 1, 2, 3] = some ([0, 1, 2, 3].map (trinomialHornerRef5 0 2 sampleCoeffs5), 77, 88) := by decide +kernel @@ -168,23 +172,8 @@ def sampleCoeffs5T1 : Fin 5 → Fin 64 /-- Run `KWise.initTableCode 5` and `KWise.evalCode 5` at `t = 1, q = 6, b = 2, L = 3` (`n_max = 64, U = 16, M = 3, base = 4`). -/ -def runKWiseEval5T1 (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := do - let base := 4 - let sp := KWise.space 64 5 - let d0 : Data 18 := - ((((((Data.init [] : Data 18).set 0 base).set 1 64).set 2 16).set 3 3).write - (base - 1) 77).write (base + sp) 88 - let (d1, _) ← (KWise.initTableCode 5).eval 16 32 d0 - let d2 ← compactData (base + sp + 2) - (((((d1.write (base + 8) 13).write (base + 9) 42).write (base + 10) 7).write - (base + 11) 59).write (base + 12) 25) - let outs ← us.mapM (fun u => do - let dIn := (d2.set 0 base).set KWise.regKey u - let (dOut, _) ← (KWise.evalCode 5).eval 16 32 dIn - forceNat (dOut.regs KWise.regOut)) - let sLo ← forceNat (d2.mem (base - 1)) - let sHi ← forceNat (d2.mem (base + sp)) - pure (outs, sLo, sHi) +def runKWiseEval5T1 (us : List ℕ) : Option (List ℕ × ℕ × ℕ) := + runKWiseEval5 16 32 64 16 3 4 sampleCoeffs5T1 us /-- Concrete `Code.eval` test at `k = 5, t = 1, q = 6, b = 2, L = 3` (`3 × 3` limb multiplication over `X ^ 6 + X ^ 3 + 1`): `KWise.evalCode 5` matches `trinomialHornerRef5 1 3 sampleCoeffs5T1` on @@ -196,8 +185,8 @@ theorem kwise_eval5_t1_b2_L3_concrete : /-- Non-vacuous theorem-level instantiation of `KWise.initTableCode_execWithin`, `KWise.resampleCode_execRand`, `KWise.initCode_execRandWithin`, `KWise.evalCode_execWithin`, and -`KWise.isKWiseIndependentHash_eval` at `k = 5`, `U = 1000`, `M = 10`, `n_max = 64`, `w = 64`, -`base = 0`. -/ +the composed Word-RAM `k`-wise independence theorem `KWise.isKWiseIndependentHash_eval` at `k = 5`, +`U = 1000`, `M = 10`, `n_max = 64`, `w = 64`, `base = 0`. -/ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) (h0 : x.regs 0 = 0) (h1 : x.regs 1 = 64) (h2 : x.regs 2 = 1000) (h3 : x.regs 3 = 10) : (∃ y : Data 18, @@ -213,7 +202,10 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) (∀ y : Data 18, KWise.TableRep 0 (KWise.chunkBits 64) 1000 10 5 y.mem → y.regs 0 = 0 → - IsKWiseIndependentHash 5 (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10) ∧ + (∀ (T : Finset (Fin 1000)), Finset.card T ≤ 5 → + ((KWise.resampleComp 0 1000 10 5 y).eval).map + (fun z => T.restrict (KWise.evalHashOut 64 0 5 1000 10 z.1)) = + PMF.uniformOfFintype (T → Fin (2 ^ 10))) ∧ ExecRand 64 (KWise.resampleCode 5) y (KWise.resampleComp 0 1000 10 5 y) ∧ ((KWise.resampleComp 0 1000 10 5 y).eval).map (fun z => KWise.extractSeeds 0 1000 10 5 z.1.mem) = @@ -223,20 +215,19 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 (KWise.extractSeeds 0 1000 10 5 z.mem) z.mem ∧ z.regs 0 = 0 ∧ - (∀ a, (a < 0 + 8 ∨ 0 + 8 + 5 ≤ a) → z.mem a = y.mem a) ∧ - (∀ u : Fin 1000, ∀ x_eval : Data 18, - x_eval.mem = z.mem → - x_eval.regs 0 = 0 → - x_eval.regs 1 = u.val → - ∃ y_eval : Data 18, - ExecWithin 64 (KWise.evalCode 5) x_eval y_eval (KWise.evalSteps 64 1000 10 5) ∧ - y_eval.regs 0 = 0 ∧ - y_eval.regs 1 = u.val ∧ - y_eval.regs 2 = - (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10 - (KWise.extractSeeds 0 1000 10 5 z.mem) u).val ∧ - y_eval.mem = z.mem)) - 35 5) := by + (∀ a, (a < 0 + 8 ∨ 0 + 8 + 5 ≤ a) → z.mem a = y.mem a)) + 35 5 ∧ + (∀ (c : Fin 5 → Fin (2 ^ trinomialDeg 1000 10)) (u : Fin 1000) (x_eval : Data 18), + KWise.Rep 0 (KWise.chunkBits 64) 1000 10 5 c x_eval.mem → + x_eval.regs 0 = 0 → + x_eval.regs 1 = u.val → + ∃ y_eval : Data 18, + ExecWithin 64 (KWise.evalCode 5) x_eval y_eval (KWise.evalSteps 64 1000 10 5) ∧ + y_eval.regs 0 = 0 ∧ + y_eval.regs 1 = u.val ∧ + y_eval.regs 2 = + (trinomialPolyHash (trinomialStep 1000 10) 5 1000 10 c u).val ∧ + y_eval.mem = x_eval.mem)) := by have htr : trinomialDeg 1000 10 = 18 := by have hclog : Nat.clog 2 (max 1000 2) ≤ 10 := (Nat.clog_le_iff_le_pow one_lt_two).mpr (by decide) @@ -250,6 +241,7 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) rw [fits_iff] dsimp [KWise.space, KWise.spaceFromB, KWise.chunkBits] decide + have hfit_end : Fits 64 (0 + 8 + 5) := by rw [fits_iff]; decide have hfit_n : Fits 64 (max 64 2) := by rw [fits_iff]; decide have hfit_U : Fits 64 (max 1000 2 + 1) := by rw [fits_iff]; decide have hfit_q : Fits 64 (2 ^ trinomialDeg 1000 10) := by @@ -259,7 +251,12 @@ theorem kwise_five_wise_wordram_nonvacuous (x : Data 18) refine ⟨⟨y, hy_exec, hy_tab⟩, KWise.initCode_execRandWithin h0 h1 h2 h3 hfit_sp hfit_n hfit_U hfit_q, fun y' htab' h0' => - KWise.isKWiseIndependentHash_eval htab' h0' hw hfit_sp⟩ + ⟨fun T hT => KWise.isKWiseIndependentHash_eval htab' h0' hw hfit_sp hT, + KWise.resampleCode_execRand htab' h0' hfit_end hfit_q, + KWise.map_extractSeeds_eval_resampleComp 0 1000 10 5 y', + KWise.resampleCode_execRandWithin htab' h0' hfit_end hfit_q, + fun c u x_eval hrep hx0 hx1 => + KWise.evalCode_execWithin u hrep hx0 hx1 hw hfit_sp⟩⟩ /-! ## 2. `RandomTable`: concrete `Code.eval` execution and theorem instantiations -/ From bc61368a8eebeb75dc85c11f1b804147908a78c8 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Fri, 30 Oct 2026 02:11:13 +0000 Subject: [PATCH 17/18] feat(Algorithms/WordRAM/Dictionary): add linear probing dictionary on randomized Word RAM --- Complexity.lean | 2 + .../WordRAM/Dictionary/LinearProbing.lean | 2274 +++++++++++++++++ .../Dictionary/LinearProbing/FiveWise.lean | 722 ++++++ ComplexityTests/WordRAMLinearProbing.lean | 335 +++ references.bib | 11 + 5 files changed, 3344 insertions(+) create mode 100644 Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean create mode 100644 Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean create mode 100644 ComplexityTests/WordRAMLinearProbing.lean diff --git a/Complexity.lean b/Complexity.lean index 976f7af96..22c0dfed5 100644 --- a/Complexity.lean +++ b/Complexity.lean @@ -78,6 +78,8 @@ public import Complexity.Algorithms.ThreeSum.ThreeSets public import Complexity.Algorithms.WordRAM.Deduplicate.Count public import Complexity.Algorithms.WordRAM.Deduplicate.Model public import Complexity.Algorithms.WordRAM.Deduplicate.Program +public import Complexity.Algorithms.WordRAM.Dictionary.LinearProbing +public import Complexity.Algorithms.WordRAM.Dictionary.LinearProbing.FiveWise public import Complexity.Algorithms.WordRAM.Dijkstra.Fibonacci public import Complexity.Algorithms.WordRAM.Dijkstra.FibonacciStatement public import Complexity.Algorithms.WordRAM.Dijkstra.GenericPQ diff --git a/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean new file mode 100644 index 000000000..d92dccd73 --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean @@ -0,0 +1,2274 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Algorithms.Lean.Dictionary +public import Complexity.Algorithms.WordRAM.Hashing.KWise +public import Complexity.Algorithms.WordRAM.Dictionary.LinearProbing.FiveWise +public import Complexity.Computability.WordRAM.Randomized.Subroutine + +/-! +# Open-Addressed Linear Probing on the Randomized Word RAM + +This file implements a self-contained open-addressed linear-probing hash table realizing +`Cslib.Algorithms.Lean.Dictionary.spec U V n_max` on the randomized Word RAM +(`Cslib.WordRAM.Randomized.SubroutineDS`) [Pagh, Pagh and Ružić, *Linear Probing with Constant +Independence*][PaghPaghRuzic2009]; [Cormen, Leiserson, Rivest and Stein, *Introduction to +Algorithms*, Section 11.4][CLRS2009]. + +## Design + +1. **Power-of-$2$ open-addressed layout ($O(n_{\max})$ words, zero tombstones):** + For capacity bound $n_{\max}$, we set $M = \mathrm{tableLog}(n_{\max}) = + \lceil \log_2 \max(8 n_{\max}, 8) \rceil$ and table size $m = 2^M$ (`tableSize n_max`), so that + $8 n_{\max} \le 2^M < 16 \max(n_{\max}, 1)$ (load factor $\alpha \le 1/8$). Memory at `base` + consists of: + - `[base, base + KWise.space n_max 5)`: embedded $5$-wise independent trinomial-field hash state + (`Cslib.Algorithms.WordRAM.Hashing.KWise`); + - `[usedBase, usedBase + 2^M)`: occupancy flags `used[0 .. 2^M - 1]` (`0` = empty, `1` = + occupied, so zero-initialized memory is immediately a valid empty table); + - `[keyBase, keyBase + 2^M)`: stored keys `key[0 .. 2^M - 1]`; + - `[valBase, valBase + 2^M)`: stored values `val[0 .. 2^M - 1]`. + Total footprint is `space n_max = KWise.space n_max 5 + 3 * tableSize n_max = O(n_max)` words. + +2. **Self-contained operations (`initSetup`, `get`, `insert`):** + - `initSetup`: runs `KWise.initCode 5` ($O(n_{\max})$ deterministic Four-Russians carryless + multiplication table setup plus $5$ `Code.rand` calls, with zero caller-supplied primes). + - `get u` and `insert u v`: evaluate $b = h_\theta(u) \in \mathrm{Fin}(2^M)$ via + `KWise.evalCode 5` in $O(L^2 + 1)$ deterministic steps ($L = \mathrm{limbCount}(n_{\max}, U, M) + = O(1)$ when $U \le n_{\max}^c$), then scan $(b + d) \bmod 2^M$ until reaching the first slot + `p` where `used[p] = 0` or `key[p] = u`. Because at most $n_{\max} \le 2^M / 8$ slots are + occupied, at least $7/8$ of all slots are empty and the probe loop terminates deterministically + in $\le n_{\max} < 2^M$ iterations on every seed. + +3. **Expected $O(1)$ probe bound under $5$-wise independence:** + If probing for $u$ from $b = h_\theta(u)$ steps over $t \ge 1$ slots occupied by keys in + $S = \mathrm{dom}(s) \setminus \{u\}$, walking backwards from $b$ to the nearest preceding empty + slot yields a contiguous cyclic interval `cyclicInterval M b a ℓ` of length + $\ell = a + t \in [t, \#S]$ whose $\ell$ stored keys in $S$ all hash into + `cyclicInterval M b a ℓ` (since no probe path can cross the preceding empty slot). Applying + `tsum_uniform_probe_le_of_cluster_witness` from + `Complexity.Algorithms.WordRAM.Dictionary.LinearProbing.FiveWise` bounds the expected probe count + by $288 = O(1)$. + +## Model, specialization, and cost scope + +- **Fixed-parameter program specialization (`initParams = Fin.elim0`):** + For a fixed key universe size `U`, value bound `V`, and capacity bound `n_max` on an admissible + external machine word width `w` (`2 * trinomialDeg U (tableLog n_max) ≤ w`, + `Fits w (base + space n_max)`, `Fits w V`), `subroutineCode U n_max` specializes configuration + constants (`tableLog n_max`, `tableSize n_max`, `KWise` trinomial-field parameters, bit masks, + and relative array offsets) as instruction literals rather than decoding them from runtime + initialization registers. +- **Oblivious-adversary charged-body cost model:** + Expected instruction bounds (`subroutineDS`, `subroutineDSStandard`, and + `seq_expected_amortized`) charge fresh initialization (`initFresh`) and subroutine operation + bodies (`SubroutineDS.ExecSteps`) across an operation sequence `ops` fixed independently of the + sampled hash seed (oblivious adversary). Caller argument delivery, caller control/dispatch loops, + and host-level code/configuration construction are outside the subroutine's charged instruction + count. +- **Insert-only dictionary specification and zero tombstones:** + `Dictionary.spec U V n_max` models an insert-only bounded dictionary (`get` and `insert` with at + most `n_max` distinct keys; deletion is deferred to a follow-up), which allows open-addressed + slots to use `{0, 1}` occupancy flags (`0` = empty, `1` = occupied) with zero tombstones; + supporting deletion would require tombstone states or backward-shift deletion. + +## References + +* [A. Pagh, R. Pagh and M. Ružić, *Linear Probing with Constant Independence*][PaghPaghRuzic2009] +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Section 11.4][CLRS2009] +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.Dictionary.LinearProbing + +open Cslib.WordRAM Cslib.WordRAM.Randomized Cslib.Probability +open Cslib.Algorithms.Lean +open Cslib.Algorithms.WordRAM.ExactLog +open Cslib.Algorithms.WordRAM.Hashing +open Finset +open scoped BigOperators ENNReal + +/-! ## Section 1: Parameters, memory layout, and cyclic probe geometry -/ + +/-- Latent seed type for the embedded $5$-wise independent trinomial-field hash family: +$5$ coefficients in $\mathrm{Fin}(2^{\mathrm{trinomialDeg}(U, \mathrm{tableLog}(n_{\max}))})$. -/ +abbrev Seed (U n_max : ℕ) : Type := + Fin 5 → Fin (2 ^ trinomialDeg U (tableLog n_max)) + +/-- Uniform reference distribution over `Seed U n_max`. -/ +@[expose] noncomputable def seedLaw (U n_max : ℕ) : PMF (Seed U n_max) := + PMF.uniformOfFintype (Seed U n_max) + +/-- The $5$-wise independent hash function $h_\theta : \mathrm{Fin}\,U \to \mathrm{Fin}(2^M)$ +associated with seed $\theta$. -/ +@[expose] noncomputable def hashFn (U n_max : ℕ) (θ : Seed U n_max) (u : Fin U) : + Fin (2 ^ tableLog n_max) := + trinomialPolyHash (trinomialStep U (tableLog n_max)) 5 U (tableLog n_max) θ u + +/-- `hashFn U n_max` is a $5$-wise independent hash family. -/ +theorem isKWiseIndependentHash_hashFn (U n_max : ℕ) : + IsKWiseIndependentHash 5 (hashFn U n_max) := + isKWiseIndependentHash_trinomialPolyHash + (le_two_pow_trinomialDeg U (tableLog n_max)) + (le_trinomialDeg U (tableLog n_max)) + +/-- Base address of the occupancy array `used[0 .. 2^M - 1]`. -/ +@[expose] def usedBase (base n_max : ℕ) : ℕ := + base + KWise.space n_max 5 + +/-- Base address of the key array `key[0 .. 2^M - 1]`. -/ +@[expose] def keyBase (base n_max : ℕ) : ℕ := + usedBase base n_max + tableSize n_max + +/-- Base address of the value array `val[0 .. 2^M - 1]`. -/ +@[expose] def valBase (base n_max : ℕ) : ℕ := + keyBase base n_max + tableSize n_max + +/-- Total contiguous memory footprint of the linear-probing dictionary (in words): +`KWise.space n_max 5 + 3 * tableSize n_max = O(n_max)`. -/ +@[expose] def space (n_max : ℕ) : ℕ := + KWise.space n_max 5 + 3 * tableSize n_max + +/-- `space n_max` is $O(n_{\max})$: specifically `space n_max ≤ 49 * max n_max 4 + 13`. -/ +theorem space_le (n_max : ℕ) : space n_max ≤ 49 * max n_max 4 + 13 := by + dsimp only [space, KWise.space, KWise.spaceFromB] + have h1 := KWise.two_pow_two_mul_chunkBits_le n_max + have h2 := tableSize_le_sixteen_mul n_max + omega + +/-- Slot probed at offset `d` from starting bucket `b : Fin (2 ^ M)`: `(b.val + d) mod 2 ^ M`. -/ +@[expose] def probeSlot (M : ℕ) (b : Fin (2 ^ M)) (d : ℕ) : Fin (2 ^ M) := + cyclicSlotMod M (b.val + d) + +/-- Slot located `a` positions cyclically before `b : Fin (2 ^ M)`: `(b.val - a) mod 2 ^ M`. -/ +@[expose] def backSlot (M : ℕ) (b : Fin (2 ^ M)) (a : ℕ) : Fin (2 ^ M) := + cyclicSlotMod M (b.val + 2 ^ M * (a + 1) - a) + +@[simp] +theorem probeSlot_zero {M : ℕ} (b : Fin (2 ^ M)) : probeSlot M b 0 = b := by + apply Fin.ext + simp [probeSlot, cyclicSlotMod, Nat.mod_eq_of_lt b.isLt] + +theorem probeSlot_add {M : ℕ} (b : Fin (2 ^ M)) (d₁ d₂ : ℕ) : + probeSlot M (probeSlot M b d₁) d₂ = probeSlot M b (d₁ + d₂) := by + apply Fin.ext + simp [probeSlot, cyclicSlotMod, Nat.add_assoc] + +theorem probeSlot_backSlot_eq {M : ℕ} (b : Fin (2 ^ M)) (a idx : ℕ) : + probeSlot M (backSlot M b a) idx = + cyclicSlotMod M (b.val + 2 ^ M * (a + 1) - a + idx) := by + apply Fin.ext + simp [probeSlot, backSlot, cyclicSlotMod] + +theorem cyclicInterval_eq_image_probeSlot {M : ℕ} (b : Fin (2 ^ M)) (a len : ℕ) : + cyclicInterval M b a len = + (Finset.range len).image (probeSlot M (backSlot M b a)) := by + ext y + simp [cyclicInterval, probeSlot_backSlot_eq] + +theorem probeSlot_backSlot_add {M : ℕ} (b : Fin (2 ^ M)) (a d : ℕ) : + probeSlot M (backSlot M b a) (a + d) = probeSlot M b d := by + apply Fin.ext + simp only [probeSlot, backSlot, cyclicSlotMod] + have h1 : a ≤ 2 ^ M * (a + 1) := by + have : 1 ≤ 2 ^ M := Nat.one_le_two_pow + calc a ≤ 1 * (a + 1) := by omega + _ ≤ 2 ^ M * (a + 1) := Nat.mul_le_mul_right (a + 1) this + have heq : (b.val + 2 ^ M * (a + 1) - a) % 2 ^ M + (a + d) = + (b.val + 2 ^ M * (a + 1) - a) % 2 ^ M + a + d := by omega + rw [Nat.mod_add_mod, show b.val + 2 ^ M * (a + 1) - a + (a + d) = + b.val + d + 2 ^ M * (a + 1) by omega, Nat.add_mul_mod_self_left] + +theorem probeSlot_backSlot_sub {M : ℕ} (b : Fin (2 ^ M)) {a idx : ℕ} (hidx : idx ≤ a) : + probeSlot M (backSlot M b a) idx = backSlot M b (a - idx) := by + apply Fin.ext + simp only [probeSlot, backSlot, cyclicSlotMod] + have h1 : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have ha_le : a ≤ 2 ^ M * (a + 1) := by + calc a ≤ 1 * (a + 1) := by omega + _ ≤ 2 ^ M * (a + 1) := Nat.mul_le_mul_right (a + 1) h1 + have hsub_le : a - idx ≤ 2 ^ M * (a - idx + 1) := by + calc a - idx ≤ 1 * (a - idx + 1) := by omega + _ ≤ 2 ^ M * (a - idx + 1) := Nat.mul_le_mul_right (a - idx + 1) h1 + have hmul : 2 ^ M * (a + 1) = 2 ^ M * (a - idx + 1) + 2 ^ M * idx := by + rw [← Nat.mul_add] + congr 1 + omega + rw [Nat.mod_add_mod, show b.val + 2 ^ M * (a + 1) - a + idx = + (b.val + 2 ^ M * (a - idx + 1) - (a - idx)) + 2 ^ M * idx by omega, + Nat.add_mul_mod_self_left] + +theorem probeSlot_injOn_range {M : ℕ} (b : Fin (2 ^ M)) {len : ℕ} (hlen : len ≤ 2 ^ M) : + Set.InjOn (probeSlot M b) (Finset.range len) := by + intro i hi j hj hij + simp only [Finset.mem_coe, Finset.mem_range] at hi hj + have hval : (b.val + i) % 2 ^ M = (b.val + j) % 2 ^ M := congrArg Fin.val hij + have hb : b.val < 2 ^ M := b.isLt + have hi_mod : (b.val + i) % 2 ^ M = + if b.val + i < 2 ^ M then b.val + i else b.val + i - 2 ^ M := by + split_ifs with h + · exact Nat.mod_eq_of_lt h + · rw [Nat.mod_eq_sub_mod (by omega), Nat.mod_eq_of_lt (by omega)] + have hj_mod : (b.val + j) % 2 ^ M = + if b.val + j < 2 ^ M then b.val + j else b.val + j - 2 ^ M := by + split_ifs with h + · exact Nat.mod_eq_of_lt h + · rw [Nat.mod_eq_sub_mod (by omega), Nat.mod_eq_of_lt (by omega)] + rw [hi_mod, hj_mod] at hval + split_ifs at hval <;> omega + +theorem card_cyclicInterval {M : ℕ} (b : Fin (2 ^ M)) (a : ℕ) {len : ℕ} (hlen : len ≤ 2 ^ M) : + #(cyclicInterval M b a len) = len := by + rw [cyclicInterval_eq_image_probeSlot, + Finset.card_image_of_injOn (probeSlot_injOn_range (backSlot M b a) hlen), Finset.card_range] + +/-! ## Section 2: Open-addressed table invariant, termination, and cluster witness -/ + +/-- Occupancy flag `used[p]` at slot `p : Fin (2 ^ tableLog n_max)`. -/ +@[expose] def usedAt (base n_max : ℕ) (mem : ℕ → ℕ) (p : Fin (2 ^ tableLog n_max)) : ℕ := + mem (usedBase base n_max + p.val) + +/-- Stored key `key[p]` at slot `p : Fin (2 ^ tableLog n_max)`. -/ +@[expose] def keyAt (base n_max : ℕ) (mem : ℕ → ℕ) (p : Fin (2 ^ tableLog n_max)) : ℕ := + mem (keyBase base n_max + p.val) + +/-- Stored value `val[p]` at slot `p : Fin (2 ^ tableLog n_max)`. -/ +@[expose] def valAt (base n_max : ℕ) (mem : ℕ → ℕ) (p : Fin (2 ^ tableLog n_max)) : ℕ := + mem (valBase base n_max + p.val) + +/-- Linear-probing table invariant at `(base, n_max)` coupling abstract state +`s : Dictionary.State U` and seed `θ : Seed U n_max` with concrete memory `mem`: +1. `(Dictionary.dom s).card ≤ n_max` and `Dictionary.ValuesBounded V s`; +2. every slot `p` has `usedAt p ∈ {0, 1}`; +3. every occupied slot `p` (`usedAt p = 1`) stores a valid key `keyAt p < U` and value + `valAt p < V` matching `s ⟨keyAt p, _⟩ = some (valAt p)`; +4. occupied slots store pairwise distinct keys; +5. every present key `u` with `s u = some v` resides at some probe offset `k < 2 ^ M` from + `hashFn U n_max θ u` such that all preceding probe offsets `j < k` are occupied (`usedAt = 1`); +6. conversely, every occupied slot `p` (`usedAt p = 1`) is reachable from its key's home bucket + `hashFn U n_max θ ⟨keyAt p, _⟩` at some offset `k < 2 ^ M` along an entirely occupied prefix + (`∀ j < k, usedAt = 1`). -/ +structure TableInv (U V n_max base : ℕ) (θ : Seed U n_max) (s : Dictionary.State U) + (mem : ℕ → ℕ) : Prop where + /-- Number of stored keys is at most `n_max`. -/ + card_dom_le : (Dictionary.dom s).card ≤ n_max + /-- Every stored value in `s` is strictly less than `V`. -/ + values_bounded : Dictionary.ValuesBounded V s + /-- Every occupancy entry is `0` or `1`. -/ + used_bit : ∀ p : Fin (2 ^ tableLog n_max), + usedAt base n_max mem p = 0 ∨ usedAt base n_max mem p = 1 + /-- Every occupied slot holds a key `< U`. -/ + occupied_key_lt : ∀ p : Fin (2 ^ tableLog n_max), + usedAt base n_max mem p = 1 → keyAt base n_max mem p < U + /-- Every occupied slot agrees with the abstract dictionary `s`. -/ + occupied_state : ∀ (p : Fin (2 ^ tableLog n_max)) (u : Fin U), + usedAt base n_max mem p = 1 → keyAt base n_max mem p = u.val → + s u = some (valAt base n_max mem p) + /-- Distinct occupied slots hold distinct keys. -/ + occupied_inj : ∀ p₁ p₂ : Fin (2 ^ tableLog n_max), + usedAt base n_max mem p₁ = 1 → usedAt base n_max mem p₂ = 1 → + keyAt base n_max mem p₁ = keyAt base n_max mem p₂ → p₁ = p₂ + /-- Every present key `u` with `s u = some v` is stored at some offset `k < 2 ^ M` from its hash + bucket with all intermediate probe slots `j < k` occupied. -/ + present_slot : ∀ (u : Fin U) (v : ℕ), s u = some v → + ∃ k < 2 ^ tableLog n_max, + let p := probeSlot (tableLog n_max) (hashFn U n_max θ u) k + usedAt base n_max mem p = 1 ∧ + keyAt base n_max mem p = u.val ∧ + valAt base n_max mem p = v ∧ + ∀ j < k, usedAt base n_max mem (probeSlot (tableLog n_max) (hashFn U n_max θ u) j) = 1 + /-- Every occupied slot `p` has a contiguous occupied probe prefix from its key's hash bucket. -/ + occupied_prefix : ∀ (p : Fin (2 ^ tableLog n_max)) (u : Fin U), + usedAt base n_max mem p = 1 → keyAt base n_max mem p = u.val → + ∃ k < 2 ^ tableLog n_max, + p = probeSlot (tableLog n_max) (hashFn U n_max θ u) k ∧ + ∀ j < k, usedAt base n_max mem (probeSlot (tableLog n_max) (hashFn U n_max θ u) j) = 1 + +/-- Any finset of occupied slots `A` whose keys belong to `K : Finset (Fin U)` has cardinality +`#A ≤ #K`. -/ +theorem TableInv.card_slots_le {U V n_max base : ℕ} {θ : Seed U n_max} {s : Dictionary.State U} + {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (A : Finset (Fin (2 ^ tableLog n_max))) (K : Finset (Fin U)) + (hused : ∀ p ∈ A, usedAt base n_max mem p = 1) + (hkey : ∀ p (hp : p ∈ A), ⟨keyAt base n_max mem p, hinv.occupied_key_lt p (hused p hp)⟩ ∈ K) : + #A ≤ #K := by + have hU_pos : 0 < U ∨ A = ∅ := by + by_cases hA : A = ∅ + · exact Or.inr hA + · obtain ⟨p, hp⟩ := Finset.nonempty_iff_ne_empty.mpr hA + exact Or.inl (Nat.lt_of_le_of_lt (Nat.zero_le _) (hinv.occupied_key_lt p (hused p hp))) + rcases hU_pos with hU | rfl + · let f : Fin (2 ^ tableLog n_max) → Fin U := fun p => + if h : keyAt base n_max mem p < U then ⟨keyAt base n_max mem p, h⟩ else ⟨0, hU⟩ + refine Finset.card_le_card_of_injOn f (fun p hp => ?_) (fun p₁ hp₁ p₂ hp₂ heq => ?_) + · have hlt := hinv.occupied_key_lt p (hused p hp) + simpa [f, hlt] using hkey p hp + · simp only [Finset.mem_coe] at hp₁ hp₂ + have hlt₁ := hinv.occupied_key_lt p₁ (hused p₁ hp₁) + have hlt₂ := hinv.occupied_key_lt p₂ (hused p₂ hp₂) + have hval : keyAt base n_max mem p₁ = keyAt base n_max mem p₂ := by + have := congrArg Fin.val heq + simpa [f, hlt₁, hlt₂] using this + exact hinv.occupied_inj p₁ p₂ (hused p₁ hp₁) (hused p₂ hp₂) hval + · simp + +/-- Every occupied slot `p` with `keyAt p ≠ u.val` maps to a key in `(Dictionary.dom s).erase u`. -/ +theorem TableInv.key_mem_erase_of_ne {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + {p : Fin (2 ^ tableLog n_max)} (hp : usedAt base n_max mem p = 1) (u : Fin U) + (hne : keyAt base n_max mem p ≠ u.val) : + ⟨keyAt base n_max mem p, hinv.occupied_key_lt p hp⟩ ∈ (Dictionary.dom s).erase u := by + rw [Finset.mem_erase] + refine ⟨?_, ?_⟩ + · intro heq + exact hne (congrArg Fin.val heq) + · rw [Dictionary.mem_dom_iff] + exact ⟨valAt base n_max mem p, + hinv.occupied_state p ⟨keyAt base n_max mem p, hinv.occupied_key_lt p hp⟩ hp rfl⟩ + +/-- Probing for key `u` from bucket `b` continues past offset `d` when slot `probeSlot M b d` is +occupied (`used = 1`) by a key different from `u`. -/ +@[expose] def probeContinue (base n_max : ℕ) (mem : ℕ → ℕ) (u : ℕ) + (b : Fin (2 ^ tableLog n_max)) (d : ℕ) : Prop := + usedAt base n_max mem (probeSlot (tableLog n_max) b d) = 1 ∧ + keyAt base n_max mem (probeSlot (tableLog n_max) b d) ≠ u + +instance instDecidableProbeContinue (base n_max : ℕ) (mem : ℕ → ℕ) (u : ℕ) + (b : Fin (2 ^ tableLog n_max)) (d : ℕ) : + Decidable (probeContinue base n_max mem u b d) := + inferInstanceAs (Decidable (_ ∧ _)) + +/-- If probing for `u : Fin U` from `b` continues across all `i < d` (`d ≤ 2 ^ M`), then +`d ≤ #((Dictionary.dom s).erase u)`. -/ +theorem TableInv.probeContinue_prefix_le {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) {d : ℕ} (hd : d ≤ 2 ^ tableLog n_max) + (hcont : ∀ i < d, probeContinue base n_max mem u.val b i) : + d ≤ #((Dictionary.dom s).erase u) := by + set A := (Finset.range d).image (probeSlot (tableLog n_max) b) + have hcardA : #A = d := by + rw [Finset.card_image_of_injOn (probeSlot_injOn_range b hd), Finset.card_range] + rw [← hcardA] + refine hinv.card_slots_le A ((Dictionary.dom s).erase u) (fun p hp => ?_) (fun p hp => ?_) + · obtain ⟨i, hi, rfl⟩ := Finset.mem_image.mp hp + exact (hcont i (Finset.mem_range.mp hi)).1 + · obtain ⟨i, hi, rfl⟩ := Finset.mem_image.mp hp + have hi' := Finset.mem_range.mp hi + exact hinv.key_mem_erase_of_ne (hcont i hi').1 u (hcont i hi').2 + +/-- Under `TableInv`, probing for any `u : Fin U` from any `b` stops within +`#((Dictionary.dom s).erase u)` steps. -/ +theorem TableInv.exists_not_probeContinue {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : + ∃ d ≤ #((Dictionary.dom s).erase u), ¬probeContinue base n_max mem u.val b d := by + by_contra! h + set K := #((Dictionary.dom s).erase u) + have hK_le : K ≤ n_max := (Finset.card_erase_le).trans hinv.card_dom_le + have h8 : 8 * n_max ≤ 2 ^ tableLog n_max := eight_mul_le_tableSize n_max + have h8' : 8 ≤ 2 ^ tableLog n_max := eight_le_tableSize n_max + have hd_le : K + 1 ≤ 2 ^ tableLog n_max := by omega + have hle := hinv.probeContinue_prefix_le u b hd_le (fun i hi => h i (by omega)) + omega + +/-- Number of probe steps from `b` before reaching an empty slot or key `u`. -/ +@[expose] noncomputable def TableInv.probeLen {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : ℕ := + Nat.find (hinv.exists_not_probeContinue u b) + +theorem TableInv.probeLen_congr {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem₁ mem₂ : ℕ → ℕ} + (h₁ : TableInv U V n_max base θ s mem₁) (h₂ : TableInv U V n_max base θ s mem₂) + (hmem : mem₁ = mem₂) (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : + h₁.probeLen u b = h₂.probeLen u b := by + subst hmem + rfl + +theorem TableInv.probeLen_le_card_erase {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : + hinv.probeLen u b ≤ #((Dictionary.dom s).erase u) := + (Nat.find_spec (hinv.exists_not_probeContinue u b)).1 + +theorem TableInv.probeLen_le_n_max {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : + hinv.probeLen u b ≤ n_max := + (hinv.probeLen_le_card_erase u b).trans (Finset.card_erase_le.trans hinv.card_dom_le) + +theorem TableInv.probeLen_lt_tableSize {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : + hinv.probeLen u b < 2 ^ tableLog n_max := + lt_of_le_of_lt (hinv.probeLen_le_n_max u b) (n_max_lt_tableSize n_max) + +theorem TableInv.not_probeContinue_probeLen {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) : + ¬probeContinue base n_max mem u.val b (hinv.probeLen u b) := + (Nat.find_spec (hinv.exists_not_probeContinue u b)).2 + +theorem TableInv.probeContinue_of_lt_probeLen {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (b : Fin (2 ^ tableLog n_max)) {d : ℕ} (hd : d < hinv.probeLen u b) : + probeContinue base n_max mem u.val b d := by + by_contra hnot + exact Nat.find_min (hinv.exists_not_probeContinue u b) hd + ⟨(le_of_lt hd).trans (hinv.probeLen_le_card_erase u b), hnot⟩ + +/-- Exact characterization of the stopping slot `p = probeSlot M (hashFn U n_max θ u) t` where +`t = hinv.probeLen u (hashFn U n_max θ u)`: +- if `usedAt p = 0`, then `s u = none`; +- if `usedAt p = 1`, then `keyAt p = u.val` and `s u = some (valAt p)`. -/ +theorem TableInv.probeStop_spec {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) : + let b := hashFn U n_max θ u + let t := hinv.probeLen u b + let p := probeSlot (tableLog n_max) b t + (usedAt base n_max mem p = 0 ∧ s u = none) ∨ + (usedAt base n_max mem p = 1 ∧ keyAt base n_max mem p = u.val ∧ + s u = some (valAt base n_max mem p)) := by + set b := hashFn U n_max θ u + set t := hinv.probeLen u b + set p := probeSlot (tableLog n_max) b t + have hstop := hinv.not_probeContinue_probeLen u b + rcases hinv.used_bit p with hp0 | hp1 + · left + refine ⟨hp0, ?_⟩ + by_contra! hne + obtain ⟨v, hv⟩ := Option.ne_none_iff_exists'.mp hne + obtain ⟨k, hk_lt, hk_used, hk_key, _, hk_pref⟩ := hinv.present_slot u v hv + rcases lt_trichotomy k t with hlt | rfl | hgt + · have hcont := hinv.probeContinue_of_lt_probeLen u b hlt + exact hcont.2 hk_key + · exact absurd hp0 (by rw [hk_used]; decide) + · have hused_t := hk_pref t hgt + exact absurd hp0 (by rw [hused_t]; decide) + · right + have hkey : keyAt base n_max mem p = u.val := by + dsimp only [probeContinue, p] at hstop + tauto + exact ⟨hp1, hkey, hinv.occupied_state p u hp1 hkey⟩ + +/-- If a slot `p` has `usedAt p = 1` and `keyAt p = u.val`, then `p` is the stopping slot +`probeSlot M (hashFn U n_max θ u) (hinv.probeLen u (hashFn U n_max θ u))`. -/ +theorem TableInv.eq_probeStop_of_key_eq {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) {p : Fin (2 ^ tableLog n_max)} + (hp1 : usedAt base n_max mem p = 1) (hkey : keyAt base n_max mem p = u.val) : + p = probeSlot (tableLog n_max) (hashFn U n_max θ u) + (hinv.probeLen u (hashFn U n_max θ u)) := by + have hst := hinv.occupied_state p u hp1 hkey + rcases hinv.probeStop_spec u with ⟨_, hnone⟩ | ⟨hstop1, hstop_key, _⟩ + · rw [hnone] at hst + cases hst + · exact hinv.occupied_inj p _ hp1 hstop1 (hkey.trans hstop_key.symm) + +/-- Zeroed occupancy array `∀ p, usedAt base n_max mem p = 0` satisfies `TableInv` for +`Dictionary.empty U`. -/ +theorem tableInv_empty (U V n_max base : ℕ) (θ : Seed U n_max) {mem : ℕ → ℕ} + (hzero : ∀ p : Fin (2 ^ tableLog n_max), usedAt base n_max mem p = 0) : + TableInv U V n_max base θ (Dictionary.empty U) mem where + card_dom_le := by simp [Dictionary.dom_empty] + values_bounded := Dictionary.valuesBounded_empty U V + used_bit p := Or.inl (hzero p) + occupied_key_lt p hp := by rw [hzero p] at hp; cases hp + occupied_state p _ hp _ := by rw [hzero p] at hp; cases hp + occupied_inj p₁ _ hp₁ _ _ := by rw [hzero p₁] at hp₁; cases hp₁ + present_slot _ _ h := by simp [Dictionary.empty] at h + occupied_prefix p _ hp _ := by rw [hzero p] at hp; cases hp + +/-- Memory update performed by `insert u v` once the stopping slot `p` has been located: +writes `used[p] ← 1`, `key[p] ← u`, and `val[p] ← v`. -/ +@[expose] def insertMem (base n_max : ℕ) (mem : ℕ → ℕ) + (p : Fin (2 ^ tableLog n_max)) (u v : ℕ) : ℕ → ℕ := + Function.update + (Function.update + (Function.update mem (usedBase base n_max + p.val) 1) + (keyBase base n_max + p.val) u) + (valBase base n_max + p.val) v + +theorem usedAt_insertMem (base n_max : ℕ) (mem : ℕ → ℕ) + (p q : Fin (2 ^ tableLog n_max)) (u v : ℕ) : + usedAt base n_max (insertMem base n_max mem p u v) q = + if q = p then 1 else usedAt base n_max mem q := by + dsimp only [usedAt, insertMem, usedBase, keyBase, valBase, tableSize] + have hp := p.isLt + have hq := q.isLt + have hne1 : base + KWise.space n_max 5 + q.val ≠ + base + KWise.space n_max 5 + 2 ^ tableLog n_max + 2 ^ tableLog n_max + p.val := by omega + have hne2 : base + KWise.space n_max 5 + q.val ≠ + base + KWise.space n_max 5 + 2 ^ tableLog n_max + p.val := by omega + rw [Function.update_of_ne hne1, Function.update_of_ne hne2] + by_cases hqp : q = p + · subst hqp + simp + · have hval_ne : q.val ≠ p.val := fun h => hqp (Fin.ext h) + rw [ite_eq_right_iff.mpr (fun h => absurd h hqp), Function.update_of_ne (by omega)] + +theorem keyAt_insertMem (base n_max : ℕ) (mem : ℕ → ℕ) + (p q : Fin (2 ^ tableLog n_max)) (u v : ℕ) : + keyAt base n_max (insertMem base n_max mem p u v) q = + if q = p then u else keyAt base n_max mem q := by + dsimp only [keyAt, insertMem, usedBase, keyBase, valBase, tableSize] + have hp := p.isLt + have hq := q.isLt + have hne1 : base + KWise.space n_max 5 + 2 ^ tableLog n_max + q.val ≠ + base + KWise.space n_max 5 + 2 ^ tableLog n_max + 2 ^ tableLog n_max + p.val := by omega + have hne2 : base + KWise.space n_max 5 + 2 ^ tableLog n_max + q.val ≠ + base + KWise.space n_max 5 + p.val := by omega + rw [Function.update_of_ne hne1] + by_cases hqp : q = p + · subst hqp + simp + · have hval_ne : q.val ≠ p.val := fun h => hqp (Fin.ext h) + rw [ite_eq_right_iff.mpr (fun h => absurd h hqp), + Function.update_of_ne (by omega), Function.update_of_ne hne2] + +theorem valAt_insertMem (base n_max : ℕ) (mem : ℕ → ℕ) + (p q : Fin (2 ^ tableLog n_max)) (u v : ℕ) : + valAt base n_max (insertMem base n_max mem p u v) q = + if q = p then v else valAt base n_max mem q := by + dsimp only [valAt, insertMem, usedBase, keyBase, valBase, tableSize] + have hp := p.isLt + have hq := q.isLt + have hne1 : base + KWise.space n_max 5 + 2 ^ tableLog n_max + 2 ^ tableLog n_max + q.val ≠ + base + KWise.space n_max 5 + 2 ^ tableLog n_max + p.val := by omega + have hne2 : base + KWise.space n_max 5 + 2 ^ tableLog n_max + 2 ^ tableLog n_max + q.val ≠ + base + KWise.space n_max 5 + p.val := by omega + by_cases hqp : q = p + · subst hqp + simp + · have hval_ne : q.val ≠ p.val := fun h => hqp (Fin.ext h) + rw [ite_eq_right_iff.mpr (fun h => absurd h hqp), Function.update_of_ne (by omega), + Function.update_of_ne hne1, Function.update_of_ne hne2] + +/-- Writing `(1, u.val, v)` at the stopping slot `p = probeSlot M (hashFn U n_max θ u) t` +preserves `TableInv` for `Function.update s u (some v)` whenever `.insert u v` is admissible. -/ +theorem TableInv.insert_mem {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (v : ℕ) (hvalid : (Dictionary.spec U V n_max).valid s (.insert u v)) : + let b := hashFn U n_max θ u + let t := hinv.probeLen u b + let p := probeSlot (tableLog n_max) b t + TableInv U V n_max base θ (Function.update s u (some v)) + (insertMem base n_max mem p u.val v) := by + set b := hashFn U n_max θ u + set t := hinv.probeLen u b + set p := probeSlot (tableLog n_max) b t + set mem' := insertMem base n_max mem p u.val v + have ht_lt : t < 2 ^ tableLog n_max := hinv.probeLen_lt_tableSize u b + have hstop_spec := hinv.probeStop_spec u + have hused_ge : ∀ q, usedAt base n_max mem q = 1 → usedAt base n_max mem' q = 1 := by + intro q hq + rw [usedAt_insertMem] + split_ifs <;> [rfl; exact hq] + have hkey_ne_u : ∀ q, q ≠ p → usedAt base n_max mem q = 1 → + keyAt base n_max mem q ≠ u.val := by + intro q hne hq heq + exact hne (hinv.eq_probeStop_of_key_eq u hq heq) + refine + { card_dom_le := Dictionary.card_dom_step_le_of_valid hinv.card_dom_le (o := .insert u v) hvalid + values_bounded := + Dictionary.valuesBounded_step_of_valid hinv.values_bounded (o := .insert u v) hvalid + used_bit := fun q => by + rw [usedAt_insertMem] + split_ifs + · exact Or.inr rfl + · exact hinv.used_bit q + occupied_key_lt := fun q hq => by + rw [keyAt_insertMem] + rw [usedAt_insertMem] at hq + split_ifs at hq ⊢ with hqp + · exact u.isLt + · exact hinv.occupied_key_lt q hq + occupied_state := fun q u' hq hkey => by + change usedAt base n_max (insertMem base n_max mem p u.val v) q = 1 at hq + change keyAt base n_max (insertMem base n_max mem p u.val v) q = u'.val at hkey + change Function.update s u (some v) u' = + some (valAt base n_max (insertMem base n_max mem p u.val v) q) + rw [usedAt_insertMem] at hq + rw [keyAt_insertMem] at hkey + rw [valAt_insertMem] + by_cases hqp : q = p + · simp only [hqp, ↓reduceIte] at hkey ⊢ + have hu' : u' = u := Fin.ext hkey.symm + simp [hu'] + · simp only [hqp, ↓reduceIte] at hq hkey ⊢ + have hne_u := hkey_ne_u q hqp hq + have hu' : u' ≠ u := fun h => hne_u (hkey.trans (congrArg Fin.val h)) + rw [Function.update_of_ne hu'] + exact hinv.occupied_state q u' hq hkey + occupied_inj := fun q₁ q₂ hq₁ hq₂ hkey => by + change usedAt base n_max (insertMem base n_max mem p u.val v) q₁ = 1 at hq₁ + change usedAt base n_max (insertMem base n_max mem p u.val v) q₂ = 1 at hq₂ + change keyAt base n_max (insertMem base n_max mem p u.val v) q₁ = + keyAt base n_max (insertMem base n_max mem p u.val v) q₂ at hkey + rw [usedAt_insertMem] at hq₁ hq₂ + rw [keyAt_insertMem, keyAt_insertMem] at hkey + by_cases h1 : q₁ = p <;> by_cases h2 : q₂ = p + · rw [h1, h2] + · simp only [h1, h2, ↓reduceIte] at hq₂ hkey + exact absurd hkey.symm (hkey_ne_u q₂ h2 hq₂) + · simp only [h1, h2, ↓reduceIte] at hq₁ hkey + exact absurd hkey (hkey_ne_u q₁ h1 hq₁) + · simp only [h1, h2, ↓reduceIte] at hq₁ hq₂ hkey + exact hinv.occupied_inj q₁ q₂ hq₁ hq₂ hkey + present_slot := fun u' v' hst' => by + by_cases hu' : u' = u + · subst hu' + rw [Function.update_self, Option.some.injEq] at hst' + subst hst' + refine ⟨t, ht_lt, ?_, ?_, ?_, ?_⟩ + · change usedAt base n_max (insertMem base n_max mem p u'.val v) p = 1 + rw [usedAt_insertMem] + simp + · change keyAt base n_max (insertMem base n_max mem p u'.val v) p = u'.val + rw [keyAt_insertMem] + simp + · change valAt base n_max (insertMem base n_max mem p u'.val v) p = v + rw [valAt_insertMem] + simp + · intro j hj + exact hused_ge _ (hinv.probeContinue_of_lt_probeLen u' b hj).1 + · rw [Function.update_of_ne hu'] at hst' + obtain ⟨k, hk_lt, hk_used, hk_key, hk_val, hk_pref⟩ := hinv.present_slot u' v' hst' + set q := probeSlot (tableLog n_max) (hashFn U n_max θ u') k + have hqp : q ≠ p := by + intro heq + rcases hstop_spec with ⟨hp0, _⟩ | ⟨_, hp_key, _⟩ + · rw [heq, hp0] at hk_used + cases hk_used + · rw [heq, hp_key] at hk_key + exact hu' (Fin.ext hk_key.symm) + refine ⟨k, hk_lt, hused_ge q hk_used, ?_, ?_, fun j hj => hused_ge _ (hk_pref j hj)⟩ + · change keyAt base n_max (insertMem base n_max mem p u.val v) q = u'.val + rw [keyAt_insertMem] + simp [hqp, hk_key] + · change valAt base n_max (insertMem base n_max mem p u.val v) q = v' + rw [valAt_insertMem] + simp [hqp, hk_val] + occupied_prefix := fun q u' hq hkey => by + change usedAt base n_max (insertMem base n_max mem p u.val v) q = 1 at hq + change keyAt base n_max (insertMem base n_max mem p u.val v) q = u'.val at hkey + rw [usedAt_insertMem] at hq + rw [keyAt_insertMem] at hkey + by_cases hqp : q = p + · simp only [hqp, ↓reduceIte] at hkey + have hu' : u' = u := Fin.ext hkey.symm + subst hu' + exact ⟨t, ht_lt, hqp, fun j hj => + hused_ge _ (hinv.probeContinue_of_lt_probeLen u' b hj).1⟩ + · simp only [hqp, ↓reduceIte] at hq hkey + obtain ⟨k, hk_lt, hk_eq, hk_pref⟩ := hinv.occupied_prefix q u' hq hkey + exact ⟨k, hk_lt, hk_eq, fun j hj => hused_ge _ (hk_pref j hj)⟩ } + +private theorem probeSlot_backSlot_pred {M : ℕ} (b : Fin (2 ^ M)) (a : ℕ) : + probeSlot M (backSlot M b a) (2 ^ M - 1) = backSlot M b (a + 1) := by + apply Fin.ext + simp only [probeSlot, backSlot, cyclicSlotMod] + have h1 : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have ha_le : a ≤ 2 ^ M * (a + 1) := by + calc a ≤ 1 * (a + 1) := by omega + _ ≤ 2 ^ M * (a + 1) := Nat.mul_le_mul_right (a + 1) h1 + have hmul : 2 ^ M * (a + 1 + 1) = 2 ^ M * (a + 1) + 2 ^ M := by ring + rw [Nat.mod_add_mod, hmul] + congr 1 + omega + +private theorem probeSlot_sub_mod {M : ℕ} (o b₀ : Fin (2 ^ M)) : + probeSlot M o ((b₀.val + 2 ^ M - o.val) % 2 ^ M) = b₀ := by + apply Fin.ext + simp only [probeSlot, cyclicSlotMod] + have ho := o.isLt + have hb := b₀.isLt + rw [Nat.add_mod, Nat.mod_mod, ← Nat.add_mod, + show o.val + (b₀.val + 2 ^ M - o.val) = b₀.val + 2 ^ M by omega, + Nat.add_mod_right, Nat.mod_eq_of_lt hb] + +private theorem TableInv.backSlot_interval_used_ne {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (a : ℕ) + (hback : ∀ r, 1 ≤ r → r ≤ a → + usedAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) r) = 1 ∧ + keyAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) r) ≠ u.val) + {idx : ℕ} (hidx : idx < a + hinv.probeLen u (hashFn U n_max θ u)) : + usedAt base n_max mem + (probeSlot (tableLog n_max) (backSlot (tableLog n_max) (hashFn U n_max θ u) a) idx) = 1 ∧ + keyAt base n_max mem + (probeSlot (tableLog n_max) (backSlot (tableLog n_max) (hashFn U n_max θ u) a) idx) ≠ + u.val := by + by_cases hlt : idx < a + · rw [probeSlot_backSlot_sub (hashFn U n_max θ u) (by omega)] + exact hback (a - idx) (by omega) (by omega) + · have hge : a ≤ idx := le_of_not_gt hlt + have heq : idx = a + (idx - a) := by omega + rw [heq, probeSlot_backSlot_add] + exact hinv.probeContinue_of_lt_probeLen u (hashFn U n_max θ u) (by omega) + +/-- **Left-anchored cluster witness theorem**: if probing key `u` from `b = hashFn U n_max θ u` +takes `t = hinv.probeLen u b ≥ 1` steps, there is a contiguous cyclic interval +`cyclicInterval M b a ℓ` (`a < ℓ`, `t ≤ ℓ ≤ #((Dictionary.dom s).erase u)`) into which at least +`ℓ` keys of `(Dictionary.dom s).erase u` hash. -/ +theorem TableInv.hasClusterWitness_probeLen {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem : ℕ → ℕ} (hinv : TableInv U V n_max base θ s mem) + (u : Fin U) (ht : 1 ≤ hinv.probeLen u (hashFn U n_max θ u)) : + ∃ a ℓ, a < ℓ ∧ + hinv.probeLen u (hashFn U n_max θ u) ≤ ℓ ∧ + ℓ ≤ #((Dictionary.dom s).erase u) ∧ + ℓ ≤ #(((Dictionary.dom s).erase u).filter fun x => + hashFn U n_max θ x ∈ cyclicInterval (tableLog n_max) (hashFn U n_max θ u) a ℓ) := by + have ht_le_S : hinv.probeLen u (hashFn U n_max θ u) ≤ #((Dictionary.dom s).erase u) := + hinv.probeLen_le_card_erase u (hashFn U n_max θ u) + have hS_le_n : #((Dictionary.dom s).erase u) ≤ n_max := + Finset.card_erase_le.trans hinv.card_dom_le + have h8n : 8 * n_max ≤ 2 ^ tableLog n_max := eight_mul_le_tableSize n_max + have h8 : 8 ≤ 2 ^ tableLog n_max := eight_le_tableSize n_max + have hex_empty : ∃ r, 1 ≤ r ∧ + r ≤ #((Dictionary.dom s).erase u) + 1 - hinv.probeLen u (hashFn U n_max θ u) ∧ + ¬(usedAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) r) = 1 ∧ + keyAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) r) ≠ u.val) := by + by_contra! hall + set a_max := #((Dictionary.dom s).erase u) + 1 - hinv.probeLen u (hashFn U n_max θ u) + set A := cyclicInterval (tableLog n_max) (hashFn U n_max θ u) a_max + (#((Dictionary.dom s).erase u) + 1) + have hcardA : #A = #((Dictionary.dom s).erase u) + 1 := + card_cyclicInterval (hashFn U n_max θ u) a_max (by omega) + have hA_le : #A ≤ #((Dictionary.dom s).erase u) := by + refine hinv.card_slots_le A ((Dictionary.dom s).erase u) (fun q hq => ?_) (fun q hq => ?_) + · dsimp only [A] at hq + rw [cyclicInterval_eq_image_probeSlot] at hq + obtain ⟨idx, hidx, rfl⟩ := Finset.mem_image.mp hq + have hidx' : idx < a_max + hinv.probeLen u (hashFn U n_max θ u) := by + have := Finset.mem_range.mp hidx + omega + exact (hinv.backSlot_interval_used_ne u a_max (fun r hr1 hr2 => hall r hr1 hr2) hidx').1 + · dsimp only [A] at hq + rw [cyclicInterval_eq_image_probeSlot] at hq + obtain ⟨idx, hidx, rfl⟩ := Finset.mem_image.mp hq + have hidx' : idx < a_max + hinv.probeLen u (hashFn U n_max θ u) := by + have := Finset.mem_range.mp hidx + omega + have hboth := + hinv.backSlot_interval_used_ne u a_max (fun r hr1 hr2 => hall r hr1 hr2) hidx' + exact hinv.key_mem_erase_of_ne hboth.1 u hboth.2 + omega + set r₀ := Nat.find hex_empty + obtain ⟨hr₀_ge, hr₀_le, hr₀_not⟩ := Nat.find_spec hex_empty + set a := r₀ - 1 + set ℓ := a + hinv.probeLen u (hashFn U n_max θ u) + have ha1 : a + 1 = r₀ := by omega + have hℓ_le_S : ℓ ≤ #((Dictionary.dom s).erase u) := by omega + have hback_occ : ∀ r, 1 ≤ r → r ≤ a → + usedAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) r) = 1 ∧ + keyAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) r) ≠ u.val := by + intro r hr1 hra + have hr_lt : r < r₀ := by omega + have hmin := Nat.find_min hex_empty hr_lt + push Not at hmin + exact hmin hr1 (by omega) + have hprev_zero : + usedAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1)) = 0 := by + rcases hinv.used_bit (backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1)) with h0 | h1 + · exact h0 + · exfalso + have hkey_u : + keyAt base n_max mem (backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1)) = + u.val := by + rw [ha1] at h1 ⊢ + tauto + have heq_stop := hinv.eq_probeStop_of_key_eq u h1 hkey_u + have h0_eq : + backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1) = + probeSlot (tableLog n_max) (backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1)) 0 := + (probeSlot_zero _).symm + have ht_eq : + probeSlot (tableLog n_max) (hashFn U n_max θ u) (hinv.probeLen u (hashFn U n_max θ u)) = + probeSlot (tableLog n_max) (backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1)) + (a + 1 + hinv.probeLen u (hashFn U n_max θ u)) := + (probeSlot_backSlot_add (hashFn U n_max θ u) (a + 1) _).symm + rw [h0_eq, ht_eq] at heq_stop + have hinj := probeSlot_injOn_range + (backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1)) + (len := 2 ^ tableLog n_max) le_rfl + (Finset.mem_coe.mpr (Finset.mem_range.mpr (by omega))) + (Finset.mem_coe.mpr (Finset.mem_range.mpr (by omega))) heq_stop + omega + refine ⟨a, ℓ, by omega, by omega, hℓ_le_S, ?_⟩ + set J := cyclicInterval (tableLog n_max) (hashFn U n_max θ u) a ℓ + have hcardJ : #J = ℓ := card_cyclicInterval (hashFn U n_max θ u) a (by omega) + rw [← hcardJ] + have hJ_occ : ∀ q ∈ J, usedAt base n_max mem q = 1 ∧ keyAt base n_max mem q ≠ u.val := by + intro q hq + dsimp only [J] at hq + rw [cyclicInterval_eq_image_probeSlot] at hq + obtain ⟨idx, hidx, rfl⟩ := Finset.mem_image.mp hq + exact hinv.backSlot_interval_used_ne u a hback_occ (Finset.mem_range.mp hidx) + refine hinv.card_slots_le J (((Dictionary.dom s).erase u).filter fun x => hashFn U n_max θ x ∈ J) + (fun q hq => (hJ_occ q hq).1) (fun q hq => ?_) + rw [Finset.mem_filter] + refine ⟨hinv.key_mem_erase_of_ne (hJ_occ q hq).1 u (hJ_occ q hq).2, ?_⟩ + set x_q : Fin U := ⟨keyAt base n_max mem q, hinv.occupied_key_lt q (hJ_occ q hq).1⟩ + obtain ⟨k, hk_lt, hq_eq, hk_pref⟩ := hinv.occupied_prefix q x_q (hJ_occ q hq).1 rfl + set b₀ := hashFn U n_max θ x_q + set o := backSlot (tableLog n_max) (hashFn U n_max θ u) a + have hq_mem : q ∈ J := hq + dsimp only [J] at hq_mem + rw [cyclicInterval_eq_image_probeSlot] at hq_mem + obtain ⟨idx, hidx_mem, hq_idx⟩ := Finset.mem_image.mp hq_mem + have hidx_lt : idx < ℓ := Finset.mem_range.mp hidx_mem + set r := (b₀.val + 2 ^ tableLog n_max - o.val) % 2 ^ tableLog n_max + have hr_lt : r < 2 ^ tableLog n_max := Nat.mod_lt _ (by positivity) + have hb₀_eq : probeSlot (tableLog n_max) o r = b₀ := probeSlot_sub_mod o b₀ + by_cases hr_lt_ℓ : r < ℓ + · dsimp only [J] + rw [cyclicInterval_eq_image_probeSlot, Finset.mem_image] + exact ⟨r, Finset.mem_range.mpr hr_lt_ℓ, hb₀_eq⟩ + · exfalso + have hq_rk : q = probeSlot (tableLog n_max) o ((r + k) % 2 ^ tableLog n_max) := by + rw [hq_eq, ← hb₀_eq, probeSlot_add] + apply Fin.ext + simp [probeSlot, cyclicSlotMod] + have hrk_eq_idx : (r + k) % 2 ^ tableLog n_max = idx := by + refine probeSlot_injOn_range o (len := 2 ^ tableLog n_max) le_rfl + (Finset.mem_coe.mpr (Finset.mem_range.mpr (Nat.mod_lt _ (by positivity)))) + (Finset.mem_coe.mpr (Finset.mem_range.mpr (by omega))) ?_ + rw [← hq_rk, hq_idx] + have hge : 2 ^ tableLog n_max ≤ r + k := by + by_contra hlt + have hmod0 : (r + k) % 2 ^ tableLog n_max = r + k := Nat.mod_eq_of_lt (not_le.mp hlt) + omega + have hmod : (r + k) % 2 ^ tableLog n_max = r + k - 2 ^ tableLog n_max := by + rw [Nat.mod_eq_sub_mod hge, Nat.mod_eq_of_lt (by omega)] + have hrk_sum : r + k = 2 ^ tableLog n_max + idx := by omega + set j := 2 ^ tableLog n_max - 1 - r + have hj_lt_k : j < k := by omega + have hused_j := hk_pref j hj_lt_k + have hslot_j : probeSlot (tableLog n_max) b₀ j = + backSlot (tableLog n_max) (hashFn U n_max θ u) (a + 1) := by + rw [← hb₀_eq, probeSlot_add, show r + j = 2 ^ tableLog n_max - 1 by omega] + exact probeSlot_backSlot_pred (hashFn U n_max θ u) a + rw [hslot_j, hprev_zero] at hused_j + cases hused_j + +noncomputable instance instDecidablePredHasClusterWitness (U n_max : ℕ) (S : Finset (Fin U)) + (u : Fin U) (θ : Seed U n_max) : + DecidablePred (HasClusterWitness (hashFn U n_max) S u θ) := + Classical.decPred _ + +/-- Maximum `t ≤ #S` satisfying `HasClusterWitness (hashFn U n_max) S u θ t` (depends only on +`θ`). -/ +@[expose] noncomputable def maxWitnessProbe (U n_max : ℕ) (S : Finset (Fin U)) (u : Fin U) + (θ : Seed U n_max) : ℕ := + have h0 : 0 ∈ (Finset.range (#S + 1)).filter (HasClusterWitness (hashFn U n_max) S u θ) := + Finset.mem_filter.mpr ⟨Finset.mem_range.mpr (by omega), fun h => by omega⟩ + ((Finset.range (#S + 1)).filter (HasClusterWitness (hashFn U n_max) S u θ)).max' ⟨0, h0⟩ + +theorem le_maxWitnessProbe {U n_max : ℕ} {S : Finset (Fin U)} {u : Fin U} + {θ : Seed U n_max} {t : ℕ} (ht_le : t ≤ #S) + (hwit : HasClusterWitness (hashFn U n_max) S u θ t) : + t ≤ maxWitnessProbe U n_max S u θ := by + dsimp only [maxWitnessProbe] + exact Finset.le_max' _ _ (Finset.mem_filter.mpr ⟨Finset.mem_range.mpr (by omega), hwit⟩) + +theorem hasClusterWitness_maxWitnessProbe (U n_max : ℕ) (S : Finset (Fin U)) (u : Fin U) + (θ : Seed U n_max) : + HasClusterWitness (hashFn U n_max) S u θ (maxWitnessProbe U n_max S u θ) := by + dsimp only [maxWitnessProbe] + exact (Finset.mem_filter.mp (Finset.max'_mem _ _)).2 + +/-- For any coupled state distribution `ν : PMF (Seed U n_max × α)` whose seed marginal is bounded +by `seedLaw U n_max`, any step counter `T_probe : Seed U n_max × α → ℕ` bounded by `hinv.probeLen` +on `ν.support` has expected value at most `288`. -/ +theorem tsum_probeLen_le {α : Type*} {U V n_max base : ℕ} + {s : Dictionary.State U} + (u : Fin U) (ν : PMF (Seed U n_max × α)) + (hseed : ∀ θ, (ν.map Prod.fst) θ ≤ seedLaw U n_max θ) + (T_probe : Seed U n_max × α → ℕ) + (hT : ∀ p ∈ ν.support, ∃ (mem : ℕ → ℕ) (hinv : TableInv U V n_max base p.1 s mem), + T_probe p ≤ hinv.probeLen u (hashFn U n_max p.1 u)) : + (∑' p, ν p * (T_probe p : ℝ≥0∞)) ≤ 288 := by + obtain ⟨p₀, hp₀⟩ := ν.support_nonempty + obtain ⟨_, hinv₀, _⟩ := hT p₀ hp₀ + have hdom_le : (Dictionary.dom s).card ≤ n_max := hinv₀.card_dom_le + set S := (Dictionary.dom s).erase u + have hu : u ∉ S := Finset.notMem_erase u _ + have hS : #S ≤ n_max := Finset.card_erase_le.trans hdom_le + have hM : 8 * n_max ≤ 2 ^ tableLog n_max := eight_mul_le_tableSize n_max + have hh := isKWiseIndependentHash_hashFn U n_max + have hpt : ∀ p, ν p * (T_probe p : ℝ≥0∞) ≤ + ν p * (maxWitnessProbe U n_max S u p.1 : ℝ≥0∞) := by + intro p + by_cases hp : p ∈ ν.support + · obtain ⟨mem, hinv, hle⟩ := hT p hp + have hwit_p : HasClusterWitness (hashFn U n_max) S u p.1 + (hinv.probeLen u (hashFn U n_max p.1 u)) := + fun ht => hinv.hasClusterWitness_probeLen u ht + have hle_max := le_maxWitnessProbe + (hinv.probeLen_le_card_erase u (hashFn U n_max p.1 u)) hwit_p + gcongr + exact hle.trans hle_max + · rw [(PMF.apply_eq_zero_iff _ _).mpr hp, zero_mul, zero_mul] + calc (∑' p, ν p * (T_probe p : ℝ≥0∞)) + ≤ ∑' p, ν p * (maxWitnessProbe U n_max S u p.1 : ℝ≥0∞) := + tsum_le_tsum hpt + _ = ∑' θ, (ν.map Prod.fst) θ * (maxWitnessProbe U n_max S u θ : ℝ≥0∞) := + (Cslib.Probability.PMF.tsum_map_mul ν Prod.fst + (fun θ => (maxWitnessProbe U n_max S u θ : ℝ≥0∞))).symm + _ ≤ ∑' θ, seedLaw U n_max θ * (maxWitnessProbe U n_max S u θ : ℝ≥0∞) := + tsum_le_tsum (fun θ => mul_le_mul_of_nonneg_right (hseed θ) zero_le) + _ ≤ 288 := + tsum_uniform_probe_le_of_cluster_witness hh S hu hS hM + (maxWitnessProbe U n_max S u) (hasClusterWitness_maxWitnessProbe U n_max S u) + +/-! ## Section 3: Word-RAM register layout, programs, and operational execution -/ + +open Cslib.Algorithms.WordRAM.Hashing.Tabulation + +/-- Total number of registers used by the linear-probing subroutine (`0 .. 17` for `KWise` and +probe scratch, `18 .. 23` for allocation, zero-fill, and operation arguments). -/ +abbrev rLP : ℕ := 24 + +/-- Register embedding `Fin 18 → Fin 24` mapping `KWise` registers `0 .. 17` to `0 .. 17`. -/ +@[expose] def kwiseMap : Fin 18 → Fin 24 := + Fin.castLE (by decide) + +theorem injective_kwiseMap : Function.Injective kwiseMap := + Fin.castLE_injective (by decide) + +/-- Allocation span program: copies `frontierReg` (`18`) to `baseReg` (`0`) and advances +`frontierReg` by `space n_max`. -/ +@[expose] def allocSpan (n_max : ℕ) : Code .exact 24 := + .seq (.op (.copy 0 18)) + (.seq (.op (.const 3 (space n_max))) + (.op (.exactBin .add 18 0 3))) + +/-- `18`-register initialization program setting `(n_max, U, tableLog n_max)` in registers +`1, 2, 3` and invoking `KWise.initCode 5`. -/ +@[expose] def initSetup18 (U n_max : ℕ) : Code .exact 18 := + .seq (.op (.const 1 n_max)) + (.seq (.op (.const 2 U)) + (.seq (.op (.const 3 (tableLog n_max))) + (KWise.initCode 5))) + +/-- `24`-register initialization program. -/ +@[expose] def initSetup (U n_max : ℕ) : Code .exact 24 := + (initSetup18 U n_max).rename kwiseMap + +/-- Single-slot probe inspection program: given current slot `pos` in `reg 2`, loads +`used[pos]` into `reg 9` (`cellReg`) and sets `reg 7` (`contReg`) to `1` if +`used[pos] = 1 ∧ key[pos] ≠ u.val`, or `0` otherwise. -/ +@[expose] def probeCheckCode : Code .exact 24 := + .seq (.op (.exactBin .add 8 4 2)) + (.seq (.op (.load 9 8)) + (.branch (.eq 9 10) + (.seq (.op (.exactBin .add 8 5 2)) + (.seq (.op (.load 8 8)) + (.branch (.eq 8 1) + (.op (.const 7 0)) + (.op (.const 7 1))))) + (.op (.const 7 0)))) + +/-- Probe setup program: initializes `maskReg` (`3`), `usedBaseReg` (`4`), `keyBaseReg` (`5`), +`valBaseReg` (`6`), and `cOneReg` (`10`), then inspects the home slot `pos = (h u).val` (`reg 2`). +-/ +@[expose] def probeSetupCode (n_max : ℕ) : Code .exact 24 := + .seq (.op (.const 3 (tableSize n_max - 1))) + (.seq (.op (.const 4 (KWise.space n_max 5))) + (.seq (.op (.exactBin .add 4 0 4)) + (.seq (.op (.const 5 (tableSize n_max))) + (.seq (.op (.exactBin .add 5 4 5)) + (.seq (.op (.const 6 (tableSize n_max))) + (.seq (.op (.exactBin .add 6 5 6)) + (.seq (.op (.const 10 1)) + probeCheckCode))))))) + +/-- Probe loop body: advances `pos ← (pos + 1) &&& (2 ^ M - 1)` in `reg 2` and inspects the new +slot via `probeCheckCode`. -/ +@[expose] def probeBody : Code .exact 24 := + .seq (.op (.exactBin .add 2 2 10)) + (.seq (.op (.exactBin .and 2 2 3)) + probeCheckCode) + +/-- Linear probe loop `while contReg == 1 do probeBody`. -/ +@[expose] def probeLoop : Code .exact 24 := + .loop (.eq 7 10) probeBody + +/-- Insertion suffix after `probeLoop` halts at slot `pos` (`reg 2`): stores `used[pos] ← 1`, +`key[pos] ← u.val` (`reg 1`), `val[pos] ← v` (`reg 23`), and sets `outReg` (`reg 2`) to `0`. -/ +@[expose] def insertFinishCode : Code .exact 24 := + .seq (.op (.exactBin .add 8 4 2)) + (.seq (.op (.store 8 10)) + (.seq (.op (.exactBin .add 8 5 2)) + (.seq (.op (.store 8 1)) + (.seq (.op (.exactBin .add 8 6 2)) + (.seq (.op (.store 8 23)) + (.op (.const 2 0))))))) + +/-- Lookup suffix after `probeLoop` halts at slot `pos` (`reg 2`) with `used[pos]` in `reg 9`: +if `reg 9 == 1`, loads `val[pos]` and writes `val[pos] + 1` into `reg 2`; otherwise writes `0` +into `reg 2`. -/ +@[expose] def getFinishCode : Code .exact 24 := + .branch (.eq 9 10) + (.seq (.op (.exactBin .add 8 6 2)) + (.seq (.op (.load 2 8)) + (.op (.exactBin .add 2 2 10)))) + (.op (.const 2 0)) + +/-- Unified operation program for `get u` (`opReg = 0`) and `insert u v` (`opReg = 1`). -/ +@[expose] def opCode (n_max : ℕ) : Code .exact 24 := + .seq ((KWise.evalCode 5).rename kwiseMap) + (.seq (probeSetupCode n_max) + (.seq probeLoop + (.branch (.eq 22 10) insertFinishCode getFinishCode))) + +/-- `SubroutineCode` bundle for linear probing. -/ +@[expose] def subroutineCode (U n_max : ℕ) : SubroutineCode .exact rLP 0 where + allocSpan := allocSpan n_max + initSetup := initSetup U n_max + code := opCode n_max + baseReg := 0 + frontierReg := 18 + zeroReg := 19 + oneReg := 20 + curReg := 21 + incCur := .exactBin .add 21 21 20 + initParams := Fin.elim0 + input i := decide (i = 1 ∨ i = 22 ∨ i = 23) + output := 2 + +/-- Standalone argument encoding for `Dictionary.Op U`. -/ +@[expose] def encodeOp {U : ℕ} (o : Dictionary.Op U) (i : Fin rLP) : ℕ := + match o with + | .get u => + if i = 1 then u.val else 0 + | .insert u v => + if i = 1 then u.val else if i = 22 then 1 else if i = 23 then v else 0 + +/-- Argument-matching predicate for `Dictionary.Op U`. -/ +@[expose] def argMatchOp {U : ℕ} (o : Dictionary.Op U) (regs : Fin rLP → ℕ) : Prop := + match o with + | .get u => regs 1 = u.val ∧ regs 22 = 0 + | .insert u v => regs 1 = u.val ∧ regs 22 = 1 ∧ regs 23 = v + +/-- Output decoder for `Dictionary.Op U`: for `get u`, `0` decodes to `none` and `k + 1` decodes to +`some k`; for `insert u v`, decodes to `some ()`. -/ +@[expose] def decodeOp {U V n_max : ℕ} (o : (Dictionary.spec U V n_max).Op) + (regs : Fin rLP → ℕ) : Option ((Dictionary.spec U V n_max).Result o) := + match o with + | .get _ => some (if regs 2 = 0 then none else some (regs 2 - 1)) + | .insert _ _ => some () + +/-- Expected instruction count for `initSetup U n_max`: +`KWise.initTableSteps n_max U (tableLog n_max) + 38 = O(n_max + log log U)`. -/ +@[expose] def setupTime (U n_max : ℕ) : ℕ := + KWise.initTableSteps n_max U (tableLog n_max) + 38 + +/-- Expected instruction charge per dictionary operation: +`KWise.evalSteps n_max U (tableLog n_max) 5 + 3483 = O(L ^ 2 + 1)`. -/ +@[expose] def charge (U n_max : ℕ) : ℕ := + KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * 288 + 27 + +/-- Full seed-coupled representation invariant `Rep U V n_max w base θ s x` for linear probing. -/ +structure Rep (U V n_max w base : ℕ) (θ : Seed U n_max) (s : Dictionary.State U) + (x : Data rLP) : Prop where + /-- Machine configuration is word-bounded at width `w`. -/ + bounded : x.Bounded w + /-- Persistent base register `reg 0` holds `base`. -/ + reg_base : x.regs 0 = base + /-- Embedded `5`-wise independent hash representation at `[base, base + KWise.space n_max 5)`. -/ + kwise_rep : KWise.Rep base (KWise.chunkBits n_max) U (tableLog n_max) 5 θ x.mem + /-- Open-addressed linear-probing table invariant at `[usedBase, valBase + tableSize n_max)`. -/ + table_inv : TableInv U V n_max base θ s x.mem + +theorem Rep.unique {U V n_max w base : ℕ} {θ : Seed U n_max} + {s₁ s₂ : Dictionary.State U} {x : Data rLP} + (h₁ : Rep U V n_max w base θ s₁ x) (h₂ : Rep U V n_max w base θ s₂ x) : + s₁ = s₂ := by + funext u + rcases h₁.table_inv.probeStop_spec u with ⟨hp0, hnone₁⟩ | ⟨hp1, hkey₁, hsome₁⟩ + · rcases h₂.table_inv.probeStop_spec u with ⟨_, hnone₂⟩ | ⟨hp1', hkey₂, _⟩ + · rw [hnone₁, hnone₂] + · have heq := h₁.table_inv.eq_probeStop_of_key_eq u hp1' hkey₂ + rw [← heq, hp1'] at hp0 + cases hp0 + · rcases h₂.table_inv.probeStop_spec u with ⟨hp0', _⟩ | ⟨hp1', hkey₂, hsome₂⟩ + · have heq := h₂.table_inv.eq_probeStop_of_key_eq u hp1 hkey₁ + rw [← heq, hp1] at hp0' + cases hp0' + · have heq := h₁.table_inv.occupied_inj _ _ hp1 hp1' (hkey₁.trans hkey₂.symm) + rw [hsome₁, hsome₂, heq] + +/-- `KWise.Rep` depends only on addresses in `[base, base + KWise.space n_max 5)`. -/ +private theorem kwise_rep_congr {U n_max base : ℕ} {θ : Seed U n_max} {mem mem' : ℕ → ℕ} + (hrep : KWise.Rep base (KWise.chunkBits n_max) U (tableLog n_max) 5 θ mem) + (heq : ∀ a, a < base + KWise.space n_max 5 → base ≤ a → mem' a = mem a) : + KWise.Rep base (KWise.chunkBits n_max) U (tableLog n_max) 5 θ mem' := by + have hk_mem : ∀ offset < KWise.space n_max 5, mem' (base + offset) = mem (base + offset) := + fun offset hlt => heq (base + offset) (by omega) (Nat.le_add_right _ _) + have hsp : KWise.space n_max 5 = 8 + 5 + 2 ^ (2 * KWise.chunkBits n_max) := rfl + have hpow_pos : 0 < 2 ^ (2 * KWise.chunkBits n_max) := by positivity + rcases hrep with ⟨htab, hseed⟩ + refine ⟨⟨?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩, ?_⟩ + · have := hk_mem 0 (by omega); rw [Nat.add_zero] at this; rw [this]; exact htab.1 + · rw [hk_mem 1 (by omega)]; exact htab.2.1 + · rw [hk_mem 2 (by omega)]; exact htab.2.2.1 + · rw [hk_mem 3 (by omega)]; exact htab.2.2.2.1 + · rw [hk_mem 4 (by omega)]; exact htab.2.2.2.2.1 + · rw [hk_mem 5 (by omega)]; exact htab.2.2.2.2.2.1 + · rw [hk_mem 6 (by omega)]; exact htab.2.2.2.2.2.2.1 + · rw [hk_mem 7 (by omega)]; exact htab.2.2.2.2.2.2.2.1 + · intro idx hidx + have h1 : base + 8 + 5 + idx = base + (8 + 5 + idx) := by omega + rw [h1, hk_mem (8 + 5 + idx) (by omega), ← h1] + exact htab.2.2.2.2.2.2.2.2 idx hidx + · intro j + have hj := j.isLt + have h1 : base + 8 + j.val = base + (8 + j.val) := by omega + rw [h1, hk_mem (8 + j.val) (by omega), ← h1] + exact hseed j + +/-- `TableInv` depends only on addresses in `[base, base + space n_max)`. -/ +private theorem tableInv_congr {U V n_max base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {mem mem' : ℕ → ℕ} + (hinv : TableInv U V n_max base θ s mem) + (heq : ∀ a ∈ (Block.mk base (space n_max)).footprint, mem' a = mem a) : + TableInv U V n_max base θ s mem' := by + have hused : ∀ p : Fin (2 ^ tableLog n_max), + usedAt base n_max mem' p = usedAt base n_max mem p := by + intro p + have hp := p.isLt + apply heq + rw [Block.mem_footprint, Block.Owns] + dsimp only [usedAt, usedBase, space, tableSize] + exact ⟨by omega, by omega⟩ + have hkey : ∀ p : Fin (2 ^ tableLog n_max), + keyAt base n_max mem' p = keyAt base n_max mem p := by + intro p + have hp := p.isLt + apply heq + rw [Block.mem_footprint, Block.Owns] + dsimp only [keyAt, keyBase, usedBase, space, tableSize] + exact ⟨by omega, by omega⟩ + have hval : ∀ p : Fin (2 ^ tableLog n_max), + valAt base n_max mem' p = valAt base n_max mem p := by + intro p + have hp := p.isLt + apply heq + rw [Block.mem_footprint, Block.Owns] + dsimp only [valAt, valBase, keyBase, usedBase, space, tableSize] + exact ⟨by omega, by omega⟩ + refine + { card_dom_le := hinv.card_dom_le + values_bounded := hinv.values_bounded + used_bit := fun p => by rw [hused]; exact hinv.used_bit p + occupied_key_lt := fun p hp => by rw [hkey]; rw [hused] at hp; exact hinv.occupied_key_lt p hp + occupied_state := fun p u hp hk => by + rw [hused] at hp + rw [hkey] at hk + rw [hval] + exact hinv.occupied_state p u hp hk + occupied_inj := fun p q hp hq hk => by + rw [hused] at hp hq + rw [hkey, hkey] at hk + exact hinv.occupied_inj p q hp hq hk + present_slot := fun u v hsu => by + obtain ⟨k, hk_lt, hp, hk, hv, hpref⟩ := hinv.present_slot u v hsu + refine ⟨k, hk_lt, by rwa [hused], by rwa [hkey], by rwa [hval], fun j hj => ?_⟩ + rw [hused] + exact hpref j hj + occupied_prefix := fun p u hp hk => by + rw [hused] at hp + rw [hkey] at hk + obtain ⟨k, hk_lt, hp_eq, hpref⟩ := hinv.occupied_prefix p u hp hk + refine ⟨k, hk_lt, hp_eq, fun j hj => ?_⟩ + rw [hused] + exact hpref j hj } + +theorem Rep.supported {U V n_max w base : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x x' : Data rLP} + (hregs : ∀ i, (subroutineCode U n_max).input i = false → x'.regs i = x.regs i) + (hmem : ∀ a ∈ (Block.mk base (space n_max)).footprint, x'.mem a = x.mem a) + (hbdd : x'.Bounded w) + (hrep : Rep U V n_max w base θ s x) : + Rep U V n_max w base θ s x' := by + have h0 : x'.regs 0 = base := by + rw [hregs 0 rfl, hrep.reg_base] + refine ⟨hbdd, h0, kwise_rep_congr hrep.kwise_rep fun a hlt hge => ?_, + tableInv_congr hrep.table_inv hmem⟩ + apply hmem + rw [Block.mem_footprint, Block.Owns] + dsimp only [space] + exact ⟨hge, by omega⟩ + +private theorem extend_kwiseMap_regs {y : Data 18} {x : Data 24} {j : Fin 24} (hj : 18 ≤ j.val) : + (y.extend kwiseMap x).regs j = x.regs j := by + change Function.extend kwiseMap y.regs x.regs j = x.regs j + apply Function.extend_apply' + rintro ⟨i, rfl⟩ + have := i.isLt + dsimp only [kwiseMap, Fin.val_castLE] at hj + omega + +private theorem fits_q_of_hw {w U n_max : ℕ} + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) : + Fits w (2 ^ trinomialDeg U (tableLog n_max)) := by + set q := trinomialDeg U (tableLog n_max) + have hq_pos : 2 ≤ q := by + dsimp only [q, trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U (tableLog n_max) := Nat.one_le_pow _ 3 (by decide) + omega + change 2 ^ q < 2 ^ w + exact (Nat.pow_lt_pow_iff_right one_lt_two).mpr (by omega) + +private theorem fits_max_U_add_one_of_hw {w U n_max : ℕ} + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) : + Fits w (max U 2 + 1) := by + set q := trinomialDeg U (tableLog n_max) + have hq_pos : 2 ≤ q := by + dsimp only [q, trinomialDeg] + have : 1 ≤ 3 ^ trinomialStep U (tableLog n_max) := Nat.one_le_pow _ 3 (by decide) + omega + have hU_le : U ≤ 2 ^ q := le_two_pow_trinomialDeg U (tableLog n_max) + have hpow_ge : 4 ≤ 2 ^ q := + calc 4 = 2 ^ 2 := by decide + _ ≤ 2 ^ q := Nat.pow_le_pow_right (by decide) hq_pos + have hmax_le : max U 2 ≤ 2 ^ q := max_le hU_le (by omega) + have hlt_2q : 2 ^ q + 1 < 2 ^ (2 * q) := by + have hle_succ : 2 ^ (q + 1) ≤ 2 ^ (2 * q) := + Nat.pow_le_pow_right (by decide) (by omega) + rw [pow_succ] at hle_succ + omega + have hle_w : 2 ^ (2 * q) ≤ 2 ^ w := Nat.pow_le_pow_right (by decide) hw + change max U 2 + 1 < 2 ^ w + omega + +private theorem fits_U_of_hw {w U n_max : ℕ} + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) : + Fits w U := + (fits_max_U_add_one_of_hw hw).of_le (by omega) + +private theorem fits_tableLog_of_hw {w U n_max : ℕ} + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) : + Fits w (tableLog n_max) := by + have hle : tableLog n_max ≤ trinomialDeg U (tableLog n_max) := + le_trinomialDeg U (tableLog n_max) + have hlt : trinomialDeg U (tableLog n_max) < 2 ^ trinomialDeg U (tableLog n_max) := + Nat.lt_two_pow_self + exact (fits_q_of_hw hw).of_le (by omega) + +private theorem fits_max_n_max_of_space {w base n_max : ℕ} + (hfit : Fits w (base + space n_max)) : + Fits w (max n_max 2) := by + have h1 : max n_max 2 ≤ max (8 * n_max) 8 := by omega + have h2a := eight_mul_le_tableSize n_max + have h2b := eight_le_tableSize n_max + have h3 : tableSize n_max ≤ base + space n_max := by + dsimp only [space] + omega + exact hfit.of_le (by omega) + +/-- `allocSpan n_max` executes in `3` steps, copying `frontierReg` (`18`) to `baseReg` (`0`) and +advancing `frontierReg` to `base + space n_max`. -/ +theorem allocSpan_execWithin {w base n_max : ℕ} {x : Data rLP} + (hfr : x.regs 18 = base) (hfit : Fits w (base + space n_max)) : + ∃ x₁ : Data rLP, + ExecWithin w (allocSpan n_max) x x₁ 3 ∧ + x₁.mem = x.mem ∧ + x₁.regs 0 = base ∧ + x₁.regs 18 = base + space n_max := by + have hfit_sp : Fits w (space n_max) := hfit.of_le (by omega) + set s1 := x.set 0 base + have e1 : Exec w (.op (.copy 0 18) : Code .exact 24) x s1 1 := by + simpa +decide [s1, hfr] using (Exec.copy (B := .exact) (w := w) (d := 0) (s := 18) (x := x)) + set s2 := s1.set 3 (space n_max) + have e2 : Exec w (.op (.const 3 (space n_max)) : Code .exact 24) s1 s2 1 := + Exec.const hfit_sp + set s3 := s2.set 18 (base + space n_max) + have e3 : Exec w (.op (.exactBin .add 18 0 3) : Code .exact 24) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1]) hfit + exact ⟨s3, ⟨3, le_rfl, e1.seq (e2.seq e3)⟩, rfl, by simp +decide [s3, s2, s1], + by simp +decide [s3]⟩ + +/-- `initSetup U n_max` initializes the `5`-wise independent hash table and leaves the zeroed +linear-probing arrays representing the empty dictionary in at most `setupTime U n_max` expected +instructions. -/ +theorem initSetup_exec {U V n_max w base : ℕ} {x₁ : Data rLP} + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) + (hfit_space : Fits w (base + space n_max)) + (hx₁ : x₁.Bounded w) + (hb : x₁.regs 0 = base) + (hfr : x₁.regs 18 = base + space n_max) + (hz : ∀ a ∈ (Block.mk base (space n_max)).footprint, x₁.mem a = 0) : + ∃ ν₀ : PMF (Seed U n_max × Data rLP), + ExecCodeDist w (initSetup U n_max) x₁ (ν₀.map Prod.snd) (setupTime U n_max) ∧ + (∀ θ, (ν₀.map Prod.fst) θ ≤ 1 * seedLaw U n_max θ) ∧ + ∀ p ∈ ν₀.support, + p.2.regs 18 = base + space n_max ∧ + Rep U V n_max w base p.1 (Dictionary.empty U) p.2 ∧ + 0 ≤ 0 ∧ + Modifies (Block.mk base (space n_max)).footprint x₁.mem p.2.mem := by + have hfit_kwise : Fits w (base + KWise.space n_max 5) := + hfit_space.of_le (by dsimp only [space]; omega) + have hfit_n : Fits w (max n_max 2) := fits_max_n_max_of_space hfit_space + have hfit_n_val : Fits w n_max := hfit_n.of_le (le_max_left _ _) + have hfit_U1 : Fits w (max U 2 + 1) := fits_max_U_add_one_of_hw hw + have hfit_U : Fits w U := fits_U_of_hw hw + have hfit_M : Fits w (tableLog n_max) := fits_tableLog_of_hw hw + have hfit_q : Fits w (2 ^ trinomialDeg U (tableLog n_max)) := fits_q_of_hw hw + set x₀ : Data 18 := x₁.comap kwiseMap + have hx₀_0 : x₀.regs 0 = base := hb + set s1 := x₀.set 1 n_max + have e1 : Exec w (.op (.const 1 n_max) : Code .exact 18) x₀ s1 1 := Exec.const hfit_n_val + set s2 := s1.set 2 U + have e2 : Exec w (.op (.const 2 U) : Code .exact 18) s1 s2 1 := Exec.const hfit_U + set s3 := s2.set 3 (tableLog n_max) + have e3 : Exec w (.op (.const 3 (tableLog n_max)) : Code .exact 18) s2 s3 1 := + Exec.const hfit_M + obtain ⟨dTab, hdTab_exec, htab, hdTab_0, hdTab_out⟩ := + KWise.initTableCode_execWithin (x := s3) + (by simp +decide [s3, s2, s1, hx₀_0]) + (by simp +decide [s3, s2, s1]) + (by simp +decide [s3, s2]) + (by simp +decide [s3]) + hfit_kwise hfit_n hfit_U1 hfit_q + have hsp_k : KWise.space n_max 5 = 8 + 5 + 2 ^ (2 * KWise.chunkBits n_max) := rfl + have hpow_pos : 0 < 2 ^ (2 * KWise.chunkBits n_max) := by positivity + have hfit_end : Fits w (base + 8 + 5) := hfit_kwise.of_le (by omega) + set m := KWise.resampleComp base U (tableLog n_max) 5 dTab + have he_rand : ExecRand w (KWise.resampleCode 5) dTab m := + KWise.resampleCode_execRand htab hdTab_0 hfit_end hfit_q + obtain ⟨m', he_rand', _, hsupp⟩ := + KWise.resampleCode_execRandWithin htab hdTab_0 hfit_end hfit_q + obtain rfl := he_rand.unique he_rand' + have hexp_le : ProbComp.expect m (fun z => (z.2 : ℝ≥0∞)) ≤ 35 := by + calc ProbComp.expect m (fun z => (z.2 : ℝ≥0∞)) + ≤ ProbComp.expect m (fun _ => (35 : ℝ≥0∞)) := + ProbComp.expect_mono_on_support (fun z hz => Nat.cast_le.mpr (hsupp z hz).2) + _ = 35 := ProbComp.expect_const m 35 + have hdist_res : ExecCodeDist w (KWise.resampleCode 5) dTab (m.eval.map Prod.fst) 35 := by + have h0 := (ExecCodeDist.of_execRand he_rand).mono hexp_le + rwa [ProbComp.eval_map] at h0 + have hdist_initCode : ExecCodeDist w (KWise.initCode 5) s3 (m.eval.map Prod.fst) + (KWise.initTableSteps n_max U (tableLog n_max) + 35) := + (ExecCodeDist.of_execWithin hdTab_exec).seq_pure hdist_res + have hdist_18 : ExecCodeDist w (initSetup18 U n_max) x₀ (m.eval.map Prod.fst) + (setupTime U n_max) := by + have h123 := + (ExecCodeDist.of_exec e1).seq_pure + ((ExecCodeDist.of_exec e2).seq_pure + ((ExecCodeDist.of_exec e3).seq_pure hdist_initCode)) + refine h123.mono ?_ + dsimp only [setupTime] + norm_cast + omega + have hdist_24 := hdist_18.rename injective_kwiseMap + let pairFn : Data 18 × ℕ → Seed U n_max × Data rLP := fun z => + (KWise.extractSeeds base U (tableLog n_max) 5 z.1.mem, z.1.extend kwiseMap x₁) + refine ⟨m.eval.map pairFn, ?_, ?_, ?_⟩ + · refine hdist_24.congr_dist ?_ le_rfl + rw [PMF.map_comp, PMF.map_comp] + rfl + · intro θ + have hmap : (m.eval.map pairFn).map Prod.fst = seedLaw U n_max := by + rw [PMF.map_comp] + exact KWise.map_extractSeeds_eval_resampleComp base U (tableLog n_max) 5 dTab + rw [hmap, one_mul] + · intro p hp + rw [PMF.mem_support_map_iff] at hp + obtain ⟨z, hz_eval, rfl⟩ := hp + rw [ProbComp.support_eval] at hz_eval + obtain ⟨⟨hkrep, hz_0, hz_out⟩, _⟩ := hsupp z hz_eval + have hp_mem_snd : z.1.extend kwiseMap x₁ ∈ + ((m.eval.map Prod.fst).map (fun y => y.extend kwiseMap x₁)).support := by + rw [PMF.mem_support_map_iff] + refine ⟨z.1, ?_, rfl⟩ + rw [PMF.mem_support_map_iff] + exact ⟨z, (ProbComp.support_eval m).symm ▸ hz_eval, rfl⟩ + have hbdd_p : (z.1.extend kwiseMap x₁).Bounded w := hdist_24.bounded hx₁ _ hp_mem_snd + have hfr_p : (z.1.extend kwiseMap x₁).regs 18 = base + space n_max := by + rw [extend_kwiseMap_regs (by decide), hfr] + have hbase_p : (z.1.extend kwiseMap x₁).regs 0 = base := by + change ((z.1.extend kwiseMap x₁).comap kwiseMap).regs 0 = base + rw [Data.comap_extend injective_kwiseMap] + exact hz_0 + have hmem_above : ∀ a, base + KWise.space n_max 5 ≤ a → z.1.mem a = x₁.mem a := by + intro a ha + rw [hz_out a (Or.inr (by omega)), hdTab_out a (Or.inr ha)] + rfl + have hempty_used : ∀ j : Fin (2 ^ tableLog n_max), + usedAt base n_max (z.1.extend kwiseMap x₁).mem j = 0 := by + intro j + have hj := j.isLt + change z.1.mem (usedBase base n_max + j.val) = 0 + rw [hmem_above (usedBase base n_max + j.val) (by dsimp only [usedBase]; omega)] + apply hz + rw [Block.mem_footprint, Block.Owns] + dsimp only [usedAt, usedBase, space, tableSize] + exact ⟨by omega, by omega⟩ + have htable_p : TableInv U V n_max base + (KWise.extractSeeds base U (tableLog n_max) 5 z.1.mem) + (Dictionary.empty U) (z.1.extend kwiseMap x₁).mem := + tableInv_empty U V n_max base _ hempty_used + have hmod_p : Modifies (Block.mk base (space n_max)).footprint x₁.mem + (z.1.extend kwiseMap x₁).mem := by + intro a ha + rw [Block.mem_footprint, Block.Owns, not_and_or, not_le, not_lt] at ha + change z.1.mem a = x₁.mem a + have ha_k : a < base ∨ base + KWise.space n_max 5 ≤ a := by + dsimp only [space] at ha + omega + have ha_s : a < base + 8 ∨ base + 8 + 5 ≤ a := by omega + rw [hz_out a ha_s, hdTab_out a ha_k] + rfl + exact ⟨hfr_p, ⟨hbdd_p, hbase_p, hkrep, htable_p⟩, le_rfl, hmod_p⟩ + +/-- Single-slot inspection `probeCheckCode` executes in at most `8` deterministic steps, loading +`usedAt base n_max x.mem pos` into `reg 9` and setting `reg 7` to `1` iff +`usedAt base n_max x.mem pos = 1 ∧ keyAt base n_max x.mem pos ≠ u.val`. -/ +theorem probeCheckCode_execWithin {U w base n_max : ℕ} (u : Fin U) + (pos : Fin (2 ^ tableLog n_max)) {x : Data rLP} + (h0 : x.regs 0 = base) + (h1 : x.regs 1 = u.val) + (h2 : x.regs 2 = pos.val) + (h3 : x.regs 3 = tableSize n_max - 1) + (h4 : x.regs 4 = usedBase base n_max) + (h5 : x.regs 5 = keyBase base n_max) + (h6 : x.regs 6 = valBase base n_max) + (h10 : x.regs 10 = 1) + (hfit_space : Fits w (base + space n_max)) : + ∃ y : Data rLP, + ExecWithin w probeCheckCode x y 8 ∧ + y.mem = x.mem ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = pos.val ∧ + y.regs 3 = tableSize n_max - 1 ∧ + y.regs 4 = usedBase base n_max ∧ + y.regs 5 = keyBase base n_max ∧ + y.regs 6 = valBase base n_max ∧ + y.regs 7 = (if usedAt base n_max x.mem pos = 1 ∧ + keyAt base n_max x.mem pos ≠ u.val then 1 else 0) ∧ + y.regs 9 = usedAt base n_max x.mem pos ∧ + y.regs 10 = 1 ∧ + y.regs 22 = x.regs 22 ∧ + y.regs 23 = x.regs 23 := by + have hpos_lt : pos.val < tableSize n_max := pos.isLt + have hfit_uaddr : Fits w (usedBase base n_max + pos.val) := + hfit_space.of_le (by dsimp only [usedBase, space]; omega) + have hfit_kaddr : Fits w (keyBase base n_max + pos.val) := + hfit_space.of_le (by dsimp only [keyBase, usedBase, space]; omega) + have hfit_0 : Fits w 0 := hfit_space.of_le (Nat.zero_le _) + have hfit_1 : Fits w 1 := + hfit_space.of_le (by dsimp only [space]; have := tableSize_pos n_max; omega) + set s1 := x.set 8 (usedBase base n_max + pos.val) + have e1 : Exec w (.op (.exactBin .add 8 4 2) : Code .exact 24) x s1 1 := + exec_exactBin (by simp +decide [h4, h2]) hfit_uaddr + set s2 := s1.set 9 (usedAt base n_max x.mem pos) + have e2 : Exec w (.op (.load 9 8) : Code .exact 24) s1 s2 1 := by + simpa +decide [s2, s1, usedAt] using + (Exec.load (B := .exact) (w := w) (d := 9) (a := 8) (x := s1)) + by_cases hused : usedAt base n_max x.mem pos = 1 + · have hc_outer : (Cond.eq (9 : Fin 24) 10).Holds s2 := by + simp +decide [Cond.Holds, s2, s1, h10, hused] + set s3 := s2.set 8 (keyBase base n_max + pos.val) + have e3 : Exec w (.op (.exactBin .add 8 5 2) : Code .exact 24) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h5, h2]) hfit_kaddr + set s4 := s3.set 8 (keyAt base n_max x.mem pos) + have e4 : Exec w (.op (.load 8 8) : Code .exact 24) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, keyAt] using + (Exec.load (B := .exact) (w := w) (d := 8) (a := 8) (x := s3)) + by_cases hkey : keyAt base n_max x.mem pos = u.val + · have hc_inner : (Cond.eq (8 : Fin 24) 1).Holds s4 := by + simp +decide [Cond.Holds, s4, s3, s2, s1, h1, hkey] + set s5 := s4.set 7 0 + have e5 : Exec w (.op (.const 7 0) : Code .exact 24) s4 s5 1 := Exec.const hfit_0 + have hall : Exec w probeCheckCode x s5 7 := + e1.seq (e2.seq (Exec.branch_pos hc_outer (e3.seq (e4.seq (Exec.branch_pos hc_inner e5))))) + refine ⟨s5, ⟨7, by decide, hall⟩, rfl, + by simp +decide [s5, s4, s3, s2, s1, h0], + by simp +decide [s5, s4, s3, s2, s1, h1], + by simp +decide [s5, s4, s3, s2, s1, h2], + by simp +decide [s5, s4, s3, s2, s1, h3], + by simp +decide [s5, s4, s3, s2, s1, h4], + by simp +decide [s5, s4, s3, s2, s1, h5], + by simp +decide [s5, s4, s3, s2, s1, h6], + by simp [s5, hused, hkey], + by simp +decide [s5, s4, s3, s2], + by simp +decide [s5, s4, s3, s2, s1, h10], + by simp +decide [s5, s4, s3, s2, s1], + by simp +decide [s5, s4, s3, s2, s1]⟩ + · have hc_inner : ¬(Cond.eq (8 : Fin 24) 1).Holds s4 := by + simp +decide [Cond.Holds, s4, s3, s2, s1, h1, hkey] + set s5 := s4.set 7 1 + have e5 : Exec w (.op (.const 7 1) : Code .exact 24) s4 s5 1 := Exec.const hfit_1 + have hall : Exec w probeCheckCode x s5 8 := + e1.seq (e2.seq (Exec.branch_pos hc_outer (e3.seq (e4.seq (Exec.branch_neg hc_inner e5))))) + refine ⟨s5, ⟨8, le_rfl, hall⟩, rfl, + by simp +decide [s5, s4, s3, s2, s1, h0], + by simp +decide [s5, s4, s3, s2, s1, h1], + by simp +decide [s5, s4, s3, s2, s1, h2], + by simp +decide [s5, s4, s3, s2, s1, h3], + by simp +decide [s5, s4, s3, s2, s1, h4], + by simp +decide [s5, s4, s3, s2, s1, h5], + by simp +decide [s5, s4, s3, s2, s1, h6], + by simp [s5, hused, hkey], + by simp +decide [s5, s4, s3, s2], + by simp +decide [s5, s4, s3, s2, s1, h10], + by simp +decide [s5, s4, s3, s2, s1], + by simp +decide [s5, s4, s3, s2, s1]⟩ + · have hc_outer : ¬(Cond.eq (9 : Fin 24) 10).Holds s2 := by + simp +decide [Cond.Holds, s2, s1, h10, hused] + set s3 := s2.set 7 0 + have e3 : Exec w (.op (.const 7 0) : Code .exact 24) s2 s3 1 := Exec.const hfit_0 + have hall : Exec w probeCheckCode x s3 5 := + e1.seq (e2.seq (Exec.branch_neg hc_outer e3)) + refine ⟨s3, ⟨5, by decide, hall⟩, rfl, + by simp +decide [s3, s2, s1, h0], + by simp +decide [s3, s2, s1, h1], + by simp +decide [s3, s2, s1, h2], + by simp +decide [s3, s2, s1, h3], + by simp +decide [s3, s2, s1, h4], + by simp +decide [s3, s2, s1, h5], + by simp +decide [s3, s2, s1, h6], + by simp [s3, hused], + by simp +decide [s3, s2], + by simp +decide [s3, s2, s1, h10], + by simp +decide [s3, s2, s1], + by simp +decide [s3, s2, s1]⟩ + +/-- `probeSetupCode n_max` executes in at most `16` deterministic steps, initializing registers +`3, 4, 5, 6, 10` and running `probeCheckCode` on the initial hash bucket `pos`. -/ +theorem probeSetupCode_execWithin {U w base n_max : ℕ} (u : Fin U) + (pos : Fin (2 ^ tableLog n_max)) {x : Data rLP} + (h0 : x.regs 0 = base) + (h1 : x.regs 1 = u.val) + (h2 : x.regs 2 = pos.val) + (hfit_space : Fits w (base + space n_max)) : + ∃ y : Data rLP, + ExecWithin w (probeSetupCode n_max) x y 16 ∧ + y.mem = x.mem ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = pos.val ∧ + y.regs 3 = tableSize n_max - 1 ∧ + y.regs 4 = usedBase base n_max ∧ + y.regs 5 = keyBase base n_max ∧ + y.regs 6 = valBase base n_max ∧ + y.regs 7 = (if usedAt base n_max x.mem pos = 1 ∧ + keyAt base n_max x.mem pos ≠ u.val then 1 else 0) ∧ + y.regs 9 = usedAt base n_max x.mem pos ∧ + y.regs 10 = 1 ∧ + y.regs 22 = x.regs 22 ∧ + y.regs 23 = x.regs 23 := by + have hts_pos : 0 < tableSize n_max := tableSize_pos n_max + have hfit_mask : Fits w (tableSize n_max - 1) := + hfit_space.of_le (by dsimp only [space]; omega) + have hfit_ksp : Fits w (KWise.space n_max 5) := + hfit_space.of_le (by dsimp only [space]; omega) + have hfit_ub : Fits w (usedBase base n_max) := + hfit_space.of_le (by dsimp only [usedBase, space]; omega) + have hfit_ts : Fits w (tableSize n_max) := + hfit_space.of_le (by dsimp only [space]; omega) + have hfit_kb : Fits w (keyBase base n_max) := + hfit_space.of_le (by dsimp only [keyBase, usedBase, space]; omega) + have hfit_vb : Fits w (valBase base n_max) := + hfit_space.of_le (by dsimp only [valBase, keyBase, usedBase, space]; omega) + have hfit_1 : Fits w 1 := + hfit_space.of_le (by dsimp only [space]; omega) + set s1 := x.set 3 (tableSize n_max - 1) + have e1 : Exec w (.op (.const 3 (tableSize n_max - 1)) : Code .exact 24) x s1 1 := + Exec.const hfit_mask + set s2 := s1.set 4 (KWise.space n_max 5) + have e2 : Exec w (.op (.const 4 (KWise.space n_max 5)) : Code .exact 24) s1 s2 1 := + Exec.const hfit_ksp + set s3 := s2.set 4 (usedBase base n_max) + have e3 : Exec w (.op (.exactBin .add 4 0 4) : Code .exact 24) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h0, usedBase]) hfit_ub + set s4 := s3.set 5 (tableSize n_max) + have e4 : Exec w (.op (.const 5 (tableSize n_max)) : Code .exact 24) s3 s4 1 := + Exec.const hfit_ts + set s5 := s4.set 5 (keyBase base n_max) + have e5 : Exec w (.op (.exactBin .add 5 4 5) : Code .exact 24) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, keyBase]) hfit_kb + set s6 := s5.set 6 (tableSize n_max) + have e6 : Exec w (.op (.const 6 (tableSize n_max)) : Code .exact 24) s5 s6 1 := + Exec.const hfit_ts + set s7 := s6.set 6 (valBase base n_max) + have e7 : Exec w (.op (.exactBin .add 6 5 6) : Code .exact 24) s6 s7 1 := + exec_exactBin (by simp +decide [s6, s5, valBase]) hfit_vb + set s8 := s7.set 10 1 + have e8 : Exec w (.op (.const 10 1) : Code .exact 24) s7 s8 1 := + Exec.const hfit_1 + obtain ⟨y, ⟨m, hm, he9⟩, hymem, hy0, hy1, hy2, hy3, hy4, hy5, hy6, hy7, hy9, hy10, hy22, hy23⟩ := + probeCheckCode_execWithin u pos (x := s8) + (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h0]) + (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h1]) + (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1, h2]) + (by simp +decide [s8, s7, s6, s5, s4, s3, s2, s1]) + (by simp +decide [s8, s7, s6, s5, s4, s3]) + (by simp +decide [s8, s7, s6, s5]) + (by simp +decide [s8, s7]) + (by simp +decide [s8]) + hfit_space + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq (e7.seq (e8.seq he9))))))) + exact ⟨y, ⟨_, by omega, hall⟩, hymem, hy0, hy1, hy2, hy3, hy4, hy5, hy6, hy7, hy9, hy10, + by simpa +decide [s8, s7, s6, s5, s4, s3, s2, s1] using hy22, + by simpa +decide [s8, s7, s6, s5, s4, s3, s2, s1] using hy23⟩ + +/-- One iteration of `probeBody` advances `pos` to `probeSlot (tableLog n_max) pos 1` and inspects +the new slot in at most `10` deterministic steps. -/ +theorem probeBody_execWithin {U w base n_max : ℕ} (u : Fin U) + (pos : Fin (2 ^ tableLog n_max)) {x : Data rLP} + (h0 : x.regs 0 = base) + (h1 : x.regs 1 = u.val) + (h2 : x.regs 2 = pos.val) + (h3 : x.regs 3 = tableSize n_max - 1) + (h4 : x.regs 4 = usedBase base n_max) + (h5 : x.regs 5 = keyBase base n_max) + (h6 : x.regs 6 = valBase base n_max) + (h10 : x.regs 10 = 1) + (hfit_space : Fits w (base + space n_max)) : + let pos' := probeSlot (tableLog n_max) pos 1 + ∃ y : Data rLP, + ExecWithin w probeBody x y 10 ∧ + y.mem = x.mem ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = pos'.val ∧ + y.regs 3 = tableSize n_max - 1 ∧ + y.regs 4 = usedBase base n_max ∧ + y.regs 5 = keyBase base n_max ∧ + y.regs 6 = valBase base n_max ∧ + y.regs 7 = (if usedAt base n_max x.mem pos' = 1 ∧ + keyAt base n_max x.mem pos' ≠ u.val then 1 else 0) ∧ + y.regs 9 = usedAt base n_max x.mem pos' ∧ + y.regs 10 = 1 ∧ + y.regs 22 = x.regs 22 ∧ + y.regs 23 = x.regs 23 := by + set pos' := probeSlot (tableLog n_max) pos 1 + have hpos_lt : pos.val < tableSize n_max := pos.isLt + have hpos'_lt : pos'.val < tableSize n_max := pos'.isLt + have hfit_p1 : Fits w (pos.val + 1) := + hfit_space.of_le (by dsimp only [space]; omega) + have hfit_pos' : Fits w pos'.val := + hfit_space.of_le (by dsimp only [space]; omega) + have hand_eq : (pos.val + 1) &&& (tableSize n_max - 1) = pos'.val := by + dsimp only [tableSize, pos', probeSlot, cyclicSlotMod] + exact Nat.and_two_pow_sub_one_eq_mod (pos.val + 1) (tableLog n_max) + set s1 := x.set 2 (pos.val + 1) + have e1 : Exec w (.op (.exactBin .add 2 2 10) : Code .exact 24) x s1 1 := + exec_exactBin (by simp +decide [h2, h10]) hfit_p1 + set s2 := s1.set 2 pos'.val + have e2 : Exec w (.op (.exactBin .and 2 2 3) : Code .exact 24) s1 s2 1 := + exec_exactBin (by simp +decide [s1, h3, hand_eq]) hfit_pos' + obtain ⟨y, ⟨m, hm, he3⟩, hymem, hy0, hy1, hy2, hy3, hy4, hy5, hy6, hy7, hy9, hy10, hy22, hy23⟩ := + probeCheckCode_execWithin u pos' (x := s2) + (by simp +decide [s2, s1, h0]) + (by simp +decide [s2, s1, h1]) + (by simp +decide [s2]) + (by simp +decide [s2, s1, h3]) + (by simp +decide [s2, s1, h4]) + (by simp +decide [s2, s1, h5]) + (by simp +decide [s2, s1, h6]) + (by simp +decide [s2, s1, h10]) + hfit_space + exact ⟨y, ⟨_, by omega, e1.seq (e2.seq he3)⟩, hymem, hy0, hy1, hy2, hy3, hy4, hy5, hy6, hy7, + hy9, hy10, + by simpa +decide [s2, s1] using hy22, + by simpa +decide [s2, s1] using hy23⟩ + +/-- `probeLoop` halts deterministically at the stopping slot +`p = probeSlot (tableLog n_max) b (hinv.probeLen u b)` in at most +`12 * (hinv.probeLen u b) + 2` instructions. -/ +theorem probeLoop_execWithin {U V w base n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x : Data rLP} + (hinv : TableInv U V n_max base θ s x.mem) + (u : Fin U) + (h0 : x.regs 0 = base) + (h1 : x.regs 1 = u.val) + (h2 : x.regs 2 = (hashFn U n_max θ u).val) + (h3 : x.regs 3 = tableSize n_max - 1) + (h4 : x.regs 4 = usedBase base n_max) + (h5 : x.regs 5 = keyBase base n_max) + (h6 : x.regs 6 = valBase base n_max) + (h7 : x.regs 7 = (if usedAt base n_max x.mem (hashFn U n_max θ u) = 1 ∧ + keyAt base n_max x.mem (hashFn U n_max θ u) ≠ u.val then 1 else 0)) + (h9 : x.regs 9 = usedAt base n_max x.mem (hashFn U n_max θ u)) + (h10 : x.regs 10 = 1) + (hfit_space : Fits w (base + space n_max)) : + let b := hashFn U n_max θ u + let t := hinv.probeLen u b + let p := probeSlot (tableLog n_max) b t + ∃ y : Data rLP, + ExecWithin w probeLoop x y (12 * t + 2) ∧ + y.mem = x.mem ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = p.val ∧ + y.regs 4 = usedBase base n_max ∧ + y.regs 5 = keyBase base n_max ∧ + y.regs 6 = valBase base n_max ∧ + y.regs 9 = usedAt base n_max x.mem p ∧ + y.regs 10 = 1 ∧ + y.regs 22 = x.regs 22 ∧ + y.regs 23 = x.regs 23 := by + set b := hashFn U n_max θ u + set t := hinv.probeLen u b + set p := probeSlot (tableLog n_max) b t + suffices hmain : ∀ rem k (d : Data rLP), + k + rem = t → + d.mem = x.mem → + d.regs 0 = base → + d.regs 1 = u.val → + d.regs 2 = (probeSlot (tableLog n_max) b k).val → + d.regs 3 = tableSize n_max - 1 → + d.regs 4 = usedBase base n_max → + d.regs 5 = keyBase base n_max → + d.regs 6 = valBase base n_max → + d.regs 7 = (if usedAt base n_max x.mem (probeSlot (tableLog n_max) b k) = 1 ∧ + keyAt base n_max x.mem (probeSlot (tableLog n_max) b k) ≠ u.val then 1 else 0) → + d.regs 9 = usedAt base n_max x.mem (probeSlot (tableLog n_max) b k) → + d.regs 10 = 1 → + d.regs 22 = x.regs 22 → + d.regs 23 = x.regs 23 → + ∃ y : Data rLP, + ExecWithin w probeLoop d y (12 * rem + 2) ∧ + y.mem = x.mem ∧ + y.regs 0 = base ∧ + y.regs 1 = u.val ∧ + y.regs 2 = p.val ∧ + y.regs 4 = usedBase base n_max ∧ + y.regs 5 = keyBase base n_max ∧ + y.regs 6 = valBase base n_max ∧ + y.regs 9 = usedAt base n_max x.mem p ∧ + y.regs 10 = 1 ∧ + y.regs 22 = x.regs 22 ∧ + y.regs 23 = x.regs 23 by + exact hmain t 0 x (Nat.zero_add t) rfl h0 h1 + (by rw [probeSlot_zero, h2]) h3 h4 h5 h6 + (by rw [probeSlot_zero, h7]) (by rw [probeSlot_zero, h9]) h10 rfl rfl + intro rem + induction rem with + | zero => + intro k d hkr hdmem hd0 hd1 hd2 _ hd4 hd5 hd6 hd7 hd9 hd10 hd22 hd23 + have hkt : k = t := by omega + subst hkt + have hstop : ¬(usedAt base n_max x.mem p = 1 ∧ keyAt base n_max x.mem p ≠ u.val) := by + rcases hinv.probeStop_spec u with ⟨hp0, _⟩ | ⟨_, hkey, _⟩ + · rintro ⟨hp1, _⟩ + rw [hp0] at hp1 + cases hp1 + · rintro ⟨_, hne⟩ + exact hne hkey + have hd7_0 : d.regs 7 = 0 := by simp [hd7, p, hstop] + have hc : ¬(Cond.eq (7 : Fin 24) 10).Holds d := by + simp [Cond.Holds, hd7_0, hd10] + exact ⟨d, ⟨2, le_rfl, Exec.loop_exit hc⟩, hdmem, hd0, hd1, hd2, hd4, hd5, hd6, hd9, hd10, + hd22, hd23⟩ + | succ rem ih => + intro k d hkr hdmem hd0 hd1 hd2 hd3 hd4 hd5 hd6 hd7 hd9 hd10 hd22 hd23 + have hkt : k < t := by omega + have hcont : usedAt base n_max x.mem (probeSlot (tableLog n_max) b k) = 1 ∧ + keyAt base n_max x.mem (probeSlot (tableLog n_max) b k) ≠ u.val := + hinv.probeContinue_of_lt_probeLen u b hkt + have hd7_1 : d.regs 7 = 1 := by simp [hd7, hcont] + have hc : (Cond.eq (7 : Fin 24) 10).Holds d := by + simp [Cond.Holds, hd7_1, hd10] + obtain ⟨d', ⟨nb, hnb, heb⟩, hd'mem, hd'0, hd'1, hd'2, hd'3, hd'4, hd'5, hd'6, hd'7, + hd'9, hd'10, hd'22, hd'23⟩ := + probeBody_execWithin u (probeSlot (tableLog n_max) b k) (x := d) + hd0 hd1 hd2 hd3 hd4 hd5 hd6 hd10 hfit_space + have hslot_succ : probeSlot (tableLog n_max) (probeSlot (tableLog n_max) b k) 1 = + probeSlot (tableLog n_max) b (k + 1) := + probeSlot_add b k 1 + rw [hslot_succ] at hd'2 hd'7 hd'9 + rw [hdmem] at hd'7 hd'9 + obtain ⟨y, ⟨nl, hnl, hel⟩, hymem, hy0, hy1, hy2, hy4, hy5, hy6, hy9, hy10, hy22, hy23⟩ := + ih (k + 1) d' (by omega) (hd'mem.trans hdmem) hd'0 hd'1 hd'2 hd'3 hd'4 hd'5 hd'6 + hd'7 hd'9 hd'10 (hd'22.trans hd22) (hd'23.trans hd23) + exact ⟨y, ⟨nb + nl + 2, by omega, Exec.loop_iter hc heb hel⟩, + hymem, hy0, hy1, hy2, hy4, hy5, hy6, hy9, hy10, hy22, hy23⟩ + +/-- `insertFinishCode` executes in `7` deterministic steps, writing `(1, u.val, v)` at slot `p` +(`insertMem base n_max x.mem p u.val v`) and setting `reg 2` (`outReg`) to `0`. -/ +theorem insertFinishCode_execWithin {U w base n_max : ℕ} (u : Fin U) (v : ℕ) + (p : Fin (2 ^ tableLog n_max)) {x : Data rLP} + (h0 : x.regs 0 = base) + (h1 : x.regs 1 = u.val) + (h2 : x.regs 2 = p.val) + (h4 : x.regs 4 = usedBase base n_max) + (h5 : x.regs 5 = keyBase base n_max) + (h6 : x.regs 6 = valBase base n_max) + (h10 : x.regs 10 = 1) + (h23 : x.regs 23 = v) + (hfit_space : Fits w (base + space n_max)) : + ∃ y : Data rLP, + ExecWithin w insertFinishCode x y 7 ∧ + y.mem = insertMem base n_max x.mem p u.val v ∧ + y.regs 0 = base ∧ + y.regs 2 = 0 := by + have hp_lt : p.val < tableSize n_max := p.isLt + have hfit_uaddr : Fits w (usedBase base n_max + p.val) := + hfit_space.of_le (by dsimp only [usedBase, space]; omega) + have hfit_kaddr : Fits w (keyBase base n_max + p.val) := + hfit_space.of_le (by dsimp only [keyBase, usedBase, space]; omega) + have hfit_vaddr : Fits w (valBase base n_max + p.val) := + hfit_space.of_le (by dsimp only [valBase, keyBase, usedBase, space]; omega) + have hfit_0 : Fits w 0 := hfit_space.of_le (Nat.zero_le _) + set s1 := x.set 8 (usedBase base n_max + p.val) + have e1 : Exec w (.op (.exactBin .add 8 4 2) : Code .exact 24) x s1 1 := + exec_exactBin (by simp +decide [h4, h2]) hfit_uaddr + set s2 := s1.write (usedBase base n_max + p.val) 1 + have e2 : Exec w (.op (.store 8 10) : Code .exact 24) s1 s2 1 := by + simpa +decide [s2, s1, h10] using + (Exec.store (B := .exact) (w := w) (a := 8) (s := 10) (x := s1)) + set s3 := s2.set 8 (keyBase base n_max + p.val) + have e3 : Exec w (.op (.exactBin .add 8 5 2) : Code .exact 24) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h5, h2]) hfit_kaddr + set s4 := s3.write (keyBase base n_max + p.val) u.val + have e4 : Exec w (.op (.store 8 1) : Code .exact 24) s3 s4 1 := by + simpa +decide [s4, s3, s2, s1, h1] using + (Exec.store (B := .exact) (w := w) (a := 8) (s := 1) (x := s3)) + set s5 := s4.set 8 (valBase base n_max + p.val) + have e5 : Exec w (.op (.exactBin .add 8 6 2) : Code .exact 24) s4 s5 1 := + exec_exactBin (by simp +decide [s4, s3, s2, s1, h6, h2]) hfit_vaddr + set s6 := s5.write (valBase base n_max + p.val) v + have e6 : Exec w (.op (.store 8 23) : Code .exact 24) s5 s6 1 := by + simpa +decide [s6, s5, s4, s3, s2, s1, h23] using + (Exec.store (B := .exact) (w := w) (a := 8) (s := 23) (x := s5)) + set s7 := s6.set 2 0 + have e7 : Exec w (.op (.const 2 0) : Code .exact 24) s6 s7 1 := + Exec.const hfit_0 + have hall := e1.seq (e2.seq (e3.seq (e4.seq (e5.seq (e6.seq e7))))) + exact ⟨s7, ⟨7, le_rfl, hall⟩, rfl, by simp +decide [s7, s6, s5, s4, s3, s2, s1, h0], + by simp +decide [s7]⟩ + +/-- `getFinishCode` executes in at most `4` deterministic steps, writing `val[p] + 1` into `reg 2` +when `s u = some (val[p])` (`used[p] = 1`) and `0` into `reg 2` when `s u = none` (`used[p] = 0`). +-/ +theorem getFinishCode_execWithin {U V w base n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x : Data rLP} + (hinv : TableInv U V n_max base θ s x.mem) + (u : Fin U) + (h0 : x.regs 0 = base) + (h2 : x.regs 2 = (probeSlot (tableLog n_max) (hashFn U n_max θ u) + (hinv.probeLen u (hashFn U n_max θ u))).val) + (h6 : x.regs 6 = valBase base n_max) + (h9 : x.regs 9 = usedAt base n_max x.mem + (probeSlot (tableLog n_max) (hashFn U n_max θ u) + (hinv.probeLen u (hashFn U n_max θ u)))) + (h10 : x.regs 10 = 1) + (hfit_space : Fits w (base + space n_max)) + (hfit_V : Fits w V) : + ∃ y : Data rLP, + ExecWithin w getFinishCode x y 4 ∧ + y.mem = x.mem ∧ + y.regs 0 = base ∧ + decodeOp (V := V) (n_max := n_max) (.get u) y.regs = some (s u) := by + set p := probeSlot (tableLog n_max) (hashFn U n_max θ u) (hinv.probeLen u (hashFn U n_max θ u)) + have hp_lt : p.val < tableSize n_max := p.isLt + have hfit_vaddr : Fits w (valBase base n_max + p.val) := + hfit_space.of_le (by dsimp only [valBase, keyBase, usedBase, space]; omega) + have hfit_0 : Fits w 0 := hfit_space.of_le (Nat.zero_le _) + rcases hinv.probeStop_spec u with ⟨hp0, hnone⟩ | ⟨hp1, _, hsome⟩ + · have hc : ¬(Cond.eq (9 : Fin 24) 10).Holds x := by + simp [Cond.Holds, h9, p, hp0, h10] + set s1 := x.set 2 0 + have e1 : Exec w (.op (.const 2 0) : Code .exact 24) x s1 1 := Exec.const hfit_0 + have ebr : Exec w getFinishCode x s1 3 := Exec.branch_neg hc e1 + refine ⟨s1, ⟨3, by decide, ebr⟩, rfl, by simp +decide [s1, h0], ?_⟩ + have hs1_2 : s1.regs 2 = 0 := by simp +decide [s1] + simp only [decodeOp, hs1_2, ↓reduceIte, hnone] + rfl + · have hc : (Cond.eq (9 : Fin 24) 10).Holds x := by + simp [Cond.Holds, h9, p, hp1, h10] + have hv_lt : valAt base n_max x.mem p < V := hinv.values_bounded u _ hsome + have hfit_vp1 : Fits w (valAt base n_max x.mem p + 1) := hfit_V.of_le (by omega) + set s1 := x.set 8 (valBase base n_max + p.val) + have e1 : Exec w (.op (.exactBin .add 8 6 2) : Code .exact 24) x s1 1 := + exec_exactBin (by simp +decide [h6, h2, p]) hfit_vaddr + set s2 := s1.set 2 (valAt base n_max x.mem p) + have e2 : Exec w (.op (.load 2 8) : Code .exact 24) s1 s2 1 := by + simpa +decide [s2, s1, valAt, p] using + (Exec.load (B := .exact) (w := w) (d := 2) (a := 8) (x := s1)) + set s3 := s2.set 2 (valAt base n_max x.mem p + 1) + have e3 : Exec w (.op (.exactBin .add 2 2 10) : Code .exact 24) s2 s3 1 := + exec_exactBin (by simp +decide [s2, s1, h10]) hfit_vp1 + have ebr : Exec w getFinishCode x s3 4 := Exec.branch_pos hc (e1.seq (e2.seq e3)) + refine ⟨s3, ⟨4, le_rfl, ebr⟩, rfl, by simp +decide [s3, s2, s1, h0], ?_⟩ + have hs3_2 : s3.regs 2 = valAt base n_max x.mem p + 1 := by simp +decide [s3] + simp only [decodeOp, hs3_2, Nat.add_one_ne_zero, ↓reduceIte, Nat.add_sub_cancel, hsome, p] + rfl + +/-- The key queried or updated by dictionary operation `o`. -/ +@[expose] def opKey {U : ℕ} : Dictionary.Op U → Fin U + | .get u => u + | .insert u _ => u + +/-- `opCode n_max` executes any admissible operation `o` from `Rep U V n_max w base θ s x` and +`argMatchOp o x.regs` in at most +`KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * t + 27` deterministic instructions (where +`t = hrep.table_inv.probeLen (opKey o) (hashFn U n_max θ (opKey o))`), preserving `Rep` for +`((Dictionary.spec U V n_max).step s o).1`, decoding to +`some ((Dictionary.spec U V n_max).step s o).2`, and modifying only +`(Block.mk base (space n_max)).footprint`. -/ +theorem opCode_execWithin {U V w base n_max : ℕ} {θ : Seed U n_max} + {s : Dictionary.State U} {x : Data rLP} + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) + (hfit_space : Fits w (base + space n_max)) + (hfit_V : Fits w V) + (hrep : Rep U V n_max w base θ s x) + (o : (Dictionary.spec U V n_max).Op) + (hvalid : (Dictionary.spec U V n_max).valid s o) + (harg : argMatchOp o x.regs) : + let u := opKey o + let b := hashFn U n_max θ u + let t := hrep.table_inv.probeLen u b + ∃ y : Data rLP, + ExecWithin w (opCode n_max) x y + (KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * t + 27) ∧ + decodeOp (V := V) (n_max := n_max) o y.regs = + some ((Dictionary.spec U V n_max).step s o).2 ∧ + Rep U V n_max w base θ ((Dictionary.spec U V n_max).step s o).1 y ∧ + Modifies (Block.mk base (space n_max)).footprint x.mem y.mem := by + set u := opKey o + set b := hashFn U n_max θ u + set t := hrep.table_inv.probeLen u b + set p := probeSlot (tableLog n_max) b t + have hfit_kwise : Fits w (base + KWise.space n_max 5) := + hfit_space.of_le (by dsimp only [space]; omega) + have hx1_key : x.regs 1 = u.val := by + cases o with + | get u₀ => exact harg.1 + | insert u₀ v => exact harg.1 + obtain ⟨y₀, heval₀, hy₀_0, hy₀_1, hy₀_2, hy₀_mem⟩ := + KWise.evalCode_execWithin u (x := x.comap kwiseMap) hrep.kwise_rep + hrep.reg_base hx1_key hw hfit_kwise + have heval1 := heval₀.rename injective_kwiseMap + set x₁ := y₀.extend kwiseMap x + have hx₁_mem : x₁.mem = x.mem := hy₀_mem + have hx₁_0 : x₁.regs 0 = base := by + change (x₁.comap kwiseMap).regs 0 = base + rw [Data.comap_extend injective_kwiseMap] + exact hy₀_0 + have hx₁_1 : x₁.regs 1 = u.val := by + change (x₁.comap kwiseMap).regs 1 = u.val + rw [Data.comap_extend injective_kwiseMap] + exact hy₀_1 + have hx₁_2 : x₁.regs 2 = b.val := by + change (x₁.comap kwiseMap).regs 2 = b.val + rw [Data.comap_extend injective_kwiseMap] + exact hy₀_2 + have hx₁_22 : x₁.regs 22 = x.regs 22 := extend_kwiseMap_regs (by decide) + have hx₁_23 : x₁.regs 23 = x.regs 23 := extend_kwiseMap_regs (by decide) + obtain ⟨x₂, he2, hx₂_mem, hx₂_0, hx₂_1, hx₂_2, hx₂_3, hx₂_4, hx₂_5, hx₂_6, hx₂_7, + hx₂_9, hx₂_10, hx₂_22, hx₂_23⟩ := + probeSetupCode_execWithin u b (x := x₁) hx₁_0 hx₁_1 hx₁_2 hfit_space + have hx₂_mem_eq : x₂.mem = x.mem := hx₂_mem.trans hx₁_mem + rw [hx₁_mem] at hx₂_7 hx₂_9 + obtain ⟨x₃, he3, hx₃_mem, hx₃_0, hx₃_1, hx₃_2, hx₃_4, hx₃_5, hx₃_6, hx₃_9, hx₃_10, + hx₃_22, hx₃_23⟩ := + probeLoop_execWithin (hinv := hx₂_mem_eq.symm ▸ hrep.table_inv) u + hx₂_0 hx₂_1 hx₂_2 hx₂_3 hx₂_4 hx₂_5 hx₂_6 + (by simpa only [hx₂_mem_eq] using hx₂_7) + (by simpa only [hx₂_mem_eq] using hx₂_9) + hx₂_10 hfit_space + have hx₃_mem_eq : x₃.mem = x.mem := hx₃_mem.trans hx₂_mem_eq + have hx₃_22_eq : x₃.regs 22 = x.regs 22 := hx₃_22.trans (hx₂_22.trans hx₁_22) + have hx₃_23_eq : x₃.regs 23 = x.regs 23 := hx₃_23.trans (hx₂_23.trans hx₁_23) + have ht_eq : (hx₂_mem_eq.symm ▸ hrep.table_inv).probeLen u b = t := + TableInv.probeLen_congr _ hrep.table_inv hx₂_mem_eq u b + rw [ht_eq] at he3 hx₃_2 hx₃_9 + rw [hx₂_mem_eq] at hx₃_9 + obtain ⟨n1, hn1, hex1⟩ := heval1 + obtain ⟨n2, hn2, hex2⟩ := he2 + obtain ⟨n3, hn3, hex3⟩ := he3 + cases o with + | get u₀ => + have hu₀ : u₀ = u := rfl + have hc_br : ¬(Cond.eq (22 : Fin 24) 10).Holds x₃ := by + simp [Cond.Holds, hx₃_22_eq, harg.2, hx₃_10] + have ht_eq3 : (hx₃_mem_eq.symm ▸ hrep.table_inv).probeLen u (hashFn U n_max θ u) = t := + TableInv.probeLen_congr _ hrep.table_inv hx₃_mem_eq u b + obtain ⟨y, ⟨n4, hn4, hex4⟩, hymem, hy0, hdec⟩ := + getFinishCode_execWithin (hinv := hx₃_mem_eq.symm ▸ hrep.table_inv) u + hx₃_0 + (by simpa only [ht_eq3] using hx₃_2) + hx₃_6 + (by simpa only [ht_eq3, hx₃_mem_eq] using hx₃_9) + hx₃_10 hfit_space hfit_V + have he_br : Exec w (.branch (.eq 22 10) insertFinishCode getFinishCode) x₃ y (n4 + 2) := + Exec.branch_neg hc_br hex4 + have hall : Exec w (opCode n_max) x y (n1 + (n2 + (n3 + (n4 + 2)))) := + hex1.seq (hex2.seq (hex3.seq he_br)) + have hex_within : ExecWithin w (opCode n_max) x y + (KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * t + 27) := + ⟨_, by omega, hall⟩ + have hymem_x : y.mem = x.mem := hymem.trans hx₃_mem_eq + refine ⟨y, hex_within, hdec, + ⟨hex_within.bounded hrep.bounded, hy0, hymem_x ▸ hrep.kwise_rep, + hymem_x ▸ hrep.table_inv⟩, ?_⟩ + intro a _ + exact hymem_x.symm ▸ rfl + | insert u₀ v => + have hu₀ : u₀ = u := rfl + have hc_br : (Cond.eq (22 : Fin 24) 10).Holds x₃ := by + simp [Cond.Holds, hx₃_22_eq, harg.2.1, hx₃_10] + obtain ⟨y, ⟨n4, hn4, hex4⟩, hymem, hy0, _⟩ := + insertFinishCode_execWithin u v p + hx₃_0 hx₃_1 hx₃_2 hx₃_4 hx₃_5 hx₃_6 hx₃_10 + (hx₃_23_eq.trans harg.2.2) hfit_space + have he_br : Exec w (.branch (.eq 22 10) insertFinishCode getFinishCode) x₃ y (n4 + 1) := + Exec.branch_pos hc_br hex4 + have hall : Exec w (opCode n_max) x y (n1 + (n2 + (n3 + (n4 + 1)))) := + hex1.seq (hex2.seq (hex3.seq he_br)) + have hex_within : ExecWithin w (opCode n_max) x y + (KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * t + 27) := + ⟨_, by omega, hall⟩ + have hymem_x : y.mem = insertMem base n_max x.mem p u.val v := by + rw [hymem, hx₃_mem_eq] + have hp_lt : p.val < tableSize n_max := p.isLt + have hkrep_y : KWise.Rep base (KWise.chunkBits n_max) U (tableLog n_max) 5 θ y.mem := + kwise_rep_congr hrep.kwise_rep fun a ha _ => by + rw [hymem_x] + dsimp only [insertMem, usedBase, keyBase, valBase, tableSize] + rw [Function.update_of_ne (by omega), Function.update_of_ne (by omega), + Function.update_of_ne (by omega)] + have htable_y : TableInv U V n_max base θ (Function.update s u₀ (some v)) y.mem := by + rw [hu₀, hymem_x] + exact hrep.table_inv.insert_mem u v (by simpa only [hu₀] using hvalid) + have hmod_y : Modifies (Block.mk base (space n_max)).footprint x.mem y.mem := by + intro a ha + rw [Block.mem_footprint, Block.Owns, not_and_or, not_le, not_lt] at ha + rw [hymem_x] + dsimp only [insertMem, usedBase, keyBase, valBase, space, tableSize] at ha ⊢ + rw [Function.update_of_ne (by omega), Function.update_of_ne (by omega), + Function.update_of_ne (by omega)] + exact ⟨y, hex_within, rfl, ⟨hex_within.bounded hrep.bounded, hy0, hkrep_y, htable_y⟩, hmod_y⟩ + +/-! ## Section 4: `SubroutineDS` certificate and expected-time sequence theorem -/ + +/-- **`SubroutineDS` certificate for open-addressed linear probing on the randomized Word RAM** +over `Cslib.WordRAM.Basis.exact`: +- **Space**: `space n_max = KWise.space n_max 5 + 3 * 2 ^ (tableLog n_max) = O(n_max)` words; +- **Initialization time**: `setupTime U n_max = KWise.initTableSteps n_max U (tableLog n_max) + 38 + = O(n_max + log log U)` expected instructions; +- **Operation charge**: `charge U n_max = KWise.evalSteps n_max U (tableLog n_max) 5 + 3483 + = O(L ^ 2 + 1)` expected instructions per `get` or `insert`, which is `O(1)` whenever + `U ≤ n_max ^ c` for any constant `c`; +- **Potential**: `0` (`seedSlack := 1`). -/ +@[expose] noncomputable def subroutineDS (U V n_max w base : ℕ) + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) + (hfit_space : Fits w (base + space n_max)) + (hfit_V : Fits w V) : + Cslib.WordRAM.Randomized.SubroutineDS (subroutineCode U n_max) + (Dictionary.spec U V n_max) w base Fin.elim0 (Dictionary.empty U) where + Seed := Seed U n_max + seedLaw := seedLaw U n_max + seedSlack := 1 + space := space n_max + spanTime := 3 + setupTime := setupTime U n_max + initPotential := 0 + charge _ _ := charge U n_max + rep θ s x := Rep U V n_max w base θ s x + potential _ _ := 0 + encode := encodeOp + argMatch := argMatchOp + decode := decodeOp + bounded h := h.bounded + rep_unique h₁ h₂ := h₁.unique h₂ + arguments_fit {s} o hvalid i hi := by + have hfit_U : Fits w U := fits_U_of_hw hw + have hfit_1 : Fits w 1 := + hfit_space.of_le (by dsimp only [space]; have := tableSize_pos n_max; omega) + have hfit_0 : Fits w 0 := hfit_space.of_le (Nat.zero_le _) + cases o with + | get u => + dsimp only [encodeOp] + split_ifs + · exact hfit_U.of_le (le_of_lt u.isLt) + · exact hfit_0 + | insert u v => + dsimp only [encodeOp] + split_ifs + · exact hfit_U.of_le (le_of_lt u.isLt) + · exact hfit_1 + · exact hfit_V.of_le (le_of_lt hvalid.2) + · exact hfit_0 + encode_matches o regs := by + cases o with + | get u => + exact ⟨rfl, rfl⟩ + | insert u v => + exact ⟨rfl, rfl, rfl⟩ + rep_supported hregs hmem hbdd hrep := hrep.supported hregs hmem hbdd + potential_supported _ _ _ := rfl + clear_nodup := by + change ([21, 19, 20, 0, 18] : List (Fin 24)).Nodup + decide + params_not_clear j := j.elim0 + incCur_eval {x} h20 hfit := by + have h20' : x.regs 20 = 1 := h20 + have hev : BinOp.add.evalExact (x.regs 21) (x.regs 20) = some (x.regs 21 + 1) := by + simp [BinOp.evalExact, h20'] + change (Op.exactBin .add 21 21 20).eval w x = some (x.set 21 (x.regs 21 + 1)) + rw [Op.eval_exactBin, hev] + exact Data.put_of_fits hfit + span_exec _ hfr _ := by + obtain ⟨x₁, hex, hmem, hb, hfr'⟩ := allocSpan_execWithin hfr hfit_space + exact ⟨x₁, hex, hmem, hb, hfr', fun j => j.elim0⟩ + setup_exec hx₁ hb hfr _ hz := + initSetup_exec hw hfit_space hx₁ hb hfr hz + step_exec {s} o ν hvalid hseed hsupp := by + classical + let u := opKey o + let hasInv : Seed U n_max × Data rLP → Prop := fun p => + Rep U V n_max w base p.1 s p.2 ∧ argMatchOp o p.2.regs + let y : Seed U n_max × Data rLP → Data rLP := fun p => + if hp : hasInv p then + Classical.choose (opCode_execWithin hw hfit_space hfit_V hp.1 o hvalid hp.2) + else p.2 + let T_probe : Seed U n_max × Data rLP → ℕ := fun p => + if hp : hasInv p then + hp.1.table_inv.probeLen u (hashFn U n_max p.1 u) + else 0 + let T_step : Seed U n_max × Data rLP → ℝ≥0∞ := fun p => + (KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * T_probe p + 27 : ℕ) + refine ⟨fun p => PMF.pure (p.1, y p), T_step, ?_, ?_, ?_⟩ + · intro p hp_mem + have hp : hasInv p := hsupp p hp_mem + have hspec := Classical.choose_spec + (opCode_execWithin hw hfit_space hfit_V hp.1 o hvalid hp.2) + rcases hspec with ⟨hex, hdec, hrep', hmod⟩ + have hy_eq : y p = + Classical.choose (opCode_execWithin hw hfit_space hfit_V hp.1 o hvalid hp.2) := by + simp only [y, hp, ↓reduceDIte] + have hT_eq : T_probe p = hp.1.table_inv.probeLen u (hashFn U n_max p.1 u) := by + simp only [T_probe, hp, ↓reduceDIte] + refine ⟨?_, ?_⟩ + · have hdist := ExecCodeDist.of_execWithin hex + refine hdist.congr_dist ?_ ?_ + · simp [hy_eq] + · dsimp only [T_step] + rw [hT_eq] + · intro q hq + rw [PMF.support_pure, Set.mem_singleton_iff] at hq + subst hq + exact ⟨by simpa only [hy_eq] using hdec, + by simpa only [hy_eq] using hrep', + by simpa only [hy_eq] using hmod⟩ + · intro θ' + have hmap_eq : (ν.bind fun p => PMF.pure (p.1, y p)).map Prod.fst = ν.map Prod.fst := by + rw [PMF.map_bind] + simp only [PMF.pure_map] + rfl + rw [hmap_eq] + exact hseed θ' + · simp only [Nat.cast_zero, mul_zero, tsum_zero, add_zero] + have htsum_probe : (∑' p, ν p * (T_probe p : ℝ≥0∞)) ≤ 288 := by + apply tsum_probeLen_le (V := V) (base := base) (s := s) u ν + · intro θ + simpa only [one_mul] using hseed θ + · intro p hp_mem + have hp : hasInv p := hsupp p hp_mem + exact ⟨p.2.mem, hp.1.table_inv, by simp only [T_probe, hp, ↓reduceDIte, le_rfl]⟩ + calc (∑' p, ν p * T_step p) + = ∑' p, ν p * + ((KWise.evalSteps n_max U (tableLog n_max) 5 + 27 : ℝ≥0∞) + + 12 * (T_probe p : ℝ≥0∞)) := by + refine tsum_congr fun p => ?_ + dsimp only [T_step] + simp only [Nat.cast_add, Nat.cast_mul, Nat.cast_ofNat] + ring_nf + _ = (∑' p, ν p * (KWise.evalSteps n_max U (tableLog n_max) 5 + 27 : ℝ≥0∞)) + + 12 * (∑' p, ν p * (T_probe p : ℝ≥0∞)) := by + simp_rw [mul_add, tsum_add, ← mul_assoc, mul_comm (ν _) 12, mul_assoc, + ENNReal.tsum_mul_left] + _ = (KWise.evalSteps n_max U (tableLog n_max) 5 + 27 : ℝ≥0∞) + + 12 * (∑' p, ν p * (T_probe p : ℝ≥0∞)) := by + rw [ENNReal.tsum_mul_right, ν.tsum_coe, one_mul] + _ ≤ (KWise.evalSteps n_max U (tableLog n_max) 5 + 27 : ℝ≥0∞) + 12 * 288 := by + gcongr + _ = (charge U n_max : ℝ≥0∞) := by + dsimp only [charge] + simp only [Nat.cast_add, Nat.cast_mul, Nat.cast_ofNat] + ring_nf + +/-- Standard-basis (`Basis.standard`) `SubroutineDS` certificate for linear probing, transported +from `subroutineDS` via `SubroutineDS.toStandard`. -/ +noncomputable def subroutineDSStandard (U V n_max w base : ℕ) + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) + (hfit_space : Fits w (base + space n_max)) + (hfit_V : Fits w V) : + Cslib.WordRAM.Randomized.SubroutineDS + (subroutineCode U n_max).toStandard + (Dictionary.spec U V n_max) w base Fin.elim0 (Dictionary.empty U) := + (subroutineDS U V n_max w base hw hfit_space hfit_V).toStandard + +/-- **Expected amortized multi-operation theorem for linear probing**: for any admissible sequence +`ops : List (Dictionary.Op U)` starting from the empty dictionary on fresh zeroed memory and fixed +independently of the sampled hash seed (oblivious adversary), `initFresh` followed by the +subroutine operation bodies (`SubroutineDS.ExecSteps`) executes with total expected instruction +count at most `3 + setupTime U n_max + ops.length * charge U n_max` (excluding caller argument +delivery, caller dispatch loops, and host-level code specialization). -/ +theorem seq_expected_amortized (U V n_max w base : ℕ) + (hw : 2 * trinomialDeg U (tableLog n_max) ≤ w) + (hfit_space : Fits w (base + space n_max)) + (hfit_V : Fits w V) + {x₀ : Data rLP} + (hx₀ : x₀.Bounded w) + (hfr : x₀.regs 18 = base) + (hz : ∀ a ∈ (Block.mk base (space n_max)).footprint, x₀.mem a = 0) + (ops : List (Dictionary.Op U)) + (hvalid : (Dictionary.spec U V n_max).ValidHistory (Dictionary.empty U) ops) : + ∃ (ν₀ ν_f : PMF (Seed U n_max × Data rLP)) + (T_ops : ℝ≥0∞), + ExecCodeDist w (subroutineCode U n_max).initFresh x₀ + (ν₀.map Prod.snd) (3 + setupTime U n_max : ℕ) ∧ + (subroutineDS U V n_max w base hw hfit_space hfit_V).ExecSteps + (Dictionary.empty U) ops ν₀ ν_f T_ops ∧ + (3 + setupTime U n_max : ℕ) + T_ops ≤ + (3 + setupTime U n_max + ops.length * charge U n_max : ℕ) := by + let C := subroutineDS U V n_max w base hw hfit_space hfit_V + obtain ⟨ν₀, ν_f, T_ops, hinit, hsteps, _, _, hbound⟩ := + C.seq_expected_amortized ops hx₀ hfr (fun j => Fin.elim0 j) hz hvalid + have htotal : (Dictionary.spec U V n_max).totalCost C.charge (Dictionary.empty U) ops = + ops.length * charge U n_max := by + have hconst : ∀ s (l : List (Dictionary.Op U)), + (Dictionary.spec U V n_max).totalCost (fun _ _ => charge U n_max) s l = + l.length * charge U n_max := by + intro s l + induction l generalizing s with + | nil => simp [Stateful.Spec.totalCost] + | cons o tl ih => + simp only [Stateful.Spec.totalCost, ih, List.length_cons] + ring + exact hconst (Dictionary.empty U) ops + have hinitTime : C.initTime = ((3 + setupTime U n_max : ℕ) : ℝ≥0∞) := by + dsimp only [Cslib.WordRAM.Randomized.SubroutineDS.initTime, C, subroutineDS] + push_cast + rfl + rw [hinitTime] at hinit hbound + have hpot0 : (∑' q, ν_f q * (C.potential q.1 q.2 : ℝ≥0∞)) = 0 := by + simp [C, subroutineDS] + have hinitPot0 : (C.initPotential : ℝ≥0∞) = 0 := Nat.cast_zero + rw [htotal, hpot0, hinitPot0, add_zero, add_zero] at hbound + have hbound' : ((3 + setupTime U n_max : ℕ) : ℝ≥0∞) + T_ops ≤ + (3 + setupTime U n_max + ops.length * charge U n_max : ℕ) := by + simpa only [← Nat.cast_add] using hbound + exact ⟨ν₀, ν_f, T_ops, hinit, hsteps, hbound'⟩ + +end Cslib.Algorithms.WordRAM.Dictionary.LinearProbing diff --git a/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean new file mode 100644 index 000000000..a01d86713 --- /dev/null +++ b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean @@ -0,0 +1,722 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Probability.KWiseIndependent +public import Complexity.Probability.PMF.Expect + +/-! +# Constant-independence cluster bounds for linear probing + +This file proves the probabilistic core of constant-time linear probing under $5$-wise independent +hashing [Pagh, Pagh and Ružić, *Linear Probing with Constant Independence*][PaghPaghRuzic2009]; +see also [Cormen, Leiserson, Rivest and Stein, *Introduction to Algorithms*, Section 11.4][CLRS2009] +and [Vadhan, *Pseudorandomness*, Section 3.5][Vadhan2012]. + +For a power-of-$2$ table of size $m = 2^M \ge 8 n_{\max}$ (load factor $\alpha = n / m \le 1/8$) and +a $5$-wise independent hash family $h : \iota \to X \to \mathrm{Fin}(2^M)$: +1. If probing key $u$ from $b = h_i(u)$ inspects $t \ge 1$ occupied slots belonging to + $S = \mathrm{dom}(s) \setminus \{u\}$, walking backwards from $b$ to the nearest preceding empty + slot yields a contiguous cyclic interval $J(b, a, \ell)$ of length $\ell = a + t \in [t, \#S]$ + containing $b$ into which at least $\ell$ keys of $S$ hash. +2. For the dyadic scale $j = \lfloor \log_2 \ell \rfloor + 1 \in \{1, \dots, M\}$, we have + $2^{j-1} \le \ell < 2^j$, and every such interval $J(b, a, \ell)$ is contained in a single + centered window $W_j(b)$ of size at most $2^{j+1}$. Consequently, at least $2^{j-1}$ keys of $S$ + hash into $W_j(b)$, whereas the expected load of $W_j(b)$ is + $\mu_{j,b} \le \#S \cdot 2^{j+1} / 2^M \le 2^{j-2}$. +3. Conditioning on $h_i(u) = b$ consumes $1$ degree of independence, leaving $4$-wise independence + on $S$. Expanding the fourth central moment via `IsKWiseIndependentHash.expect_prod` on sets + $\mathrm{insert}\ u\ (\mathrm{image}\ t)$ of size at most $5$ bounds the overload probability at + scale $j$ by $18 / (2^{j-2})^2 = 288 / 4^j$, and summing $\sum_{j=1}^M 2^j \cdot (288 / 4^j)$ + yields an expected probe count of at most $288 = O(1)$. + +## Main definitions + +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.tableLog`, + `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.tableSize`: power-of-$2$ table bit-width + $M = \lceil \log_2 \max(8 n_{\max}, 8) \rceil$ and capacity $2^M$. +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.cyclicSlotMod`, + `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.cyclicInterval`, + `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.scaleWindow`: cyclic slot indexing, contiguous + cyclic intervals containing $b$, and centered dyadic-scale windows $W_j(b)$. +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.scaleOverload`, + `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.HasClusterWitness`: scale-$j$ window overload + event $2^{j-1} \le \#\{x \in S \mid h_i(x) \in W_j(h_i(u))\}$ and left-anchored cluster-witness + predicate. + +## Main results + +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.eight_mul_le_tableSize`, + `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.tableSize_le_sixteen_mul`: load-factor and + linear-space bounds for `tableSize`. +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.expect_indicator_mul_sum_pow_four_le`: + fourth-moment bound for centered bucket indicators conditioned on $h_i(u) = b$ under $5$-wise + independence. +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.expect_scaleOverload_le`: scale-$j$ overload + probability bound $\le 288 / 4^j$. +* `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.expect_probe_le_of_cluster_witness`, + `Cslib.Algorithms.WordRAM.Dictionary.LinearProbing.tsum_uniform_probe_le_of_cluster_witness`: + expected probe count $\le 288$ under `Finset.expect` and `PMF.uniformOfFintype`. + +## References + +* [A. Pagh, R. Pagh and M. Ružić, *Linear Probing with Constant Independence*][PaghPaghRuzic2009] +* [T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, *Introduction to Algorithms*, + Section 11.4][CLRS2009] +* [S. P. Vadhan, *Pseudorandomness*, Section 3.5][Vadhan2012] +-/ + +public section + +namespace Cslib.Algorithms.WordRAM.Dictionary.LinearProbing + +open Cslib.Probability Finset Fintype +open scoped BigOperators ENNReal + +/-! ## Section 1: Table size parameters and load-factor bounds -/ + +/-- Base-2 logarithm $M = \lceil \log_2 \max(8 n_{\max}, 8) \rceil$ of the linear-probing table +size, ensuring $M \ge 3$ and $8 n_{\max} \le 2^M < 16 \max(n_{\max}, 1)$. -/ +@[expose] def tableLog (n_max : ℕ) : ℕ := + Nat.clog 2 (max (8 * n_max) 8) + +/-- Power-of-$2$ table size $m = 2^{\mathrm{tableLog}(n_{\max})}$. -/ +@[expose] def tableSize (n_max : ℕ) : ℕ := + 2 ^ tableLog n_max + +/-- `tableLog n_max` is always at least `3`. -/ +theorem three_le_tableLog (n_max : ℕ) : 3 ≤ tableLog n_max := by + dsimp only [tableLog] + have h8 : 8 ≤ max (8 * n_max) 8 := le_max_right _ _ + calc 3 = Nat.clog 2 8 := by decide + _ ≤ Nat.clog 2 (max (8 * n_max) 8) := Nat.clog_mono_right 2 h8 + +/-- `tableLog n_max` is always positive. -/ +theorem tableLog_pos (n_max : ℕ) : 0 < tableLog n_max := by + have := three_le_tableLog n_max + omega + +/-- `tableSize n_max` is always positive. -/ +theorem tableSize_pos (n_max : ℕ) : 0 < tableSize n_max := by + dsimp only [tableSize] + positivity + +/-- `tableSize n_max` is always at least `8`. -/ +theorem eight_le_tableSize (n_max : ℕ) : 8 ≤ tableSize n_max := by + dsimp only [tableSize] + calc 8 = 2 ^ 3 := by decide + _ ≤ 2 ^ tableLog n_max := Nat.pow_le_pow_right (by decide) (three_le_tableLog n_max) + +/-- The table size is at least $8 n_{\max}$ (load factor $\le 1/8$). -/ +theorem eight_mul_le_tableSize (n_max : ℕ) : 8 * n_max ≤ tableSize n_max := by + dsimp only [tableSize, tableLog] + calc 8 * n_max ≤ max (8 * n_max) 8 := le_max_left _ _ + _ ≤ 2 ^ Nat.clog 2 (max (8 * n_max) 8) := Nat.le_pow_clog (by decide) _ + +/-- `n_max` is strictly less than `tableSize n_max`, so at least $7/8$ of the table slots are +always empty. -/ +theorem n_max_lt_tableSize (n_max : ℕ) : n_max < tableSize n_max := by + have h8 := eight_le_tableSize n_max + have hmul := eight_mul_le_tableSize n_max + omega + +/-- `tableSize n_max` is strictly less than $16 \max(n_{\max}, 1)$. -/ +theorem tableSize_lt_sixteen_mul (n_max : ℕ) : tableSize n_max < 16 * max n_max 1 := by + dsimp only [tableSize, tableLog] + have hpos : 1 < max (8 * n_max) 8 := by + have : 8 ≤ max (8 * n_max) 8 := le_max_right _ _ + omega + have hlt : 2 ^ (Nat.clog 2 (max (8 * n_max) 8) - 1) < max (8 * n_max) 8 := + Nat.pow_pred_clog_lt_self (by decide) hpos + have hclog_pos : 1 ≤ Nat.clog 2 (max (8 * n_max) 8) := by + have := three_le_tableLog n_max + dsimp only [tableLog] at this + omega + have hpow : 2 ^ Nat.clog 2 (max (8 * n_max) 8) = + 2 * 2 ^ (Nat.clog 2 (max (8 * n_max) 8) - 1) := by + conv_lhs => rw [← Nat.sub_add_cancel hclog_pos, Nat.pow_succ'] + have hmax : max (8 * n_max) 8 = 8 * max n_max 1 := by omega + omega + +/-- `tableSize n_max` is at most $16 \max(n_{\max}, 1) = O(n_{\max})$. -/ +theorem tableSize_le_sixteen_mul (n_max : ℕ) : tableSize n_max ≤ 16 * max n_max 1 := + le_of_lt (tableSize_lt_sixteen_mul n_max) + +/-! ## Section 2: Fourth central moment under $5$-wise independence conditioned on $h_i(u) = b$ -/ + +/-- Tuples of `n` elements of `S` with at most `m ≤ n` distinct entries factor through `Fin m`, so +there are at most `m ^ n * #S ^ m` of them. -/ +private theorem card_filter_card_image_le {X : Type*} [DecidableEq X] (S : Finset X) {m n : ℕ} + (hmn : m ≤ n) : + #{t ∈ piFinset fun _ : Fin n => S | #(univ.image t) ≤ m} ≤ m ^ n * #S ^ m := by + classical + calc #{t ∈ piFinset fun _ : Fin n => S | #(univ.image t) ≤ m} + ≤ #(((piFinset fun _ : Fin n => (univ : Finset (Fin m))) ×ˢ + (piFinset fun _ : Fin m => S)).image fun q => q.2 ∘ q.1) := by + refine card_le_card fun t ht => ?_ + rw [mem_filter, mem_piFinset] at ht + obtain ⟨htS, hcard⟩ := ht + set I := univ.image t + let e := I.equivFin + have hmem (j : Fin n) : t j ∈ I := mem_image_of_mem t (mem_univ j) + let v (r : Fin m) : X := if hr : r.1 < #I then (e.symm ⟨r, hr⟩).1 else + t ⟨0, lt_of_lt_of_le (Nat.zero_lt_of_lt r.2) hmn⟩ + refine mem_image.2 ⟨(fun j => Fin.castLE hcard (e ⟨t j, hmem j⟩), v), ?_, ?_⟩ + · refine mem_product.2 ⟨by simp, mem_piFinset.2 fun r => ?_⟩ + by_cases hr : r.1 < #I + · obtain ⟨j, -, hj⟩ := mem_image.1 (e.symm ⟨r, hr⟩).2 + simp only [v, hr, ↓reduceDIte, ← hj] + exact htS j + · simp only [v, hr, ↓reduceDIte] + exact htS _ + · funext j + have hlt : (Fin.castLE hcard (e ⟨t j, hmem j⟩)).1 < #I := (e ⟨t j, hmem j⟩).2 + simp only [Function.comp_apply, v, hlt, ↓reduceDIte] + have : (⟨(Fin.castLE hcard (e ⟨t j, hmem j⟩)).1, hlt⟩ : Fin #I) = e ⟨t j, hmem j⟩ := + Fin.ext rfl + rw [this, Equiv.symm_apply_apply] + _ ≤ m ^ n * #S ^ m := card_image_le.trans (by simp [card_piFinset]) + +/-- Centered indicator of a bucket subset `W ⊆ Y`: `1 - #W / |Y|` on `W` and `- #W / |Y|` outside +`W`. -/ +@[expose] noncomputable def bucketCentered {Y : Type*} [Fintype Y] [DecidableEq Y] + (W : Finset Y) (y : Y) : ℝ := + (if y ∈ W then (1 : ℝ) else 0) - (#W : ℝ) / (Fintype.card Y : ℝ) + +/-- The uniform expectation of `bucketCentered W` over a nonempty finite type `Y` is `0`. -/ +theorem expect_bucketCentered {Y : Type*} [Fintype Y] [DecidableEq Y] [Nonempty Y] + (W : Finset Y) : + 𝔼 y : Y, bucketCentered W y = 0 := by + have hpos : (0 : ℝ) < Fintype.card Y := Nat.cast_pos.mpr Fintype.card_pos + rw [Fintype.expect_eq_sum_div_card] + simp only [bucketCentered, sum_sub_distrib, sum_boole, filter_mem_eq_inter, univ_inter, + sum_const, card_univ, nsmul_eq_mul] + rw [mul_div_cancel₀ _ hpos.ne', sub_self, zero_div] + +/-- `bucketCentered W y` always has absolute value at most `1`. -/ +theorem abs_bucketCentered_le_one {Y : Type*} [Fintype Y] [DecidableEq Y] + (W : Finset Y) (y : Y) : + |bucketCentered W y| ≤ 1 := by + have hcard : (#W : ℝ) ≤ (Fintype.card Y : ℝ) := by + exact_mod_cast W.card_le_univ + have hp0 : 0 ≤ (#W : ℝ) / (Fintype.card Y : ℝ) := by positivity + have hp1 : (#W : ℝ) / (Fintype.card Y : ℝ) ≤ 1 := div_le_one_of_le₀ hcard (by positivity) + dsimp only [bucketCentered] + rw [abs_le] + split_ifs <;> constructor <;> linarith + +/-- The second moment of `bucketCentered W` is at most `p = #W / |Y|`. -/ +theorem expect_bucketCentered_sq_le {Y : Type*} [Fintype Y] [DecidableEq Y] [Nonempty Y] + (W : Finset Y) : + 𝔼 y : Y, (bucketCentered W y) ^ 2 ≤ (#W : ℝ) / (Fintype.card Y : ℝ) := by + set p : ℝ := (#W : ℝ) / (Fintype.card Y : ℝ) + have hpos : (0 : ℝ) < Fintype.card Y := Nat.cast_pos.mpr Fintype.card_pos + have hp0 : 0 ≤ p := by positivity + have hcard : (#W : ℝ) ≤ (Fintype.card Y : ℝ) := by exact_mod_cast W.card_le_univ + have hp1 : p ≤ 1 := div_le_one_of_le₀ hcard (by positivity) + have hle_pt (y : Y) : (bucketCentered W y) ^ 2 ≤ + (if y ∈ W then (1 : ℝ) else 0) * (1 - p) + (1 - (if y ∈ W then (1 : ℝ) else 0)) * p ^ 2 := by + dsimp only [bucketCentered, p] + split_ifs <;> nlinarith + have hmean : (𝔼 y : Y, if y ∈ W then (1 : ℝ) else 0) = p := by + rw [Fintype.expect_eq_sum_div_card] + simp only [sum_boole, filter_mem_eq_inter, univ_inter, p] + calc 𝔼 y : Y, (bucketCentered W y) ^ 2 + ≤ 𝔼 y : Y, ((if y ∈ W then (1 : ℝ) else 0) * (1 - p) + + (1 - (if y ∈ W then (1 : ℝ) else 0)) * p ^ 2) := + expect_le_expect fun y _ => hle_pt y + _ = (𝔼 y : Y, if y ∈ W then (1 : ℝ) else 0) * (1 - p) + + (1 - (𝔼 y : Y, if y ∈ W then (1 : ℝ) else 0)) * p ^ 2 := by + rw [Fintype.expect_eq_sum_div_card, Fintype.expect_eq_sum_div_card] + simp only [sum_add_distrib, ← sum_mul, add_div, mul_div_right_comm, sub_div, + sum_sub_distrib, sum_const, card_univ, nsmul_eq_mul, mul_one] + rw [div_self hpos.ne'] + _ = p * (1 - p) + (1 - p) * p ^ 2 := by rw [hmean] + _ ≤ p := by nlinarith + +/-- Any moment of degree `c ≥ 2` of `bucketCentered W` has absolute value at most +`p = #W / |Y|`. -/ +theorem abs_expect_bucketCentered_pow_le {Y : Type*} [Fintype Y] [DecidableEq Y] [Nonempty Y] + (W : Finset Y) {c : ℕ} (hc : 2 ≤ c) : + |𝔼 y : Y, (bucketCentered W y) ^ c| ≤ (#W : ℝ) / (Fintype.card Y : ℝ) := by + refine (abs_expect_le _ _).trans ((expect_le_expect fun y _ => ?_).trans + (expect_bucketCentered_sq_le W)) + rw [abs_pow] + have habs := abs_bucketCentered_le_one W y + have h0 : 0 ≤ |bucketCentered W y| := abs_nonneg _ + have hpow : |bucketCentered W y| ^ c ≤ |bucketCentered W y| ^ 2 := + pow_le_pow_of_le_one h0 habs hc + simpa only [sq_abs] using hpow + +/-- **Conditioned fourth-moment bound under $5$-wise independence**: for a $5$-wise independent +family `h : ι → X → Y`, a finite set `S ⊆ X`, a query point `u ∉ S`, a target bucket `b : Y`, and a +window `W ⊆ Y` of density `p = #W / |Y|`, the expectation of +`(if h i u = b then 1 else 0) * (∑ x ∈ S, bucketCentered W (h i x)) ^ 4` is at most +`(𝔼 y, if y = b then 1 else 0) * (#S * p + 16 * (#S * p) ^ 2)`. -/ +theorem expect_indicator_mul_sum_pow_four_le {ι X Y : Type*} [Fintype ι] [Fintype Y] + [DecidableEq Y] [Nonempty Y] {h : ι → X → Y} + (hh : IsKWiseIndependentHash 5 h) (S : Finset X) {u : X} (hu : u ∉ S) + (b : Y) (W : Finset Y) : + let p : ℝ := (#W : ℝ) / (Fintype.card Y : ℝ) + 𝔼 i, (if h i u = b then (1 : ℝ) else 0) * (∑ x ∈ S, bucketCentered W (h i x)) ^ 4 ≤ + (𝔼 y : Y, if y = b then (1 : ℝ) else 0) * + ((#S : ℝ) * p + 16 * ((#S : ℝ) * p) ^ 2) := by + classical + intro p + have := hh.nonempty + have hp0 : 0 ≤ p := by positivity + have hcardW : (#W : ℝ) ≤ (Fintype.card Y : ℝ) := by exact_mod_cast W.card_le_univ + have hp1 : p ≤ 1 := div_le_one_of_le₀ hcardW (by positivity) + have hb0 : 0 ≤ (𝔼 y : Y, if y = b then (1 : ℝ) else 0) := + expect_nonneg fun y _ => by split_ifs <;> positivity + simp only [sum_pow', mul_sum] + rw [expect_sum_comm] + have hterm : ∀ t ∈ piFinset fun _ : Fin 4 => S, + 𝔼 i, (if h i u = b then (1 : ℝ) else 0) * ∏ r : Fin 4, bucketCentered W (h i (t r)) ≤ + (𝔼 y : Y, if y = b then (1 : ℝ) else 0) * + ((if #(univ.image t) ≤ 1 then p else 0) + + (if #(univ.image t) ≤ 2 then p ^ 2 else 0)) := by + intro t ht + set I := univ.image t + have htS : ∀ x ∈ I, x ∈ S := by + intro x hx + obtain ⟨r, -, rfl⟩ := mem_image.1 hx + exact mem_piFinset.1 ht r + have huI : u ∉ I := fun hmem => hu (htS u hmem) + have hcard_I : #I ≤ 4 := card_image_le.trans (by simp) + have hcard_ins : #(insert u I) ≤ 5 := by + rw [card_insert_of_notMem huI] + omega + let f : X → Y → ℝ := fun z y => + if z = u then (if y = b then 1 else 0) + else (bucketCentered W y) ^ #{r : Fin 4 | t r = z} + have hprod_i (i : ι) : + (if h i u = b then (1 : ℝ) else 0) * (∏ r : Fin 4, bucketCentered W (h i (t r))) = + ∏ z ∈ insert u I, f z (h i z) := by + rw [prod_insert huI, prod_comp (fun x => bucketCentered W (h i x)) t] + simp only [f, ↓reduceIte] + congr 1 + refine prod_congr rfl fun x hx => ?_ + have hne : x ≠ u := fun h => huI (h ▸ hx) + simp only [hne, ↓reduceIte] + simp only [hprod_i] + rw [hh.expect_prod hcard_ins f, prod_insert huI] + have hfu : (𝔼 y : Y, f u y) = 𝔼 y : Y, if y = b then (1 : ℝ) else 0 := by + simp only [f, ↓reduceIte] + have hfI : (∏ x ∈ I, 𝔼 y : Y, f x y) = + ∏ x ∈ I, 𝔼 y : Y, (bucketCentered W y) ^ #{r : Fin 4 | t r = x} := by + refine prod_congr rfl fun x hx => ?_ + have hne : x ≠ u := fun h => huI (h ▸ hx) + simp only [f, hne, ↓reduceIte] + rw [hfu, hfI] + refine mul_le_mul_of_nonneg_left ?_ hb0 + by_cases hall2 : ∀ x ∈ I, 2 ≤ #{r : Fin 4 | t r = x} + · have hsum_mult := card_eq_sum_card_image t (univ : Finset (Fin 4)) + have h2I : 2 * #I ≤ 4 := by + have hle : ∑ _x ∈ I, 2 ≤ ∑ x ∈ I, #{r : Fin 4 | t r = x} := + sum_le_sum fun x hx => hall2 x hx + rw [← hsum_mult, sum_const, card_univ, Fintype.card_fin, smul_eq_mul] at hle + omega + have hI_le2 : #I ≤ 2 := by omega + have hI_pos : 1 ≤ #I := by + have : t 0 ∈ I := mem_image_of_mem t (mem_univ 0) + exact Finset.card_pos.mpr ⟨t 0, this⟩ + have hprod_le : (∏ x ∈ I, 𝔼 y : Y, (bucketCentered W y) ^ #{r : Fin 4 | t r = x}) ≤ + p ^ #I := by + refine (le_abs_self _).trans ?_ + rw [abs_prod] + calc ∏ x ∈ I, |𝔼 y : Y, (bucketCentered W y) ^ #{r : Fin 4 | t r = x}| + ≤ ∏ _x ∈ I, p := by + gcongr with x hx + exact abs_expect_bucketCentered_pow_le W (hall2 x hx) + _ = p ^ #I := by rw [prod_const] + by_cases hI1 : #I ≤ 1 + · have hI_eq1 : #I = 1 := by omega + rw [hI_eq1, pow_one] at hprod_le + simp only [hI1, hI_le2, ↓reduceIte] + linarith [sq_nonneg p] + · have hI_eq2 : #I = 2 := by omega + rw [hI_eq2] at hprod_le + simp only [hI1, hI_le2, ↓reduceIte, zero_add] + exact hprod_le + · push Not at hall2 + obtain ⟨x, hx, hlt2⟩ := hall2 + obtain ⟨r, -, rfl⟩ := mem_image.1 hx + have hpos : 0 < #{r' : Fin 4 | t r' = t r} := card_pos.2 ⟨r, by simp⟩ + have hc1 : #{r' : Fin 4 | t r' = t r} = 1 := by omega + have hzero : (∏ z ∈ I, 𝔼 y : Y, (bucketCentered W y) ^ #{r' : Fin 4 | t r' = z}) = 0 := + prod_eq_zero hx (by simp only [hc1, pow_one, expect_bucketCentered]) + rw [hzero] + positivity + calc ∑ t ∈ piFinset (fun _ : Fin 4 => S), + 𝔼 i, (if h i u = b then (1 : ℝ) else 0) * ∏ r : Fin 4, bucketCentered W (h i (t r)) + ≤ ∑ t ∈ piFinset (fun _ : Fin 4 => S), + (𝔼 y : Y, if y = b then (1 : ℝ) else 0) * + ((if #(univ.image t) ≤ 1 then p else 0) + + (if #(univ.image t) ≤ 2 then p ^ 2 else 0)) := + sum_le_sum hterm + _ = (𝔼 y : Y, if y = b then (1 : ℝ) else 0) * + ((∑ t ∈ piFinset (fun _ : Fin 4 => S), if #(univ.image t) ≤ 1 then p else 0) + + (∑ t ∈ piFinset (fun _ : Fin 4 => S), if #(univ.image t) ≤ 2 then p ^ 2 else 0)) := by + rw [← mul_sum, sum_add_distrib] + _ = (𝔼 y : Y, if y = b then (1 : ℝ) else 0) * + ((#{t ∈ piFinset (fun _ : Fin 4 => S) | #(univ.image t) ≤ 1} : ℝ) * p + + (#{t ∈ piFinset (fun _ : Fin 4 => S) | #(univ.image t) ≤ 2} : ℝ) * p ^ 2) := by + simp only [sum_ite, sum_const_zero, add_zero, sum_const, nsmul_eq_mul] + _ ≤ (𝔼 y : Y, if y = b then (1 : ℝ) else 0) * + ((#S : ℝ) * p + 16 * ((#S : ℝ) * p) ^ 2) := by + refine mul_le_mul_of_nonneg_left ?_ hb0 + have hc1 : (#{t ∈ piFinset (fun _ : Fin 4 => S) | #(univ.image t) ≤ 1} : ℝ) ≤ #S := by + have h1 := card_filter_card_image_le S (m := 1) (n := 4) (by decide) + simpa using (Nat.cast_le (α := ℝ)).mpr h1 + have hc2 : (#{t ∈ piFinset (fun _ : Fin 4 => S) | #(univ.image t) ≤ 2} : ℝ) ≤ + 16 * (#S : ℝ) ^ 2 := by + have h2 := card_filter_card_image_le S (m := 2) (n := 4) (by decide) + exact_mod_cast h2 + have h1 : (#{t ∈ piFinset (fun _ : Fin 4 => S) | #(univ.image t) ≤ 1} : ℝ) * p ≤ + (#S : ℝ) * p := mul_le_mul_of_nonneg_right hc1 hp0 + have h2 : (#{t ∈ piFinset (fun _ : Fin 4 => S) | #(univ.image t) ≤ 2} : ℝ) * p ^ 2 ≤ + 16 * (#S : ℝ) ^ 2 * p ^ 2 := mul_le_mul_of_nonneg_right hc2 (sq_nonneg p) + linarith + +/-! ## Section 3: Cyclic intervals, scale windows, and expected probe bound -/ + +/-- Slot in `Fin (2 ^ M)` corresponding to natural number `x` modulo `2 ^ M`. -/ +@[expose] def cyclicSlotMod (M : ℕ) (x : ℕ) : Fin (2 ^ M) := + ⟨x % 2 ^ M, Nat.mod_lt _ (by positivity)⟩ + +/-- Contiguous cyclic interval of length `len` starting `a` slots before `b` in `Fin (2 ^ M)`: +$\{(b - a + \mathrm{idx}) \bmod 2^M \mid 0 \le \mathrm{idx} < \mathrm{len}\}$. -/ +@[expose] def cyclicInterval (M : ℕ) (b : Fin (2 ^ M)) (a len : ℕ) : Finset (Fin (2 ^ M)) := + (Finset.range len).image fun idx => cyclicSlotMod M (b.val + 2 ^ M * (a + 1) - a + idx) + +/-- Centered dyadic-scale window $W_j(b)$ of width $2^{j+1}$ around $b$ in `Fin (2 ^ M)`: +$\{(b - 2^j + d) \bmod 2^M \mid 0 \le d < 2^{j+1}\}$. -/ +@[expose] def scaleWindow (M : ℕ) (b : Fin (2 ^ M)) (j : ℕ) : Finset (Fin (2 ^ M)) := + (Finset.range (2 ^ (j + 1))).image fun d => + cyclicSlotMod M (b.val + 2 ^ M * 2 ^ j - 2 ^ j + d) + +/-- The scale window `scaleWindow M b j` has at most `2 ^ (j + 1)` elements. -/ +theorem card_scaleWindow_le (M : ℕ) (b : Fin (2 ^ M)) (j : ℕ) : + #(scaleWindow M b j) ≤ 2 ^ (j + 1) := by + dsimp only [scaleWindow] + exact card_image_le.trans (by simp) + +/-- Every cyclic interval `cyclicInterval M b a len` with `a < 2 ^ j` and `len ≤ 2 ^ j` is a +subset of the centered window `scaleWindow M b j`. -/ +theorem cyclicInterval_subset_scaleWindow {M : ℕ} (b : Fin (2 ^ M)) {a len j : ℕ} + (ha : a < 2 ^ j) (hlen : len ≤ 2 ^ j) : + cyclicInterval M b a len ⊆ scaleWindow M b j := by + intro y hy + simp only [cyclicInterval, mem_image, mem_range] at hy + obtain ⟨idx, hidx, rfl⟩ := hy + simp only [scaleWindow, mem_image, mem_range] + refine ⟨2 ^ j - a + idx, ?_, ?_⟩ + · rw [Nat.pow_succ] + omega + · apply Fin.ext + simp only [cyclicSlotMod] + have h1 : 1 ≤ 2 ^ M := Nat.one_le_two_pow + have hmul : 2 ^ M * 2 ^ j = 2 ^ M * (a + 1) + 2 ^ M * (2 ^ j - (a + 1)) := by + rw [← Nat.mul_add, Nat.add_sub_cancel' (by omega)] + have ha_le : a ≤ 2 ^ M * (a + 1) := by + calc a ≤ 1 * (a + 1) := by omega + _ ≤ 2 ^ M * (a + 1) := Nat.mul_le_mul_right (a + 1) h1 + have hj_le : 2 ^ j ≤ 2 ^ M * 2 ^ j := by + calc 2 ^ j = 1 * 2 ^ j := (Nat.one_mul _).symm + _ ≤ 2 ^ M * 2 ^ j := Nat.mul_le_mul_right (2 ^ j) h1 + have heq : b.val + 2 ^ M * 2 ^ j - 2 ^ j + (2 ^ j - a + idx) = + (b.val + 2 ^ M * (a + 1) - a + idx) + 2 ^ M * (2 ^ j - (a + 1)) := by + omega + rw [heq, Nat.add_mul_mod_self_left] + +/-- Scale-$j$ window overload event: at least $2^{j-1}$ keys of $S$ hash into the centered window +`scaleWindow M (h i u) j` around $h_i(u)$. -/ +@[expose] def scaleOverload {ι X : Type*} (M : ℕ) (h : ι → X → Fin (2 ^ M)) + (S : Finset X) (u : X) (j : ℕ) (i : ι) : Prop := + 2 ^ (j - 1) ≤ #(S.filter fun x => h i x ∈ scaleWindow M (h i u) j) + +instance instDecidablePredScaleOverload {ι X : Type*} (M : ℕ) (h : ι → X → Fin (2 ^ M)) + (S : Finset X) (u : X) (j : ℕ) : DecidablePred (scaleOverload M h S u j) := + fun _ => inferInstanceAs (Decidable (_ ≤ _)) + +/-- Pointwise sum identity for `bucketCentered W`: summing `bucketCentered W (g x)` over `x ∈ S` +equals `#(S.filter (g · ∈ W)) - #S * (#W / |Y|)`. -/ +private theorem sum_bucketCentered_eq {X Y : Type*} [Fintype Y] [DecidableEq Y] + (S : Finset X) (W : Finset Y) (g : X → Y) : + ∑ x ∈ S, bucketCentered W (g x) = + (#(S.filter fun x => g x ∈ W) : ℝ) - (#S : ℝ) * ((#W : ℝ) / (Fintype.card Y : ℝ)) := by + simp only [bucketCentered, sum_sub_distrib, sum_boole, sum_const, nsmul_eq_mul] + +/-- **Scale-$j$ overload probability bound**: when `8 * n_max ≤ 2 ^ M`, `#S ≤ n_max`, `u ∉ S`, +`hh : IsKWiseIndependentHash 5 h`, and `1 ≤ j`, the probability of `scaleOverload M h S u j` is at +most `288 / (2 ^ j : ℝ) ^ 2`. -/ +theorem expect_scaleOverload_le {ι X : Type*} [Fintype ι] + {M n_max j : ℕ} {h : ι → X → Fin (2 ^ M)} + (hh : IsKWiseIndependentHash 5 h) (S : Finset X) {u : X} (hu : u ∉ S) + (hS : #S ≤ n_max) (hM : 8 * n_max ≤ 2 ^ M) (hj : 1 ≤ j) : + (𝔼 i, if scaleOverload M h S u j i then (1 : ℝ) else 0) ≤ + 288 / ((2 ^ j : ℝ) ^ 2) := by + have := hh.nonempty + set δ : ℝ := (2 ^ (j - 1) : ℝ) / 2 + have hpow_pos : (0 : ℝ) < 2 ^ (j - 1) := by positivity + have hδ_pos : 0 < δ := by dsimp only [δ]; positivity + have hδ_ge_half : (1 / 2 : ℝ) ≤ δ := by + dsimp only [δ] + have h1 : (1 : ℝ) ≤ (2 ^ (j - 1) : ℝ) := by + exact_mod_cast (Nat.one_le_two_pow (n := j - 1)) + linarith + have h2j_eq : (2 ^ j : ℝ) = 4 * δ := by + have hj_eq : j = (j - 1) + 1 := by omega + conv_lhs => rw [hj_eq, pow_succ] + dsimp only [δ] + ring + have hcardY : (Fintype.card (Fin (2 ^ M)) : ℝ) = 2 ^ M := by simp + have h2M_pos : (0 : ℝ) < 2 ^ M := by positivity + have hmean_le (b : Fin (2 ^ M)) : + (#S : ℝ) * ((#(scaleWindow M b j) : ℝ) / (Fintype.card (Fin (2 ^ M)) : ℝ)) ≤ δ := by + rw [hcardY, ← mul_div_assoc, div_le_iff₀ h2M_pos] + have hW : (#(scaleWindow M b j) : ℝ) ≤ (2 ^ (j + 1) : ℝ) := by + exact_mod_cast card_scaleWindow_le M b j + have hj1 : (2 ^ (j + 1) : ℝ) = 4 * (2 ^ (j - 1) : ℝ) := by + have : j + 1 = (j - 1) + 2 := by omega + rw [this, pow_add] + ring + have hSn : (#S : ℝ) ≤ n_max := by exact_mod_cast hS + have h8n : (8 * (n_max : ℝ)) ≤ 2 ^ M := by exact_mod_cast hM + calc (#S : ℝ) * (#(scaleWindow M b j) : ℝ) + ≤ (n_max : ℝ) * (4 * (2 ^ (j - 1) : ℝ)) := by + rw [← hj1] + exact mul_le_mul hSn hW (by positivity) (by positivity) + _ = (8 * (n_max : ℝ)) * δ := by dsimp only [δ]; ring + _ ≤ δ * (2 ^ M : ℝ) := by nlinarith + have hpt_le (i : ι) : + δ ^ 4 * (if scaleOverload M h S u j i then (1 : ℝ) else 0) ≤ + ∑ b : Fin (2 ^ M), + (if h i u = b then (1 : ℝ) else 0) * + (∑ x ∈ S, bucketCentered (scaleWindow M b j) (h i x)) ^ 4 := by + have hsum_b : + (∑ b : Fin (2 ^ M), + (if h i u = b then (1 : ℝ) else 0) * + (∑ x ∈ S, bucketCentered (scaleWindow M b j) (h i x)) ^ 4) = + (∑ x ∈ S, bucketCentered (scaleWindow M (h i u) j) (h i x)) ^ 4 := by + simp only [boole_mul, Fintype.sum_ite_eq] + rw [hsum_b] + by_cases hov : scaleOverload M h S u j i + · simp only [hov, ↓reduceIte, mul_one] + have hov_real : (2 ^ (j - 1) : ℝ) ≤ + (#(S.filter fun x => h i x ∈ scaleWindow M (h i u) j) : ℝ) := by + exact_mod_cast hov + have hsum_eq := sum_bucketCentered_eq S (scaleWindow M (h i u) j) (h i) + have hm := hmean_le (h i u) + have hdiff : δ ≤ ∑ x ∈ S, bucketCentered (scaleWindow M (h i u) j) (h i x) := by + rw [hsum_eq] + have : (2 ^ (j - 1) : ℝ) = 2 * δ := by dsimp only [δ]; ring + linarith + exact pow_le_pow_left₀ hδ_pos.le hdiff 4 + · simp only [hov, ↓reduceIte, mul_zero] + positivity + have hexp_b (b : Fin (2 ^ M)) : + 𝔼 i, (if h i u = b then (1 : ℝ) else 0) * + (∑ x ∈ S, bucketCentered (scaleWindow M b j) (h i x)) ^ 4 ≤ + (𝔼 y : Fin (2 ^ M), if y = b then (1 : ℝ) else 0) * (18 * δ ^ 2) := by + have h4 := expect_indicator_mul_sum_pow_four_le hh S hu b (scaleWindow M b j) + refine h4.trans (mul_le_mul_of_nonneg_left ?_ + (expect_nonneg fun y _ => by split_ifs <;> positivity)) + set μ := (#S : ℝ) * ((#(scaleWindow M b j) : ℝ) / (Fintype.card (Fin (2 ^ M)) : ℝ)) + have hμ_le : μ ≤ δ := hmean_le b + have hμ_nonneg : 0 ≤ μ := by dsimp only [μ]; positivity + have hμ_sq : μ ^ 2 ≤ δ ^ 2 := pow_le_pow_left₀ hμ_nonneg hμ_le 2 + have hδ_le_2sq : δ ≤ 2 * δ ^ 2 := by nlinarith + linarith + have hsum_exp_b : + (∑ b : Fin (2 ^ M), (𝔼 y : Fin (2 ^ M), if y = b then (1 : ℝ) else 0) * (18 * δ ^ 2)) = + 18 * δ ^ 2 := by + rw [← sum_mul] + have hsum1 : (∑ b : Fin (2 ^ M), 𝔼 y : Fin (2 ^ M), if y = b then (1 : ℝ) else 0) = 1 := by + rw [← expect_sum_comm] + simp only [Fintype.sum_ite_eq, Fintype.expect_const] + rw [hsum1, one_mul] + have hmarkov : + δ ^ 4 * (𝔼 i, if scaleOverload M h S u j i then (1 : ℝ) else 0) ≤ 18 * δ ^ 2 := by + calc δ ^ 4 * (𝔼 i, if scaleOverload M h S u j i then (1 : ℝ) else 0) + = 𝔼 i, δ ^ 4 * (if scaleOverload M h S u j i then (1 : ℝ) else 0) := by + rw [Fintype.expect_eq_sum_div_card, Fintype.expect_eq_sum_div_card, ← mul_sum, + mul_div_assoc] + _ ≤ 𝔼 i, ∑ b : Fin (2 ^ M), + (if h i u = b then (1 : ℝ) else 0) * + (∑ x ∈ S, bucketCentered (scaleWindow M b j) (h i x)) ^ 4 := + expect_le_expect fun i _ => hpt_le i + _ = ∑ b : Fin (2 ^ M), + 𝔼 i, (if h i u = b then (1 : ℝ) else 0) * + (∑ x ∈ S, bucketCentered (scaleWindow M b j) (h i x)) ^ 4 := + expect_sum_comm _ _ _ + _ ≤ ∑ b : Fin (2 ^ M), + (𝔼 y : Fin (2 ^ M), if y = b then (1 : ℝ) else 0) * (18 * δ ^ 2) := + sum_le_sum fun b _ => hexp_b b + _ = 18 * δ ^ 2 := hsum_exp_b + have hδ4_pos : 0 < δ ^ 4 := pow_pos hδ_pos 4 + rw [← le_div_iff₀' hδ4_pos] at hmarkov + refine hmarkov.trans_eq ?_ + rw [h2j_eq] + field_simp [hδ_pos.ne'] + ring + +/-- Left-anchored cluster witness predicate for hash family `h` at `(S, u, i, t)`: if `1 ≤ t`, +there is a cyclic interval `cyclicInterval M (h i u) a ℓ` of length `ℓ ∈ [t, #S]` with `a < ℓ` +into which at least `ℓ` keys of `S` hash. -/ +@[expose] def HasClusterWitness {ι X : Type*} {M : ℕ} (h : ι → X → Fin (2 ^ M)) + (S : Finset X) (u : X) (i : ι) (t : ℕ) : Prop := + 1 ≤ t → + ∃ a ℓ, a < ℓ ∧ t ≤ ℓ ∧ ℓ ≤ #S ∧ + ℓ ≤ #(S.filter fun x => h i x ∈ cyclicInterval M (h i u) a ℓ) + +private theorem probe_le_sum_scaleOverload {ι X : Type*} + {M n_max : ℕ} (h : ι → X → Fin (2 ^ M)) (S : Finset X) (u : X) + (hS : #S ≤ n_max) (hM : 8 * n_max ≤ 2 ^ M) (i : ι) (t : ℕ) + (hwit : HasClusterWitness h S u i t) : + (t : ℝ) ≤ + ∑ j ∈ Finset.Icc 1 M, + (2 ^ j : ℝ) * (if scaleOverload M h S u j i then (1 : ℝ) else 0) := by + by_cases ht : 1 ≤ t + · obtain ⟨a, ℓ, ha_lt_ℓ, ht_le_ℓ, hℓ_le_S, hℓ_card⟩ := hwit ht + have hℓ_pos : 0 < ℓ := by omega + set j := Nat.log 2 ℓ + 1 + have hj_ge_one : 1 ≤ j := Nat.le_add_left 1 _ + have h2j1_le : 2 ^ (j - 1) ≤ ℓ := by + dsimp only [j] + exact Nat.pow_log_le_self 2 (by omega) + have hℓ_lt_2j : ℓ < 2 ^ j := by + dsimp only [j] + exact Nat.lt_pow_succ_log_self (by decide) ℓ + have hℓ_lt_2M : ℓ < 2 ^ M := by + have : 1 ≤ 2 ^ M := Nat.one_le_two_pow + omega + have hj_le_M : j ≤ M := by + have hpow_lt : 2 ^ (j - 1) < 2 ^ M := lt_of_le_of_lt h2j1_le hℓ_lt_2M + have hj1_lt : j - 1 < M := (Nat.pow_lt_pow_iff_right (by decide)).mp hpow_lt + omega + have hj_mem : j ∈ Finset.Icc 1 M := Finset.mem_Icc.mpr ⟨hj_ge_one, hj_le_M⟩ + have hov : scaleOverload M h S u j i := by + dsimp only [scaleOverload] + have hsub : cyclicInterval M (h i u) a ℓ ⊆ scaleWindow M (h i u) j := + cyclicInterval_subset_scaleWindow (h i u) (by omega) (by omega) + have hfilter : + #(S.filter fun x => h i x ∈ cyclicInterval M (h i u) a ℓ) ≤ + #(S.filter fun x => h i x ∈ scaleWindow M (h i u) j) := by + refine card_le_card fun x hx => ?_ + simp only [mem_filter] at hx ⊢ + exact ⟨hx.1, hsub hx.2⟩ + omega + calc (t : ℝ) ≤ (ℓ : ℝ) := by exact_mod_cast ht_le_ℓ + _ ≤ (2 ^ j : ℝ) := by exact_mod_cast hℓ_lt_2j.le + _ = (2 ^ j : ℝ) * (if scaleOverload M h S u j i then (1 : ℝ) else 0) := by + simp [hov] + _ ≤ ∑ k ∈ Finset.Icc 1 M, + (2 ^ k : ℝ) * (if scaleOverload M h S u k i then (1 : ℝ) else 0) := by + refine single_le_sum + (f := fun k => (2 ^ k : ℝ) * (if scaleOverload M h S u k i then (1 : ℝ) else 0)) + (fun k _ => ?_) hj_mem + split_ifs <;> positivity + · have ht0 : t = 0 := by omega + rw [ht0, Nat.cast_zero] + refine sum_nonneg fun k _ => ?_ + split_ifs <;> positivity + +private theorem sum_Icc_two_pow_mul_div_sq_eq (M : ℕ) : + (∑ j ∈ Finset.Icc 1 M, (2 ^ j : ℝ) * (288 / (2 ^ j : ℝ) ^ 2)) = + 288 - 288 / (2 ^ M : ℝ) := by + induction M with + | zero => + simp + | succ M ih => + rw [Finset.sum_Icc_succ_top (Nat.le_add_left 1 M), ih, pow_succ'] + have hpowM : (2 ^ M : ℝ) ≠ 0 := by positivity + field_simp [hpowM] + ring + +/-- If a nonnegative probe count `probe : ι → ℕ` satisfies the **left-anchored cluster witness** +property — every seed `i` with `1 ≤ probe i` has an occupied cyclic interval +`cyclicInterval M (h i u) a ℓ` of length `ℓ ∈ [probe i, #S]` with `0 ≤ a < ℓ` into which at +least `ℓ` keys of `S` hash — then under a `5`-wise independent hash family at load factor +`8 * n_max ≤ 2 ^ M` and `#S ≤ n_max`, the expected probe count is at most `288`. -/ +theorem expect_probe_le_of_cluster_witness {ι X : Type*} [Fintype ι] + {M n_max : ℕ} {h : ι → X → Fin (2 ^ M)} + (hh : IsKWiseIndependentHash 5 h) + (S : Finset X) {u : X} (hu : u ∉ S) + (hS : #S ≤ n_max) (hM : 8 * n_max ≤ 2 ^ M) + (probe : ι → ℕ) + (hwit : ∀ i, HasClusterWitness h S u i (probe i)) : + 𝔼 i, (probe i : ℝ) ≤ 288 := by + have hpt (i : ι) : + (probe i : ℝ) ≤ + ∑ j ∈ Finset.Icc 1 M, + (2 ^ j : ℝ) * (if scaleOverload M h S u j i then (1 : ℝ) else 0) := + probe_le_sum_scaleOverload h S u hS hM i (probe i) (hwit i) + calc 𝔼 i, (probe i : ℝ) + ≤ 𝔼 i, ∑ j ∈ Finset.Icc 1 M, + (2 ^ j : ℝ) * (if scaleOverload M h S u j i then (1 : ℝ) else 0) := + expect_le_expect fun i _ => hpt i + _ = ∑ j ∈ Finset.Icc 1 M, + 𝔼 i, (2 ^ j : ℝ) * (if scaleOverload M h S u j i then (1 : ℝ) else 0) := + expect_sum_comm _ _ _ + _ ≤ ∑ j ∈ Finset.Icc 1 M, (2 ^ j : ℝ) * (288 / (2 ^ j : ℝ) ^ 2) := by + refine sum_le_sum fun j hj => ?_ + rw [Finset.mem_Icc] at hj + rw [Fintype.expect_eq_sum_div_card, ← mul_sum, mul_div_assoc, + ← Fintype.expect_eq_sum_div_card] + exact mul_le_mul_of_nonneg_left + (expect_scaleOverload_le hh S hu hS hM hj.1) (by positivity) + _ = 288 - 288 / (2 ^ M : ℝ) := sum_Icc_two_pow_mul_div_sq_eq M + _ ≤ 288 := by + have : (0 : ℝ) ≤ 288 / (2 ^ M : ℝ) := by positivity + linarith + +/-- Natural-number sum form of `expect_probe_le_of_cluster_witness`: the sum of `probe i` over all +seeds `i : ι` is at most `288 * Fintype.card ι`. -/ +theorem sum_probe_le_of_cluster_witness {ι X : Type*} [Fintype ι] + {M n_max : ℕ} {h : ι → X → Fin (2 ^ M)} + (hh : IsKWiseIndependentHash 5 h) + (S : Finset X) {u : X} (hu : u ∉ S) + (hS : #S ≤ n_max) (hM : 8 * n_max ≤ 2 ^ M) + (probe : ι → ℕ) + (hwit : ∀ i, HasClusterWitness h S u i (probe i)) : + (∑ i : ι, probe i) ≤ 288 * Fintype.card ι := by + have := hh.nonempty + have hexp := expect_probe_le_of_cluster_witness hh S hu hS hM probe hwit + rw [Fintype.expect_eq_sum_div_card] at hexp + have hcard_pos : (0 : ℝ) < (Fintype.card ι : ℝ) := Nat.cast_pos.mpr Fintype.card_pos + rw [div_le_iff₀ hcard_pos] at hexp + exact_mod_cast hexp + +/-- `ℝ≥0∞` expectation form of `expect_probe_le_of_cluster_witness` under `PMF.uniformOfFintype`: +`∑' i, (PMF.uniformOfFintype ι) i * (probe i : ℝ≥0∞) ≤ 288`. -/ +theorem tsum_uniform_probe_le_of_cluster_witness {ι X : Type*} [Fintype ι] [Nonempty ι] + {M n_max : ℕ} {h : ι → X → Fin (2 ^ M)} + (hh : IsKWiseIndependentHash 5 h) + (S : Finset X) {u : X} (hu : u ∉ S) + (hS : #S ≤ n_max) (hM : 8 * n_max ≤ 2 ^ M) + (probe : ι → ℕ) + (hwit : ∀ i, HasClusterWitness h S u i (probe i)) : + (∑' i : ι, (PMF.uniformOfFintype ι) i * (probe i : ℝ≥0∞)) ≤ 288 := by + have hsum := sum_probe_le_of_cluster_witness hh S hu hS hM probe hwit + calc (∑' i : ι, (PMF.uniformOfFintype ι) i * (probe i : ℝ≥0∞)) + = ∑ i : ι, (Fintype.card ι : ℝ≥0∞)⁻¹ * (probe i : ℝ≥0∞) := by + rw [tsum_fintype] + simp [PMF.uniformOfFintype_apply] + _ = (Fintype.card ι : ℝ≥0∞)⁻¹ * ((∑ i : ι, probe i : ℕ) : ℝ≥0∞) := by + rw [← Finset.mul_sum, Nat.cast_sum] + _ ≤ (Fintype.card ι : ℝ≥0∞)⁻¹ * ((288 * Fintype.card ι : ℕ) : ℝ≥0∞) := by + gcongr + _ = 288 := by + push_cast + rw [mul_comm (288 : ℝ≥0∞), ← mul_assoc, ENNReal.inv_mul_cancel, one_mul] + · exact Nat.cast_ne_zero.mpr Fintype.card_ne_zero + · exact ENNReal.natCast_ne_top _ + +end Cslib.Algorithms.WordRAM.Dictionary.LinearProbing diff --git a/ComplexityTests/WordRAMLinearProbing.lean b/ComplexityTests/WordRAMLinearProbing.lean new file mode 100644 index 000000000..1e0e2949d --- /dev/null +++ b/ComplexityTests/WordRAMLinearProbing.lean @@ -0,0 +1,335 @@ +/- +Copyright (c) 2026 Renfei Zhou. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: Renfei Zhou +-/ + +module +-- upstream: candidate + +public import Complexity.Init +public import Complexity.Algorithms.WordRAM.Dictionary.LinearProbing + +/-! +# Linear Probing Word-RAM Dictionary Tests + +Exercises the open-addressed linear-probing dictionary implementation in +`Complexity.Algorithms.WordRAM.Dictionary.LinearProbing` and its 5-wise independent dyadic +cluster analysis in `Complexity.Algorithms.WordRAM.Dictionary.LinearProbing.FiveWise`: +* concrete space, table-size (`2 ^ tableLog n_max`), setup-time, and per-operation charge bounds; +* multi-key linear-probe cluster collisions when distinct keys share the same home bucket, forcing + sequential probing across contiguous occupied runs; +* in-place key updates at full capacity `(Dictionary.dom s).card = n_max`; +* absent-key misses (`get u` halting at the first empty slot `used[pos] = 0` and decoding to + `none`); +* deterministic step-bounded Word-RAM execution `opCode_execWithin` across inserts, cluster + collisions, in-place updates at capacity, hits, and misses; +* instantiation of `subroutineDS`, `subroutineDSStandard`, and `seq_expected_amortized` on a + concrete 6-operation trace. +-/ + +@[expose] public section + +namespace ComplexityTests.WordRAMLinearProbing + +open Cslib Cslib.WordRAM Cslib.WordRAM.Randomized Cslib.Probability +open Cslib.Algorithms.Lean +open Cslib.Algorithms.WordRAM.Hashing +open Cslib.Algorithms.WordRAM.Dictionary.LinearProbing +open scoped ENNReal + +/-! ## 1. Concrete parameter and table-sizing sanity checks -/ + +example : tableLog 0 = 3 := by decide +example : tableSize 0 = 8 := by decide +example : tableLog 3 = 5 := by decide +example : tableSize 3 = 32 := by decide +example : space 3 ≤ 49 * max 3 4 + 13 := space_le 3 + +/-- Concrete machine parameters `U = 16`, `V = 100`, `n_max = 3`, `w = 64`, `base = 32` satisfy +the word-width and address-fit hypotheses of `LinearProbing.subroutineDS`. -/ +theorem hw_example : 2 * trinomialDeg 16 (tableLog 3) ≤ 64 := by + have hle := trinomialDeg_le_three_mul 16 (tableLog 3) + have hval : 3 * max (Nat.clog 2 (max 16 2)) (max (tableLog 3) 1) = 15 := by decide + omega + +theorem hfit_space_example : Fits 64 (32 + space 3) := by + have hsp := space_le 3 + have h241 : Fits 64 241 := by dsimp only [Fits]; decide + exact h241.of_le (by omega) + +theorem hfit_V_example : Fits 64 100 := by + dsimp only [Fits] + decide + +/-! ## 2. Multi-key cluster collisions, in-place updates at capacity `n_max`, and misses -/ + +abbrev k1 : Fin 16 := ⟨1, by decide⟩ +abbrev k2 : Fin 16 := ⟨2, by decide⟩ +abbrev k3 : Fin 16 := ⟨5, by decide⟩ +abbrev k4 : Fin 16 := ⟨9, by decide⟩ + +/-- Under the all-zero 5-wise seed `θZero`, every polynomial coefficient is `0`, so all keys +hash to the same home bucket and form a single contiguous linear-probing cluster. -/ +def θZero : Seed 16 3 := + fun _ => ⟨0, Nat.two_pow_pos _⟩ + +/-- Concrete 3-key insertion sequence reaching capacity `n_max = 3`, followed by an in-place +update of `k3` at full capacity, a present-key lookup on `k3`, and an absent-key lookup on `k2`. -/ +def sampleOps : List (Dictionary.Op 16) := + [ .insert k1 10 + , .insert k3 20 + , .insert k4 30 + , .insert k3 25 + , .get k3 + , .get k2 ] + +/-- Abstract states along `sampleOps`: +- `s1`: `{k1 ↦ 10}` (size `1 < 3`); +- `s2`: `{k1 ↦ 10, k3 ↦ 20}` (size `2 < 3`); +- `s3`: `{k1 ↦ 10, k3 ↦ 20, k4 ↦ 30}` (at full capacity `3 = n_max`); +- `s4`: `{k1 ↦ 10, k3 ↦ 25, k4 ↦ 30}` (in-place update of `k3` at full capacity `3 = n_max`). -/ +def s0 : Dictionary.State 16 := Dictionary.empty 16 +def s1 : Dictionary.State 16 := Function.update s0 k1 (some 10) +def s2 : Dictionary.State 16 := Function.update s1 k3 (some 20) +def s3 : Dictionary.State 16 := Function.update s2 k4 (some 30) +def s4 : Dictionary.State 16 := Function.update s3 k3 (some 25) + +theorem valid_insert_k1 : (Dictionary.spec 16 100 3).valid s0 (.insert k1 10) := by + refine ⟨Or.inl ?_, by decide⟩ + simp [s0, Dictionary.dom_empty] + +theorem valid_insert_k3 : (Dictionary.spec 16 100 3).valid s1 (.insert k3 20) := by + refine ⟨Or.inl ?_, by decide⟩ + have hdom : Dictionary.dom s1 = {k1} := by + simp [s1, s0, Dictionary.dom_insert, Dictionary.dom_empty] + simp [hdom] + +theorem valid_insert_k4 : (Dictionary.spec 16 100 3).valid s2 (.insert k4 30) := by + refine ⟨Or.inl ?_, by decide⟩ + have hdom : Dictionary.dom s2 = insert k3 {k1} := by + simp [s2, s1, s0, Dictionary.dom_insert, Dictionary.dom_empty] + rw [hdom] + decide + +/-- At full capacity `(Dictionary.dom s3).card = 3 = n_max`, updating the existing key `k3` +in-place satisfies `(Dictionary.spec 16 100 3).valid` via `(s3 k3).isSome = true`. -/ +theorem card_dom_s3_eq_capacity : (Dictionary.dom s3).card = 3 := by + have hdom : Dictionary.dom s3 = insert k4 (insert k3 {k1}) := by + simp [s3, s2, s1, s0, Dictionary.dom_insert, Dictionary.dom_empty] + rw [hdom] + decide + +theorem valid_update_k3_at_capacity : (Dictionary.spec 16 100 3).valid s3 (.insert k3 25) := by + refine ⟨Or.inr ?_, by decide⟩ + have hk3 : s3 k3 = some 20 := by + dsimp only [s3, s2] + rw [Function.update_of_ne (by decide), Function.update_self] + simp [hk3] + +/-- After the in-place update at capacity `n_max = 3`, `s4 k3 = some 25` (hit) and `s4 k2 = none` +(absent-key miss). -/ +theorem s4_lookup_hit_and_miss : s4 k3 = some 25 ∧ s4 k2 = none := by + refine ⟨?_, ?_⟩ + · dsimp only [s4] + rw [Function.update_self] + · dsimp only [s4, s3, s2, s1, s0] + rw [Function.update_of_ne (by decide), Function.update_of_ne (by decide), + Function.update_of_ne (by decide), Function.update_of_ne (by decide)] + rfl + +/-- The full 6-operation trace `sampleOps` (3 inserts to capacity, 1 in-place update at capacity, +1 hit lookup, 1 miss lookup) is a valid history of `Dictionary.spec 16 100 3`. -/ +theorem sampleOps_validHistory : + (Dictionary.spec 16 100 3).ValidHistory (Dictionary.empty 16) sampleOps := by + refine ⟨valid_insert_k1, valid_insert_k3, valid_insert_k4, + valid_update_k3_at_capacity, trivial, trivial, trivial⟩ + +/-! ## 3. Table invariant preservation across colliding clusters and updates -/ + +/-- Starting from zeroed memory and any seed `θ`, inserting `k1`, `k3`, `k4` and then updating `k3` +in-place at full capacity preserves `TableInv` and produces a cluster witness for any probe length +`t ≥ 1`. -/ +theorem tableInv_cluster_and_update (θ : Seed 16 3) : + ∃ (mem4 : ℕ → ℕ) (hinv4 : TableInv 16 100 3 32 θ s4 mem4), + HasClusterWitness (hashFn 16 3) ((Dictionary.dom s4).erase k2) k2 θ + (hinv4.probeLen k2 (hashFn 16 3 θ k2)) := by + have hinv0 : TableInv 16 100 3 32 θ s0 (fun _ => 0) := + tableInv_empty 16 100 3 32 θ (fun _ => rfl) + let p1 := probeSlot (tableLog 3) (hashFn 16 3 θ k1) (hinv0.probeLen k1 (hashFn 16 3 θ k1)) + let mem1 := insertMem 32 3 (fun _ => 0) p1 k1.val 10 + have hinv1 : TableInv 16 100 3 32 θ s1 mem1 := hinv0.insert_mem k1 10 valid_insert_k1 + let p2 := probeSlot (tableLog 3) (hashFn 16 3 θ k3) (hinv1.probeLen k3 (hashFn 16 3 θ k3)) + let mem2 := insertMem 32 3 mem1 p2 k3.val 20 + have hinv2 : TableInv 16 100 3 32 θ s2 mem2 := hinv1.insert_mem k3 20 valid_insert_k3 + let p3 := probeSlot (tableLog 3) (hashFn 16 3 θ k4) (hinv2.probeLen k4 (hashFn 16 3 θ k4)) + let mem3 := insertMem 32 3 mem2 p3 k4.val 30 + have hinv3 : TableInv 16 100 3 32 θ s3 mem3 := hinv2.insert_mem k4 30 valid_insert_k4 + let p4 := probeSlot (tableLog 3) (hashFn 16 3 θ k3) (hinv3.probeLen k3 (hashFn 16 3 θ k3)) + let mem4 := insertMem 32 3 mem3 p4 k3.val 25 + have hinv4 : TableInv 16 100 3 32 θ s4 mem4 := + hinv3.insert_mem k3 25 valid_update_k3_at_capacity + exact ⟨mem4, hinv4, fun ht => hinv4.hasClusterWitness_probeLen k2 ht⟩ + +/-- Concrete collision check under `θZero`: when `k1` and `k3` hash to the same bucket `b`, +inserting `k1` occupies slot `b`, so `k3` must probe past `b` (`probeLen = 1`) and land in the +next slot `probeSlot (tableLog 3) b 1`. -/ +theorem collision_probeLen_pos : + let hinv0 : TableInv 16 100 3 32 θZero s0 (fun _ => 0) := + tableInv_empty 16 100 3 32 θZero (fun _ => rfl) + let b1 := hashFn 16 3 θZero k1 + let p1 := probeSlot (tableLog 3) b1 (hinv0.probeLen k1 b1) + let mem1 := insertMem 32 3 (fun _ => 0) p1 k1.val 10 + let hinv1 : TableInv 16 100 3 32 θZero s1 mem1 := hinv0.insert_mem k1 10 valid_insert_k1 + hinv1.probeLen k3 b1 = 1 := by + intro hinv0 b1 p1 mem1 hinv1 + have ht0 : hinv0.probeLen k1 b1 = 0 := by + by_contra hne + have hpos : 0 < hinv0.probeLen k1 b1 := Nat.pos_of_ne_zero hne + have hcont := hinv0.probeContinue_of_lt_probeLen k1 b1 hpos + dsimp only [probeContinue, usedAt] at hcont + exact absurd hcont.1 (by decide) + have hp1_eq : p1 = b1 := by + dsimp only [p1] + rw [ht0, probeSlot_zero] + have hcont0 : probeContinue 32 3 mem1 k3.val b1 0 := by + dsimp only [probeContinue] + rw [probeSlot_zero, ← hp1_eq] + refine ⟨?_, ?_⟩ + · dsimp only [mem1] + rw [usedAt_insertMem, ite_eq_left_iff.mpr (fun h => (h rfl).elim)] + · dsimp only [mem1] + rw [keyAt_insertMem, ite_eq_left_iff.mpr (fun h => (h rfl).elim)] + decide + have hnot1 : ¬probeContinue 32 3 mem1 k3.val b1 1 := by + intro ⟨hused1, _⟩ + dsimp only [mem1] at hused1 + rw [usedAt_insertMem] at hused1 + have hne : probeSlot (tableLog 3) b1 1 ≠ p1 := by + rw [hp1_eq] + intro heq + have heq' : probeSlot (tableLog 3) b1 1 = probeSlot (tableLog 3) b1 0 := by + simpa only [probeSlot_zero] using heq + have hinj := probeSlot_injOn_range b1 (len := 2) (by decide) + (x₁ := 1) (Finset.mem_coe.mpr (Finset.mem_range.mpr (by decide))) + (x₂ := 0) (Finset.mem_coe.mpr (Finset.mem_range.mpr (by decide))) heq' + omega + rw [ite_eq_right_iff.mpr (fun h => absurd h hne)] at hused1 + dsimp only [usedAt] at hused1 + exact absurd hused1 (by decide) + have hle1 : hinv1.probeLen k3 b1 ≤ 1 := by + by_contra! hgt + exact hnot1 (hinv1.probeContinue_of_lt_probeLen k3 b1 hgt) + have hge1 : 1 ≤ hinv1.probeLen k3 b1 := by + by_contra! hlt + have heq0 : hinv1.probeLen k3 b1 = 0 := by omega + have hnot0 := hinv1.not_probeContinue_probeLen k3 b1 + rw [heq0] at hnot0 + exact hnot0 hcont0 + omega + +/-! ## 4. Word-RAM `opCode_execWithin`, `subroutineDS`, `subroutineDSStandard`, and +`seq_expected_amortized` -/ + +/-- Concrete `SubroutineDS` instance for `(U = 16, V = 100, n_max = 3, w = 64, base = 32)` on +`Basis.exact`. -/ +noncomputable def sampleDS : + Cslib.WordRAM.Randomized.SubroutineDS (subroutineCode 16 3) (Dictionary.spec 16 100 3) 64 32 + Fin.elim0 (Dictionary.empty 16) := + subroutineDS 16 100 3 64 32 hw_example hfit_space_example hfit_V_example + +/-- Concrete `SubroutineDS` instance on `Basis.standard`. -/ +noncomputable def sampleDSStandard : + Cslib.WordRAM.Randomized.SubroutineDS (subroutineCode 16 3).toStandard + (Dictionary.spec 16 100 3) 64 32 Fin.elim0 (Dictionary.empty 16) := + subroutineDSStandard 16 100 3 64 32 hw_example hfit_space_example hfit_V_example + +/-- End-to-end expected-time execution of `initFresh` followed by the 6-operation trace +`sampleOps` (3 inserts to capacity `n_max = 3`, 1 in-place update at capacity, 1 hit lookup, and +1 miss lookup) on a zeroed Word-RAM machine state. -/ +theorem sampleOps_seq_expected_amortized : + let x₀ : Data rLP := + { regs := fun i => if i = 18 then 32 else 0 + mem := fun _ => 0 } + ∃ (ν₀ _ν_f : PMF (Seed 16 3 × Data rLP)) (T_ops : ℝ≥0∞), + ExecCodeDist 64 (subroutineCode 16 3).initFresh x₀ + (ν₀.map Prod.snd) (3 + setupTime 16 3 : ℕ) ∧ + sampleDS.ExecSteps (Dictionary.empty 16) sampleOps ν₀ _ν_f T_ops ∧ + ((3 + setupTime 16 3 : ℕ) : ℝ≥0∞) + T_ops ≤ + (3 + setupTime 16 3 + sampleOps.length * charge 16 3 : ℕ) := by + intro x₀ + have hfit_32 : Fits 64 32 := by dsimp only [Fits]; decide + have hx₀ : x₀.Bounded 64 := by + refine ⟨fun i => ?_, fun _ => fits_zero 64⟩ + dsimp only [x₀] + split_ifs + · exact hfit_32 + · exact fits_zero 64 + have hfr : x₀.regs 18 = 32 := by simp [x₀] + have hz : ∀ a ∈ (Block.mk 32 (space 3)).footprint, x₀.mem a = 0 := fun _ _ => rfl + exact seq_expected_amortized 16 100 3 64 32 + hw_example hfit_space_example hfit_V_example + hx₀ hfr hz sampleOps sampleOps_validHistory + +/-- Direct deterministic execution test of `opCode_execWithin` for an in-place update at full +capacity `n_max = 3`, followed by a hit lookup (`get k3 = some 25`) and an absent-key miss lookup +(`get k2 = none`), each completing within `KWise.evalSteps 3 16 (tableLog 3) 5 + 12 * 3 + 27` +deterministic instructions on every seed. -/ +theorem opCode_update_hit_miss_from_rep (θ : Seed 16 3) (x : Data rLP) + (hrep : Rep 16 100 3 64 32 θ s3 x) : + let x_upd := sampleDS.toDynamicProgram.prepare (.insert k3 25) x + let B := KWise.evalSteps 3 16 (tableLog 3) 5 + 12 * 3 + 27 + ∃ y_upd y_hit y_miss : Data rLP, + ExecWithin 64 (opCode 3) x_upd y_upd B ∧ + Rep 16 100 3 64 32 θ s4 y_upd ∧ + ExecWithin 64 (opCode 3) (sampleDS.toDynamicProgram.prepare (.get k3) y_upd) y_hit B ∧ + decodeOp (V := 100) (n_max := 3) (.get k3) y_hit.regs = some (some 25) ∧ + ExecWithin 64 (opCode 3) (sampleDS.toDynamicProgram.prepare (.get k2) y_hit) y_miss B ∧ + decodeOp (V := 100) (n_max := 3) (.get k2) y_miss.regs = some none := by + intro x_upd B + have hrep_upd_in : Rep 16 100 3 64 32 θ s3 x_upd := + sampleDS.prepare_rep hrep valid_update_k3_at_capacity + have harg_upd : argMatchOp (.insert k3 25) x_upd.regs := + sampleDS.prepare_argMatch (.insert k3 25) x + obtain ⟨y_upd, ⟨n1, hn1, hex_upd⟩, _, hrep_upd, _⟩ := + opCode_execWithin hw_example hfit_space_example hfit_V_example + hrep_upd_in (.insert k3 25) valid_update_k3_at_capacity harg_upd + change Rep 16 100 3 64 32 θ s4 y_upd at hrep_upd + have hle1 : hrep_upd_in.table_inv.probeLen (opKey (.insert k3 25)) + (hashFn 16 3 θ (opKey (.insert k3 25))) ≤ 3 := + hrep_upd_in.table_inv.probeLen_le_n_max _ _ + let x_hit := sampleDS.toDynamicProgram.prepare (.get k3) y_upd + have hrep_hit_in : Rep 16 100 3 64 32 θ s4 x_hit := + sampleDS.prepare_rep hrep_upd trivial + have harg_hit : argMatchOp (.get k3) x_hit.regs := + sampleDS.prepare_argMatch (.get k3) y_upd + obtain ⟨y_hit, ⟨n2, hn2, hex_hit⟩, hdec_hit, hrep_hit, _⟩ := + opCode_execWithin hw_example hfit_space_example hfit_V_example + hrep_hit_in (.get k3) trivial harg_hit + change Rep 16 100 3 64 32 θ s4 y_hit at hrep_hit + have hle2 : hrep_hit_in.table_inv.probeLen (opKey (.get k3)) + (hashFn 16 3 θ (opKey (.get k3))) ≤ 3 := + hrep_hit_in.table_inv.probeLen_le_n_max _ _ + let x_miss := sampleDS.toDynamicProgram.prepare (.get k2) y_hit + have hrep_miss_in : Rep 16 100 3 64 32 θ s4 x_miss := + sampleDS.prepare_rep hrep_hit trivial + have harg_miss : argMatchOp (.get k2) x_miss.regs := + sampleDS.prepare_argMatch (.get k2) y_hit + obtain ⟨y_miss, ⟨n3, hn3, hex_miss⟩, hdec_miss, _, _⟩ := + opCode_execWithin hw_example hfit_space_example hfit_V_example + hrep_miss_in (.get k2) trivial harg_miss + have hle3 : hrep_miss_in.table_inv.probeLen (opKey (.get k2)) + (hashFn 16 3 θ (opKey (.get k2))) ≤ 3 := + hrep_miss_in.table_inv.probeLen_le_n_max _ _ + have hhit_val : ((Dictionary.spec 16 100 3).step s4 (.get k3)).2 = some 25 := + s4_lookup_hit_and_miss.1 + have hmiss_val : ((Dictionary.spec 16 100 3).step s4 (.get k2)).2 = none := + s4_lookup_hit_and_miss.2 + rw [hhit_val] at hdec_hit + rw [hmiss_val] at hdec_miss + refine ⟨y_upd, y_hit, y_miss, + ⟨n1, by dsimp only [B]; omega, hex_upd⟩, hrep_upd, + ⟨n2, by dsimp only [B]; omega, hex_hit⟩, hdec_hit, + ⟨n3, by dsimp only [B]; omega, hex_miss⟩, hdec_miss⟩ + +end ComplexityTests.WordRAMLinearProbing diff --git a/references.bib b/references.bib index ffd09ecfe..b203ecebc 100644 --- a/references.bib +++ b/references.bib @@ -3489,6 +3489,17 @@ @article{Oum2005 doi = {10.1016/j.jctb.2005.03.003} } +@article{PaghPaghRuzic2009, + author = {Pagh, Anna and Pagh, Rasmus and Ru{\v{z}}i{\'c}, Milan}, + title = {Linear Probing with Constant Independence}, + journal = {SIAM Journal on Computing}, + volume = {39}, + number = {3}, + pages = {1107--1120}, + year = {2009}, + doi = {10.1137/070702278} +} + @article{Papadimitriou1994PPAD, author = {Papadimitriou, Christos H.}, title = {On the Complexity of the Parity Argument and Other Inefficient Proofs of Existence}, From ae034db16c733f2b8d6ce082f2ca9f81fb7097a9 Mon Sep 17 00:00:00 2001 From: Renfei Zhou Date: Fri, 30 Oct 2026 02:53:10 +0000 Subject: [PATCH 18/18] docs(LinearProbing): add Pagh-Pagh-Ruzic 2009 source statement, locators, and model mapping --- .../WordRAM/Dictionary/LinearProbing.lean | 96 +++++++++++++------ .../Dictionary/LinearProbing/FiveWise.lean | 63 ++++++++---- 2 files changed, 112 insertions(+), 47 deletions(-) diff --git a/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean index d92dccd73..b3772fd76 100644 --- a/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean +++ b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing.lean @@ -18,8 +18,10 @@ public import Complexity.Computability.WordRAM.Randomized.Subroutine This file implements a self-contained open-addressed linear-probing hash table realizing `Cslib.Algorithms.Lean.Dictionary.spec U V n_max` on the randomized Word RAM -(`Cslib.WordRAM.Randomized.SubroutineDS`) [Pagh, Pagh and Ružić, *Linear Probing with Constant -Independence*][PaghPaghRuzic2009]; [Cormen, Leiserson, Rivest and Stein, *Introduction to +(`Cslib.WordRAM.Randomized.SubroutineDS`), adapting [Pagh, Pagh and Ružić, *Linear Probing with +Constant Independence*, *SIAM J. Comput.* 39(3):1107–1120, 2009, Section 2.3 (Lemma 2.1, +pp. 1110–1111) and Section 4 (Lemma 4.1, Theorem 4.3, and Theorem 4.5, +pp. 1114–1118)][PaghPaghRuzic2009]; [Cormen, Leiserson, Rivest and Stein, *Introduction to Algorithms*, Section 11.4][CLRS2009]. ## Design @@ -52,32 +54,68 @@ Algorithms*, Section 11.4][CLRS2009]. $S = \mathrm{dom}(s) \setminus \{u\}$, walking backwards from $b$ to the nearest preceding empty slot yields a contiguous cyclic interval `cyclicInterval M b a ℓ` of length $\ell = a + t \in [t, \#S]$ whose $\ell$ stored keys in $S$ all hash into - `cyclicInterval M b a ℓ` (since no probe path can cross the preceding empty slot). Applying + `cyclicInterval M b a ℓ` (`TableInv.hasClusterWitness_probeLen`, adapting the occupied-run + argument in the proof of Lemma 4.1, p. 1114 of `[PaghPaghRuzic2009]`). Applying `tsum_uniform_probe_le_of_cluster_witness` from - `Complexity.Algorithms.WordRAM.Dictionary.LinearProbing.FiveWise` bounds the expected probe count - by $288 = O(1)$. - -## Model, specialization, and cost scope - -- **Fixed-parameter program specialization (`initParams = Fin.elim0`):** - For a fixed key universe size `U`, value bound `V`, and capacity bound `n_max` on an admissible - external machine word width `w` (`2 * trinomialDeg U (tableLog n_max) ≤ w`, - `Fits w (base + space n_max)`, `Fits w V`), `subroutineCode U n_max` specializes configuration - constants (`tableLog n_max`, `tableSize n_max`, `KWise` trinomial-field parameters, bit masks, - and relative array offsets) as instruction literals rather than decoding them from runtime - initialization registers. -- **Oblivious-adversary charged-body cost model:** - Expected instruction bounds (`subroutineDS`, `subroutineDSStandard`, and - `seq_expected_amortized`) charge fresh initialization (`initFresh`) and subroutine operation - bodies (`SubroutineDS.ExecSteps`) across an operation sequence `ops` fixed independently of the - sampled hash seed (oblivious adversary). Caller argument delivery, caller control/dispatch loops, - and host-level code/configuration construction are outside the subroutine's charged instruction - count. -- **Insert-only dictionary specification and zero tombstones:** - `Dictionary.spec U V n_max` models an insert-only bounded dictionary (`get` and `insert` with at - most `n_max` distinct keys; deletion is deferred to a follow-up), which allows open-addressed - slots to use `{0, 1}` occupancy flags (`0` = empty, `1` = occupied) with zero tombstones; - supporting deletion would require tombstone states or backward-shift deletion. + `Complexity.Algorithms.WordRAM.Dictionary.LinearProbing.FiveWise` bounds the expected number of + skipped occupied slots `TableInv.probeLen` by $288 = O(1)$ (`tsum_probeLen_le`). + +## Source mapping, specialization, and cost scope + +In `[PaghPaghRuzic2009]`, Section 4 (Theorems 4.3 and 4.5, pp. 1115–1118, built on Lemma 2.1, +pp. 1110–1111, and Lemma 4.1, p. 1114) proves that for any sequence of insertions, deletions, and +lookups in a linear-probing hash table of size $r$ using a hash function chosen uniformly at random +from a $5$-wise independent family $\mathcal{H} : U \to [r]$ with maximum load $\bar{\alpha} < 1$, +the expected number of probes made during an operation is $O((1 - \bar{\alpha})^{-5/2})$ +(Theorem 4.3) and $O((1 - \bar{\alpha})^{-13/6})$ (Theorem 4.5), which is $O(1)$ for bounded load +factor. The declarations in this module and `LinearProbing.FiveWise` map to `[PaghPaghRuzic2009]` as +follows: +- **Hash family (`isKWiseIndependentHash_hashFn`):** + Uniform $5$-wise independent trinomial-field polynomial family `hashFn U n_max` + (`trinomialPolyHash` over $\mathbb{F}_{2^q}$ projected to $M$ bits, initialized and evaluated on + the Word RAM via `KWise.initCode 5` and `KWise.evalCode 5`), realizing exact uniform $5$-wise + independence (`IsKWiseIndependentHash 5`) with maximum load $\bar{\alpha} = \alpha = \#S / 2^M$. +- **Operation history and oblivious adversary (`ValidHistory`, `seq_expected_amortized`):** + An admissible operation sequence `ops : List (Dictionary.Op U)` fixed independently of the sampled + hash seed $\theta \sim \mathrm{seedLaw}\ U\ n_{\max}$, matching the fixed-set / oblivious + operation model of `[PaghPaghRuzic2009]`, Section 4. +- **Load-factor specialization ($\alpha \le 1/8$):** + While `[PaghPaghRuzic2009]` Theorems 4.3 and 4.5 apply to any maximum load $\bar{\alpha} < 1$, we + fix $M = \mathrm{tableLog}(n_{\max}) = \lceil \log_2 \max(8 n_{\max}, 8) \rceil$ so that + $\alpha = \#\mathrm{dom}(s) / 2^M \le n_{\max} / 2^M \le 1/8$ (`eight_mul_le_tableSize`) and + `space n_max = O(n_max)` (`space_le`). +- **Probe count (`TableInv.probeLen`) vs. executable Word-RAM body charge (`charge`):** + Whereas `[PaghPaghRuzic2009]` Section 4 (p. 1114) counts + $1 + \max\{l \mid h(x) + [l] \subseteq P\}$ table probes, `TableInv.probeLen` counts the number of + skipped occupied slots before the stopping slot ($\mathbb{E}[\mathrm{probeLen}] \le 288$ via + `tsum_probeLen_le`). The final stopping-slot inspection, loop/branch overhead ($12 t + 27$ + instructions for $t$ skipped slots), and embedded `KWise.evalCode 5` hash evaluation + (`KWise.evalSteps n_max U (tableLog n_max) 5`) are all included in the executable Word-RAM + operation charge + `charge U n_max = KWise.evalSteps n_max U (tableLog n_max) 5 + 12 * 288 + 27 = + KWise.evalSteps n_max U (tableLog n_max) 5 + 3483` (`opCode_execWithin`). +- **Original proof adaptation and explicit constants (`288` and `3483`):** + Rather than formalizing the interval-grid decompositions of Lemmas 4.2 and 4.4 in + `[PaghPaghRuzic2009]`, `LinearProbing.FiveWise` uses an original simplified proof adaptation at + load $\le 1/8$ that covers any length-$\ell$ cyclic interval containing $b$ ($2^{j-1} \le \ell < + 2^j$) by a single centered dyadic-scale window `scaleWindow M b j` of size at most $2^{j+1}$ + (`card_scaleWindow_le`), yielding the explicit scale-$j$ overload bound + $18 / (2^{j-2})^2 = 288 / 4^j$ (`expect_scaleOverload_le`), the probe-step bound $288$, and the + instruction constant $12 \cdot 288 + 27 = 3483$. +- **Fixed-parameter program specialization and charged-body scope (`initParams = Fin.elim0`):** + For fixed `(U, V, n_max)` on an admissible external machine word width `w` + (`2 * trinomialDeg U (tableLog n_max) ≤ w`, `Fits w (base + space n_max)`, `Fits w V`), + `subroutineCode U n_max` specializes configuration constants as instruction literals. Expected + instruction bounds (`subroutineDS`, `subroutineDSStandard`, and `seq_expected_amortized`) charge + fresh initialization (`initFresh`) and subroutine operation bodies (`SubroutineDS.ExecSteps`), + excluding caller argument delivery, caller control/dispatch loops, and host-level code + construction. +- **Insert-only dictionary specification vs. source deletions:** + Whereas `[PaghPaghRuzic2009]` Section 1 (p. 1107) and Section 4 (Theorems 4.3 and 4.5) include + deletions via greedy backward-shift rearrangement, `Dictionary.spec U V n_max` specifies an + insert-only bounded dictionary (`get` and `insert` with at most `n_max` distinct keys and no + deletion operation), so open-addressed slots use `{0, 1}` occupancy flags (`0` = empty, `1` = + occupied) with zero tombstones. ## References @@ -1071,7 +1109,7 @@ into `reg 2`. -/ | .insert _ _ => some () /-- Expected instruction count for `initSetup U n_max`: -`KWise.initTableSteps n_max U (tableLog n_max) + 38 = O(n_max + log log U)`. -/ +`KWise.initTableSteps n_max U (tableLog n_max) + 38 = O(n_max + log U)`. -/ @[expose] def setupTime (U n_max : ℕ) : ℕ := KWise.initTableSteps n_max U (tableLog n_max) + 38 @@ -2064,7 +2102,7 @@ theorem opCode_execWithin {U V w base n_max : ℕ} {θ : Seed U n_max} over `Cslib.WordRAM.Basis.exact`: - **Space**: `space n_max = KWise.space n_max 5 + 3 * 2 ^ (tableLog n_max) = O(n_max)` words; - **Initialization time**: `setupTime U n_max = KWise.initTableSteps n_max U (tableLog n_max) + 38 - = O(n_max + log log U)` expected instructions; + = O(n_max + log U)` expected instructions; - **Operation charge**: `charge U n_max = KWise.evalSteps n_max U (tableLog n_max) 5 + 3483 = O(L ^ 2 + 1)` expected instructions per `get` or `insert`, which is `O(1)` whenever `U ≤ n_max ^ c` for any constant `c`; diff --git a/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean index a01d86713..2f5d8058f 100644 --- a/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean +++ b/Complexity/Algorithms/WordRAM/Dictionary/LinearProbing/FiveWise.lean @@ -14,26 +14,53 @@ public import Complexity.Probability.PMF.Expect # Constant-independence cluster bounds for linear probing This file proves the probabilistic core of constant-time linear probing under $5$-wise independent -hashing [Pagh, Pagh and Ružić, *Linear Probing with Constant Independence*][PaghPaghRuzic2009]; -see also [Cormen, Leiserson, Rivest and Stein, *Introduction to Algorithms*, Section 11.4][CLRS2009] -and [Vadhan, *Pseudorandomness*, Section 3.5][Vadhan2012]. - -For a power-of-$2$ table of size $m = 2^M \ge 8 n_{\max}$ (load factor $\alpha = n / m \le 1/8$) and -a $5$-wise independent hash family $h : \iota \to X \to \mathrm{Fin}(2^M)$: -1. If probing key $u$ from $b = h_i(u)$ inspects $t \ge 1$ occupied slots belonging to +hashing, adapting [Pagh, Pagh and Ružić, *Linear Probing with Constant Independence*, *SIAM J. +Comput.* 39(3):1107–1120, 2009, Section 2.3 (Lemma 2.1, pp. 1110–1111) and Section 4 (Lemma 4.1, +Theorem 4.3, and Theorem 4.5, pp. 1114–1118)][PaghPaghRuzic2009]; see also [Cormen, Leiserson, +Rivest and Stein, *Introduction to Algorithms*, Section 11.4][CLRS2009] and [Vadhan, +*Pseudorandomness*, Section 3.5][Vadhan2012]. + +## Source statement and proof adaptation + +In `[PaghPaghRuzic2009]`, Lemma 2.1 (Section 2.3, pp. 1110–1111) shows that for a stored key set +$S \subseteq U$ of size $n$, a $5$-wise independent hash family $\mathcal{H} : U \to [r]$ with +maximum load $\bar{\alpha} < 1$, any subset $Q \subset [r]$ of size $q$, any $d > 0$, and any fixed +$x \in U \setminus S$, conditioning on $h(x) = \rho$ leaves a $4$-wise independent family on +$U \setminus \{x\}$ and the centered fourth-moment Markov bound gives +$\Pr[|\{y \in S : h(y) \in h(x) + Q\}| \ge \bar{\alpha} q + d] < 4 \bar{\alpha} q^2 / d^4$. +Combining Lemma 2.1 with the occupied-run witness in Lemma 4.1 (Section 4, p. 1114), Theorem 4.3 +(Section 4.1, pp. 1115–1116) and Theorem 4.5 (Section 4.2, p. 1118) bound the expected number of +probes per operation by $O((1 - \bar{\alpha})^{-5/2})$ and $O((1 - \bar{\alpha})^{-13/6})$ for any +maximum load $\bar{\alpha} < 1$. + +Our formalization specializes to a power-of-$2$ table of size $m = 2^M \ge 8 n_{\max}$ (exact +uniform load factor $\bar{\alpha} = \alpha = \#S / 2^M \le 1/8$) and an exact uniform $5$-wise +independent family $h : \iota \to X \to \mathrm{Fin}(2^M)$ (`IsKWiseIndependentHash 5 h`), using an +original simplified dyadic-window proof adaptation in place of the interval-grid decompositions of +Lemmas 4.2 and 4.4 in `[PaghPaghRuzic2009]`: +1. **Left-anchored cluster witness (`HasClusterWitness`, adapting the proof of Lemma 4.1, + p. 1114):** + If probing key $u$ from $b = h_i(u)$ steps over $t \ge 1$ occupied slots belonging to $S = \mathrm{dom}(s) \setminus \{u\}$, walking backwards from $b$ to the nearest preceding empty - slot yields a contiguous cyclic interval $J(b, a, \ell)$ of length $\ell = a + t \in [t, \#S]$ - containing $b$ into which at least $\ell$ keys of $S$ hash. -2. For the dyadic scale $j = \lfloor \log_2 \ell \rfloor + 1 \in \{1, \dots, M\}$, we have - $2^{j-1} \le \ell < 2^j$, and every such interval $J(b, a, \ell)$ is contained in a single - centered window $W_j(b)$ of size at most $2^{j+1}$. Consequently, at least $2^{j-1}$ keys of $S$ - hash into $W_j(b)$, whereas the expected load of $W_j(b)$ is - $\mu_{j,b} \le \#S \cdot 2^{j+1} / 2^M \le 2^{j-2}$. -3. Conditioning on $h_i(u) = b$ consumes $1$ degree of independence, leaving $4$-wise independence + slot yields a contiguous cyclic interval $J(b, a, \ell)$ (`cyclicInterval M b a ℓ`) of length + $\ell = a + t \in [t, \#S]$ containing $b$ into which at least $\ell$ keys of $S$ hash. +2. **Centered dyadic-scale window (`scaleWindow`, original proof adaptation at load $\le 1/8$):** + For the dyadic scale $j = \lfloor \log_2 \ell \rfloor + 1 \in \{1, \dots, M\}$, we have + $2^{j-1} \le \ell < 2^j$. Every cyclic interval $J(b, a, \ell)$ containing $b$ is contained in a + single centered window $W_j(b)$ (`scaleWindow M b j`) of size at most $2^{j+1}$ + (`cyclicInterval_subset_scaleWindow`, `card_scaleWindow_le`). Consequently, at least $2^{j-1}$ + keys of $S$ hash into $W_j(b)$, whereas at load $\#S \le 2^M / 8$ the expected load of $W_j(b)$ + is $\mu_{j,b} \le \#S \cdot 2^{j+1} / 2^M \le 2^{j-2}$, leaving deviation slack + $2^{j-1} - \mu_{j,b} \ge 2^{j-2}$. +3. **Conditioned fourth-moment tail bound (`expect_indicator_mul_sum_pow_four_le` and + `expect_scaleOverload_le`, adapting Lemma 2.1, pp. 1110–1111):** + Conditioning on $h_i(u) = b$ consumes $1$ degree of independence, leaving $4$-wise independence on $S$. Expanding the fourth central moment via `IsKWiseIndependentHash.expect_prod` on sets - $\mathrm{insert}\ u\ (\mathrm{image}\ t)$ of size at most $5$ bounds the overload probability at - scale $j$ by $18 / (2^{j-2})^2 = 288 / 4^j$, and summing $\sum_{j=1}^M 2^j \cdot (288 / 4^j)$ - yields an expected probe count of at most $288 = O(1)$. + $\mathrm{insert}\ u\ (\mathrm{image}\ t)$ of size at most $5$ bounds the scale-$j$ overload + probability (`scaleOverload`) by $18 / (2^{j-2})^2 = 288 / 4^j$, and summing + $\sum_{j=1}^M 2^j \cdot (288 / 4^j)$ yields the original explicit constant upper bound + $288 = O(1)$ on the expected number of skipped occupied slots + (`expect_probe_le_of_cluster_witness`, `tsum_uniform_probe_le_of_cluster_witness`). ## Main definitions