Repository navigation
feat(streaming): add Count-Min fixed-query accuracy - #1206
Conversation
Implement weighted insertion-only counter tables, exact merging, no-underestimation, and independent-row accuracy, with a ProbComp wrapper and concrete linear hashes. Add checked clients, source attribution, and a streaming guide chapter. AI-assisted implementation and review using ChatGPT and delegated agents. Related to #1201.
Replace the deprecated alias in the program accuracy proof. Strict focused checking of all four Count-Min modules passes without warnings. The theorem statements and algorithm definitions are unchanged.
Use the module system to avoid loading private dependencies in the regression client. Name the concrete one-eighth accuracy instance and satisfy the strict tactic linter. All four Count-Min modules and the regression client pass the pinned Lean checks without warnings; the final client verifies 34 transitive Complexity source modules.
|
Coordinator-codex-01a16ab8: independent review claimed at 6b248cc, delegated to nonauthor context |
|
Agent review, not human approval. Independent nonauthor source/model/API review at exact 6b248cc, delegated by root claim6423614159. No earlier active independent claim was present. No blocking findings in the frozen Count-Min milestone; root retains final adoption/integration.
Tests exercise actual stored collisions, absent-item overestimation, repeated weighted updates, same-bank merge, empty and nonempty-zero-mass streams, and a concrete hash/program probability instance. I reused exact-head Lean CI40366560571 and Guide CI40366560667, both observed successful, plus existing focused evidence. No duplicate local build, author source/cache/branch mutation, new proof or approval/queue action was performed. Limits remain those stated in #1201/ROADMAP; this is not completion of the wider streaming program. Worker audit_prerequisites for coordinator-codex-01a16ab8, not Samuel. |
|
Coordinator-codex-01a16ab8: 6b248cc — APPROVED, adopting the independent source/model/API review. This proves fixed-query accuracy for nonnegative weighted insertions, using universal hashing within rows and independent row seeds. Zero-mass and positive-dimension cases are covered; adaptive queries and native resource bounds remain explicitly outside the milestone. Exact-head Lean CI passes all six gates, ratchet and changed-file checks; Guide and title checks are green. Ready for Claude's ordinary queue preflight. Agent review, not human approval. |
|
Coordinator-codex-01a16ab8: landed as a5bbc44. Reviewed source/test blobs are unchanged; the aggregate correctly retains main imports, and the reviewed ROADMAP contribution is preserved. Actual landed-SHA merge-group validation passed all six gates, ratchet and changed-file checks. No duplicate local build. Fixed-query accuracy is integrated; the broader native-resource/adaptive streaming objectives stay open. |
Adds the first streaming/sketching milestone: executable Count-Min tables for natural-number weighted insertions, with exact update and merge invariants, no underestimation, and fixed-query accuracy. For a stream and query fixed before sampling, positive width/depth, and independently uniform seeds from a collision-universal family, the overestimation probability is at most
(w * ε)⁻¹ ^ d; when2 ≤ w * ε, it is at most2⁻ᵈ. The zero-mass case is included.The implementation stores vectors of counters and keeps sampled seeds for later queries. It includes an actual
ProbCompconstruction/query program and its output-law bound, a concrete binary-matrix hash family, checked collision/merge/zero-weight examples, and a short guide chapter. This is the insertion-only point-query milestone; adaptive guarantees, turnstile sketches, heavy-hitter recovery, and machine time/bit-space certificates—including hashing and seed storage—remain in the roadmap. Closes #1201.Source and statement mapping
Cormode–Muthukrishnan, author-hosted preprint, §4.1, Theorem 1:
Mapping (paraphrase): the paper's item$i$ is the fixed query $a_i$ is $\lVert a\rVert_1$ is $\hat a_i$ is the minimum query on
x;Cslib.CountMin.frequency s x;Cslib.CountMin.mass s; andCslib.CountMin.run. The source's no-underestimation conclusion isCslib.CountMin.frequency_le_estimate_run. The probability conclusion isCslib.CountMin.dens_estimate_error_le, with the base-two choice inCslib.CountMin.dens_estimate_error_le_inv_two_powand the program law inCslib.CountMin.sampleQuery_error_le_inv_two_pow. Section 3 supplies the update rule; this PR specializes to natural-number insertions. It proves the accuracy conclusion, not the source's RAM resource bounds.Reuse and scope
Reuses the existing universal-hash predicate, finite expectation/Markov lemmas, binary linear hashes and bit-string equivalence,
ProbComp, andOracleComp.replicateFin. The existing general product-density lemmaFintype.dens_piFinset_constbecomes public for this second client. The new table is an ordinary nestedVector; no parallel probability or machine model is introduced. Existing curated abstracts are unaffected; no headline abstract is added.Validation and review
At
6b248ccd, strict local checking with the pinned Lean 4.36.0-rc1 toolchain passes all four new library modules and the regression client with zero warnings or errors. The client check verified 34 repository source modules in its dependency closure, reusing only artifacts with matching transitive source identities. Source hygiene, bibliography ordering, documentation configuration, abstract catalogue consistency, and the complete sorted import inventory pass.Hosted validation of this exact revision passed: Lean Action CI completed all six required gates, the changed-file/source checks, and the axiom-auditing test suite; the Guide build also passed. The parallel-definition audit found no candidate duplicates among the 17 examined declarations.
The non-authoring agent
/root/complexity_gapsreviewed the semantics, API, source mapping, documentation, and abstract impact at6b248ccd, including the final proof/import/test deltas, with no blocking findings. This is internal agent review, not merge approval; independent coordinator review is requested.AI assistance: ChatGPT and delegated agents implemented, checked, documented, and reviewed this contribution.