Repository navigation
feat(store): v1 SQLite store with init migration, queries and pinned query plans (AFDEV-12) - #4
Merged
Merged
Conversation
…query plans (AFDEV-12) The v1 store at internal/store: single init migration with the application_id stamp, every query the v1 core needs (lookups, dedupe window, list with filters and cursors, stats, marks, tombstones, export), and a test that pins the query plans. The v3 store moves unchanged to internal/v3/store so the running service and its e2e gate stay green until the v1 core, HTTP and tooling packages replace them.
…p and plan checks (AFDEV-12) Review findings: the stamp check and the schema_version bootstrap run in one IMMEDIATE transaction so concurrent first opens agree on one row; the series bucket floors microseconds to whole seconds instead of passing a fractional unixepoch that date() rounds to milliseconds (the last microsecond of a day crossed midnight and year 9999 became NULL); a foreign database is refused for any schema object, not only tables; the interrupted-first-open test writes the real stamp; the plan test matches whole index names; the q clause documents the driver's NUL limit and a test pins it.
Collaborator
Author
|
Review round (cross-model panel, 4 reviewers) addressed in the second commit:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The v1 store:
internal/storeis the SQLite package the v1 API's core will run on.internal/store/migrations/001_init.sqlis the only migration: onesubmissionstable for every kind,uid(UUIDv7) beside the integerid, the friction payload'scategoryandfix_statusas VIRTUAL generated columns, and ten indexes (keyed uniqueness on(kind, key), the hash + created_at pair for the keyless dedupe window, the open queue, and the filter columns).internal/store/queries.go: inserts (API and id-preserving restore), lookups by id, uid,(kind, key)and the 24 h same-hash window that skips processed rows,List/Countwith every filter ofGET /api/v1/submissionsand both cursors, the stats pieces (StatsTotals,StatsGroupsfor up to three keys,StatsRecurring,StatsSeriesby day or ISO week), set-based mark/unmark, redaction tombstones, ascending export iteration.internal/store/store.go: the pools and migration loader as before, plus aPRAGMA application_idstamp written before the first migration. A non-empty database without the stamp is refused at open withErrForeignDatabasenaming the path and the way out (a freshDATABASE_PATH). Both the previous store and this one number their init migration 1, so the schema version alone cannot tell them apart; without the stamp a binary on the old file would fail on its first query instead of at start.qis implemented as a registered deterministic SQLite function (af_lower, backed bypkg/schema.LowerSimple) oversummaryand every string value in the payload viajson_tree, so "case-insensitive" means the same Unicode mapping the contract uses for tokens; keys, numbers and booleans are not searched.internal/store/plan_test.gopinsEXPLAIN QUERY PLANfor 45 statements: the queue and its pages, the identity lookups, every indexed list filter,qalone and with a filter, each stats grouping, recurring, series and export. Where the schema has an index the named index must appear and the table is not scanned; where it has none the scan is recorded as expected.Coexistence with the running v3 service
The v3 service (
internal/core,internal/api,cmd/agentfeedback) still compiles against the previous store, andscripts/e2e.shexercises it in CI. That package moves unchanged tointernal/v3/store(import paths only; its own tests move with it) so the new store can take the path the v1 layout names. The tickets that replace the core, HTTP and tooling packages removeinternal/v3/when its last consumer goes. The e2e suite proves the move changed nothing (55/55).Docs and contract
docs/develop.md: layout lines for both stores, the write-once rule now names the processing fields and the tombstone, the migration rule names the stamp.docs/openapi.yaml: thebyandqdescriptions state the rules the store enforces (rows without a grouping key are in no group; whatqsearches). Descriptions only; no shape change.Out of scope, by design: identity decisions (200/201/409), warnings, the HTTP layer. The store exposes the unique index, the window lookup and the snapshot read; the core decides.
Verification
Locally, all green:
just checkwith a clean tree,go test -race -count=1 ./...,just fuzz, live e2e 55/55, shellcheck (CI form and the develop.md form), skill tests 124/124,just contract(89 fixtures, Redocly valid), staticcheck 2026.2.1 clean.Store tests cover: byte-exact payload and context round trips (spacing, member order, large integers, exponent spelling, U+FFFD, escapes),
""⇔ NULL, keyed uniqueness and the uid index, the dedupe window (inside/outside, processed skipped, newest wins), every list filter with its count, both cursors includingafter_id=0, count and list inside one read snapshot, every grouping key and multi-key groups, numericcategorygrouped as text, recurring, day and week series with Sunday/Monday and year-turn boundaries, mark keep/replace/clear and unmark, tombstones (hash and identity kept, category gone, still markable), export ordering and filters,qfrom more reader connections than the pool holds, and the foreign-database and interrupted-first-open cases.