Skip to content

fix: SQLite index compaction dropping incomparable versions - #91

Open
cpruijsen wants to merge 1 commit into
electric-sql:mainfrom
cpruijsen:fix/issue-57
Open

cpruijsen wants to merge 1 commit into
electric-sql:mainfrom
cpruijsen:fix/issue-57

Conversation

@cpruijsen

Copy link
Copy Markdown

Summary

SQLIndex.compact skips identity advances when grouping rows to move, so values are kept when a version advances to itself under an incomparable frontier (issue #57). The existing compactKey / deleteOldVersionsData statements never insert-then-delete the same primary key.

SQLIndex.compact grouped that identity version with the versions that move onto it, inserted the summed row, then deleted every source version, including the destination. In-memory Index.compact already deleted before rewriting.

Adds the reporter's case to the shared in-memory/SQLite version-index tests.

Provenance: Kevin's reproduction on #57 (in-memory [[10, 3]], SQLite []), matching compaction as meet-of-joins.

Grouping skips identity advances instead of rewriting compact as materialize → delete sources → insert (match in-memory order). Smallest reversible change; same compacted collection as in-memory. Can switch to the rewrite if the SQL path should share control flow with Index.compact.

Fixes #57

Test plan

  • packages/d2ts tests/version-index.test.ts: in-memory and sqlite, including "compaction with incomparable versions"
  • Confirm the test fails on sqlite (result1 === []) with the source change reverted, and passes with it restored
  • Confirm the sqlite case still matches in-memory after a second compact on a later frontier

SQLIndex.compact moved then deleted versions that advance onto themselves, so reconstructed values disappeared after compaction under an incomparable frontier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem with compaction in SQLite

1 participant