Skip to content

[Issue #21] Coalesce transaction-local primary-key changes in Retina batches - #22

Draft
AntiO2 wants to merge 1 commit into
masterfrom
fix/retina-coalesce-transaction-keys
Draft

AntiO2 wants to merge 1 commit into
masterfrom
fix/retina-coalesce-transaction-keys

Conversation

@AntiO2

@AntiO2 AntiO2 commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

A transaction that updates one primary key twice currently sends both updates in one Retina operation group. Retina resolves the same original row for both replacements and leaves the intermediate replacement unhidden.

This change folds each same-source-transaction, same-table, same-key sequence within a detached RPC batch into its net I/U/D effect. It retains the earliest before image, final after image and NULL flags. Transactions and RPC boundaries remain separate; snapshots and unidentified transactions are not folded. Operation groups also split on repeated keys. Existing bucket routing handles primary-key changes before folding.

Canceled writes still advance the original event counters only after a successful RPC acknowledgment, including an entirely empty batch. The coalescer has no shared state and adds no locks. This is separate from the native visibility concurrency fix in pixelsdb/pixels#1412.

Validation:

  • mvn -q test: 80 tests passed before the additional split-primary-key/NULL regression.

  • mvn -q -Dtest=RetinaPayloadBuilderTest,TableCrossTxCoalescingTest test: 14 tests passed, including 100 concurrent batches, transaction/RPC boundaries and success/failure acknowledgments for canceled writes.

  • mvn -q -DskipTests package: passed.

  • Captured cds26 PostgreSQL/Debezium events replayed through the new coalescer and payload builder: seven transactions, sixteen source changes, five expected net writes. This checks actual captured input; it is not a full Retina storage integration test or steady-state GC benchmark.

  • Combined live HyBench SF100 Proto/Retina/GC validation with the private native fixes: 22,945,933 input rows and 452,616 admitted transactions fully committed; two GC cycles and three partial-live group rewrites. After draining, savingaccount/checkingaccount each retained 30,200,000 active versions, customer 30,000,000 and company 200,000. Sink shutdown produced no error entries. This is functional integration validation, not a steady-state or isolated performance result.

Closes #21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[pixels-sink] Coalesce same-transaction primary-key changes within Retina RPC batches

1 participant