Repository navigation
Conversation
Author
|
The — Written by Claude Fable 5.1 via Claude Code, on behalf of ParyonUSD. |
The agent's mempool scan archives node_transaction rows accepted (or replaced) by blocks, then reads rows expiring soon and arms an exact expiry timer for each. A row inserted between those two statements with an old validated_at (e.g. a confirmed transaction re-announced or re-validated) got an immediate timer and was archived to node_transaction_history with replaced_at = expiry, recording a confirmed transaction as dropped. The e2e test "archives stale mempool transactions already accepted by blocks" hit this intermittently. archiveMempoolTransaction now guards its DELETE with NOT EXISTS checks for an inclusion of the transaction, or of a conflicting spend of one of its outpoints, in a block accepted by the same node. The guard is evaluated in the archiving statement itself, so no earlier read can go stale. Such rows are left for archiveMempoolTransactionsAcceptedByBlocks, which records replaced_at NULL (confirmed) or the block's accepted_at (replaced) on the next scan. Adds a deterministic regression test that inserts rows between the two scan steps and fires the expiry archive directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…an test The test creates test_output_search_index next to the managed output_search_index. The two are identical, so the planner's choice is a tie that flips once Postgres has auto-analyzed output (the managed index then has expression statistics, the fresh copy none): reproducible on this branch's e2e runs, passing on paryon only by timing. Either index satisfies what the test checks (the 25-byte prefix index, no sequential scan). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Archiving an expired mempool transaction also archives its same-node descendants (the node_transaction_history trigger's recursive cascade). The agent fired one archive statement per expiry timer, racing on every pool connection: a parent's expiry locked its own row and then waited in the cascade for a descendant's row, while the descendant's own expiry held that row and waited (in its cascade) for a row the first had locked. A restarted agent whose 6-day-stale mempool expired all at once hit >1,300 deadlocks (40P01) and made no net progress (2026-10-13 backfill lab). - archiveMempoolTransactions: one set-based statement per batch; takes a transaction-level advisory lock before any row lock (serializes expiry across agents sharing the database), locks rows in (node, transaction) order, keeps the accepted-by-block guards, and retries a deadlock or serialization failure against other writers with exponential backoff. - expireMempoolTransaction: in-process queue; timers firing together are archived in batches of up to 500, one batch at a time on one connection. - archiveMempoolTransaction keeps its signature (a one-row batch). - New spec reproduces the lab: 60 x 40 chained/fanned-out mempool transactions for two nodes, all expiring at once on a 16-connection pool (failed with dozens of 40P01 before; passes 5/5 now). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… log line When two mock nodes announce the same block, the "Saved new block" line lists only the nodes whose headers arrived before the save; a later announcer is recorded via "accepted existing blocks" and never appears in a save line. Header order across separate peer connections is not deterministic, so waiting for "nodes: node1, node3" (and similar) can time out even though the agent behaved correctly. Poll node_block for the expected nodes instead, for each multi-node announcement in the reorg sequence (heights 3001, 3002, 3006, 3007, 3008). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ci: use current major versions of GitHub actions
test(e2e): deflake re-org tests — check node acceptance in DB, not the save log line
node.internal_id and node_transaction.node_internal_id are integer, so a node id above 32767 overflowed the smallint[] parameter and failed the whole expiry batch. Node registration consumes sequence values even for known nodes, so such ids are routine. The expiry test schemas now use the production column types and the concurrency fixture uses node ids above the smallint range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y or their block time The expiry statement deleted only the expired rows and left descendants to the history trigger's cascade, which loses two things: - the trigger only reaches descendants still in node_transaction, so a descendant expired in the same batch kept its own, later expiry (and passed it on), instead of its ancestor's earlier one; - the trigger does not check blocks, so a descendant also replaced by a block accepted by the node was archived with its ancestor's expiry instead of the block's accepted_at, and the accepted-block archive could no longer correct it. The statement now walks the same-node descendants itself: each gets the earliest expiry among its archived ancestors, or, if resolved by a block accepted by the node, the replaced_at the accepted-block archive would give it (NULL ends the cascade there). All rows are locked in (node, transaction) order and the trigger finds nothing left to cascade. Only expired rows without an expired parent expiring no later seed the walk, so an expired chain is walked once. The expiry queue takes the earliest expiries first, so an ancestor is never in a later batch than its descendant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test(sql): deflake search index plan test (accept either output_search_index copy)
…lacement in expiry Blocks accepted before Chaingraph began monitoring have a NULL accepted_at, which MIN(accepted_at) could not tell apart from no conflicting block at all. Expiry then archived a root replaced by such a block with its expiry, and a descendant replaced by one with its ancestor's expiry. The accepted-block resolution now returns whether a conflicting spend exists separately from its (nullable) earliest accepted_at: such a root is left for the accepted-block archive, and such a descendant gets that archive's NULL replaced_at. The test schemas allow a NULL node_block.accepted_at, as production does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al statement The advisory lock was taken inside the archival statement, whose snapshot predates the wait, so the accepted-block guards missed block inclusions committed while expiry waited, and an expired parent deleted meanwhile by block acceptance (confirmed) still seeded the descendant walk: its row lock skipped the parent, but its unconfirmed child was archived with the parent's expiry. Expiry now runs as one transaction: take the advisory lock, lock the expired rows, then archive in a separate statement, whose READ COMMITTED snapshot is taken after both waits. The cleanup spec discards its temporary schema after each test instead of rolling back an enclosing transaction, as expiry commits its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lock The archival statement walks descendants in its snapshot but locks them only afterwards. For P -> C -> G with only P expiring, C confirmed concurrently (deleted with replaced_at NULL, ending the cascade) was skipped by the row locks, but the walk had already reached G through the old C, so G was archived with P's expiry. The statement now deletes nothing unless every walked row was still present when locked; otherwise the transaction is rolled back and retried with a fresh snapshot under the existing backoff. If retries are exhausted it throws, leaving the rows for the next expiration scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(db): never expire a mempool transaction already accepted by a block
Attach pool and per-client error listeners so a Postgres restart (57P01 etc.) is logged instead of crashing the agent via uncaughtException; the incomplete-block repair scan retries after a lost connection. Block-save failures stay fatal. Adds unit tests and an e2e test that terminates every agent backend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fixes bitauth#80) (#7) Headers the agent already knows (e.g. the shared prefix of a deep reorg's headers message) are skipped before replacing blocks and reporting stale hashes, so valid node_block rows are no longer removed on deep reorgs. Adds regression tests for chain extensions, shallow reorgs and deep reorgs. Fixes bitauth#80. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itauth#81) (#8) After a reorg and on (re)connect, the agent requests each node's mempool (BIP35, rate-limited) so transactions returned to the mempool are recorded again. An expiry now carries the validated_at it was scheduled for, so a validation restored by the resync is not archived by a stale expiry timer. Adds e2e and database regression tests. Fixes bitauth#81. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#10) Block conversion reuses the bytes received from the node instead of re-serialising blocks and transactions: transaction hashes and sizes come from the wire bytes, one-shot hashing is used where available and transaction fields are read directly. Adds parity tests for decoding and conversion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ar SyncState (#12) The startup incomplete-block repair scan records, per node, the highest block internal_id it has fully audited and skips those blocks on later starts (falling back to the full scan when any node has no marker). SyncState tracking becomes linear. Adds e2e coverage for markers, the bounded per-node scan and the full-audit fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…opy) (#13) Adds an opt-in block write path (CHAINGRAPH_WRITE_PATH=copy, default sql) that streams rows with binary COPY into per-connection temporary staging tables, pipelined in one round trip, then inserts them with the same statements and conflict handling as the sql path, in one transaction per block. CI runs the full suite on both write paths; adds encoder unit tests and a real-server COPY failure-recovery test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aky tests (#27) The expiry archival statement's descendant walk now looks up node_transaction by its full key behind an OFFSET 0 barrier, so archiving a stale mempool stays linear even when node_transaction statistics are outdated (was quadratic). Adds a plan-based regression test, shrinks the concurrency spec's mempool, and replaces fixed sleeps in two e2e tests with polling for the saved transactions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #71
Closes #72
Closes #73
Closes #74
Closes #75
Closes #55
Closes #77
Closes #78
Closes #79
Also improves performance by bounding mempool-cleanup conflict lookups to the outpoints currently being cleaned up, instead of scanning all conflicting inputs.
Leaving these here for others who may find them helpful, courtesy of ParyonUSD.
Code by various Codex (5.5 - 6.1) and Claude (Fable 5, Opus 5.5) models. See commit messages for attribution.