A handful of pull requests...

[brettwooldridge]

I am sorry to have spammed your pull requests. We migrated from Xodus to NitriteDB in our latest product release. Unfortunately, our automated testing did not uncover any issues, but when we got into production all hell broke loose.

We encountered several crashes, and performance issues that effectively locked our servers. All of these were traced down to their root causes. We have generated pull requests for these. One or two of the pull requests are minor performance improvements, but the majority are either critical path bugs or critical path performance issues.

I hope you look upon these pull requests favorably, in the spirit in which they are intended. We need NitriteDB to be solid.

In addition, we are currently developing a multitheaded load test harness to try to uncover any additonal issues. When it is done, we will be happy to share it.

Even with all of the above fixes, our AI is uncovering issues like this:

Nitrite disabling retention time and version keeping on every store is risky in H2, since a retention time of zero means freed chunks can be overwritten immediately—dangerous with concurrent readers if a disk page from a reused chunk gets read before it's evicted from cache, potentially causing corruption.

(and that corruption was observed during a load test)

Retention time is the key factor — the 45s run survived two restarts while every retention-zero variant (including stock H2 2.4.240) failed at the first close, matching two still-open H2 issues.

All five zero-retention runs corrupted on first close, while all runs with default H2 settings survived — confirming Nitrite's adapter has forced those settings to zero since 2020.

Thanks.

[anidotnet]

No problem, I'll be happy to help. I'll see what I xan do by this weekend.

[brettwooldridge]

One more PR incoming related to the retention time. Our soak test could recreate db corruption on 5 out 5 runs.

[brettwooldridge]

branch	what it fixes	class	in patch jar
fix/store-catalog-copy-on-write	Catalog mutated the set held in an MVStore page; races background serialisation, panics the store until restart.	crash	yes
fix/mvstore-retention-defaults	Adapter forced retention and versions-to-keep to 0; file refuses to open after a close.	corruption	yes
fix/index-scan-skips-removed-documents	Document removed between index lookup and fetch reached filters as a null row.	crash	yes
fix/collection-factory-lock-convoy	Factory-wide lock held while checking collection state; every lookup queues behind one long index rebuild.	hang	no
fix/deep-copy-on-read	clone() shared nested containers and getById() returned the stored instance; a caller's edit races serialisation or bypasses indexes.	crash, latent	no
fix/mvstore-page-split-size-default	Default page split size was 16 bytes instead of 16 KB.	performance	no, product sets 64 KB
fix/skip-unchanged-index-updates	Index rewritten even when the indexed value did not change.	performance	no
fix/numbers-compare-fast-path	Same-type numeric comparisons went through BigDecimal.	performance	no
fix/unique-index-single-id	Unique index stored a one-element list per key; also fixes clear() leaving composite rows that later violate the unique constraint.	performance, plus one correctness bug	no
feat/lazy-index-scan	Index scans materialised every id; now streamed, and size() answers from the index.	performance	no

I'll be honest, these are Claude Fable 5.1 fixes on Max. It represents the result of 16 straight hours of panicked analysis of production failure logs and creation of a soak test harness. The "in patch jar" is not relavent to you, it is our internal patched jar we're injecting back into production.

[anidotnet]

I have addressed all of them but I have not released yet to maven. You can build the main branch code and run the soak tests and report back in case of new findings. Some your prs are superseded by a combined pr which contains code rabbit suggested fixes, a stale test fixes and others.