Skip to content

Separate 11 legacy article and supplementary evidence caches - #1997

Merged
realmarcin merged 3 commits into
mainfrom
fix/separate-legacy-supplements
Oct 13, 2026
Merged

realmarcin merged 3 commits into
mainfrom
fix/separate-legacy-supplements

Conversation

@realmarcin

@realmarcin realmarcin commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #1994. Fixes #1998. Fixes #1999.

Separate all 11 mixed article caches. Retain ten existing supplementary sidecars unchanged and extract the missing Lupinus sidecar byte-for-byte. Preserve article source text except one recorded append-separator LF per file. The large diff is mechanical movement/deduplication of tabular source text, not scientific curation.

Add hash-bound migration, boundary regression tests, reproducible source/snippet receipts and accurate validator guidance. Historical cache checks now verify their recorded Git blobs rather than freezing current files. CI explicitly fetches the five immutable pre-squash review bases, with fresh-clone control/mutation/restoration coverage.

No canonical YAML, scientific claim, schema, validator, curation history or generated community page changes. Old ledgers and prior saved reviews remain immutable. No publisher fetch, publication PDF, paid provider call or unrelated local cache is included.

Review Evidence

Adversarial self-review, not independent approval or scientific endorsement. Immutable YAML and rendered Markdown pairs:

  • reviews/structured/20261013T043255Z-1994-source-boundaries-before/
  • reviews/structured/20261013T044252Z-1994-source-boundaries-after/
  • reviews/structured/20261013T052657Z-1994-integration-before/
  • reviews/structured/20261013T052919Z-1994-integration-after/
  • reviews/structured/20261013T060629Z-1994-portability-before/
  • reviews/structured/20261013T060932Z-1994-portability-after/
  • Reproducible receipts: reports/reference_cache_separation/20261013-1994/

#1998 records both the original six integration failures and the subsequent four fresh-checkout failures. Both were addressed without changing historical digests or weakening checks; successor reviews preserve the failed observations.

Verification

Reviewed head: 109fb0d.

  • All seven required PR checks passed.
  • Full CI: 4888 passed, 86 skipped, 8 deselected; structured-review contract: 4 passed. Run: https://github.com/CultureBotAI/CommunityMech/actions/runs/38355277269
  • Wider local causal/cache/writer suite: 1080 passed.
  • Latest focused integration/provenance/workflow suite: 80 passed.
  • All 17 immutable review bundles validate; lint passes Black 389 files, Ruff and mypy 49 files.
  • Fresh transport clone lacked all five historical bases; the actual CI fetch command retrieved them from GitHub; all 19 historical cache hashes verified.
  • Byte receipts confirm 11 article outputs, ten unchanged sidecars, one exact extraction and 11 unchanged records. Ignored-inclusive article scan is clean.
  • Historical reproduction matches all 284 snippet-validation rows and input hashes.
  • All seven protected merge-group checks passed on 8df8937. Merged through the queue at 2026-10-13T07:38:26Z. Queue full suite: 4888 passed, 86 skipped, 8 deselected.

Explicit Limitation

Native reference validation reports 0 failures on the old mixed caches and 79 article-only failures after separation. All 79 have retained verbatim case/whitespace-normalized locations in separate supplements. Diagnostic matching against those selected sources is not a normal CLI pass. #1996 remains open for explicit source-specific routing. No concatenation, union/fallback, threshold change or waiver was added. Two other pre-existing snippets require native punctuation normalization and are recorded transparently.

Branch-Cleanup Provenance

#1999 records the pre-merge cleanup check. Following the established #1979 procedure, annotated tag review-provenance/20261013-1994 retains the exact reviewed head 109fb0d and its source bases eabe585/7c0413e2. Remote tag object: 19b26db004b856950611e8a8fa239a1410df447d. No code or saved-review bytes changed. The first queue run was intentionally withdrawn/canceled for this check, not failed by a test. The replacement protected queue run passed and the PR merged. Local main is fast-forwarded; local and remote task branches are deleted. The archival provenance tag is retained, and #1996 remains open.

@github-actions

Copy link
Copy Markdown

Network integrity findings

Warnings only — a member with no interaction yet, or a participant matched by ontology id rather than by name, or one on a community-level interaction that resolves to no member. Reported, but does not fail the build.

Network Integrity Audit Report
================================================================================

0 error, 57 warning across 21 records with findings
Only error-severity findings fail the build.

Algal_Methanotroph_Biogas_Valorization_Coculture
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Leptolyngbya' has no interactions

Total: 1 issues (0 error, 1 warning)

At_RSPHERE_SynCom
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Flavobacterium sp.' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas sp.' has no interactions
  [warning] DISCONNECTED: Taxon 'Rhodococcus sp.' has no interactions
  [warning] DISCONNECTED: Taxon 'Streptomyces sp.' has no interactions
  [warning] DISCONNECTED: Taxon 'Variovorax sp.' has no interactions

Total: 5 issues (0 error, 5 warning)

Australian_Lead_Zinc_Polymetallic
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Acidithiobacillus ferrooxidans' has no interactions
  [warning] DISCONNECTED: Taxon 'Leptospirillum ferriphilum' has no interactions

Total: 2 issues (0 error, 2 warning)

Bacillus_G12_Y4_X25_Tobacco_Biocontrol_SynCom
--------------------------------------------------------------------------------
  [warning] UNKNOWN_TARGET: Target taxon 'tobacco' not found in taxonomy section (community-level scope)
  [warning] UNKNOWN_TARGET: Target taxon 'tobacco' not found in taxonomy section (community-level scope)

Total: 2 issues (0 error, 2 warning)

Bacillus_siamensis_vallismortis_HT_Masson_Pine_SynCom
--------------------------------------------------------------------------------
  [warning] UNKNOWN_TARGET: Target taxon 'Fusarium oxysporum' not found in taxonomy section (community-level scope)
  [warning] UNKNOWN_TARGET: Target taxon 'Pinus massoniana' not found in taxonomy section (community-level scope)
  [warning] UNKNOWN_TARGET: Target taxon 'Pinus massoniana' not found in taxonomy section (community-level scope)

Total: 3 issues (0 error, 3 warning)

Brocadia_Low_Nitrogen_Anammox_Consortium
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Ca. Kuenenia early-acclimation lineage' has no interactions

Total: 1 issues (0 error, 1 warning)

Dangl_SynComm_35
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Escherichia coli DH5α' has no interactions
  [warning] DISCONNECTED: Taxon 'Flavobacterium species' has no interactions
  [warning] DISCONNECTED: Taxon 'Microbacterium species' has no interactions
  [warning] DISCONNECTED: Taxon 'Stenotrophomonas species' has no interactions
  [warning] DISCONNECTED: Taxon 'Xanthomonadales members' has no interactions

Total: 5 issues (0 error, 5 warning)

Deepwater_Horizon_Deep_Sea_Oil_Plume_Succession
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Methylophaga' has no interactions

Total: 1 issues (0 error, 1 warning)

East_River_Floodplain_Core_Microbiome
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Acidobacteriota floodplain representatives' has no interactions
  [warning] DISCONNECTED: Taxon 'Nitrospirae core floodplain members' has no interactions

Total: 2 issues (0 error, 2 warning)

GLBRC_UFMP_Fermentation_Community
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'RUG023 sp. (MAG SPH2)' has no interactions

Total: 1 issues (0 error, 1 warning)

KBase_Models_for_Zahmeeth_Original_PLOS
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Klebsiella' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas fluorescens' has no interactions

Total: 2 issues (0 error, 2 warning)

Lake_Washington_Methane_Oxygen_Methylotroph_Community
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Methylosarcina' has no interactions

Total: 1 issues (0 error, 1 warning)

Lotus_LjSC3
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Burkholderiaceae bacterium' has no interactions
  [warning] DISCONNECTED: Taxon 'Mesorhizobium sp. LjNodule210' has no interactions
  [warning] DISCONNECTED: Taxon 'Mesorhizobium sp. LjNodule215' has no interactions
  [warning] DISCONNECTED: Taxon 'Mesorhizobium sp. LjNodule218' has no interactions
  [warning] DISCONNECTED: Taxon 'Microbacteriaceae bacterium' has no interactions
  [warning] DISCONNECTED: Taxon 'Oxalobacteriaceae bacterium' has no interactions
  [warning] DISCONNECTED: Taxon 'Phyllobacteriaceae bacterium' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonadaceae bacterium' has no interactions
  [warning] DISCONNECTED: Taxon 'Rhizobiaceae bacterium' has no interactions

Total: 9 issues (0 error, 9 warning)

Nitrifying_Wastewater_Ammonia_Oxidizing_Consortium
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Legionella MAGs' has no interactions

Total: 1 issues (0 error, 1 warning)

Panzhihua_Vanadium_Titanium_Tailings
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Polaromonas species' has no interactions
  [warning] DISCONNECTED: Taxon 'Rhizobium pisi' has no interactions
  [warning] DISCONNECTED: Taxon 'Rhizobium selenitireducens' has no interactions

Total: 3 issues (0 error, 3 warning)

Phormidium_Alkaline_Consortium
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Bacteroidota members' has no interactions

Total: 1 issues (0 error, 1 warning)

Polaromonas_Vanadium_Reduction_Community
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Desulfitobacterium species' has no interactions
  [warning] DISCONNECTED: Taxon 'Desulfovibrio species' has no interactions
  [warning] DISCONNECTED: Taxon 'Geobacter species' has no interactions

Total: 3 issues (0 error, 3 warning)

Pseudomonas_Aeruginosa_EightStrain_Full_Factorial_SynCom
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Pseudomonas aeruginosa MDR6-CAZ' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas aeruginosa NfxB-CAZ' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas aeruginosa OrfN' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas aeruginosa PA14' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas aeruginosa mexZ' has no interactions

Total: 5 issues (0 error, 5 warning)

Sclerotinia_Sclerotia_12Strain_Biocontrol_SynCom
--------------------------------------------------------------------------------
  [warning] UNKNOWN_TARGET: Target taxon 'Sclerotinia sclerotiorum' not found in taxonomy section (community-level scope)
  [warning] UNKNOWN_TARGET: Target taxon 'Sclerotinia sclerotiorum' not found in taxonomy section (community-level scope)

Total: 2 issues (0 error, 2 warning)

Space_Habitat_SevenMember_Stress_Tolerance_SynCom
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Acinetobacter radioresistens 50V1' has no interactions
  [warning] DISCONNECTED: Taxon 'Micrococcus luteus W4-Al-K1' has no interactions
  [warning] DISCONNECTED: Taxon 'Propionibacterium cyclohexanicum TA-12T' has no interactions
  [warning] DISCONNECTED: Taxon 'Pseudomonas antarctica CMS 35T' has no interactions
  [warning] DISCONNECTED: Taxon 'Staphylococcus capitis K1-2-2-23' has no interactions
  [warning] DISCONNECTED: Taxon 'Streptococcus halotolerans HTS9T' has no interactions

Total: 6 issues (0 error, 6 warning)

Tinto_River_Iron_Cycling_Community
--------------------------------------------------------------------------------
  [warning] DISCONNECTED: Taxon 'Basidiomycota' has no interactions

Total: 1 issues (0 error, 1 warning)

The full report is attached to the workflow run as an artifact.

@realmarcin realmarcin left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial self-review of exact head eabe585, not independent approval.

No additional blocking defect found in this bounded source-provenance repair. Checked the 28-file scope, every before/after cache hash, the two non-identical existing sidecars, preservation of all ten existing sidecar files, byte-exact extraction of the missing Lupinus sidecar, and the explicitly recorded one-LF append-separator removal per article. A post-commit historical verifier confirms all 11 canonical records remain unchanged and an ignored-inclusive filesystem scan finds no mixed article caches.

The migration plans the entire hash-bound cohort before writing, refuses changed inputs/conflicting sidecars/ambiguous boundaries, and writes new separate source bytes before trimming articles. The new gate has clean-control, applied-mutation and immediate-restore arms. The corpus red-before result was exactly 11 offenders; 80 focused tests and 4 review-contract tests pass after repair. Black, Ruff, mypy and staged diff checks pass. A fresh historical control reproduced all 284 diagnostic rows and the final manifest.

The native reference result is intentionally not called green: 79 article-only failures remain, each located in a preserved separate supplement. #1996 records source-specific routing as follow-up. No scientific claims, schema, validator, threshold, fallback or curation history were altered. The structured before and successor assessments are under reviews/structured/20261013T043255Z-1994-source-boundaries-before/ and reviews/structured/20261013T044252Z-1994-source-boundaries-after/.

Required PR and merge-group CI must still pass; no bypass is requested. This review resolves #1994 only, not scientific correctness or #1996.

@realmarcin realmarcin left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial self-review of exact head 7c0413e (same implementation agent; not independent approval).

Confirmed and addressed #1998: original required CI and local control both reproduced five old-cache/live-file digest failures plus guide wording. All 19 historical cache hashes match the ledgers recorded commits. The fix reads immutable Git blobs, enforces base_tree except explicitly legacy commit-only batch 15, never falls back to current files, and preserves live record/non-cache assertions. Twelve new control/mutation/restoration cases exercise current-file evolution, bad hashes, missing/mutable/unknown revisions, tree mismatch, missing path and restricted cache scope. 51 focused tests pass. Lint, immutable review validation (15 bundles) and source byte verification pass.

No old ledgers, scientific records, source repair bytes or earlier saved reviews changed in this successor. The source-boundary review and #1996 native supplement-routing limitation remain explicit. New immutable before/after integration records are linked in the PR body.

No additional merge-blocking issue found in this bounded diff review. Wider local tests and protected CI remain pending; this comment is not a claim of CI readiness. Auto-merge remains disabled until results are checked.

@realmarcin

Copy link
Copy Markdown
Contributor Author

Successor verification for exact head 7c0413e: all causal-review batch tests plus source/cache/writer tests passed (1080 passed, 12 pre-existing dependency warnings, 353.02s). Separately, 51 focused integration/provenance tests passed. No further diff or finding. The 15 saved review bundles, lint and byte-preservation verification also pass.

Protected CI is still running; full CI is not yet claimed green. Enabling normal auto-merge for this exact reviewed head, without bypassing PR checks or merge-queue checks. #1996 remains an explicit separate limitation.

@realmarcin realmarcin left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial self-review of exact head 109fb0d (same agent; not independent approval).

The previous CI failure was a missing fetch prerequisite, not a waived hash failure. This successor adds only an explicit five-SHA git fetch before tests, two workflow/fresh-clone regression tests, and immutable review records. No historical hash helper, ledger, record or source bytes changed. CI still fails if any exact historical commit is unavailable; no live-byte fallback, branch-name substitution or continue-on-error exists.

Fresh single-branch transport clone: all five bases confirmed absent before; actual new workflow command fetched all five from GitHub; all 19 historical cache hashes verified afterward. Synthetic fresh-clone control/mutation/restoration rejects one unknown SHA and restores green. 80 focused integration/provenance/workflow tests pass; lint and all 17 immutable review bundles pass. Earlier 1080 causal/cache tests remain recorded for the unchanged source/test behavior.

Saved portability review pair: reviews/structured/20261013T060629Z-1994-portability-before/ and reviews/structured/20261013T060932Z-1994-portability-after/. They explicitly supersede the local-only #1998 disposition without rewriting it. No additional issue found in this bounded successor review. Protected CI must still pass; #1996 remains open.

@realmarcin
realmarcin enabled auto-merge October 13, 2026 06:11
@realmarcin

Copy link
Copy Markdown
Contributor Author

PR validation is now green on exact reviewed head 109fb0d. All seven required checks passed. Full validate-strict run https://github.com/CultureBotAI/CommunityMech/actions/runs/38355277269: 4888 passed, 86 skipped, 8 deselected; structured-review contract 4 passed. The explicit historical-base fetch succeeded on the runner. No gates were bypassed and no validator fallback was introduced. Proceeding through the protected merge queue; #1996 remains open.

@realmarcin
realmarcin added this pull request to the merge queue Oct 13, 2026
@realmarcin
realmarcin removed this pull request from the merge queue due to a manual request Oct 13, 2026

@realmarcin realmarcin left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Publication/cleanup adversarial follow-up on unchanged head 109fb0d: #1999 is addressed by the repository-standard annotated review-provenance/20261013-1994 archive. A fresh squash-candidate clone plus old archive tags failed canonical provenance validation on eabe585; fetching the new tag restored all 17 unchanged review bundles to valid. Exact remote tag/head binding and actual CI tag-fetch refspec are verified. No source, test, workflow, ledger or saved-review byte changed; all seven PR checks remain green. Self-review, not independent approval. Resume protected merge-group validation before deleting the feature branch; retain the archival tag.

@realmarcin
realmarcin added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit 8df8937 Oct 13, 2026
8 checks passed
@realmarcin
realmarcin deleted the fix/separate-legacy-supplements branch October 13, 2026 07:40
@realmarcin

Copy link
Copy Markdown
Contributor Author

Completed through protected merge queue: 8df8937 at 2026-10-13T07:38:26Z. All seven PR checks and all seven merge-group checks passed. Queue full suite: 4888 passed, 86 skipped, 8 deselected; structured review contract 4 passed. #1994, #1998 and #1999 are closed.

Primary main is fast-forwarded. Final main-side verification confirms all 11 article outputs, ten unchanged sidecars, one exact extraction, 11 unchanged records and no mixed article cache, including ignored files. All 17 immutable review bundles validate. Existing local untracked work was preserved; the matching pre-existing Lupinus supplement was backed up before it became tracked.

Local and remote fix/separate-legacy-supplements branches are deleted, and task worktrees/fresh-clone probes are removed. Annotated review-provenance/20261013-1994 remains to preserve review input history. No unrelated worktree or branch was changed.

#1996 remains open: 79 supplement-only native reference-validation failures have retained source locations. No fallback or waiver was added. Saved reviews are explicitly self-review/provenance assessments, not independent scientific approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant