Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ merge only with the maintainer's go-ahead. Changes merged to `main` deploy.
- `_config.yml` - site configuration, `header_pages` navigation order, plugins, `exclude:`
- `Gemfile` - Ruby dependencies
- `_layouts/default.html`, `_layouts/home.html`, `_layouts/page.html`, `_includes/head.html`, `_includes/footer.html` - Minima overrides; the site header comes from the Minima theme itself
- `assets/custom.css` - the site palette and styles (`--ink`, `--muted`, `--accent`, `--card` and the rest), which the fleet page also uses; the twelve Mech identity colours (`--mech-<name>` and their `-ink` variants) are defined in `_fleet/fleet_fragment.html`
- `assets/custom.css` - the site palette and styles (`--ink`, `--muted`, `--accent`, `--card` and the rest), which the fleet page also uses; the thirteen Mech identity colours (`--mech-<name>` and their `-ink` variants) are defined in `_fleet/fleet_fragment.html`

### Other
- `.github/workflows/fleet-page.yml` - the only CI; runs the tests and the fleet page checks, and nightly the checks against CLAW's live manifest and the Mech sites
Expand All @@ -76,9 +76,9 @@ Content pages use Jekyll front matter with `layout`, `title`, `description` and
`resources.md` to follow the data flow from recipes to ingredients to communities.

### The X-Mech Suite page
The suite has twelve Mechs, all under https://github.com/CultureBotAI, plus the
The suite has thirteen Mechs, all under https://github.com/CultureBotAI, plus the
orchestrator `culturebotai-claw`. Three have pages here; the others link to their
own GitHub Pages sites. DUFMech's card links its family browser and seed
own GitHub Pages sites, except CMMMech, which links its repository records. DUFMech's card links its family browser and seed
snapshot, the source of its count. Its native FamilyRecord corpus now has a
LinkML schema, history and individual pages; the census still measures the
frozen seed worklist. CLAW membership is read from `_fleet/data/manifest.json`, independently of
Expand All @@ -104,17 +104,21 @@ Cells without a published record route link the pinned source file, including
CommunityMech isolates. Complete cell lists remain separate from graph
`indexed_voc` and do not change overlap methodology. Rows sort by vocabulary
count, descending, with ties in biological scale order; DOI and PMID remain
first, followed by columns in descending Mech coverage.
first, followed by columns in descending dataset coverage. The heatmap also
compares kg-microbe's pinned 2025-03-07 core release node table, always last
with a small gap above it, counted with
`build_kg_microbe.py`. Its source manifest, census and cell assets are separate
from Mech membership, scientific-record totals and overlap indexes.

Sources, refreshed differently:
- **Fleet membership and capabilities:** the membership badges and capability
table come from a snapshot of culturebotai-claw's fleet manifest,
`_fleet/data/manifest.json`, refreshed with
`scripts/fleet/refresh_manifest.py --claw-root <claw checkout>`, separately
from the census. Its tooltips quote claw's reasons verbatim. The site lists
twelve Mechs; the membership count and undeclared-project prose are derived
thirteen Mechs; the membership count and undeclared-project prose are derived
from the snapshot, not maintained as a separate count.
- **All twelve corpora:** PathwayMech records and DUFMech seed families use the
- **All thirteen corpora:** PathwayMech records and DUFMech seed families use the
common census, subset, statistics and site-audit pipeline. DUFMech's frozen
JSON worklist is validated and projected into individual family documents;
its numeric card counts seed families, not curated mechanism records.
Expand All @@ -123,7 +127,7 @@ Sources, refreshed differently:
vocabulary and merged-PR tiles, regenerated by the pipeline: record counts and
vocabularies over snapshots of each Mech pinned at its GitHub `main`, merged-PR
counts from GitHub search when `mech_stats.py` runs. The headline PR total
includes all twelve Mechs plus CLAW and this website; the two supporting
includes all thirteen Mechs plus CLAW and this website; the two supporting
repositories are stored separately from the Mech record statistics. The census,
the subsets, the cell lists, the stats and `_fleet/data/site_audit.json` record
the revisions they read, and
Expand All @@ -147,7 +151,9 @@ Sources, refreshed differently:
date and other sites unchanged. Schema-documentation prose reflects the
website verdict, never an inference about whether a repository has a schema.

Last full data refresh was 2026-10-07, pinned at 19:36 UTC. All twelve Mechs
The original twelve source revisions remain from the 2026-10-07 refresh, pinned
at 19:36 UTC. CMMMech was added at its October 8 revision, with a separate
per-repository pin time. All thirteen Mechs
participate in the census, overlap assets and repository statistics, including
PathwayMech's 156 records and DUFMech's 6,532 seed families. TraitMech has 1,059
records and CellStructureMech 906 at those pins. TaxonMech joined the census in
Expand All @@ -161,7 +167,7 @@ neither refresh is permission to redate or recount the other inputs.
The suite licensing policy is CC BY 4.0 for data and BSD-3-Clause for code.
Redistributed source material retains its applicable terms.

### The twelve Mechs
### The thirteen Mechs
- **HabitatMech** (https://culturebotai.github.io/HabitatMech/) - habitats harmonized from GOLD, BacDive, PREGO and Madin et al. into ontology-grounded records (ENVO, UBERON, FOODON, BTO, PO).
- **CommunityMech** (`/communitymech/`) - microbial communities, their interactions and cultivation conditions, in LinkML.
- **TaxonMech** (https://culturebotai.github.io/TaxonMech/) - taxa and strains at species level and below, resolving NCBI Taxonomy, GTDB, LPSN and BacDive onto one NCBI-grounded identity.
Expand All @@ -172,11 +178,12 @@ Redistributed source material retains its applicable terms.
- **DUFMech** (https://culturebotai.github.io/DUFMech/, repository https://github.com/CultureBotAI/DUFMech) - domains and protein families of unknown function, with evidence gathering and triage worklists.
- **NaturalProductMech** (https://culturebotai.github.io/NaturalProductMech/) - one record per natural product structure with producer, gene cluster and bioactivity; grounded in ChEBI, MIBiG and NCBI Taxonomy. NPAtlas is a cross-reference target only: its CC BY-NC licence (from release 2024_09) bars ingestion into a CC BY 4.0 corpus.
- **AntibioticMech** (https://culturebotai.github.io/AntibioticMech/) - one record per antimicrobial structure, harmonizing ChEBI and CARD/ARO; record content is CC BY 4.0.
- **CMMMech** (https://github.com/CultureBotAI/CMMMech) - critical-material microbial extraction, transformation and recovery, with three element-level pilot records, ChEBI identities, taxon references and evidence. No published website; card and record links point to the repository.
- **MediaIngredientMech** (`/mediaingredientmech/`) - LLM-assisted curation of media-ingredient mappings; its records map mostly to ChEBI, then MeSH, NCIT, MicrO, FOODON and ENVO. Where curation found no ontology term, about 175 mapped ingredients carry kg-microbe registry ids, mostly for preparations and mixtures, and about 85 carry CAS Registry Numbers; about 55 more carry kg-microbe placeholder ids pending curation. None map to PubChem or METPO (#150, #186, #215).
- **CultureMech** (`/culturemech/`) - versioned, ontology-grounded culture-media recipes from MediaDive, TogoMedium, KOMODO and the major collections.

Wherever the suite is enumerated (index.md, resources.md, each page's Related
Tools), list all twelve and link to `/mechs/`. Name ontologies consistently across
Tools), list all thirteen and link to `/mechs/`. Name ontologies consistently across
pages (ChEBI, NCBITaxon, METPO, ENVO and so on), and describe a Mech's groundings
by what its records actually use, checked at its pinned revision.

Expand Down
72 changes: 67 additions & 5 deletions _fleet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,14 @@

## Membership and capability updates

The site lists twelve Mechs. Membership counts, badges and prose about
The site lists thirteen Mechs. Membership counts, badges and prose about
undeclared projects come from `data/manifest.json`; suite coverage does not
imply CLAW admission. A project's capability row remains undeclared until the
snapshot contains its declaration. Tests exercise both an eleven-member
fixture without DUFMech and a twelve-member fixture with synthetic DUFMech
declarations, independently of the current snapshot. The October 7 vocabulary
census, shared-term assets and repository statistics cover all twelve Mechs.
snapshot contains its declaration. Tests exercise both a twelve-member
fixture without DUFMech and a thirteen-member fixture with synthetic DUFMech
declarations, independently of the current snapshot. The October 8 addition
extends the census, shared-term assets and statistics to CMMMech, retaining
the original twelve Mech source revisions from October 7.
DUFMech's contribution to that census is a measured seed worklist, labeled as
families rather than curated mechanism records. Its schema-documentation prose
uses the website feature verdict, not membership or an inferred schema absence.
Expand Down Expand Up @@ -443,3 +444,64 @@ The DUF nightly card check resolves the latest dated worklist on `main` through
GitHub directory metadata. Its pinned audit still records the frozen source
used for this page, so a new worklist cannot leave the live freshness check
reading the old snapshot indefinitely.


## CMMMech and the kg-microbe comparison (October 8, 2026)

CMMMech joins every Mech surface: ladder, card, graph node, vocabulary census,
shared-term edges, statistics and both support tables. The canonical corpus is
`data/records/**/*.yaml` and `*.yml`, matching its native CLI and CLAW manifest;
fixtures, reviews and history sidecars are not additional records. Its three
records at `905388523b6e9184783989484896e4cc20297832` have no REVIEWED-bearing
status slot. A null statistic makes no claim about scientific review artifacts.
There is no published website, so links use the pinned GitHub records and the
website-feature row explicitly says so. Its card checker counts a complete Git
tree inventory, rejects truncated API responses, and compares canonical paths
and blob hashes with `git ls-tree` at the pin. It never treats a README's
hand-maintained list as the live corpus size.

The original twelve Mech source revisions are retained. All thirteen corpora
were rescanned for this addition: CELEX and registry aliases observed in the
kg-microbe node table apply consistently to every row. CMMMech carries its own
`pinned_at_utc` in the audit and snapshot manifest; the original twelve keep
their October 7 pin time and nightly growth allowance. Merged-PR activity was
queried again. The existing website feature audits retain their own dates.

kg-microbe is a **heatmap comparison**, not a fourteenth Mech. It never joins
`roots.ORDER`, the Mech statistics, cards, capability table or shared-term edge
index. `_fleet/kg_microbe_source.json` pins the published **2025-03-07 core**
release asset by SHA-256, not current main or a locally rebuilt graph. Reproduce
its counts with:

```bash
python3 scripts/fleet/build_kg_microbe.py /path/to/kg-microbe-core.tar.gz
python3 scripts/fleet/build_data.py
python3 scripts/fleet/assemble_page.py
```

The script verifies the archive hash and reads the nested `merged-kg.tar.gz`
without extracting archive paths. It records the node-table hash and treats
one KGX node as one record. Counts scan all node fields (including xrefs and
prose) with the same prefix regex and normalization registry as Mech records;
repeat identifiers are repeat occurrences. Matching-node totals count each
node once per vocabulary. Edge rows, graph-local identifiers, metamodel terms
and namespaces outside the suite registry are excluded. This is coverage of
the tracked namespaces, not an exhaustive inventory of graph namespaces or a
count of distinct ontology terms. The release has 1,379,336 node records;
headers and blank lines are not nodes.

`data/kg_microbe_census.json` records the counts and source, independently of
Mech commit pins. `assets/fleet/comparison-cells/` contains up to 300 node links
per populated cell with total counts and hashes, following the same cell format
as Mech records. Links open external term pages where available, otherwise the
source release with the node identifier shown in the label. Mech cell rebuilds
own only `assets/fleet/cells/` and cannot delete comparison assets.

`build_data.py` merges the comparison into `heat`, `voc` and `cells`, keeping
`order`, `indexed_voc` and `vocab_edges` exclusively Mech-derived. The embedded
`comparisons` metadata supplies the row identity and source note. Columns sort
by dataset coverage after DOI/PMID; Mech rows sort by tracked vocabulary count, with
biological-order ties. kg-microbe is always the final row, with a small gap
above it. Tests check
source hashes, complete cell coverage, matching counts and isolation from
fleet membership and graph indexes.
Loading
Loading