Skip to content

feat(kb): consume the Encyclopedia Machine KB (machine-kb-v0.1.0) - #10

Merged
GreenGrassBlueOcean merged 3 commits into
mainfrom
feat/machine-kb-integration
Sep 28, 2026
Merged

GreenGrassBlueOcean merged 3 commits into
mainfrom
feat/machine-kb-integration

Conversation

@GreenGrassBlueOcean

@GreenGrassBlueOcean GreenGrassBlueOcean commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a consumer for the OpenWebNet Encyclopedia Machine KB (machine-kb-v0.1.0), as offered in OpenWebNet-Encyclopedia discussion #36.

New tools

  • search_knowledge(query, kind, area, epistemic_status, limit): BM25 over the 1,173 retrieval chunks and 7,449 claims.
  • get_knowledge_record(record_id): resolves any ownkb: stable ID, with cautions and open questions filled in.
  • get_knowledge_status(): shows the snapshot (location, generator and schema versions, whether the hashes verified, counts).

New resource: kb://manifest. rescan_documentation now reloads the KB.

Flow

 tool call (search_knowledge / get_knowledge_record / get_knowledge_status)
                    |
                    v
        _get_kb()  -- loaded? --yes--------------------------------+
                    | no                                           |
                    v                                              |
      locate the knowledge/ dir                                    |
        1. OPENWEBNET_KB_PATH                                      |
        2. download cache (newest tag, hash-verified)              |
        3. sibling OpenWebNet-Encyclopedia checkout                |
                    |                                              |
          found? --no--> error: how to fetch (retried next call)   |
                    | yes                                          |
                    v                                              |
      read manifest.json                                           |
        schema_compatibility_version 0.1.x? --no--> refuse         |
                    |                                              |
                    v                                              |
      for each manifest artifact (path inside knowledge/)          |
        SHA-256 matches? --no--> warn  (OPENWEBNET_KB_STRICT=1:    |
                    |                   refuse)                    |
                    v                                              |
      index chunks + claims (BM25), key by ownkb: id,              |
      registries, id-registry aliases / retirements                |
                    |                                              |
                    v   (built aside, swapped in when complete)    |
                   KB <--------------------------------------------+
                    |
        +-----------+---------------------------+
        v                                       v
  search(query, filters)                 lookup(id)
    tokens + WHO/WHAT/DIMENSION            alias? -> canonical id
    exact-reference bonus                  retired? -> replacement
    rank (lexical, not evidence)                 |
        |                                        |
        +------------------+---------------------+
                           v
        format: epistemic status, confidence, applicability,
        provenance, cautions, open questions, claim links
        (#37 truncated statements point to the intact chunk)

 offline, once per release:
   kb_fetch [TAG] -> manifest -> download each artifact -> verify SHA-256
                  -> write to cache only if ALL verify

How it follows knowledge/CONSUMING.md

  • It reads the manifest first and refuses an unsupported schema_compatibility_version.
  • It checks the SHA-256 of every data artifact. A mismatch is logged and shown in the status; OPENWEBNET_KB_STRICT=1 refuses to load instead.
  • Every result keeps its qualifications: epistemic status, confidence, applicability (including the version expression and reason), namespace context, provenance, cautions, open questions and claim links.
  • unresolved, inferred, rejected and superseded records are labelled as not established. Rank is labelled as lexical relevance, not evidence strength.
  • An empty result says that the absence of a record is not a negative assertion.

Finding the KB

The KB is looked up in this order: OPENWEBNET_KB_PATH, then the hash-verified download cache (newest release), then a sibling OpenWebNet-Encyclopedia/knowledge checkout (an unverified working tree). The default uvx install ships without the corpus: run openwebnet-mcp-kb-fetch [TAG] (or python -m openwebnet_mcp.kb_fetch [TAG]) to fill the cache; it writes nothing unless every hash verifies, validates the tag and manifest paths, and the tools pick it up on the next call. A failed load is retried on every call. Without a KB, the three tools return an error and nothing else changes.

Contract details

  • Artifacts are loaded from the manifest paths (escapes outside knowledge/ are rejected), not a hardcoded layout.
  • id-registry.json is honoured: aliases resolve, deprecated and retired IDs are annotated, retired IDs show their replacement.
  • A reload builds the new index off to the side and swaps it in, so a rescan never exposes an empty KB.
  • Exact WHO n / WHAT n / DIMENSION n references in a query earn a flat ranking bonus.
  • prompt_boost points agents at the KB tools.

Known 0.1.0 limits (shown, not hidden)

  • 1,120 claim statements contain leaked parser output (OpenWebNet-Encyclopedia#37). They are cut cleanly and point to their intact chunk.
  • DALI / WHO 24 is not in this corpus. The status says so, keyed to the 0.1.0 content hash.

Also fixed here

  • who_catalog.json WHO 1 WHAT 11-15 were wrong; they are timed ON for 1-5 minutes. Added 16, 17 (unresolved), 18, 20-29 blink and 30/31 level steps, as documented in the KB.

Testing

  • 106 tests pass and 1 is skipped by default, 95% coverage (CI requires 95%).
  • tests/test_kb.py builds a synthetic KB with real hashes. It covers loading, search filters and the exact-reference boost, hash mismatch and strict mode, version rejection, a missing KB and retry after a miss, id-registry aliases and lifecycle, manifest-path containment, the #37 truncation, kb_fetch verification and path/tag validation, cache release ordering and every server tool.
  • An opt-in live test (OPENWEBNET_LIVE_TESTS=1 pytest -m network) fetches the real machine-kb-v0.1.0, verifies every hash, loads in strict mode and searches. It passes.
  • Ruff was not run (it isn't installed in the dev venv).

🤖 Generated with Claude Code

GreenGrassBlueOcean and others added 2 commits September 28, 2026 22:58
Add search_knowledge, get_knowledge_record, get_knowledge_status and the
kb://manifest resource, backed by the OpenWebNet Encyclopedia Machine KB.

Loading follows knowledge/CONSUMING.md: manifest first, schema
compatibility check, SHA-256 verification of every data artifact, then
BM25 over retrieval chunks and claims with all records keyed by stable ID.
Every result keeps epistemic status, confidence, applicability (including
version expression), namespace context, provenance, and hydrated cautions
and open questions. Claim statements hit by Encyclopedia#37 are truncated
cleanly and point to their intact source chunk.

The KB is found via OPENWEBNET_KB_PATH, a sibling Encyclopedia checkout,
or a cache filled by `python -m openwebnet_mcp.kb_fetch`, which verifies
hashes before writing. Without a KB the new tools return an error and the
rest of the server is unaffected. rescan_documentation reloads it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontainment)

- Retry KB load on every call until it succeeds (a miss was sticky), and
  ship an openwebnet-mcp-kb-fetch console script for uvx users
- Honour id-registry.json aliases, deprecations and retirements
- Load artifacts from manifest paths instead of a hardcoded layout
- Build the index off to the side and swap it in (no empty window on rescan)
- Prefer the hash-verified cache over a sibling checkout
- Validate fetch tag and manifest paths (no writes outside knowledge/)
- Also truncate the JSON-style {"text": leak; mention KB tools in prompt_boost
- Lock the real DALI known-gap hash in a test

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@GreenGrassBlueOcean

GreenGrassBlueOcean commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Follow-up improvements to #10 (eb7cefb)

Pushed a second commit that hardens the Machine KB integration. Summary of what changed and why:

Easier to adopt

  • Retry on every call. A missing KB is no longer remembered. Run kb_fetch (or set OPENWEBNET_KB_PATH) and the next tool call picks it up, with no rescan or restart.
  • openwebnet-mcp-kb-fetch console script, so uvx users can fetch the corpus without python -m. The README now says plainly that the default install ships without the corpus.
  • Better lookup order: OPENWEBNET_KB_PATH → hash-verified cache → sibling checkout. A verified release now wins over an unverified working tree.

More faithful to CONSUMING.md

  • id-registry.json is honoured. get_knowledge_record follows aliases and shows deprecated/retired notes and replacements, so IDs stay stable across future tags.
  • Manifest paths drive loading instead of a hardcoded folder layout, and paths that escape knowledge/ are rejected.

More robust

  • Atomic reload. The new index is built off to the side and swapped in, so a rescan never exposes an empty KB to a concurrent search.
  • Safer kb_fetch. The tag must be a single safe path segment and manifest paths are validated before anything is downloaded or written.
  • The {"text": variant of the #37 leak is truncated too.

Better agent discovery

  • prompt_boost now points agents at search_knowledge / get_knowledge_record, with a reminder that unresolved, inferred, rejected and superseded records are not established facts.

Verification

104 tests at the time of eb7cefb, 95% coverage (up from 98 tests). New tests cover retry after a miss, id-registry aliases and lifecycle, moved manifest layouts, path containment, tag validation and the pinned DALI known-gap hash. Also checked against a hash-verified export of machine-kb-v0.1.0 (1,173 chunks, 7,449 claims).

Also in c2bca26

  • Exact-reference search boost: WHO n / WHAT n / DIMENSION n in a query now outranks records that merely contain the number (WHO 1 WHAT 17 finds the WHAT 17 duration claims first).
  • who_catalog.json corrected: WHO 1 WHAT 11-15 are timed ON (1-5 min), not dim up/down, toggle, timed and blink. Also added 16, 17 (unresolved), 18, 20-29 blink and 30/31 level steps, matching the Machine KB.
  • Live kb_fetch test: opt-in (OPENWEBNET_LIVE_TESTS=1 pytest -m network); it fetches machine-kb-v0.1.0, verifies every hash, loads in strict mode and searches. It passed against the real release.

107 tests with the live test enabled (106 plus 1 skipped by default), 95% coverage.

…kb_fetch test

- search: "WHO n" / "WHAT n" / "DIMENSION n" (also "dim n") in a query now
  earns a flat bonus on records that contain the same reference (WHO also
  from who-N- source paths), so "WHO 1 WHAT 17" no longer loses to any
  document that merely contains a 17
- who_catalog.json WHO 1: WHAT 11-15 are timed ON (1-5 min), not dim
  up/down, toggle, timed, blink; add 16, 17 (unresolved), 18, 20 blink
  (20..29) and 30/31 level steps, as documented in the Machine KB
- opt-in live test of kb_fetch against machine-kb-v0.1.0
  (OPENWEBNET_LIVE_TESTS=1, marker "network")

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@GreenGrassBlueOcean
GreenGrassBlueOcean merged commit 8c1891b into main Sep 28, 2026
16 checks passed
@GreenGrassBlueOcean
GreenGrassBlueOcean deleted the feat/machine-kb-integration branch September 28, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant