Gate the reverse lot index exactly as /traceforward is gated

CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## [Unreleased]
+
+### Changed
+- **The reverse lot index carries the same gate as trace-forward**: `GET /list-artifact/reverse/{geoid}`
+  answered "which lots is this field in" for any hub login, with no grant and no audit entry;
+  measured live, that was five of the six lots the gated `/traceforward` returns (review
+  ledger AG-015). It now requires a grant for the seed or an authority credential with a
+  scope, exactly as `/traceforward` does, and an authority answer is written to the MEAL
+  ledger before it is returned. Callers holding only a login receive 403. Pancake's audit
+  route, which calls this index, already forwards the caller's authority token.
+- `docs/MCP_SURFACE.md` gating note for `get_list_artifact_reverse`; the MCP tool takes
+  `grant_token`, `authority_token` and `scope`. README endpoint table corrected: it had
+  described a gate the code did not have.
+
+### Added
+- `app/tests/testkit/grants.py`: one shared minter for field-access grants, replacing the
+  copy in `test_traceforward.py`. Six direct tests of the reverse index, including the
+  control that a bare login is refused and the proof that the same check can fire.
+
 ## [v0.9-review]
 
 ### Added

README.md

@@ -65,8 +65,10 @@ Follow these steps to run the Node locally alongside the `ar2-hub` and `Pancake`
 | `GET`  | `/geoid/{geoid}/eudr-export` | Exports an EUDR-compliant GeoJSON artifact. | Yes (Grant) |
 | `POST` | `/list-artifact` | Registers a content-derived list (a lot/case/pallet/retail item) over GeoIDs, nested lists (`L:`), or regions (`R:`). Append-only. | Yes (Hub JWT) |
 | `POST` | `/region-artifact` | Registers a content-derived region from a WKT boundary (no acreage cap) or from member GeoIDs/lists. | Yes (Hub JWT) |
-| `GET`  | `/list-artifact/{list_id}` | Returns the members of a list artifact. | Yes (Hub JWT) |
-| `GET`  | `/list-artifact/reverse/{geoid}` | Trace-forward: every list/region containing a GeoID, climbed recursively to terminal products. | Yes (Hub JWT + `trace-forward` capability + seed authorization; accredited authority credential for the identity tier) |
+| `GET`  | `/list-artifact/{list_id}` | Returns the members of a list artifact. | Yes (Hub JWT, then a grant for the list or an authority credential) |
+| `GET`  | `/list-artifact/reverse/{geoid}` | The first hop of trace-forward: the lists that name a GeoID directly. | Yes (Hub JWT, then a grant for the seed or an authority credential with `?scope=`; the same gate as `/traceforward`) |
+| `POST` | `/traceforward` | Trace-forward: every list/region containing a GeoID, climbed recursively to terminal products. | Yes (Hub JWT with the `trace-forward` capability or a grant, then a grant for the seed or an authority credential; identities only on the authority path, audited) |
+| `GET`  | `/list-artifact/{list_id}/traceback` | Trace-back: a lot walked down to its input lots and fields, one hop per lot. | Yes (Hub JWT, then a grant for the list or an authority credential) |
 
 See [TRACEABILITY.md](./TRACEABILITY.md) for how these endpoints compose into trace-back and trace-forward.
 

app/mcp_server.py

@@ -613,16 +613,21 @@ async def get_list_artifact(
             "GET /list-artifact/reverse/{geoid}. Which lots a given field "
             "appears in. The reverse index behind a recall: given a "
             "contaminated field, the lots that carried its produce. Requires a "
-            "hub token. The seed is expanded through its equivalence set "
-            "first, so a v1 Geo Id, a superseded alias and the canonical Geo Id "
-            "all return the same lots."
+            "hub token, and gated beyond that exactly like run_traceforward: "
+            "grant_token for this field, or authority_token with a scope. The "
+            "seed is expanded through its equivalence set first, so a v1 Geo "
+            "Id, a superseded alias and the canonical Geo Id all return the "
+            "same lots."
         )
     )
     async def get_list_artifact_reverse(
-        geoid: str, token: str | None = None,
+        geoid: str, grant_token: str | None = None,
+        authority_token: str | None = None, scope: str | None = None,
+        token: str | None = None,
     ) -> dict[str, Any]:
         return await _handle(
-            trace.get_list_artifact_reverse, geoid=geoid,
+            trace.get_list_artifact_reverse, geoid=geoid, scope=scope,
+            x_grant_token=grant_token, x_authority_token=authority_token,
             user=_required(token, "the reverse lot index"))
 
     # -- GET /list-artifact/{list_id}/traceback ---------------------------

app/routers/traceforward.py

@@ -230,22 +230,49 @@ class ListReverseResponse(BaseModel):
 @router.get("/list-artifact/reverse/{geoid}", response_model=ListReverseResponse)
 def get_list_artifact_reverse(
     geoid: str,
+    scope: str | None = None,
+    x_grant_token: str | None = Header(None),
+    x_authority_token: str | None = Header(None),
     user: dict = Depends(require_hub_user),
     db: Session = Depends(get_db)
 ):
     """
     Retrieves all list IDs that contain the given geoid.
 
+    Gated exactly as POST /traceforward is: a verified grant for this seed, or
+    an authority credential. This route answers the first hop of the same
+    question -- which lots is this field in -- and until 2026-10 it answered it
+    for any hub login, with no grant and no audit entry. Measured live, that
+    was five of the six lots the gated route returns (review ledger AG-015).
+    A second door to a gated answer has to carry the same gate.
+
     Expands the seed through its equivalence set first, so a v1 GeoID, a
     superseded content-hash alias and the canonical GeoID all return the same
     lists. Without this the reverse lookup silently disagrees with
     /traceforward about what counts as the same field.
     """
+    from app.auth import authorize_artifact
+    from app.meal_logger import log_traceforward
+    if x_authority_token and not scope:
+        raise HTTPException(status_code=400, detail="scope is required with an authority credential")
+    auth_result = authorize_artifact(
+        grant_token=x_grant_token,
+        authority_token=x_authority_token,
+        geoid=geoid,
+        scope=scope,
+        raise_404_on_fail=False,
+    )
+
     seeds = _equivalence_set(db, geoid)
-    list_ids = db.execute(
+    list_ids = sorted(set(db.execute(
         select(ListMemberEdge.list_id).where(ListMemberEdge.geoid.in_(seeds))
-    ).scalars().all()
-    return ListReverseResponse(list_ids=sorted(set(list_ids)))
+    ).scalars().all()))
+    if auth_result.get("used_authority"):
+        # Same ledger event as the gated route: an authority asked which lots
+        # a field is in, and got an answer. Audit before disclosure.
+        log_traceforward(user.get("sub"), auth_result.get("authority_jti"),
+                         geoid, scope, len(list_ids), list_ids)
+    return ListReverseResponse(list_ids=list_ids)
 
 
 # --------------------------------------------------------------------------

app/tests/test_geoid_v2_live.py

@@ -46,6 +46,7 @@
     ListMemberEdge,
 )
 from app.s2_services import S2Service
+from app.tests.testkit.grants import grant_header_for  # noqa: E402
 from app.utils import Utils
 
 app.dependency_overrides[require_hub_user] = lambda: {
@@ -457,7 +458,7 @@ def test_reverse_lookup_finds_a_list_through_a_v1_identifier():
     v1_id = "legacy-" + uuid.uuid4().hex[:12]
     _alias_v1(v1_id, v2_id)
 
-    r = client.get(f"/list-artifact/reverse/{v1_id}")
+    r = client.get(f"/list-artifact/reverse/{v1_id}", headers=grant_header_for(v1_id))
     assert r.status_code == 200, r.text
     assert list_id in r.json()["list_ids"], (
         "the v1 identifier must reach lists registered against its v2 identity"
@@ -501,6 +502,6 @@ def test_a_list_registered_against_a_v1_geoid_is_still_traceable():
     finally:
         db.close()
 
-    r = client.get(f"/list-artifact/reverse/{v2_id}")
+    r = client.get(f"/list-artifact/reverse/{v2_id}", headers=grant_header_for(v2_id))
     assert r.status_code == 200, r.text
     assert list_id in r.json()["list_ids"]

app/tests/test_point_polygon_peers.py

@@ -35,10 +35,18 @@
 from app import geoid_v2  # noqa: E402
 from app.auth import require_hub_user  # noqa: E402
 from app.main import app  # noqa: E402
+from app.tests.testkit.grants import grant_header_for  # noqa: E402
 
 app.dependency_overrides[require_hub_user] = lambda: {"sub": "peers@demo.com"}
 client = TestClient(app)
 
+
+@pytest.fixture(autouse=True)
+def set_test_status_list_dir(monkeypatch):
+    # The reverse lot index takes a grant, and a grant's status list is
+    # resolved locally under test rather than fetched from Pancake.
+    monkeypatch.setenv("TEST_STATUS_LIST_DIR", TESTKIT_DIR)
+
 # Two plots in the Honduran coffee belt, far enough apart that neither resolves
 # onto the other and neither collides with anything registered by another test.
 POINT_LAT, POINT_LNG = 14.812345, -88.312345
@@ -245,7 +253,7 @@ def test_both_kinds_join_one_list_and_prove_membership(plots):
     assert sorted(fetched.json()["members"]) == members
 
     for kind, geo_id in plots.items():
-        reverse = client.get(f"/list-artifact/reverse/{geo_id}")
+        reverse = client.get(f"/list-artifact/reverse/{geo_id}", headers=grant_header_for(geo_id))
         assert reverse.status_code == 200, kind
         assert list_id in reverse.json()["list_ids"], f"{kind} is not findable in its own list"
 

app/tests/test_traceforward.py

@@ -1,4 +1,3 @@
-import json
 import os
 import uuid
 
@@ -221,35 +220,7 @@ def _hub(capabilities):
     app.dependency_overrides[require_hub_user] = lambda: {
         "sub": "authority@demo.agstack.org", "capabilities": capabilities}
 
-def valid_grant_for(geoid: str) -> str:
-    import base64
-    import hashlib
-    import secrets
-    import time
-
-    import jwt
-
-    def _b64url(data: bytes) -> str:
-        return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
-
-    private_pem = (Path(__file__).parent / "testkit" / "dev_keys" / "dev_issuer_private.pem").read_bytes()
-
-    salt = _b64url(secrets.token_bytes(16))
-    encoded_disclosure = _b64url(json.dumps([salt, "fields.0", geoid]).encode("utf-8"))
-    digest = _b64url(hashlib.sha256(encoded_disclosure.encode("ascii")).digest())
-
-    claims = {
-        "iss": "did:web:pancake.test",
-        "sub": "owner@demo.com",
-        "iat": int(time.time()),
-        "exp": int(time.time()) + 3600,
-        "vct": "agstack.org/credentials/field-access-grant/v1",
-        "status": {"status_list": {"uri": "http://localhost:8100/grants/status-list", "idx": 1}},
-        "_sd": [digest],
-        "_sd_alg": "sha-256"
-    }
-    token = jwt.encode(claims, private_pem, algorithm="EdDSA", headers={"typ": "vc+sd-jwt", "kid": "pancake-test-1"})
-    return f"{token}~{encoded_disclosure}~"
+from app.tests.testkit.grants import valid_grant_for  # noqa: E402
 
 # 1. Gate A: no capability -> 403
 def test_gate_a_missing_capability_403():
@@ -373,3 +344,92 @@ def test_authority_round_trip():
             fwd = client.post("/traceforward", json={"seed_geoid": geoids[0], "scope": "demo-recall"}, headers=auth)
             assert fwd.status_code == 200
             assert retail_list_id in [m["list_id"] for m in fwd.json()["matches"]]
+
+
+# ---------------------------------------------------------------------------
+# The second door. GET /list-artifact/reverse/{geoid} answers the first hop of
+# the same question /traceforward answers, and until 2026-10 it answered it
+# for any hub login with no grant and no audit entry (review ledger AG-015:
+# five of the six lots the gated route returned, live). These tests hold the
+# gate on the second door to the same shape as the first.
+# ---------------------------------------------------------------------------
+
+def _lot_of(geoid: str) -> str:
+    r = client.post("/list-artifact", json={"members": [geoid]})
+    assert r.status_code == 200, r.text
+    return r.json()["list_id"]
+
+
+def test_reverse_refuses_a_bare_login():
+    """The control: a login with every capability and no credential gets nothing."""
+    _hub(["trace-forward"])
+    seed = create_test_geoid(["111"])
+    lot = _lot_of(seed)
+    r = client.get(f"/list-artifact/reverse/{seed}")
+    assert r.status_code == 403, r.text
+    assert lot not in r.text
+
+
+def test_reverse_refuses_a_grant_for_a_different_field():
+    _hub(["trace-forward"])
+    seed = create_test_geoid(["111"])
+    other = create_test_geoid(["222"])
+    _lot_of(seed)
+    r = client.get(f"/list-artifact/reverse/{seed}",
+                   headers={"X-Grant-Token": valid_grant_for(other)})
+    assert r.status_code == 403
+
+
+def test_reverse_answers_a_grant_for_the_seed_and_the_check_can_fire():
+    """The same lot the control must not reveal is revealed here, so the
+    control is shown to be testing a real answer rather than an empty index."""
+    _hub([])  # no capability at all: the grant is sufficient, as on /traceforward
+    seed = create_test_geoid(["111"])
+    lot = _lot_of(seed)
+    r = client.get(f"/list-artifact/reverse/{seed}",
+                   headers={"X-Grant-Token": valid_grant_for(seed)})
+    assert r.status_code == 200, r.text
+    assert r.json()["list_ids"] == [lot]
+
+
+def test_reverse_with_an_authority_credential_needs_a_scope_and_is_audited():
+    _hub(["trace-forward"])
+    seed = create_test_geoid(["111"])
+    lot = _lot_of(seed)
+    auth = {"X-Authority-Token": _read("valid_authority")}
+
+    no_scope = client.get(f"/list-artifact/reverse/{seed}", headers=auth)
+    assert no_scope.status_code == 400
+
+    packets = []
+    with patch("app.meal_logger._append_to_meal_chain", side_effect=packets.append):
+        r = client.get(f"/list-artifact/reverse/{seed}", params={"scope": "demo-recall"}, headers=auth)
+    assert r.status_code == 200, r.text
+    assert r.json()["list_ids"] == [lot]
+    assert packets and packets[-1]["seed_geoid"] == seed, "an authority answer must leave a MEAL entry"
+
+
+def test_reverse_with_a_grant_is_not_audited():
+    """Tier 1 on /traceforward is not logged; the second door matches it."""
+    _hub([])
+    seed = create_test_geoid(["111"])
+    _lot_of(seed)
+    packets = []
+    with patch("app.meal_logger._append_to_meal_chain", side_effect=packets.append):
+        r = client.get(f"/list-artifact/reverse/{seed}",
+                       headers={"X-Grant-Token": valid_grant_for(seed)})
+    assert r.status_code == 200
+    assert packets == []
+
+
+def test_reverse_and_traceforward_agree_on_the_first_hop():
+    """Same seed, same grant: the second door's answer is contained in the first's."""
+    _hub([])
+    seed = create_test_geoid(["111"])
+    lot = _lot_of(seed)
+    parent = client.post("/list-artifact", json={"members": [f"L:{lot}"]}).json()["list_id"]
+    g = {"X-Grant-Token": valid_grant_for(seed)}
+    rev = set(client.get(f"/list-artifact/reverse/{seed}", headers=g).json()["list_ids"])
+    fwd = set(client.post("/traceforward", json={"seed_geoid": seed}, headers=g).json()["list_ids"])
+    assert rev == {lot}
+    assert rev < fwd and parent in fwd

app/tests/testkit/grants.py

@@ -0,0 +1,52 @@
+"""Mint a field-access grant for one GeoID with the dev issuer key.
+
+Shared by every suite that has to pass the grant gate on a trace route. Lives
+here rather than in one test module so that a second copy cannot drift from
+the first; the credential shape is the one Pancake's testkit mints
+(vct agstack.org/credentials/field-access-grant/v1, one selective disclosure
+naming the field, a status-list reference the tests resolve locally).
+"""
+from __future__ import annotations
+
+import base64
+import hashlib
+import json
+import secrets
+import time
+from pathlib import Path
+
+import jwt
+
+DEV_KEYS = Path(__file__).parent / "dev_keys"
+GRANT_VCT = "agstack.org/credentials/field-access-grant/v1"
+STATUS_URI = "http://localhost:8100/grants/status-list"
+
+
+def _b64url(data: bytes) -> str:
+    return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
+
+
+def valid_grant_for(geoid: str, *, ttl_seconds: int = 3600) -> str:
+    """An SD-JWT grant whose single disclosure names ``geoid``."""
+    private_pem = (DEV_KEYS / "dev_issuer_private.pem").read_bytes()
+    salt = _b64url(secrets.token_bytes(16))
+    disclosure = _b64url(json.dumps([salt, "fields.0", geoid]).encode("utf-8"))
+    digest = _b64url(hashlib.sha256(disclosure.encode("ascii")).digest())
+    now = int(time.time())
+    claims = {
+        "iss": "did:web:pancake.test",
+        "sub": "owner@demo.com",
+        "iat": now,
+        "exp": now + ttl_seconds,
+        "vct": GRANT_VCT,
+        "status": {"status_list": {"uri": STATUS_URI, "idx": 1}},
+        "_sd": [digest],
+        "_sd_alg": "sha-256",
+    }
+    token = jwt.encode(claims, private_pem, algorithm="EdDSA",
+                       headers={"typ": "vc+sd-jwt", "kid": "pancake-test-1"})
+    return f"{token}~{disclosure}~"
+
+
+def grant_header_for(geoid: str) -> dict[str, str]:
+    return {"X-Grant-Token": valid_grant_for(geoid)}

docs/MCP_SURFACE.md

@@ -25,7 +25,7 @@ Endpoint: `POST /mcp`, streamable HTTP, bearer token from the hub. Discovery at
 | `GET /geoid/{geo_id}/eudr-export` | `eudr_export` | optional | refuses without | The only read route that refuses without a grant rather than answering coarsely, because a masked cell is not a filing. |
 | `POST /get-percentage-overlap-two-fields` | `get_percentage_overlap_two_fields` | none | -- | Answers whether two Geo Ids name the same ground while disclosing only the one number. |
 | `POST /list-artifact` | `register_list_artifact` | hub token | -- | FSMA 204 lot. `location_geo_id` is the traceability lot code source (21 CFR 1.1330(a)(14)); omitting it records a gap rather than failing. |
-| `GET /list-artifact/reverse/{geoid}` | `get_list_artifact_reverse` | hub token | -- |  |
+| `GET /list-artifact/reverse/{geoid}` | `get_list_artifact_reverse` | hub token | -- | Gated twice, like run_traceforward: a hub token to call at all, then a grant for this field or an authority credential with a scope. The first hop of trace-forward, carrying the same gate. |
 | `GET /list-artifact/{list_id}` | `get_list_artifact` | hub token | -- | Gated twice: a hub token to call at all, then a grant or authority credential to see the members. |
 | `GET /list-artifact/{list_id}/traceback` | `trace_back` | hub token | -- | One hop is one lot, which is the unit 21 CFR 1.1320(a) records. |
 | `POST /populate-country-in-geo-ids` | `populate_country_in_geo_ids` | none | -- | Maintenance. Walks the whole table; not for answering a question. |

scripts/mcp_surface.py

@@ -84,6 +84,10 @@
     "get_list_artifact":
         "Gated twice: a hub token to call at all, then a grant or authority "
         "credential to see the members.",
+    "get_list_artifact_reverse":
+        "Gated twice, like run_traceforward: a hub token to call at all, then a "
+        "grant for this field or an authority credential with a scope. The first "
+        "hop of trace-forward, carrying the same gate.",
     "trace_back":
         "One hop is one lot, which is the unit 21 CFR 1.1320(a) records.",
     "run_traceforward":