fix(search): fan out only over values the leaf can read

[ddeboer]

Follow-up to #801, found while adapting Linked Open Limburg’s schema to ADR 26 against its real data.

The defect

A role is sometimes stated twice on one node – once as a string, once as a Wikidata entity. Beeld & Geluid does this, and LOL’s own schema comment noted the IRI “costs nothing and takes nothing away”, which was true while the entry held a list.

Under fan-out it stopped being true. tuplesOf split on every framed value, but a keyword leaf reads literals and passes over an IRI – so the IRI minted a second entry the role was absent from:

// role: ["fotograaf", <wikidata:Q33231>], one agent
[ { "role": "fotograaf", "creator": {…}, "creator_id": "…/a/1" },
  {                      "creator": {…}, "creator_id": "…/a/1" } ]

The same endpoint, apparently in no role at all – shown to a reader, and matched by a filter for that agent. Not a crash and not a hang: a phantom entry that reads as data.

Two things the shape happens to get right, worth stating so the fix is not over-drawn: keyword does drop the IRI, so no raw http://… is rendered as a role label, and applyIdentityCompanion dedupes, so the agent’s id appears once rather than twice.

The fix

The split follows the reader. splittable() narrows a leaf’s framed values to those its kind can read before the product is taken.

Only keyword needs it:

• text has no filter operator (filterOperatorFor maps it to undefined), so nothing welds it and it is never a tuple position;
• reference is one, and both of its shapes are meaningful – an IRI, and, for a local lookup, a node the graph named inline, which has no IRI and is a referent all the same.

Notes

• This is the ninth defect the ADR 26 work has turned up, and the first found by pointing it at a real downstream schema rather than by review. The pattern from the earlier rounds holds: the fan-out itself is right, and what goes wrong is a step around it disagreeing with what the projection actually does.
• Downstream: LOL’s schema change (codeberg limburg/lol, its issue 162) is what surfaced this. It needs @lde/search@0.25.0 plus this fix.

[ddeboer]

Closing: this guards a shape the data does not have.

The phantom entry needs two or more values on one weldable leaf, at least one of which the leaf cannot read – a role stated as a literal and as an IRI on the same node. I took that case from a comment in Linked Open Limburg's schema, which says Beeld & Geluid "sometimes states a Wikidata IRI beside the string". Its own documentation says the opposite – "sometimes a Wikidata IRI rather than a string" – so I queried the source rather than pick one.

Against https://cat.apis.beeldengeluid.nl/sparql:

SELECT ?r ?lit ?iri WHERE {
  ?r schema:roleName ?lit , ?iri .
  FILTER(isLiteral(?lit) && isIRI(?iri))
} LIMIT 3

returns nothing, and a sample of roleName values is IRIs throughout:

http://www.wikidata.org/entity/Q36834
http://www.wikidata.org/entity/Q27939
https://data.muziekschatten.nl/som/um1836
…

One value per node means no split, so no entry is minted that the leaf is absent from. The fix would change no document in the register.

It remains true that schema.org gives roleName a range of Text or URL, so a publisher stating both is conforming and would hit this. If one ever appears, the change is eight lines and this branch has them. Until then it is a guard with no subject, and the projection is easier to reason about without a per-kind exception in the fan-out.

The measurement did turn up something worth acting on, in the other direction. B&G's roles being IRIs only means a keyword leaf reads no value from any of them: those roles are not degraded in the index, they are absent. The schema comment that sent me here – "an IRI a keyword field reads no value from, so it costs nothing and takes nothing away" – is wrong about the cost. Modelling the role as a term, so the IRI becomes concept identity rather than something discarded, is the fix; tracked downstream.