From bb8bc5603bd08834492df177909a4a57cbb8f97c Mon Sep 17 00:00:00 2001 From: David de Boer Date: Thu, 10 Sep 2026 07:25:57 +0200 Subject: [PATCH 1/3] feat(search)!: resolve a reference across several targets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A lookup’s target (and an idOnly’s labelSource) may name several Root Types, in order of precedence. Each referent resolves against every named collection and is read through the declaration of the one that holds it – label, key and facet policy per value – and a stored referent carries an rdf:type-derived discriminator so an unidentified one is still served as the kind it was stated to be. A GraphQL surface emits an interface named for the set, implemented by each target’s own reference type, so __typename is truthful per referent; the field’s filter is named for the set too. joinable is refused on such a lookup. BREAKING CHANGE: labelSourceNameOf, labelTargetNameOf, localLookupTypeOf and inheritedFacetKeys are replaced by labelSourceNamesOf, labelTargetNamesOf, localLookupTargetsOf and inheritedFacetPolicies, which return lists (or a map) rather than a single value; _target is a reserved field name. --- ...olve-a-reference-across-several-targets.md | 201 ++++++ docs/reference/search-api-graphql.md | 22 +- docs/reference/search-pipeline.md | 7 +- docs/reference/search-typesense.md | 18 +- docs/reference/search.md | 59 +- .../search-api-graphql/src/build-schema.ts | 248 ++++++- packages/search-api-graphql/src/projection.ts | 44 +- .../test/polymorphic-lookup.test.ts | 360 ++++++++++ packages/search-api-graphql/vite.config.ts | 2 +- packages/search-pipeline/src/extraction.ts | 146 ++-- .../search-pipeline/test/extraction.test.ts | 36 +- .../test/polymorphic-extraction.test.ts | 151 +++++ .../src/collection-definition.ts | 135 +++- packages/search-typesense/src/lookup.ts | 100 ++- .../search-typesense/src/query-compiler.ts | 18 +- packages/search-typesense/src/search.ts | 232 ++++--- .../test/polymorphic-lookup.test.ts | 394 +++++++++++ .../test/qualified-relation.test.ts | 10 +- packages/search/src/adapter.ts | 9 +- packages/search/src/engine.ts | 18 + packages/search/src/index.ts | 1 + packages/search/src/join-graph.ts | 6 +- packages/search/src/project.ts | 260 +++++-- packages/search/src/query.ts | 55 +- packages/search/src/schema.ts | 374 +++++++--- packages/search/src/testing.ts | 25 +- .../search/test/polymorphic-lookup.test.ts | 641 ++++++++++++++++++ .../search/test/qualified-relation.test.ts | 8 +- packages/search/test/schema.test.ts | 33 +- packages/search/vite.config.ts | 2 +- 30 files changed, 3136 insertions(+), 479 deletions(-) create mode 100644 docs/decisions/0027-resolve-a-reference-across-several-targets.md create mode 100644 packages/search-api-graphql/test/polymorphic-lookup.test.ts create mode 100644 packages/search-pipeline/test/polymorphic-extraction.test.ts create mode 100644 packages/search-typesense/test/polymorphic-lookup.test.ts create mode 100644 packages/search/test/polymorphic-lookup.test.ts diff --git a/docs/decisions/0027-resolve-a-reference-across-several-targets.md b/docs/decisions/0027-resolve-a-reference-across-several-targets.md new file mode 100644 index 00000000..24955fce --- /dev/null +++ b/docs/decisions/0027-resolve-a-reference-across-several-targets.md @@ -0,0 +1,201 @@ +# 27. Resolve a reference across several targets + +Date: 2026-09-10 + +## Status + +Accepted + +Extends [ADR 20 (Resolve a reference’s fields from the target’s own collection)](./0020-resolve-a-references-fields-from-the-targets-own-collection.md), +whose `lookup` names one target, and +[ADR 24 (Carry data on a reference edge)](./0024-carry-data-on-a-reference-edge.md), +whose `local` lookup stores a referent through that one target’s declaration. +Amends [ADR 21 (Type a filter by what its field keys on)](./0021-type-a-filter-by-what-its-field-keys-on.md), +which rejected two of the things decided here. Leaves +[ADR 19 (Filter across collections through declared joins)](./0019-filter-across-collections-through-declared-joins.md) +untouched, and states what it cannot reach. + +## Context + +SCHEMA-AP-NDE ranges `CreativeWork.creator` over `Person` **or** +`Organization`. A `lookup` names one `target`, so a deployment declared +`target: 'Person'` and got exactly half of the field: every person labelled and +typed `PersonReference`, every organization unlabelled and _also_ typed +`PersonReference`. Measured on the reporting deployment’s facet, the split was +exact – seven labelled values, all persons; three null, all organizations. A +consumer routing a click on `creator.id` to a person page followed it for an +organization and landed nowhere, with nothing in the response to warn them. + +Two things have moved since the issue was filed. First, the declaration now +sits **inside an edge**: `creator` is an inline `CreatorRole` carrying the role +a publisher wrapped the agent in, and the polymorphic lookup is the edge’s +nested `creator`, reached through `identity`. Second, `local: true` stores +what the work states about its creator, so an organization is no longer +_unlabelled_ – the referring document names it. That narrowed the defect to +two things a label mechanism can never fix: the facet buckets, still labelled +through the single target, and `__typename`, still claiming `PersonReference`. + +Retargeting is not a workaround. Pointing `creator` at the type most referents +are co-typed with (`Term`, in that corpus) took the facet from seven labelled +values to five. No single target covers a range the profile itself declares as +a disjunction. + +ADR 21 had rejected both a list of targets per reference and an output-side +interface, on the grounds that coarse discovery already answered the input-side +question and that a shared fragment was “buildable but unearned” on the output +side. Both grounds were about convenience. A type claim that is false for half +the values is not a convenience problem. + +## Decision + +### A lookup names several targets, in order of precedence + +`target` – and an `idOnly`’s `labelSource` – accepts a `string` or a +`readonly string[]`. One reading serves both: a single name is the list of +one, and no consumer branches on which it was given. Every named type must be a +Root Type with a label field; a name declared twice is rejected; `joinable` is +refused when there is more than one (below). + +**Declaration order is precedence.** A referent two collections hold – two +Root Types whose `class` selections overlap – belongs to the first target +declared. So does a stored referent whose `rdf:type` matches none of them. The +rule is per value: one field serves persons and organizations side by side. +Precedence is declaration order rather than an option because it is the only +lever an author already has, and because overlap is the author’s own choice – +two Root Types selecting overlapping classes is legal for facets and joins +already, and silently choosing is what the rest of the system does. + +### Each referent is read through the target it belongs to + +Three declarations flow through _naming the target_: the label field, the +document key ([ADR 22](./0022-key-a-root-type-on-a-declared-field.md)) and the +facet policy. The reporting deployment’s two targets disagree on the one that +changes what is stored – `Person` is keyed on an authority IRI, `Organization` +on nothing – so requiring the targets to agree would reject the schema the +issue came from. + +Each referent is therefore treated as a document of whichever target it +matches: re-keyed through that target’s key, admitted to a facet by that +target’s policy, labelled from that target’s collection. Nothing changes for +the targets themselves; `Person` keeps its key without knowing it appears in a +polymorphic lookup. The interface an author learns is one sentence longer than +before, which is the point of doing the dispatch per entry rather than +demanding alike targets. + +### A stored referent carries a discriminator + +A `local` lookup stores what the referring document states, and in the +reporting corpus seven creators in eight are never identified. No collection +will ever answer for them, so the collections cannot be the only evidence of +kind. The extraction reads the referent’s `rdf:type` – one `OPTIONAL` triple +per referent, emitted as `rdf:type` itself so framing carries it as `@type` – +and the projection matches it against each target’s `class` in declaration +order. The entry is projected through the matching declaration and records +which under the reserved physical name `_target`, which no type may declare. + +That discriminator is also what re-keys the entry and admits it to a facet, so +the three readings cannot disagree. An engine adapter reads it back when no +collection answers, so an unidentified organization is served as an +organization. A single-target lookup stores none: there is nothing to tell +apart. + +The nested object an engine declares is the union of the targets’ fields, so +two targets declaring one field must declare it alike – same kind, same arity, +same Roles, and for a reference the same strategy and referent – since every +one of those decides the field’s physical shape, and a shape that depended on +which target was listed first would store one target’s referents wrongly. +`searchSchema` rejects the pair otherwise; a field only one target declares is +fine, the other simply never fills it. This holds for a `local` lookup today +and for every lookup once +[#818](https://github.com/ldelements/lde/issues/818) drops the flag, which is +why the multi-target projection lives in the shared nesting body rather than +behind a `local`-specific switch. + +### The surface serves an interface, and a filter named for the set + +A GraphQL surface serves a lookup naming several targets as an **interface** +named for the set in declaration order – `PersonOrOrganizationReference` – +implemented by each target’s own `‹Target›Reference`. Two fields naming the +same targets share one interface, exactly as two lookups on one target share +one reference type ([ADR 20](./0020-resolve-a-references-fields-from-the-targets-own-collection.md)). +The name is derived, not declared: no new option, and the same mechanism that +derives `‹Target›Reference`. + +The interface carries only what is true of every referent: `id`, nullable if +any member’s is, and every `output` field all targets declare alike, with the +weakest nullability any of them keeps – an implementation may promise more than +its interface, never less. Per-type fields stay on the members, reached +through an inline fragment. `__typename` resolves per referent to the target +whose collection answered for it, or, for a stored referent no collection +answers for, to the target its discriminator names. The port marks each nested +document with that target under a symbol key, so the mark can never collide +with a declared field. A bare IRI no collection holds is the one referent with +no kind to report; an interface must still resolve to some object type, so it +is served as the first target declared. + +An interface rather than a union, because union members share no fields, so +even `creator { id name }` would need a fragment and every existing consumer +would break. An interface keeps that selection working unchanged. A `type` +scalar on a flat reference was rejected in the issue for a reason that still +holds: the moment a reference carries more than a label, per-type fields have +nowhere to live. + +On the input side, the field’s filter is named for the set too – +`PersonOrOrganizationFilter { in: [IRI!] }` – rather than for the first target, +which would lie about half the values, or `IRIFilter`, which would claim the +ids belong to no collection here. ADR 21’s coarse discovery still finds it, +since its element type is `IRI`. Refined discovery, which resolves through one +target’s own `id`, does not reach it; ADR 21 already called refined “a +precision tool, not a completeness claim” and named polymorphic ranges as +where it under-reports. This makes the under-report visible in the name. + +### No join + +A reference naming several targets refuses `joinable`. An engine reference +names one collection, and ids that live in several have no single collection +to reference. Its labels, facets and id filters all work from the referring +document. What stays out of reach is a **join predicate** – a condition on the +referent’s own fields, “works whose creator died before 1900”. The issue lists +three ways to recover one (a shared collection of the common fields; the same +collection denormalised to the union of fields with a discriminator; one +reference field per target disjoined at query time), none needed for what this +record delivers and each costing a second copy of every agent or two +unverified engine behaviours. Deferred, to +[#845](https://github.com/ldelements/lde/issues/845) and a record of its own. + +## Consequences + +- A field whose referent may be of several kinds labels every value, types + every value truthfully, and serves each kind’s own fields through a fragment. + Facet buckets are labelled from whichever collection holds the value. +- **Breaking** for a schema-level reader: `labelSourceNameOf` and + `labelTargetNameOf` become `labelSourceNamesOf` and `labelTargetNamesOf`, + returning a list; `localLookupTypeOf` becomes `localLookupTargetsOf`; + `inheritedFacetKeys` becomes `inheritedFacetPolicies`, keyed by target. A + declaration naming one target is unchanged, stores what it stored, and emits + the GraphQL it emitted. +- A multi-target lookup costs one query per target collection per level where + a single-target one costs one – still one round-trip per level, the queries + running concurrently – and asks every collection even for a selection + reduced to `id`, because the answer is what types the referent. Its stored + entries cost one short string each, and its extraction one `OPTIONAL` triple + per referent plus every keyed target’s key hop. +- The framing depth of a type nesting a multi-target lookup is the furthest + any target reaches: the referent is framed once, and the frame has to hold + whichever declaration it turns out to match. +- `_target` joins `id`, `and` and `or` as a name no type may declare. +- Label precedence is settled **per type**, in the order its fields first name + the collections: a page’s labels are one map keyed by IRI, so two fields of + one type ordering the same targets differently cannot each get their own + answer for an IRI both collections hold. Before, the same overlap was + labelled by whichever collection answered last – now it is deterministic + and documented, and the projected lookup (which does resolve per field) + can disagree with a bucket label only in that corner. +- A declared type may not be named as the joined name of a target set + (`PersonOrOrganization` beside a lookup over `Person` and `Organization`): + the interface and the type’s filter would share a name, so the GraphQL + surface refuses the schema, naming both. +- Amends ADR 21: a list of targets per reference and an output-side interface + are both accepted, for a reason ADR 21 did not weigh – a type claim that is + false for half the values. Its input-side reasoning stands: an interface is + output-only, and the `IRI` scalar remains the input-side abstraction. diff --git a/docs/reference/search-api-graphql.md b/docs/reference/search-api-graphql.md index be036f31..85986c4d 100644 --- a/docs/reference/search-api-graphql.md +++ b/docs/reference/search-api-graphql.md @@ -167,12 +167,23 @@ editor. `id`, since a referent needs no identity – so a client selects a nested object’s fields directly and renders one referent at a time. A `local` lookup gets the nullable `id` too, and for the same reason: it carries what the - document states about an endpoint whether or not the endpoint is identified; scalars/booleans + document states about an endpoint whether or not the endpoint is identified. + A lookup naming [several targets](./search#a-referent-of-several-kinds) is + served as an **interface** named for the set (`creator` over `Person` and + `Organization` → `PersonOrOrganizationReference`), implemented by each + target’s own reference type; it carries `id` plus every field the targets + declare alike, with the weakest nullability any of them keeps, and resolves + per referent to the type whose collection answered for it, or that a stored + referent was projected through – so `__typename` is true per referent, and + `... on PersonReference { birthDate }` reaches what only a person has. An + IRI no collection holds has no kind to report and is served as the first + target declared, since an interface must resolve to some type; scalars/booleans per kind; `date` → ISO 8601 string; nullability from `required` / `array` / `kind`. - **`where`** one input per `filterable` field, typed by what the field keys on: a `keyword` holds literals (`KeywordFilter`), a `reference` holds identity - (`‹Target›Filter`, or `IRIFilter` when it names no target), and the numeric + (`‹Target›Filter`, or `IRIFilter` when it names no target, or a filter named + for the set – `PersonOrOrganizationFilter` – when it names several), and the numeric kinds take `IntRange` / `FloatRange` / `DateRange`, a `boolean` a plain `Boolean`. Every type also gets **`id: ‹Type›Filter`** – the document’s IRI, declared by no type and filterable on all of them @@ -328,7 +339,12 @@ when a shorter, higher-precision list is what you want. **Known limit**: the refined strategy resolves only when the target is itself a root collection. A `ref` to a type no collection serves has no `‹Type›Where.id` to match against, so fall back to the coarse strategy – which is also the right -one for a reference declared with no target at all (`IRIFilter`). +one for a reference declared with no target at all (`IRIFilter`), and for one +declared with [several](./search#a-referent-of-several-kinds): its filter is +named for the set (`PersonOrOrganizationFilter`) rather than for any one +member’s `id`, so resolving through `PersonWhere.id` does not reach it, by +design – the name is truthful about what the field admits. Its `in` element is +still `IRI`, so coarse discovery finds it. Two further notes. `IRI` is wire-compatible with `String`, but GraphQL checks variable usage **nominally**, so a variable must be declared `[IRI!]` rather than diff --git a/docs/reference/search-pipeline.md b/docs/reference/search-pipeline.md index 6260b907..9988e6d6 100644 --- a/docs/reference/search-pipeline.md +++ b/docs/reference/search-pipeline.md @@ -424,7 +424,12 @@ guarantees one output triple per genuine value: because a `local` lookup stores its endpoint by id whether or not the document says anything about it – conjoining would drop the id along with the absent fields. It subsumes the key hop below, the key field being one of the target's - own. + own. A lookup naming [several targets](./search#a-referent-of-several-kinds) + expands through every one of them inside the same `OPTIONAL` union – a + referent is of one kind, and the other targets' branches bind nothing for it – + and reads the referent’s `rdf:type` in an `OPTIONAL` of its own, emitted as + `rdf:type` so framing carries it as `@type`: that is what the projection + matches against each target’s `class` to tell which kind the referent is. - **A key hop stays inside its branch.** A reference naming a target that declares a [document key](./search#document-key) gets its branch extended with an `OPTIONAL` hop reading the referent’s key field, emitted under the target’s diff --git a/docs/reference/search-typesense.md b/docs/reference/search-typesense.md index 1dc62891..07baccc2 100644 --- a/docs/reference/search-typesense.md +++ b/docs/reference/search-typesense.md @@ -200,15 +200,25 @@ down. - runs the search; - resolves reference (and reference-facet) labels **per reference field** from the collection of the `SearchType` its `labelSource` names – all - sources bundled into a single lookup. A reference without a `labelSource` - stays id-only. With `labelCacheTtlMs` set, each label-source collection is - instead loaded once into an in-memory cache; + sources bundled into a single lookup. A reference naming + [several targets](./search#a-referent-of-several-kinds) sends its IRIs to + every one of their collections in the same round-trip. An IRI two of them + hold is labelled by the collection the type’s fields name first – label + precedence is settled per type, not per field, since one page’s labels are + one map keyed by IRI. A reference without a + `labelSource` stays id-only. With `labelCacheTtlMs` set, each label-source + collection is instead loaded once into an in-memory cache; - reconstructs the logical `SearchResult` (`parseSearchResponse`) – language maps, labelled references, labelled facet buckets, and one nested Search Document per referent of a surfaced inline reference (each referent’s values grouped, `id` only where the referent had one). Nesting is rebuilt here, below every API surface, so a second surface inherits it rather than - reimplementing it. + reimplementing it. A projected lookup naming several targets fetches from + each target’s collection concurrently, still one round-trip per level, and + marks each nested document with the target whose collection answered for it + (`NESTED_DOCUMENT_TYPE`, a symbol key, so it never surfaces as a field); a + stored referent no collection answers for is read through the target its + stored discriminator names. A label source is just another `SearchType` in the schema (with an `output`, `searchable` text field under its diff --git a/docs/reference/search.md b/docs/reference/search.md index fff09ff6..28ed9c4a 100644 --- a/docs/reference/search.md +++ b/docs/reference/search.md @@ -338,7 +338,9 @@ fields are read from, and the name its emitted type derives from (GraphQL: named per query rather than per declaration, by a [projection](#projecting-what-a-lookup-carries); asked for nothing in particular, it carries the target's label. Only an `inline` reference’s -`ref.typeName` resolves to a declared Reference Type. +`ref.typeName` resolves to a declared Reference Type. Where the referent may be +of several kinds, `target` lists them – see +[A referent of several kinds](#a-referent-of-several-kinds). Note what this makes true of `kind`: **a `reference` holds identity, a `keyword` holds a literal.** A field over an IRI-valued property is a `reference` whatever @@ -603,6 +605,61 @@ See [ADR 26](../decisions/0026-fan-out-a-qualified-edge-into-one-entry-per-tuple Out of scope for now: faceting an edge's own values, which the current engine cannot serve correctly. +#### A referent of several kinds + +A profile often ranges one property over several classes – SCHEMA-AP-NDE’s +`creator` is a `Person` _or_ an `Organization`. A lookup naming one target can +label and type only half of such a field: every organization comes back +unlabelled, and typed as a person. So a `lookup` (and an `idOnly`’s +`labelSource`) may name **several** targets: + +```ts +{ + name: 'creator', + kind: 'reference', + path: `${SCHEMA}creator`, + output: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'], local: true }, +} +``` + +Each referent then resolves against every named collection and is read through +the declaration of whichever one holds it: its label, its own fields, its +[key](#document-key) and its [facet policy](#facet-policy) are those of _its_ +target, per value. A GraphQL surface serves the field as an interface named for +the set (`PersonOrOrganizationReference`), implemented by each target’s own +reference type, so `__typename` says which kind each referent is and an inline +fragment reaches the fields only one kind has. The interface itself carries +`id` and whatever every target declares alike. + +**Order is precedence.** A referent two collections hold – two Root Types +whose `class` selections overlap – belongs to the first target listed, and so +does a stored referent whose `rdf:type` matches none of them. List the most +specific type first. + +**A stored referent says which kind it is.** A [`local`](#data-on-the-edge) +lookup stores what the referring document states, and most such referents are +never identified, so no collection can ever say what they are. The extraction +therefore reads the referent’s `rdf:type` and the projection matches it against +each target’s `class`, projecting the entry through the matching declaration +and recording which under a reserved physical name (`_target`, which no type +may declare). A stored referent that no collection answers for – unidentified, +or identified but not indexed – is then still served as the kind it was stated +to be. The nested object declares the union of the targets’ fields, so two +targets that both declare a field must declare it alike – same `kind`, same +arity, same Roles, and for a reference the same strategy and referent; +`searchSchema` rejects the pair otherwise. + +**No join.** A reference naming several targets can never be +[`joinable`](#filtering-across-collections): an engine reference names one +collection, and ids that live in several have no single collection to +reference. Its labels, facets and id filters all work from the referring +document; a condition on the referent’s own fields (“works whose creator died +before 1900”) does not, and needs a representation of its own +([#845](https://github.com/ldelements/lde/issues/845) lists three). + +See [ADR 27](../decisions/0027-resolve-a-reference-across-several-targets). + #### Naming the label field That word is `label` by default, but a type may name its own display field with diff --git a/packages/search-api-graphql/src/build-schema.ts b/packages/search-api-graphql/src/build-schema.ts index 9dae0ff8..08ca9952 100644 --- a/packages/search-api-graphql/src/build-schema.ts +++ b/packages/search-api-graphql/src/build-schema.ts @@ -5,6 +5,7 @@ import { GraphQLFloat, GraphQLInputObjectType, GraphQLInt, + GraphQLInterfaceType, GraphQLList, GraphQLNonNull, GraphQLObjectType, @@ -19,9 +20,11 @@ import { type GraphQLOutputType, } from 'graphql'; import { + NESTED_DOCUMENT_TYPE, type Criterion, type Filter, type LocalizedValue, + type NestedDocument, type RootType, type SearchEngine, type ReferenceField, @@ -35,8 +38,10 @@ import { AND_KEY, facetableFields, filterableFields, - labelTargetNameOf, - localLookupTypeOf, + fieldNamed, + labelSourceNamesOf, + labelTargetNamesOf, + localLookupTargetsOf, referenceFields, filterOn, filterOperatorFor, @@ -407,6 +412,15 @@ export function buildGraphQLSchema( // collide: searchSchema resolves its typeName to a declared Reference Type // and rejects duplicate names schema-wide. const referenceTypes = new Map(); + /** + * The interface a lookup naming several targets is served as, by the + * targets’ joined name – a namespace of its own, apart from the object + * types above, so an inline Reference Type that happens to be named + * `PersonOrOrganization` can never be mistaken for it. Beside each, the + * targets behind it. + */ + const interfaceTypes = new Map(); + const interfaceMembers = new Map(); /** * The lookup targets whose emitted type must carry a NULLABLE `id`: those * some field reaches through a {@link ReferenceStrategy.local local} lookup, @@ -433,12 +447,14 @@ export function buildGraphQLSchema( walked.add(searchType.name); for (const field of referenceFields(searchType)) { if (field.ref?.strategy === 'lookup' && field.ref.local === true) { - nullableIdTargets.add(field.ref.target); + for (const target of labelSourceNamesOf(field)) { + nullableIdTargets.add(target); + } } - const nested = - nestedReferenceType(schema, field) ?? - localLookupTypeOf(field, schema); - if (nested !== undefined) { + const referenceType = nestedReferenceType(schema, field); + for (const nested of referenceType === undefined + ? localLookupTargetsOf(field, schema) + : [referenceType]) { collect(nested); } } @@ -478,14 +494,26 @@ export function buildGraphQLSchema( const nestedFilters = new Map(); const referenceFilters = new Map(); - /** The name a reference’s emitted type is keyed under: a `lookup`’s target, - * an `inline`’s reference type, an `idOnly`’s declared name (which names a - * filter’s target, never an object type – an `idOnly` surfaces as its bare - * IRI). */ - function referencedTypeName( - ref: NonNullable, - ): string | undefined { - return ref.strategy === 'lookup' ? ref.target : ref.typeName; + /** The name a reference’s emitted type is keyed under: a `lookup`’s target + * – or its targets joined with `Or` where it names several, the one name + * its interface and its filter share – an `inline`’s reference type, an + * `idOnly`’s declared name (which names a filter’s target, never an object + * type – an `idOnly` surfaces as its bare IRI). */ + function referencedTypeName(field: ReferenceField): string | undefined { + const ref = field.ref; + if (ref === undefined) { + return undefined; + } + return ref.strategy === 'lookup' + ? polymorphicName(labelSourceNamesOf(field)) + : ref.typeName; + } + + /** The one name several targets share: `PersonOrOrganization`. Derived, in + * declaration order, so two fields naming the same targets meet one type – + * exactly as two lookups on one target share `‹Target›Reference`. */ + function polymorphicName(targets: readonly string[]): string { + return targets.join('Or'); } /** @@ -505,13 +533,159 @@ export function buildGraphQLSchema( // as the IRI itself rather than as an object – there is no type to // register, and no name needed to register one under. field.ref.strategy === 'idOnly' || - referencedTypeName(field.ref) === undefined + referencedTypeName(field) === undefined ) { return; } + // A lookup naming several targets is served as an INTERFACE over the + // targets’ own reference types, each registered as it would be for a + // lookup naming it alone – so a consumer selecting only what every target + // carries (`id`, the label) needs no fragment, and `__typename` says + // which target a referent came from. + const targets = labelSourceNamesOf(field); + if (field.ref.strategy === 'lookup' && targets.length > 1) { + for (const target of targets) { + registerObjectType(target, field, owner); + } + registerInterfaceType(targets, field, owner); + return; + } // Guaranteed by the guard above: idOnly is out, and the other two strategies // each name their referent. - const typeName = referencedTypeName(field.ref) as string; + registerObjectType(referencedTypeName(field) as string, field, owner); + } + + /** + * The interface a lookup naming several targets is served as: named for the + * targets (`PersonOrOrganizationReference`), carrying what every one of them + * carries – `id`, and every `output` field they all declare alike – and + * resolved per referent to the target whose collection answered for it, + * which the port marks on each nested document. + * + * A field the targets share is offered with the weakest nullability any of + * them keeps: an implementation may promise more than its interface, never + * less. + */ + function registerInterfaceType( + targets: readonly string[], + field: SearchField, + owner: SearchType, + ): void { + const typeName = polymorphicName(targets); + if (interfaceTypes.has(typeName)) { + return; + } + const graphQLName = `${typeName}Reference`; + // The joined name is derived, so a declared type may spell the same + // thing – `PersonOrOrganization` as a Reference Type – and its filter + // would then share a name with this interface’s. Refused, naming both. + if (takenTypeNames.has(graphQLName) || takenTypeNames.has(typeName)) { + throw new Error( + `Reference type “${typeName}” (field “${field.name}” of “${owner.name}”) would be served as “${graphQLName}”, which collides with another type name; rename one.`, + ); + } + takenTypeNames.add(graphQLName); + // Reserve the joined name too, so a Reference Type spelling it that + // registers LATER is refused with the same message. + takenTypeNames.add(typeName); + const members = targets.map( + (target) => rootTypesByName.get(target) as RootType, + ); + const nullableId = members.some((member) => + nullableIdTargets.has(member.name), + ); + interfaceMembers.set(typeName, targets); + interfaceTypes.set( + typeName, + new GraphQLInterfaceType({ + name: graphQLName, + description: `A reference to a ${targets.join(' or a ')}; select \`__typename\` to tell which, and the fields of one through an inline fragment on its type.`, + resolveType: (source: Source) => { + const resolved = (source as NestedDocument)[NESTED_DOCUMENT_TYPE]; + // A document the port did not type – a referent no collection + // holds, or one built by hand – is read as the first target + // declared: an interface must resolve to SOME object type, and the + // first target is the precedence every other reading of several + // targets applies. + return `${ + typeof resolved === 'string' && targets.includes(resolved) + ? resolved + : targets[0] + }Reference`; + }, + fields: (): Record< + string, + GraphQLFieldConfig + > => ({ + id: { + type: nullableId ? iriScalar : new GraphQLNonNull(iriScalar), + }, + ...Object.fromEntries( + outputFields(members[0]) + .map((declared) => sharedField(declared, members, nullableId)) + .filter((shared): shared is SearchField => shared !== undefined) + .map((shared) => [shared.name, outputFieldConfig(shared)]), + ), + }), + }), + ); + } + + /** + * A field every member declares alike – same kind, same arity – with the + * nullability relaxed unless every member requires it. `undefined` where the + * members do not all carry it: the interface promises only what holds of + * every referent. + */ + function sharedField( + declared: SearchField, + members: readonly RootType[], + nullableId: boolean, + ): SearchField | undefined { + const counterparts = members.map((member) => + fieldNamed(member, declared.name), + ); + if ( + counterparts.some( + (counterpart) => + counterpart === undefined || + counterpart.output !== true || + counterpart.kind !== declared.kind || + (counterpart.array === true) !== (declared.array === true), + ) + ) { + return undefined; + } + // A reference field is shared only where every member serves it as the + // same emitted type – same strategy, same referent – or the interface + // would offer one field of two types: an `idOnly` surfaces as an `IRI` + // where a lookup to the same name surfaces as an object. + if ( + declared.kind === 'reference' && + counterparts.some( + (counterpart) => + (counterpart as ReferenceField).ref?.strategy !== + declared.ref?.strategy || + referencedTypeName(counterpart as ReferenceField) !== + referencedTypeName(declared), + ) + ) { + return undefined; + } + const required = + !nullableId && + counterparts.every((counterpart) => counterpart?.required === true); + return { ...declared, required } as SearchField; + } + + /** Register the object type one referenced shape is served as, under + * `typeName`: a lookup’s target root type, or an inline reference’s + * Reference Type. */ + function registerObjectType( + typeName: string, + field: ReferenceField, + owner: SearchType, + ): void { if (referenceTypes.has(typeName)) { // Fields sharing a referent share one emitted type, and cannot disagree // about its FIELDS: a lookup's come from the target that names the type, @@ -536,14 +710,22 @@ export function buildGraphQLSchema( // searchSchema rejects a lookup whose target it cannot find, and an inline // reference that resolves to no Reference Type. const nested = ( - field.ref.strategy === 'lookup' - ? rootTypesByName.get(field.ref.target) + field.ref?.strategy === 'lookup' + ? rootTypesByName.get(typeName) : nestedReferenceType(schema, field) ) as SearchType; + // Every interface this type will implement: one per lookup naming this + // target among several. Resolved lazily, so registration order between + // the interface and its members does not matter. + const implemented = (): GraphQLInterfaceType[] => + [...interfaceMembers] + .filter(([, members]) => members.includes(typeName)) + .map(([key]) => interfaceTypes.get(key) as GraphQLInterfaceType); referenceTypes.set( typeName, new GraphQLObjectType({ name: graphQLName, + interfaces: implemented, // A thunk, so a Reference Type nesting another one resolves whatever // the registration order is (the graph is acyclic by searchSchema). fields: (): Record< @@ -655,10 +837,13 @@ export function buildGraphQLSchema( resolve: (source) => iriOf(source[field.name]) ?? null, }; } - const referenceType = referenceTypes.get( - (field.ref === undefined - ? undefined - : referencedTypeName(field.ref)) ?? '', + const typeName = referencedTypeName(field) ?? ''; + // A lookup naming several targets is served as their interface. + const referenceType = ( + field.ref?.strategy === 'lookup' && + labelSourceNamesOf(field).length > 1 + ? interfaceTypes.get(typeName) + : referenceTypes.get(typeName) )!; return field.array === true ? { @@ -737,14 +922,14 @@ export function buildGraphQLSchema( if (existing !== undefined) { return existing; } - const identityTarget = labelTargetNameOf(field, schema); + const identityTargets = labelTargetNamesOf(field, schema); const nestedWhere = nestedWhereInputFor(referenceType); const created = new GraphQLInputObjectType({ name: `${referenceType.name}Filter`, description: `A condition on ${referenceType.name}: the ids its entries reference, or a condition on one entry.`, isOneOf: true, fields: () => ({ - ...(identityTarget !== undefined && { + ...(identityTargets.length > 0 && { in: { type: new GraphQLList(new GraphQLNonNull(iriScalar)) }, }), where: { type: nestedWhere }, @@ -792,9 +977,12 @@ export function buildGraphQLSchema( return keywordFilter; } // A lookup keys on its `target`, an idOnly/inline on its `typeName`: - // one reading, so a filter is typed by whatever names the referent. - const target = - field.ref === undefined ? undefined : referencedTypeName(field.ref); + // one reading, so a filter is typed by whatever names the referent. A + // lookup naming several targets keys on all of them, so its filter + // is named for the set (`PersonOrOrganizationFilter`): truthful, and + // still an `IRI` list, so coarse discovery finds it – refined + // discovery, which resolves through one target’s own `id`, does not. + const target = referencedTypeName(field); return target === undefined ? iriFilter : targetFilter(target); } case 'range': @@ -1153,6 +1341,10 @@ export function buildGraphQLSchema( return new GraphQLSchema({ query: new GraphQLObjectType({ name: 'Query', fields: queryFields }), + // An interface's implementations are reachable only through it, so they + // are listed explicitly; every other reference type is reachable already + // and listing it twice is harmless. + types: [...referenceTypes.values(), ...interfaceTypes.values()], }); } diff --git a/packages/search-api-graphql/src/projection.ts b/packages/search-api-graphql/src/projection.ts index 6558f967..858cb80b 100644 --- a/packages/search-api-graphql/src/projection.ts +++ b/packages/search-api-graphql/src/projection.ts @@ -13,7 +13,7 @@ import { import { fieldNamed, nestedReferenceType, - rootTypeNamed, + referencedTargetsOf, } from '@lde/search/adapter'; type Fragments = Readonly>; @@ -164,34 +164,40 @@ function fromSelections( if (field.kind !== 'reference' || field.ref?.strategy !== 'lookup') { continue; } - // The target's own declaration decides what its selections mean, so the - // level below is read against it rather than against this type. + // The targets' own declarations decide what the selections mean, so the + // level below is read against them rather than against this type. // `searchSchema` rejects a lookup whose target it cannot resolve, so a // schema always resolves its own – which is why this reads the schema - // rather than taking a resolver that could answer nothing. - const target = rootTypeNamed(schema, field.ref.target) as SearchType; - // Only what the target actually serves. A selection carries more than + // rather than taking a resolver that could answer nothing. A lookup + // naming several targets is served as an interface, and a client + // reaches one target’s own fields through an inline fragment on its + // type; the fragment’s fields are read like any other, and each target + // takes the ones it serves. + const targets = referencedTargetsOf(field, schema); + // Only what a target actually serves. A selection carries more than // that: `id` is on the referring document already, and every GraphQL // client worth the name injects `__typename` into every selection set – // asking the engine for either would fail the query at the port's guard. const wanted = fieldsOf(selected.selectionSet, fragments) .map((node) => node.name.value) - .filter((name) => servesField(target, name)); + .filter((name) => targets.some((target) => servesField(target, name))); const entry = (projection[selected.name.value] ??= {}); entry.fields = [...new Set([...(entry.fields ?? []), ...wanted])]; if (selected.selectionSet !== undefined) { - const below = fromSelections( - [selected.selectionSet], - target, - fragments, - schema, - ); - if (Object.keys(below).length > 0) { - // Merged level by level, not key by key: one lookup selected twice – - // two fragments each spreading it – must union what each asked for, - // or the second selection silently replaces the first and a field - // the client asked for is never fetched. - entry.resolve = mergeProjections(entry.resolve, below); + for (const target of targets) { + const below = fromSelections( + [selected.selectionSet], + target, + fragments, + schema, + ); + if (Object.keys(below).length > 0) { + // Merged level by level, not key by key: one lookup selected twice + // – two fragments each spreading it – must union what each asked + // for, or the second selection silently replaces the first and a + // field the client asked for is never fetched. + entry.resolve = mergeProjections(entry.resolve, below); + } } } } diff --git a/packages/search-api-graphql/test/polymorphic-lookup.test.ts b/packages/search-api-graphql/test/polymorphic-lookup.test.ts new file mode 100644 index 00000000..59341d77 --- /dev/null +++ b/packages/search-api-graphql/test/polymorphic-lookup.test.ts @@ -0,0 +1,360 @@ +import { describe, expect, it } from 'vitest'; +import { + graphql, + Kind, + parse, + printSchema, + type FragmentDefinitionNode, + type OperationDefinitionNode, +} from 'graphql'; +import { + NESTED_DOCUMENT_TYPE, + searchSchema, + type FacetsOutcome, + type SearchEngine, + type SearchField, + type SearchQuery, + type SearchResult, + type SearchType, +} from '@lde/search'; +import { buildGraphQLSchema } from '../src/build-schema.js'; +import { projectionFor } from '../src/projection.js'; + +const person: SearchType = { + name: 'Person', + class: 'https://schema.org/Person', + labelField: 'name', + fields: [ + { + name: 'name', + kind: 'text', + locales: ['nl'], + output: true, + searchable: { weight: 1 }, + }, + { name: 'birthDate', kind: 'keyword', output: true }, + // Shared with Organization, but required only here: the interface offers + // it with the weaker promise. + { name: 'homepage', kind: 'keyword', output: true, required: true }, + { + name: 'birthPlace', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: 'Place' }, + }, + // Shared, and pointing at one type on both: reaches the interface. + { + name: 'address', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: 'Place' }, + }, + // Shared in name only: each member points it at a different type. + { + name: 'affiliation', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: 'Organization' }, + }, + ], +}; +const organization: SearchType = { + name: 'Organization', + class: 'https://schema.org/Organization', + labelField: 'name', + fields: [ + { + name: 'name', + kind: 'text', + locales: ['nl'], + output: true, + searchable: { weight: 1 }, + }, + { name: 'location', kind: 'keyword', output: true }, + { name: 'homepage', kind: 'keyword', output: true }, + { + name: 'address', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: 'Place' }, + }, + { + name: 'affiliation', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: 'Place' }, + }, + ], +}; +const place: SearchType = { + name: 'Place', + class: 'https://schema.org/Place', + fields: [ + { + name: 'label', + kind: 'text', + locales: ['nl'], + output: true, + searchable: { weight: 1 }, + }, + ], +}; +const work: SearchType = { + name: 'CreativeWork', + class: 'https://schema.org/CreativeWork', + fields: [ + { name: 'title', kind: 'text', locales: ['nl'], output: true }, + { + name: 'creator', + kind: 'reference', + array: true, + output: true, + filterable: true, + facetable: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }, + // A second field naming the same targets meets the same interface. + { + name: 'contributor', + kind: 'reference', + array: true, + output: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }, + ], +}; +const schema = searchSchema(work, person, organization, place); +const gqlSchema = buildGraphQLSchema(schema); +const sdl = printSchema(gqlSchema); + +describe('serving a lookup over several targets', () => { + it('emits one interface per target set, implemented by each target’s reference type', () => { + expect(sdl).toMatch( + /interface PersonOrOrganizationReference \{\s+id: IRI!\s+name: \[LanguageString!\]!\s+homepage: String\s+address: PlaceReference\s+\}/, + ); + expect(sdl).toMatch( + /type PersonReference implements PersonOrOrganizationReference \{/, + ); + expect(sdl).toMatch( + /type OrganizationReference implements PersonOrOrganizationReference \{/, + ); + expect(sdl).toMatch(/creator: \[PersonOrOrganizationReference!\]!/); + expect(sdl).toMatch(/contributor: \[PersonOrOrganizationReference!\]!/); + expect( + sdl.match(/^interface PersonOrOrganizationReference /gm), + ).toHaveLength(1); + // Only what every target carries alike reaches the interface. + expect(sdl).not.toMatch( + /interface PersonOrOrganizationReference \{[^}]*birthDate/, + ); + expect(sdl).not.toMatch( + /interface PersonOrOrganizationReference \{[^}]*location/, + ); + // The members keep everything of their own, promises included. + expect(sdl).toMatch( + /type PersonReference implements[^}]*homepage: String!/, + ); + expect(sdl).toMatch( + /type PersonReference implements[^}]*birthDate: String/, + ); + }); + + it('refuses a declaration spelling the derived name itself', () => { + // A Reference Type named `PersonOrOrganization` would put its filter + // under the interface's filter name; refused rather than served wrongly. + const spelled: SearchType = { + name: 'PersonOrOrganization', + fields: [{ name: 'note', kind: 'keyword', output: true }], + }; + const nesting = (fields: readonly SearchField[]): SearchType => ({ + ...work, + fields, + }); + const note: SearchField = { + name: 'note', + kind: 'reference', + output: true, + ref: { strategy: 'inline', typeName: 'PersonOrOrganization' }, + }; + // Whichever registers first, the other is refused. + for (const fields of [ + [...work.fields, note], + [note, ...work.fields], + ]) { + expect(() => + buildGraphQLSchema( + searchSchema(nesting(fields), person, organization, place, spelled), + ), + ).toThrow(/collides with another type name/); + } + }); + + it('shares a reference field only where every target points it at one type', () => { + // Person.affiliation looks up an Organization, Organization.affiliation a + // Place: two emitted types, so the interface offers neither. + expect(sdl).not.toMatch( + /interface PersonOrOrganizationReference \{[^}]*affiliation/, + ); + }); + + it('refuses a declaration whose derived interface name is taken', () => { + expect(() => + buildGraphQLSchema( + searchSchema(work, person, organization, place, { + name: 'PersonOrOrganizationReference', + class: 'https://schema.org/Thing', + fields: [{ name: 'note', kind: 'keyword', output: true }], + }), + ), + ).toThrow(/would be served as “PersonOrOrganizationReference”/); + }); + + it('types the filter by the target set, still over IRIs', () => { + expect(sdl).toMatch( + /input PersonOrOrganizationFilter \{\s+in: \[IRI!\]\s+\}/, + ); + expect(sdl).toMatch( + /input CreativeWorkCriterion @oneOf \{[^}]*creator: PersonOrOrganizationFilter/, + ); + }); + + it('reports the target each referent came from, and serves its own fields through a fragment', async () => { + const engine: SearchEngine = { + schema, + async search(): Promise { + return { + total: 1, + facets: {}, + hits: [ + { + id: 'https://w/1', + document: { + title: { nl: ['Werk'] }, + creator: [ + { + id: 'https://p/1', + name: { nl: ['Frank Koel'] }, + birthDate: '1901', + [NESTED_DOCUMENT_TYPE]: 'Person', + }, + { + id: 'https://o/1', + name: { nl: ['Zusters Franciscanessen'] }, + location: 'Roermond', + [NESTED_DOCUMENT_TYPE]: 'Organization', + }, + // Untyped – a hand-built document – reads as the first. + { id: 'https://x/1', name: { nl: ['Onbekend'] } }, + ], + }, + }, + ], + }; + }, + searchFacets: async ( + _searchType: SearchType, + queries: readonly SearchQuery[], + ): Promise => + queries.map(() => ({ facets: {} })), + }; + + const result = await graphql({ + schema: gqlSchema, + source: `{ + creativeWorks { + items { + creator { + __typename + id + name { value } + ... on PersonReference { birthDate } + ... on OrganizationReference { location } + } + } + } + }`, + contextValue: { engine, acceptLanguage: ['nl'] }, + }); + + expect(result.errors).toBeUndefined(); + const [item] = (result.data as { creativeWorks: { items: unknown[] } }) + .creativeWorks.items; + expect(item).toEqual({ + creator: [ + { + __typename: 'PersonReference', + id: 'https://p/1', + name: [{ value: 'Frank Koel' }], + birthDate: '1901', + }, + { + __typename: 'OrganizationReference', + id: 'https://o/1', + name: [{ value: 'Zusters Franciscanessen' }], + location: 'Roermond', + }, + { + __typename: 'PersonReference', + id: 'https://x/1', + name: [{ value: 'Onbekend' }], + birthDate: null, + }, + ], + }); + }); +}); + +describe('projecting a selection over several targets', () => { + function infoFor(query: string) { + const document = parse(query); + const operation = document.definitions.find( + (definition): definition is OperationDefinitionNode => + definition.kind === Kind.OPERATION_DEFINITION, + ); + const fragments = Object.fromEntries( + document.definitions + .filter( + (definition): definition is FragmentDefinitionNode => + definition.kind === Kind.FRAGMENT_DEFINITION, + ) + .map((fragment) => [fragment.name.value, fragment]), + ); + return { + fieldNodes: operation!.selectionSet.selections.filter( + (selection) => selection.kind === Kind.FIELD, + ), + fragments, + }; + } + + it('asks for what any target serves, fragments included, and descends each target’s own lookups', () => { + expect( + projectionFor( + infoFor(`{ + creativeWorks { + items { + creator { + __typename + id + name { value } + ... on PersonReference { birthDate birthPlace { label { value } } } + ... on OrganizationReference { location } + address { label { value } } + } + } + } + }`) as never, + work, + schema, + ), + ).toEqual({ + creator: { + fields: ['name', 'birthDate', 'birthPlace', 'location', 'address'], + resolve: { + birthPlace: { fields: ['label'] }, + address: { fields: ['label'] }, + }, + }, + }); + }); +}); diff --git a/packages/search-api-graphql/vite.config.ts b/packages/search-api-graphql/vite.config.ts index d4ea5d9a..2708d450 100644 --- a/packages/search-api-graphql/vite.config.ts +++ b/packages/search-api-graphql/vite.config.ts @@ -29,7 +29,7 @@ export default mergeConfig( // Re-anchored for the selection-set projection: its defensive reads // (an unresolvable target, an absent field list) are reachable only // from a direct caller, since GraphQL validates the query first. - branches: 97.05, + branches: 97.32, statements: 100, }, }, diff --git a/packages/search-pipeline/src/extraction.ts b/packages/search-pipeline/src/extraction.ts index abe3109d..d9ef358e 100644 --- a/packages/search-pipeline/src/extraction.ts +++ b/packages/search-pipeline/src/extraction.ts @@ -14,13 +14,11 @@ import { fieldNamed, irAlias, isInlineReference, - labelSourceNameOf, - localLookupTypeOf, + localLookupTargetsOf, + referencedTargetsOf, referenceTypeNamed, - rootTypeNamed, } from '@lde/search/adapter'; import type { - ReferenceField, RootType, SearchField, SearchSchema, @@ -31,6 +29,8 @@ const factory = new AstFactory(); const parser = new Parser(); const generator = new Generator(); +const RDF_TYPE = 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type'; + /** Options for {@link extractionQuery}. */ export interface ExtractionOptions { /** @@ -81,6 +81,12 @@ export interface ExtractionOptions { * `OPTIONAL`, so a referent with no key candidate keeps its row. The root side * needs nothing: a key field is a declared field, so its own branch and * template triple are already there. + * - **references naming several targets**: the referent may be of any of them, + * so its branch also reads the referent’s `rdf:type` (`OPTIONAL`, emitted + * as is – framing carries it as `@type`), which is what the projection + * matches against each target’s `class` to tell which one it is. Every + * target then contributes its own expansion or key hop, so whichever the + * referent turns out to be, its declaration is in the frame. * * Wire the result into a `SparqlConstructReader` (see `searchStages`), which * runs it per batch with the roots injected as VALUES. @@ -196,36 +202,43 @@ function buildFor( factory.gen(), ), ]; - const local = localTargetOf(field, schema, onPath); - if (local === undefined) { - // Only where no local expansion follows: a `local` lookup reads the - // target’s every path-bearing field, and a keyed target’s key field is - // one of them, so emitting the hop as well would state it twice. - const keyed = keyedTargetOf(field, schema); - if (keyed !== undefined) { - const built = buildKeyHop( - keyed.target, - keyed.keyField, - value, - counter, - ); - template.push(built.triple); - patterns.push(built.pattern); + const targets = referencedTargetsOf(field, schema); + if (targets.length > 1) { + const built = buildTypeHop(value, counter); + template.push(built.triple); + patterns.push(built.pattern); + } + const local = localTargetsOf(field, schema, onPath); + const expansions: PatternGroup[] = []; + for (const target of targets) { + if (!local.includes(target)) { + // Only where no local expansion follows: a `local` lookup reads the + // target’s every path-bearing field, and a keyed target’s key field + // is one of them, so emitting the hop as well would state it twice. + const keyed = keyedFieldOf(target); + if (keyed !== undefined) { + const built = buildKeyHop(target, keyed, value, counter); + template.push(built.triple); + patterns.push(built.pattern); + } + continue; } - } else { - const nested = buildFor(local, value, schema, counter, onPath); + const nested = buildFor(target, value, schema, counter, onPath); template.push(...nested.template); - // `OPTIONAL`, unlike an inline reference’s conjoined nesting: this - // referent is stored by id whether or not the referring document says - // anything about it, and conjoining would drop both together. - if (nested.branches.length > 0) { - patterns.push( - factory.patternOptional( - [factory.patternUnion(nested.branches, factory.gen())], - factory.gen(), - ), - ); - } + expansions.push(...nested.branches); + } + // `OPTIONAL`, unlike an inline reference’s conjoined nesting: this + // referent is stored by id whether or not the referring document says + // anything about it, and conjoining would drop both together. Several + // targets’ expansions share the one UNION: a referent is of one kind, + // and the branches of the others simply bind nothing for it. + if (expansions.length > 0) { + patterns.push( + factory.patternOptional( + [factory.patternUnion(expansions, factory.gen())], + factory.gen(), + ), + ); } branches.push(factory.patternGroup(patterns, factory.gen())); } @@ -234,45 +247,60 @@ function buildFor( } /** - * The Root Type a {@link ReferenceStrategy.local} lookup expands into here, or - * `undefined` where the field declares none – or where that type is already on - * this path, which is where the recursion stops. + * The Root Types a {@link ReferenceStrategy.local} lookup expands into here: + * none where the field declares none, and never one already on this path, + * which is where the recursion stops. */ -function localTargetOf( +function localTargetsOf( field: SearchField, schema: SearchSchema, visiting: ReadonlySet, -): RootType | undefined { - const local = localLookupTypeOf(field, schema); - return local === undefined || visiting.has(local.name) ? undefined : local; +): readonly RootType[] { + return localLookupTargetsOf(field, schema).filter( + (target) => !visiting.has(target.name), + ); } /** - * The keyed Root Type a reference points at, with the field its key is read - * from: a `lookup`’s `target` or an `idOnly`’s `labelSource` - * ({@link labelSourceNameOf}) that declares a {@link RootType.key}, or - * `undefined` for every other field. Naming the target is exactly the boundary - * the projection re-keys along, so the extraction reads the same declarations - * rather than a rule of its own: a reference that names no target keeps the - * node IRI, and needs no hop. + * The field a Root Type’s key is read from, when it declares a + * {@link RootType.key}. Naming the target is exactly the boundary the + * projection re-keys along ({@link referencedTargetsOf}), so the extraction + * reads the same declarations rather than a rule of its own: a reference that + * names no target keeps the node IRI, and needs no hop. */ -function keyedTargetOf( - field: SearchField, - schema: SearchSchema, -): { readonly target: RootType; readonly keyField: KeyedField } | undefined { - if (field.kind !== 'reference') { - return undefined; - } - const targetName = labelSourceNameOf(field as ReferenceField); - const target = - targetName === undefined ? undefined : rootTypeNamed(schema, targetName); - if (target?.key === undefined) { +function keyedFieldOf(target: RootType): KeyedField | undefined { + if (target.key === undefined) { return undefined; } // `searchSchema` guarantees a declared, path-bearing key field, so the target // – which came out of the schema – always has one. - const keyField = fieldNamed(target, target.key.field) as KeyedField; - return { target, keyField }; + return fieldNamed(target, target.key.field) as KeyedField; +} + +/** + * The one-triple hop that reads a referent’s `rdf:type`, for a reference whose + * referent may be of several kinds: emitted under `rdf:type` itself, which + * framing turns into the node’s `@type`, and bound in an `OPTIONAL` so an + * untyped referent keeps its row and falls back to the first target declared. + */ +function buildTypeHop( + referent: TermVariable, + counter: VariableCounter, +): { readonly triple: TripleNesting; readonly pattern: Pattern } { + const type = factory.termVariable(`t${counter.next++}`, factory.gen()); + const predicate = factory.termNamed(factory.gen(), RDF_TYPE); + return { + triple: factory.triple(referent, predicate, type), + pattern: factory.patternOptional( + [ + factory.patternBgp( + [factory.triple(referent, predicate, type)], + factory.gen(), + ), + ], + factory.gen(), + ), + }; } /** A key field as the schema guarantees it: path-bearing. */ diff --git a/packages/search-pipeline/test/extraction.test.ts b/packages/search-pipeline/test/extraction.test.ts index 0ed63c05..127c02ea 100644 --- a/packages/search-pipeline/test/extraction.test.ts +++ b/packages/search-pipeline/test/extraction.test.ts @@ -11,10 +11,9 @@ import { defineSearchType, searchSchema } from '@lde/search'; import { fieldNamed, irAlias, - labelSourceNameOf, - localLookupTypeOf, + localLookupTargetsOf, + referencedTargetsOf, referenceTypeNamed, - rootTypeNamed, } from '@lde/search/adapter'; import type { SearchSchema, SearchType } from '@lde/search'; import { extractionQuery, extractionQueryString } from '../src/extraction.js'; @@ -720,26 +719,27 @@ describe('extraction ⟷ projection contract', () => { } // A `local` lookup: the projection shapes the referent through the // TARGET’s own declaration, so it reads that type’s aliases off it. - const local = localLookupTypeOf(field, schema); - if (local !== undefined && !onPath.has(local.name)) { - for (const alias of projectionReads(local, schema, onPath)) { + const local = localLookupTargetsOf(field, schema).filter( + (target) => !onPath.has(target.name), + ); + for (const target of local) { + for (const alias of projectionReads(target, schema, onPath)) { aliases.add(alias); } - continue; } // A reference into a keyed type: the projection reads the referent’s key // candidates off the frame, under the TARGET’s alias for its key field. - const targetName = labelSourceNameOf(field); - const target = - targetName === undefined - ? undefined - : rootTypeNamed(schema, targetName); - const keyField = - target?.key === undefined - ? undefined - : fieldNamed(target, target.key.field); - if (target !== undefined && keyField !== undefined) { - aliases.add(irAlias(target, keyField)); + for (const target of referencedTargetsOf(field, schema)) { + if (local.includes(target)) { + continue; + } + const keyField = + target.key === undefined + ? undefined + : fieldNamed(target, target.key.field); + if (keyField !== undefined) { + aliases.add(irAlias(target, keyField)); + } } } return aliases; diff --git a/packages/search-pipeline/test/polymorphic-extraction.test.ts b/packages/search-pipeline/test/polymorphic-extraction.test.ts new file mode 100644 index 00000000..406f7c7d --- /dev/null +++ b/packages/search-pipeline/test/polymorphic-extraction.test.ts @@ -0,0 +1,151 @@ +import { describe, expect, it } from 'vitest'; +import { defineSearchType, searchSchema } from '@lde/search'; +import { irAlias } from '@lde/search/adapter'; +import { extractionQueryString } from '../src/extraction.js'; + +const SCHEMA = 'https://schema.org/'; +const RDF_TYPE = 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type'; + +/** Keyed on an authority IRI: what makes the hop load-bearing. */ +const person = defineSearchType({ + name: 'Person', + class: `${SCHEMA}Person`, + labelField: 'name', + key: { field: 'sameAs' }, + fields: [ + { + name: 'name', + kind: 'text', + path: `<${SCHEMA}name>`, + locales: ['nl'], + output: true, + searchable: { weight: 1 }, + }, + { + name: 'sameAs', + kind: 'reference', + path: `<${SCHEMA}sameAs>`, + array: true, + }, + ], +}); + +const organization = defineSearchType({ + name: 'Organization', + class: `${SCHEMA}Organization`, + labelField: 'name', + fields: [ + { + name: 'name', + kind: 'text', + path: `<${SCHEMA}name>`, + locales: ['nl'], + output: true, + searchable: { weight: 1 }, + }, + { + name: 'location', + kind: 'keyword', + path: `<${SCHEMA}location>`, + output: true, + }, + ], +}); + +describe('extracting a lookup that names several targets', () => { + it('reads the referent’s rdf:type, so the projection can tell which target it is', () => { + const work = defineSearchType({ + name: 'Work', + class: `${SCHEMA}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + path: `<${SCHEMA}creator>`, + array: true, + output: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }, + ], + }); + const query = extractionQueryString( + work, + searchSchema(work, person, organization), + ); + + // Emitted as `rdf:type` itself, which framing carries as `@type`, and + // bound OPTIONAL so an untyped referent keeps its row. + expect(query).toContain(`<${RDF_TYPE}>`); + expect(query).toContain('OPTIONAL'); + // Every keyed target contributes its key hop, under its own alias – a + // plain lookup expands nothing else. + expect(query).toContain(irAlias(person, person.fields[1])); + expect(query).not.toContain(irAlias(organization, organization.fields[1])); + }); + + it('expands a local lookup through every target, in one OPTIONAL union', () => { + const edge = defineSearchType({ + name: 'CreatorEdge', + fields: [ + { + name: 'creator', + kind: 'reference', + path: `<${SCHEMA}creator>`, + output: true, + ref: { + strategy: 'lookup', + target: ['Person', 'Organization'], + local: true, + }, + }, + ], + }); + const work = defineSearchType({ + name: 'Work', + class: `${SCHEMA}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + path: `<${SCHEMA}creator>`, + array: true, + output: true, + ref: { strategy: 'inline', typeName: 'CreatorEdge' }, + }, + ], + }); + const query = extractionQueryString( + work, + searchSchema(work, edge, person, organization), + ); + + expect(query).toContain(`<${RDF_TYPE}>`); + // Each target's fields under that target's own aliases: the projection + // reads a referent through whichever declaration its type matches. + for (const field of person.fields) { + expect(query).toContain(irAlias(person, field)); + } + for (const field of organization.fields) { + expect(query).toContain(irAlias(organization, field)); + } + }); + + it('reads no rdf:type for a lookup naming one target', () => { + const work = defineSearchType({ + name: 'Work', + class: `${SCHEMA}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + path: `<${SCHEMA}creator>`, + output: true, + ref: { strategy: 'lookup', target: 'Person' }, + }, + ], + }); + expect( + extractionQueryString(work, searchSchema(work, person)), + ).not.toContain(RDF_TYPE); + }); +}); diff --git a/packages/search-typesense/src/collection-definition.ts b/packages/search-typesense/src/collection-definition.ts index c794d385..b42d78bc 100644 --- a/packages/search-typesense/src/collection-definition.ts +++ b/packages/search-typesense/src/collection-definition.ts @@ -11,7 +11,8 @@ import { isInlineReference, isInternalField, joinGraph, - localLookupTypeOf, + localLookupTargetsOf, + TARGET_FIELD, nestedFieldName, nestedReferenceType, physicalFields, @@ -148,25 +149,24 @@ export function buildCollectionDefinition( } /** - * The type a field nests, or `undefined` when it nests nothing. Throws when the - * type carries an inline reference the caller gave no schema to resolve: the - * collection would silently store the reference as a string the projection - * never writes, so every document would fail to import. + * The types a field nests – one for an inline reference or a single-target + * `local` lookup, several for a `local` lookup naming several targets – or + * none when it nests nothing. Throws when the type carries an inline reference + * the caller gave no schema to resolve: the collection would silently store + * the reference as a string the projection never writes, so every document + * would fail to import. */ -function nestedTypeOf( +function nestedTypesOf( searchType: SearchType, field: SearchField, schema: SearchSchema | undefined, -): SearchType | undefined { +): readonly SearchType[] { const nested = - schema === undefined - ? undefined - : (nestedReferenceType(schema, field) ?? - localLookupTypeOf(field, schema)); + schema === undefined ? [] : nestedTypesOfNestedField(field, schema); // Reached only for a field carrying a Role, so an inline reference here // stores entries: a Role-less one is an internal reading device, pruned // before the writer. - if (nested === undefined && isInlineReference(field)) { + if (nested.length === 0 && isInlineReference(field)) { throw new Error( `Building the collection for “${searchType.name}” needs the search schema its surfaced inline reference “${field.name}” resolves against; pass it as the collection-definition option “schema”.`, ); @@ -246,13 +246,14 @@ function typesenseFields( if (isInternalField(field)) { return []; } - const nested = nestedTypeOf(searchType, field, schema); - if (nested !== undefined) { + const nested = nestedTypesOf(searchType, field, schema); + if (nested.length > 0) { return [ - ...nestedFields( + ...nestedObjectFields( field.name, field, nested, + nested.length > 1, schema as SearchSchema, defaultLocale, false, @@ -373,6 +374,67 @@ function typesenseFields( * the children rather than declared, and this is the one place that invariant * can be got wrong. */ +function nestedObjectFields( + prefix: string, + reference: SearchField, + nestedTypes: readonly SearchType[], + polymorphic: boolean, + schema: SearchSchema, + defaultLocale: string | undefined, + withinArray: boolean, + onPath: ReadonlySet, +): CollectionFieldSchema[] { + // `polymorphic` is about the DECLARATION, `nestedTypes` about what is + // descended into here: a lookup naming several targets of which only one is + // still off the path stores a discriminator all the same. + if (!polymorphic) { + return nestedFields( + prefix, + reference, + nestedTypes[0], + schema, + defaultLocale, + withinArray, + onPath, + ); + } + // A lookup naming several targets stores a referent of whichever kind it + // turns out to be under ONE nested object, so the object declares the union + // of the targets’ fields – each declared once, and `searchSchema` has + // already refused two targets declaring one name differently – plus the + // discriminator the projection writes to say which kind an entry is. + const byName = new Map(); + for (const nestedType of nestedTypes) { + for (const declared of nestedFields( + prefix, + reference, + nestedType, + schema, + defaultLocale, + withinArray, + onPath, + )) { + if (!byName.has(declared.name)) { + byName.set(declared.name, declared); + } + } + } + const flattensToArray = withinArray || reference.array === true; + const children = [...byName.values()].filter( + (declared) => declared.name !== prefix, + ); + children.push(targetField(prefix, flattensToArray)); + return [ + { + name: prefix, + type: flattensToArray ? 'object[]' : 'object', + index: children.some((child) => child.index !== false), + optional: reference.required !== true, + }, + ...children, + ]; +} + function nestedFields( prefix: string, reference: SearchField, @@ -413,8 +475,11 @@ function nestedFields( if (isInternalField(field)) { continue; } - const deeper = nestedTypeOfNestedField(field, schema); - if (deeper !== undefined && walked.has(deeper.name)) { + const deeper = nestedTypesOfNestedField(field, schema); + // Descend into the types not yet on the path; where every one is, the + // cycle stops here. + const descend = deeper.filter((type) => !walked.has(type.name)); + if (deeper.length > 0 && descend.length === 0) { // The cycle stops here, and so does the frame – but a value is still // stored: the extraction falls back to the target's key hop, and the // projection writes the referent as an `{id}` object beside its identity @@ -432,6 +497,9 @@ function nestedFields( index: false, optional: true, }, + // A cut lookup naming several targets still says which one it + // stored, exactly as an expanded one does. + ...(deeper.length > 1 ? [targetField(boundary, boundaryArray)] : []), { // Always a Root Type, so always identified: `searchSchema` rejects // inline cycles, so the only way to arrive at a type already on the @@ -445,12 +513,13 @@ function nestedFields( ); continue; } - if (deeper !== undefined) { + if (descend.length > 0) { children.push( - ...nestedFields( + ...nestedObjectFields( nestedFieldName(prefix, field.name), field, - deeper, + descend, + deeper.length > 1, schema, defaultLocale, flattensToArray, @@ -563,14 +632,28 @@ function nestedIdentityFields( ]; } -/** The type a *nested* field itself nests: another inline reference’s type, or - * the Root Type a {@link ReferenceStrategy.local local} lookup projects its - * endpoint through. */ -function nestedTypeOfNestedField( +/** The stored discriminator of a lookup naming several targets + * ({@link TARGET_FIELD}): a stored value, never indexed. */ +function targetField(prefix: string, array: boolean): CollectionFieldSchema { + return { + name: nestedFieldName(prefix, TARGET_FIELD), + type: array ? 'string[]' : 'string', + index: false, + optional: true, + }; +} + +/** The types a field nests: another inline reference’s type, or the Root + * Types a {@link ReferenceStrategy.local local} lookup projects its endpoint + * through. */ +function nestedTypesOfNestedField( field: SearchField, schema: SearchSchema, -): SearchType | undefined { - return nestedReferenceType(schema, field) ?? localLookupTypeOf(field, schema); +): readonly SearchType[] { + const referenceType = nestedReferenceType(schema, field); + return referenceType === undefined + ? localLookupTargetsOf(field, schema) + : [referenceType]; } /** diff --git a/packages/search-typesense/src/lookup.ts b/packages/search-typesense/src/lookup.ts index 4521981d..db8fbe7b 100644 --- a/packages/search-typesense/src/lookup.ts +++ b/packages/search-typesense/src/lookup.ts @@ -10,7 +10,7 @@ import { fieldNamed, labelFieldOf, nestedReferenceType, - rootTypeNamed, + referencedTargetsOf, } from '@lde/search/adapter'; import { escapeFilterValue } from './query-compiler.js'; @@ -31,18 +31,26 @@ import { escapeFilterValue } from './query-compiler.js'; export type ResolvedReferents = | { readonly via: 'lookup'; - /** The Root Type these referents are declared by – reconstruction reads - * them through the target’s own declaration, never the referrer’s. */ - readonly target: RootType; - readonly documents: ReadonlyMap>; - /** Keyed by the reference field’s name on the type this level carries. */ - readonly children: ReadonlyMap; + /** By IRI. A lookup naming several targets holds referents of any of + * them here, each saying which. */ + readonly documents: ReadonlyMap; } | { readonly via: 'nested'; readonly children: ReadonlyMap; }; +/** One referent a lookup level fetched. */ +export interface ResolvedReferent { + /** The Root Type this referent is declared by – the target whose collection + * answered for it. Reconstruction reads it through that target’s own + * declaration, never the referrer’s. */ + readonly target: RootType; + readonly document: Record; + /** The levels below, keyed by the reference field’s name on `target`. */ + readonly children: ReadonlyMap; +} + /** Typesense caps a filter list; the same batch size the label lookup uses. */ const BATCH_SIZE = 200; @@ -104,45 +112,67 @@ export async function resolveProjection( // hand-built query from throwing here rather than at the port’s guard. return undefined; } - const target = rootTypeNamed(schema, field.ref.target); - const collection = - target === undefined ? undefined : collections.get(target.class); - if (target === undefined || collection === undefined) { - return undefined; - } - const include = includeFields(target, level.fields, level.resolve); + const targets = referencedTargetsOf(field, schema).filter((target) => + collections.has(target.class), + ); const iris = distinctIris(parents, name); - // Nothing to read: no referent named, or the selection reduced to the - // `id` the referring document already carries. - if (iris.length === 0 || include.length <= 1) { + if (targets.length === 0 || iris.length === 0) { return undefined; } - const documents = await fetchReferents( - client, - collection, - iris, - include, - onError, - ); - return [ - name, - { - via: 'lookup', - target, - documents, + // One fetch per target, concurrently: a referent lives in one of the + // collections, and which one is what a lookup naming several targets + // exists to find out – so with several, every collection is asked even + // for a selection reduced to `id`, since the answer types the referent. + // With one target, a selection reduced to the `id` the referring + // document already carries reads nothing. + const perTarget = await Promise.all( + targets.map(async (target) => { + const include = includeFields(target, level.fields, level.resolve); + if (targets.length === 1 && include.length <= 1) { + return undefined; + } + const fetched = await fetchReferents( + client, + collections.get(target.class) as string, + iris, + include, + onError, + ); // The level below reads the IRIs off the documents this one just // fetched – one more round-trip for the page, whatever its size. - children: await resolveProjection( + const children = await resolveProjection( client, level.resolve, target, schema, collections, - [...documents.values()], + [...fetched.values()], onError, - ), - }, - ] as const; + ); + return { target, fetched, children }; + }), + ); + // Declaration order is precedence: an IRI two collections hold belongs + // to the target declared first. + const documents = new Map(); + for (const resolved of perTarget) { + if (resolved === undefined) { + continue; + } + for (const [iri, document] of resolved.fetched) { + if (!documents.has(iri)) { + documents.set(iri, { + target: resolved.target, + document, + children: resolved.children, + }); + } + } + } + if (perTarget.every((resolved) => resolved === undefined)) { + return undefined; + } + return [name, { via: 'lookup', documents }] as const; }), ); for (const level of levels) { diff --git a/packages/search-typesense/src/query-compiler.ts b/packages/search-typesense/src/query-compiler.ts index cdc0bc2f..e05791db 100644 --- a/packages/search-typesense/src/query-compiler.ts +++ b/packages/search-typesense/src/query-compiler.ts @@ -23,7 +23,7 @@ import { isRangeFacet, isWelded, joinGraph, - localLookupTypeOf, + localLookupTargetsOf, pageForOffset, physicalFields, nestedReferenceType, @@ -268,16 +268,24 @@ function collectSearchable( if (isInternalField(field)) { continue; } + const referenceType = nestedReferenceType(schema, field); const nested = - nestedReferenceType(schema, field) ?? localLookupTypeOf(field, schema); + referenceType === undefined + ? localLookupTargetsOf(field, schema) + : [referenceType]; // Cut on the CHILD, not on entry, and against the set this level was // reached with – the same boundary the collection's own walk cuts at // (`nestedFields`). Returning on entry instead skipped the companions of // the level the collection HAD declared, leaving them indexed and absent - // from `query_by` wherever a type reached itself. - if (nested !== undefined && !onPath.has(nested.name)) { + // from `query_by` wherever a type reached itself. Several targets each + // contribute their searchable fields: the one nested object declares them + // all, and free text has to reach a referent of either kind. + for (const nestedType of nested) { + if (onPath.has(nestedType.name)) { + continue; + } collectSearchable( - nested, + nestedType, locale, schema, qualify(prefix, field.name), diff --git a/packages/search-typesense/src/search.ts b/packages/search-typesense/src/search.ts index b1caa61c..961af012 100644 --- a/packages/search-typesense/src/search.ts +++ b/packages/search-typesense/src/search.ts @@ -1,6 +1,7 @@ import type { Client } from 'typesense'; import type { SearchParams } from 'typesense/lib/Typesense/Documents.js'; import { + NESTED_DOCUMENT_TYPE, type FacetBucket, type FacetsOutcome, type LocalizedValue, @@ -31,13 +32,15 @@ import { isUnsatisfiable, joinGraph, labelFieldOf, - labelSourceNameOf, - labelTargetNameOf, - localLookupTypeOf, + labelSourceNamesOf, + labelTargetNamesOf, + localLookupTargetsOf, nestedReferenceType, outputFields, physicalFields, + referencedTargetsOf, referenceFields, + storedTargetOf, } from '@lde/search/adapter'; import { buildSearchParams, @@ -45,7 +48,11 @@ import { type BuildSearchParamsOptions, } from './query-compiler.js'; import { deriveCollectionName } from './collection-name.js'; -import { resolveProjection, type ResolvedReferents } from './lookup.js'; +import { + resolveProjection, + type ResolvedReferent, + type ResolvedReferents, +} from './lookup.js'; /** Where the engine reads documents – plus every query-compiler knob * ({@link BuildSearchParamsOptions}), declared once there and forwarded @@ -189,7 +196,12 @@ export function createTypesenseSearchEngine< const typesByName = new Map( [...schema.values()].map((searchType) => [searchType.name, searchType]), ); - const labelSources = new Map>( + // Per field, one source per target it names, in order of precedence – a + // reference naming several targets reads labels from every one of them. + const labelSources = new Map< + string, + ReadonlyMap + >( [...schema.values()].map((searchType) => [ searchType.class, new Map( @@ -197,21 +209,19 @@ export function createTypesenseSearchEngine< // An INLINE reference names its label target one level in, through // its identity companion: the companion holds that field's ids, so // that field's target is what can label them. - .filter((field) => labelTargetNameOf(field, schema) !== undefined) - .map((field) => { - const source = typesByName.get( - labelTargetNameOf(field, schema) as string, - ) as RootType; - const labelField = labelFieldOf(source) as TextField; - return [ - field.name, - { + .filter((field) => labelTargetNamesOf(field, schema).length > 0) + .map((field) => [ + field.name, + labelTargetNamesOf(field, schema).map((targetName) => { + const source = typesByName.get(targetName) as RootType; + const labelField = labelFieldOf(source) as TextField; + return { collection: collections.get(source.class) as string, labelField, queryBy: physicalFields(labelField).search.join(','), - }, - ]; - }), + }; + }), + ]), ), ]), ); @@ -242,7 +252,9 @@ export function createTypesenseSearchEngine< type, [ ...new Map( - [...sources.values()].map((source) => [source.collection, source]), + [...sources.values()] + .flat() + .map((source) => [source.collection, source]), ).values(), ], ]), @@ -252,7 +264,7 @@ export function createTypesenseSearchEngine< // not re-derive or re-resolve them on every search. const outputReferenceSources = new Map< string, - readonly { name: string; source: LabelSource }[] + readonly { name: string; sources: readonly LabelSource[] }[] >( [...schema.values()].map((searchType) => { const sources = labelSources.get(searchType.class); @@ -272,13 +284,13 @@ export function createTypesenseSearchEngine< // the entry – and their FACET buckets are labelled by a separate // path that reads the companion. nestedReferenceType(schema, field) === undefined && - localLookupTypeOf(field, schema) === undefined, + localLookupTargetsOf(field, schema).length === 0, ) .map((field) => ({ name: field.name, - source: (sources as Map).get( + sources: (sources as Map).get( field.name, - ) as LabelSource, + ) as readonly LabelSource[], })), ]; }), @@ -441,6 +453,7 @@ export function createTypesenseSearchEngine< [response], labelSources.get(searchType.class), outputReferenceSources.get(searchType.class) ?? [], + distinctLabelSources.get(searchType.class) ?? [], ), ); // What the caller asked to resolve, level by level. Independent of the @@ -520,6 +533,7 @@ export function createTypesenseSearchEngine< responses, labelSources.get(searchType.class), outputReferenceSources.get(searchType.class) ?? [], + distinctLabelSources.get(searchType.class) ?? [], ), ); // multi_search reports a failed entry inline instead of rejecting the @@ -632,37 +646,44 @@ async function loadAllLabels( */ function labelLookupGroups( responses: readonly TypesenseSearchResponse[], - sources: ReadonlyMap | undefined, - outputSources: readonly { name: string; source: LabelSource }[], + sources: ReadonlyMap | undefined, + outputSources: readonly { name: string; sources: readonly LabelSource[] }[], + ordered: readonly LabelSource[], ): LabelLookupGroup[] { if (sources === undefined || sources.size === 0) { return []; } - const irisByCollection = new Map< - string, - { source: LabelSource; iris: Set } - >(); - const add = (source: LabelSource, iri: string): void => { - let group = irisByCollection.get(source.collection); - if (group === undefined) { - group = { source, iris: new Set() }; - irisByCollection.set(source.collection, group); + // One group per collection, in the order the TYPE’s fields first name them + // – seeded up front rather than in the order the hits happen to mention + // them, because `fetchLabels` reads group order as precedence and a label + // is resolved per type, not per field: an IRI two collections hold gets the + // label of the one the type declares first, whichever field carried it. + const irisByCollection = new Map( + ordered.map((source) => [ + source.collection, + { source, iris: new Set() }, + ]), + ); + // An IRI of a field naming several targets travels to EVERY one of their + // collections: which holds it is not known until one answers. + const add = (fieldSources: readonly LabelSource[], iri: string): void => { + for (const source of fieldSources) { + irisByCollection.get(source.collection)?.iris.add(iri); } - group.iris.add(iri); }; for (const response of responses) { // Hits only carry labels for OUTPUT reference fields (reconstructDocument // skips non-output fields); `outputSources` pairs each with its resolved // source, precomputed per type. for (const hit of response.hits ?? []) { - for (const { name, source } of outputSources) { + for (const { name, sources: fieldSources } of outputSources) { const raw = hit.document[name]; if (Array.isArray(raw)) { for (const value of raw) { - add(source, String(value)); + add(fieldSources, String(value)); } } else if (typeof raw === 'string') { - add(source, raw); + add(fieldSources, raw); } } } @@ -670,19 +691,18 @@ function labelLookupGroups( // like `class`); resolve them in the same lookup. Skip a non-source facet // (e.g. a keyword facet) in one check instead of probing every bucket. for (const facet of response.facet_counts ?? []) { - const source = sources.get(facet.field_name); - if (source === undefined) { + const facetSources = sources.get(facet.field_name); + if (facetSources === undefined) { continue; } for (const bucket of facet.counts) { - add(source, bucket.value); + add(facetSources, bucket.value); } } } - return [...irisByCollection.values()].map(({ source, iris }) => ({ - source, - iris: [...iris], - })); + return [...irisByCollection.values()] + .filter(({ iris }) => iris.size > 0) + .map(({ source, iris }) => ({ source, iris: [...iris] })); } /** @@ -797,12 +817,20 @@ export async function fetchLabels( return; } for (const hit of result.hits ?? []) { + const id = String(hit.document.id); + // First group wins: the groups arrive in the order a field declares its + // targets, so an IRI two collections hold is labelled by the target + // declared first – the same precedence the lookup and the projection + // apply. + if (labels.has(id)) { + continue; + } const label = localizedValue( hit.document, groupPerSearch[index].source.labelField, ); if (label !== undefined) { - labels.set(String(hit.document.id), label); + labels.set(id, label); } } }); @@ -823,7 +851,10 @@ function mergeLabels( const merged = new Map(); for (const map of maps) { for (const [iri, label] of map) { - merged.set(iri, label); + // First source wins, as it does for a fetched lookup (`fetchLabels`). + if (!merged.has(iri)) { + merged.set(iri, label); + } } } return merged; @@ -889,7 +920,7 @@ export function parseSearchResponse( // its IRIs. const referenceFacets = new Set( referenceFields(searchType) - .filter((field) => labelTargetNameOf(field, schema) !== undefined) + .filter((field) => labelTargetNamesOf(field, schema).length > 0) .map((field) => field.name), ); const facets: Record = {}; @@ -965,19 +996,19 @@ function logicalValue( nested, labels, schema, - resolved?.children, + resolved?.via === 'nested' ? resolved.children : undefined, ); } // Checked BEFORE the plain lookup below, because a `local` lookup // resolves through the same kind of level while storing a different // shape: its stored value is the endpoint's own document, not an id, so // reading it as one would stringify the object into a bogus `id`. - const localType = localLookupTypeOf(field, schema); - if (localType !== undefined) { + const localTargets = localLookupTargetsOf(field, schema); + if (localTargets.length > 0) { return localLookupValue( flat[field.name], field, - localType, + localTargets, resolved, labels, schema, @@ -1071,18 +1102,9 @@ function nestedValue( (referent): referent is Record => typeof referent === 'object' && referent !== null, ); - const documents: NestedDocument[] = referents.map((referent) => { - const document = reconstructDocument( - referent, - referenceType, - labels, - schema, - children, - ); - return typeof referent.id === 'string' - ? { id: referent.id, ...document } - : document; - }); + const documents: NestedDocument[] = referents.map((referent) => + nestedDocument(referent, referenceType, labels, schema, children), + ); if (documents.length === 0) { return undefined; } @@ -1111,14 +1133,16 @@ function nestedValue( function localLookupValue( raw: unknown, field: ReferenceField, - target: SearchType, + targets: readonly RootType[], resolved: ResolvedReferents | undefined, labels: ReadonlyMap, schema: SearchSchema, ): SearchValue | undefined { const fetched = - resolved?.via === 'lookup' ? resolved.documents : new Map(); - const entries = (Array.isArray(raw) ? raw : [raw]) + resolved?.via === 'lookup' + ? resolved.documents + : new Map(); + const documents = (Array.isArray(raw) ? raw : [raw]) .filter( (entry): entry is Record => typeof entry === 'object' && entry !== null, @@ -1126,16 +1150,67 @@ function localLookupValue( .map((entry) => { const id = typeof entry.id === 'string' ? entry.id : undefined; const authoritative = id === undefined ? undefined : fetched.get(id); - return authoritative ?? entry; + // An entry no collection answered for is read through the declaration + // it was stored under – which it says itself where there were several + // to choose from, and the only one otherwise. + const referent: ResolvedReferent = authoritative ?? { + target: storedTargetOf(entry, targets), + document: entry, + children: new Map(), + }; + return typedDocument( + targets.length > 1, + referent.target, + nestedDocument( + referent.document, + referent.target, + labels, + schema, + referent.children, + ), + ); }); - return nestedValue( - entries, - field, - target, + if (documents.length === 0) { + return undefined; + } + return field.array === true ? documents : documents[0]; +} + +/** + * A nested document read through one of a lookup’s **several** targets, marked + * with that type’s name ({@link NESTED_DOCUMENT_TYPE}) so a surface can tell + * which it is – the marker is a symbol, so it never surfaces as a field. A + * single-target lookup marks nothing: there is nothing to tell apart. + */ +function typedDocument( + polymorphic: boolean, + target: RootType, + document: NestedDocument, +): NestedDocument { + return polymorphic + ? { ...document, [NESTED_DOCUMENT_TYPE]: target.name } + : document; +} + +/** One nested document rebuilt from a flat referent, carrying its `id` where + * it has one – the shared body of every nesting that reads a referent. */ +function nestedDocument( + referent: Record, + searchType: SearchType, + labels: ReadonlyMap, + schema: SearchSchema, + children: ReadonlyMap | undefined, +): NestedDocument { + const document = reconstructDocument( + referent, + searchType, labels, schema, - resolved?.children, + children, ); + return typeof referent.id === 'string' + ? { id: referent.id, ...document } + : document; } /** @@ -1157,20 +1232,21 @@ function lookupValue( return undefined; } const iris = Array.isArray(raw) ? (raw as string[]) : [String(raw)]; + const polymorphic = referencedTargetsOf(field, schema).length > 1; const documents: NestedDocument[] = iris.map((iri) => { const referent = resolved.documents.get(iri); return referent === undefined ? { id: iri } - : { + : typedDocument(polymorphic, referent.target, { id: iri, ...reconstructDocument( - referent, - resolved.target, + referent.document, + referent.target, labels, schema, - resolved.children, + referent.children, ), - }; + }); }); return field.array === true ? documents : documents[0]; } @@ -1191,7 +1267,7 @@ function referenceValue( // a label, even if the (cached, full-collection) map happens to hold this // IRI from another source. const label = - labelSourceNameOf(field) === undefined ? undefined : labels.get(iri); + labelSourceNamesOf(field).length === 0 ? undefined : labels.get(iri); return label === undefined ? { id: iri } : { id: iri, label }; }); return field.array === true ? references : references[0]; diff --git a/packages/search-typesense/test/polymorphic-lookup.test.ts b/packages/search-typesense/test/polymorphic-lookup.test.ts new file mode 100644 index 00000000..023f5620 --- /dev/null +++ b/packages/search-typesense/test/polymorphic-lookup.test.ts @@ -0,0 +1,394 @@ +import { describe, expect, it } from 'vitest'; +import { + defineSearchType, + NESTED_DOCUMENT_TYPE, + searchSchema, + type NestedDocument, + type SearchQuery, +} from '@lde/search'; +import { TARGET_FIELD } from '@lde/search/adapter'; +import { createTypesenseSearchEngine } from '../src/search.js'; +import { buildCollectionDefinition } from '../src/collection-definition.js'; +import { fakeTypesenseClient, filterByIds } from './fake-typesense-client.js'; + +const person = defineSearchType({ + name: 'Person', + class: 'https://example.org/Person', + labelField: 'name', + fields: [ + { + name: 'name', + kind: 'text', + locales: ['nl', 'und'], + output: true, + searchable: { weight: 1 }, + }, + { name: 'birthDate', kind: 'keyword', output: true }, + ], +}); + +const organization = defineSearchType({ + name: 'Organization', + class: 'https://example.org/Organization', + labelField: 'name', + fields: [ + { + name: 'name', + kind: 'text', + locales: ['nl', 'und'], + output: true, + searchable: { weight: 1 }, + }, + { name: 'location', kind: 'keyword', output: true }, + ], +}); + +/** A plain lookup at the root, faceted: what the issue was filed against. */ +const work = defineSearchType({ + name: 'CreativeWork', + class: 'https://example.org/CreativeWork', + fields: [ + { name: 'title', kind: 'text', locales: ['nl'], output: true }, + { + name: 'creator', + kind: 'reference', + array: true, + output: true, + filterable: true, + facetable: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }, + ], +}); + +/** The same range one level in: an edge nesting a `local` lookup. */ +const creatorEdge = defineSearchType({ + name: 'CreatorEdge', + fields: [ + { name: 'role', kind: 'keyword', output: true }, + { + name: 'creator', + kind: 'reference', + output: true, + ref: { + strategy: 'lookup', + target: ['Person', 'Organization'], + local: true, + }, + }, + ], +}); + +const edgedWork = defineSearchType({ + name: 'Painting', + class: 'https://example.org/Painting', + fields: [ + { + name: 'creator', + kind: 'reference', + array: true, + output: true, + filterable: true, + ref: { strategy: 'inline', typeName: 'CreatorEdge', identity: 'creator' }, + }, + ], +}); + +const schema = searchSchema(work, edgedWork, person, organization, creatorEdge); +const collections = { + Person: 'people', + Organization: 'organizations', + CreativeWork: 'works', + Painting: 'paintings', +}; + +const base: SearchQuery = { + where: [], + orderBy: [], + limit: 10, + offset: 0, + facets: [], + locale: 'nl', +}; + +/** `p/1` is a person, `o/1` an organization, `both` is in both collections. */ +const people: Record> = { + 'https://p/1': { + id: 'https://p/1', + name_nl: 'Frank Koel', + birthDate: '1901', + }, + 'https://both': { id: 'https://both', name_nl: 'Als persoon' }, +}; +const organizations: Record> = { + 'https://o/1': { + id: 'https://o/1', + name_nl: 'Zusters Franciscanessen', + location: 'Roermond', + }, + 'https://both': { id: 'https://both', name_nl: 'Als organisatie' }, +}; + +/** The lookups a search made, told from the label round-trip by their + * `include_fields`. */ +function client(rootResponse: Record) { + const searches: Record[] = []; + const fake = fakeTypesenseClient({ + multiSearch: (search) => { + if (search.query_by_weights !== undefined) { + return rootResponse; + } + if (search.include_fields !== undefined) { + searches.push(search); + } + const documents = search.collection === 'people' ? people : organizations; + const include = + search.include_fields === undefined + ? undefined + : new Set(String(search.include_fields).split(',')); + const hits = filterByIds(String(search.filter_by)) + .filter((id) => documents[id] !== undefined) + .map((id) => ({ + document: + include === undefined + ? documents[id] + : Object.fromEntries( + Object.entries(documents[id]).filter(([key]) => + include.has(key), + ), + ), + })); + return { found: hits.length, hits }; + }, + }); + return { fake, searches }; +} + +const typeOf = (document: unknown) => + (document as NestedDocument)[NESTED_DOCUMENT_TYPE]; + +describe('resolving a lookup over several targets', () => { + const hits = { + found: 1, + hits: [ + { + document: { + id: 'https://w/1', + title_nl: 'Werk', + creator: [ + 'https://p/1', + 'https://o/1', + 'https://both', + 'https://gone', + ], + }, + }, + ], + }; + + it('asks every target’s collection, and types each referent by the one that answered', async () => { + const { fake, searches } = client(hits); + const engine = createTypesenseSearchEngine(fake.client, schema, { + collections, + }); + + const result = await engine.search(work as never, { + ...base, + resolve: { creator: { fields: ['name', 'birthDate', 'location'] } }, + }); + const creators = (result.hits[0].document as Record) + .creator as readonly NestedDocument[]; + + expect(searches.map((search) => search.collection).sort()).toEqual([ + 'organizations', + 'people', + ]); + expect(creators.map(typeOf)).toEqual([ + 'Person', + 'Organization', + // Held by both: the target declared first wins. + 'Person', + // Held by neither: a bare id, of no type at all. + undefined, + ]); + expect(creators[0]).toMatchObject({ + id: 'https://p/1', + name: { nl: ['Frank Koel'] }, + birthDate: '1901', + }); + expect(creators[1]).toMatchObject({ + id: 'https://o/1', + name: { nl: ['Zusters Franciscanessen'] }, + location: 'Roermond', + }); + expect(creators[2]).toMatchObject({ name: { nl: ['Als persoon'] } }); + expect(creators[3]).toEqual({ id: 'https://gone' }); + // Each collection is asked only for what its own declaration serves. + const included = Object.fromEntries( + searches.map((search) => [search.collection, search.include_fields]), + ); + expect(included.people).not.toContain('location'); + expect(included.organizations).not.toContain('birthDate'); + }); + + it('fetches from every collection even for a selection reduced to id, since the answer types the referent', async () => { + const { fake, searches } = client(hits); + const engine = createTypesenseSearchEngine(fake.client, schema, { + collections, + }); + + const result = await engine.search(work as never, { + ...base, + resolve: { creator: { fields: [] } }, + }); + const creators = (result.hits[0].document as Record) + .creator as readonly NestedDocument[]; + + expect(searches).toHaveLength(2); + expect(creators.map(typeOf)).toEqual([ + 'Person', + 'Organization', + 'Person', + undefined, + ]); + }); + + it('labels facet buckets from whichever collection holds the value', async () => { + const { fake } = client({ + ...hits, + facet_counts: [ + { + field_name: 'creator', + counts: [ + { value: 'https://p/1', count: 3 }, + { value: 'https://o/1', count: 2 }, + { value: 'https://both', count: 1 }, + ], + }, + ], + }); + const engine = createTypesenseSearchEngine(fake.client, schema, { + collections, + }); + + const result = await engine.search(work as never, { + ...base, + facets: ['creator'], + }); + + expect((result.facets as Record).creator).toEqual([ + { value: 'https://p/1', count: 3, label: { nl: ['Frank Koel'] } }, + { + value: 'https://o/1', + count: 2, + label: { nl: ['Zusters Franciscanessen'] }, + }, + { value: 'https://both', count: 1, label: { nl: ['Als persoon'] } }, + ]); + }); +}); + +describe('a stored referent of several possible kinds', () => { + const hits = { + found: 1, + hits: [ + { + document: { + id: 'https://w/2', + creator: [ + // Identified and indexed: the collection's record replaces it. + { + role: 'schilder', + creator: { + id: 'https://o/1', + name_und: 'Zusters', + [TARGET_FIELD]: 'Organization', + }, + }, + // Identified, not indexed: what was stated, read through the + // declaration it was stored under. + { + role: 'drukker', + creator: { + id: 'https://o/gone', + name_und: 'Drukkerij', + location: 'Venlo', + [TARGET_FIELD]: 'Organization', + }, + }, + // Never identified: same, minus the id. + { + role: 'auteur', + creator: { name_und: 'Jan Jansen', [TARGET_FIELD]: 'Person' }, + }, + ], + creator_id: ['https://o/1', 'https://o/gone'], + }, + }, + ], + }; + + it('reads an entry no collection answers for through its stored discriminator', async () => { + const { fake } = client(hits); + const engine = createTypesenseSearchEngine(fake.client, schema, { + collections, + }); + + const result = await engine.search(edgedWork as never, { + ...base, + resolve: { + creator: { resolve: { creator: { fields: ['name', 'location'] } } }, + }, + }); + const entries = (result.hits[0].document as Record) + .creator as readonly Record[]; + const endpoints = entries.map((entry) => entry.creator as NestedDocument); + + expect(endpoints.map(typeOf)).toEqual([ + 'Organization', + 'Organization', + 'Person', + ]); + expect(endpoints[0]).toEqual({ + id: 'https://o/1', + name: { nl: ['Zusters Franciscanessen'] }, + location: 'Roermond', + [NESTED_DOCUMENT_TYPE]: 'Organization', + }); + expect(endpoints[1]).toEqual({ + id: 'https://o/gone', + name: { und: ['Drukkerij'] }, + location: 'Venlo', + [NESTED_DOCUMENT_TYPE]: 'Organization', + }); + expect(endpoints[2]).toEqual({ + name: { und: ['Jan Jansen'] }, + [NESTED_DOCUMENT_TYPE]: 'Person', + }); + // The discriminator is a storage detail, never a field. + for (const endpoint of endpoints) { + expect(endpoint).not.toHaveProperty(TARGET_FIELD); + } + }); +}); + +describe('declaring the collection for a stored referent of several kinds', () => { + it('declares the union of the targets’ fields once, plus the discriminator', () => { + const definition = buildCollectionDefinition(edgedWork, { schema }); + const names = definition.fields?.map((field) => field.name) ?? []; + + expect(names).toContain('creator.creator.id'); + expect(names).toContain('creator.creator.birthDate'); + expect(names).toContain('creator.creator.location'); + expect(names).toContain(`creator.creator.${TARGET_FIELD}`); + // The shared label field is declared once, not once per target. + expect( + names.filter((name) => name === 'creator.creator.name_[^_]+'), + ).toHaveLength(1); + expect( + definition.fields?.find( + (field) => field.name === `creator.creator.${TARGET_FIELD}`, + ), + ).toMatchObject({ type: 'string[]', index: false, optional: true }); + }); +}); diff --git a/packages/search-typesense/test/qualified-relation.test.ts b/packages/search-typesense/test/qualified-relation.test.ts index 5cec28ee..b672c0f5 100644 --- a/packages/search-typesense/test/qualified-relation.test.ts +++ b/packages/search-typesense/test/qualified-relation.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest'; import { defineSearchType, searchSchema } from '@lde/search'; -import { labelTargetNameOf } from '@lde/search/adapter'; +import { labelTargetNamesOf } from '@lde/search/adapter'; import { buildCollectionDefinition } from '../src/collection-definition.js'; import { buildSearchParams } from '../src/query-compiler.js'; @@ -765,9 +765,9 @@ describe('a facet policy over the companion', () => { describe('labelling the buckets of an edge’s facet', () => { it('reads the label target through the identity companion', () => { // An inline reference names no label source of its own, so reading only - // `labelSourceNameOf` left its buckets unlabelled – the facet policy was + // `labelSourceNamesOf` left its buckets unlabelled – the facet policy was // inherited one level in, but the labels were not. - expect(labelTargetNameOf(work.fields[0], schema)).toBe('Person'); + expect(labelTargetNamesOf(work.fields[0], schema)).toEqual(['Person']); }); it('names nothing for an inline reference without an identity', () => { @@ -787,11 +787,11 @@ describe('labelling the buckets of an edge’s facet', () => { }); expect( - labelTargetNameOf( + labelTargetNamesOf( displayOnly.fields[0], searchSchema(displayOnly, person, creatorEdge), ), - ).toBeUndefined(); + ).toEqual([]); }); }); diff --git a/packages/search/src/adapter.ts b/packages/search/src/adapter.ts index 3437e99f..bfe312e4 100644 --- a/packages/search/src/adapter.ts +++ b/packages/search/src/adapter.ts @@ -22,7 +22,10 @@ export { isInlineReference, nestedReferenceType, nestedFieldName, - localLookupTypeOf, + localLookupTargetsOf, + referencedTargetsOf, + storedTargetOf, + TARGET_FIELD, identityFieldOf, identityFieldName, referenceFields, @@ -36,8 +39,8 @@ export { OR_KEY, labelFieldOf, labelFieldNameOf, - labelSourceNameOf, - labelTargetNameOf, + labelSourceNamesOf, + labelTargetNamesOf, documentKeyOf, DEFAULT_LABEL_FIELD, isRangeFacet, diff --git a/packages/search/src/engine.ts b/packages/search/src/engine.ts index e9335473..3fca8171 100644 --- a/packages/search/src/engine.ts +++ b/packages/search/src/engine.ts @@ -169,8 +169,26 @@ export type SearchValue = */ export interface NestedDocument { readonly [field: string]: SearchValue | undefined; + /** + * The `name` of the Root Type this document is read through, where the + * field that carries it is a `lookup` naming **several** targets – the + * declaration it was resolved or stored against. Such a lookup serves + * referents of any of its targets under one field, and this is what tells a + * surface which one each referent is, so it can type the value per referent + * instead of claiming one kind for all of them. Absent where there is + * nothing to tell apart: a single-target lookup, or an inline reference’s + * entry. Keyed by a symbol so it can never collide with a declared field + * name. + */ + readonly [NESTED_DOCUMENT_TYPE]?: string; } +/** The key under which a {@link NestedDocument} carries the `name` of the + * Root Type it is read through. */ +export const NESTED_DOCUMENT_TYPE: unique symbol = Symbol( + 'lde.search.nestedDocumentType', +); + /** * A JSON-LD-style language map (`@container: @language`, `@set` arrays); the key * `und` carries untagged (`@none`) values. The surface flattens it to a diff --git a/packages/search/src/index.ts b/packages/search/src/index.ts index 73e5072f..93746734 100644 --- a/packages/search/src/index.ts +++ b/packages/search/src/index.ts @@ -80,6 +80,7 @@ export type { // Engine port + the logical result document returned across it. An engine is // bound to the whole SearchSchema at construction by its adapter factory. +export { NESTED_DOCUMENT_TYPE } from './engine.js'; export type { SearchEngine, SearchResult, diff --git a/packages/search/src/join-graph.ts b/packages/search/src/join-graph.ts index d2def7bd..83e84a0c 100644 --- a/packages/search/src/join-graph.ts +++ b/packages/search/src/join-graph.ts @@ -1,5 +1,5 @@ import { - labelSourceNameOf, + labelSourceNamesOf, referenceFields, type ReferenceField, type RootType, @@ -177,7 +177,9 @@ function declaredEdges( // field, which a Reference Type cannot carry (a nested field is `output` // only). So a joinable edge always has a collection at the far end without // a rule of its own. - const target = byName.get(labelSourceNameOf(field) as string) as RootType; + // One target, never several: `validateSearchType` refuses `joinable` on a + // reference naming more than one. + const target = byName.get(labelSourceNamesOf(field)[0]) as RootType; const claimed = claimedBy.get(target.name); if (claimed !== undefined) { throw new Error( diff --git a/packages/search/src/project.ts b/packages/search/src/project.ts index db242275..3488f283 100644 --- a/packages/search/src/project.ts +++ b/packages/search/src/project.ts @@ -10,21 +10,25 @@ import { displayFieldName, documentKeyOf, fieldNamed, - inheritedFacetKeys, + identityFieldOf, + inheritedFacetPolicies, inlineFramingDepth, irAlias, isAbsoluteIri, isInternalField, isInlineReference, isoToUnixSeconds, - labelSourceNameOf, - localLookupTypeOf, + localLookupTargetsOf, physicalFields, + referencedTargetsOf, referenceTypeNamed, - rootTypeNamed, + storedTargetOf, + TARGET_FIELD, type KeywordField, type ProjectionValue, + type FacetKeys, type ReferenceField, + type ReferenceType, type RootType, type SearchField, type SearchSchema, @@ -143,16 +147,27 @@ function pruneInternalFields( if (schema === undefined || field.kind !== 'reference') { continue; } - const nestedType = + const nestedTypes: readonly SearchType[] = field.ref?.strategy === 'inline' - ? referenceTypeNamed(schema, field.ref.typeName) - : localLookupTypeOf(field, schema); + ? [referenceTypeNamed(schema, field.ref.typeName)].filter( + (type): type is ReferenceType => type !== undefined, + ) + : localLookupTargetsOf(field, schema); const nested = document[field.name]; - if (nestedType === undefined || nested === undefined) { + if (nestedTypes.length === 0 || nested === undefined) { continue; } for (const referent of Array.isArray(nested) ? nested : [nested]) { - pruneInternalFields(referent as ProjectedNode, nestedType, schema); + pruneInternalFields( + referent as ProjectedNode, + nestedTypes.length === 1 + ? nestedTypes[0] + : storedTargetOf( + referent as ProjectedNode, + nestedTypes as readonly RootType[], + ), + schema, + ); } } } @@ -359,13 +374,13 @@ function applyField( // is an entry like any other, minus its `id`. Same reason as above for // needing a schema: without one there is no target to project through. if (field.kind === 'reference' && schema !== undefined) { - const localType = localLookupTypeOf(field, schema); - if (localType !== undefined) { + const localTargets = localLookupTargetsOf(field, schema); + if (localTargets.length > 0) { const endpoints = applyNestedReferents( document, valuesOf(node, alias), field, - localType, + localTargets, schema, context, ); @@ -382,13 +397,12 @@ function applyField( return applyText(document, langValuesOf(node, alias), field); case 'keyword': return applyFacet(document, literalsOf(node, alias), field, schema); - case 'reference': - return applyFacet( - document, - referenceValues(node, alias, field, schema), - field, - schema, - ); + case 'reference': { + const targetsById = new Map(); + const values = referenceValues(node, alias, field, schema, targetsById); + rememberTargets(document, targetsById); + return applyFacet(document, values, field, schema, targetsById); + } case 'integer': return setNumber( document, @@ -451,32 +465,120 @@ function referenceValues( alias: string, field: ReferenceField, schema: SearchSchema | undefined, + targetsById: Map, ): readonly string[] { - const targetName = labelSourceNameOf(field); - if (schema === undefined || targetName === undefined) { - return irisOf(node, alias); - } // Guaranteed declared: `searchSchema` validates that every named target // resolves, and the projection only ever runs against a type of its schema. - const target = rootTypeNamed(schema, targetName) as RootType; - if (target.key === undefined) { + const targets = referencedTargetsOf(field, schema); + if (targets.length === 0) { return irisOf(node, alias); } - const keyFieldName = target.key.field; return valuesOf(node, alias) .map((value) => { const iri = iriString(value); - return iri === undefined - ? undefined - : documentKeyOf( - target, - iri, - isObject(value) ? keyCandidatesOf(value, target, keyFieldName) : [], - ); + if (iri === undefined) { + return undefined; + } + // Each referent is re-keyed through the target IT belongs to – the one + // its `rdf:type` matches – so a person keyed on an authority IRI and an + // organization keyed on nothing store, under one field, the ids their + // own collections file them under. + const target = targetOfReferent(value, targets); + const keyed = + target.key === undefined + ? iri + : documentKeyOf( + target, + iri, + isObject(value) + ? keyCandidatesOf(value, target, target.key.field) + : [], + ); + // Remembered only where there was a choice: a single-target reference + // stays byte-identical to what it was, symbol keys included. Keyed by + // what the field STORES – the value after its own `transform`, which + // `applyFacet` applies before it reads this map. + if (keyed !== undefined && targets.length > 1) { + targetsById.set( + field.transform === undefined ? keyed : field.transform(keyed), + target, + ); + } + return keyed; }) .filter((value): value is string => value !== undefined); } +/** + * The Root Type a framed referent belongs to, among the targets its reference + * names: the first whose `class` is among the referent’s `rdf:type`s, which + * the extraction emits and framing carries as `@type`; else the first target + * declared. One target needs no evidence at all. This is the one reading of + * *which of several targets is this*, so re-keying, the facet policy and the + * stored discriminator ({@link TARGET_FIELD}) cannot disagree. + */ +function targetOfReferent( + value: unknown, + targets: readonly RootType[], +): RootType { + if (targets.length === 1 || !isObject(value)) { + return targets[0]; + } + const raw = value['@type']; + const classes = new Set( + (Array.isArray(raw) ? raw : [raw]).filter( + (type): type is string => typeof type === 'string', + ), + ); + return targets.find((target) => classes.has(target.class)) ?? targets[0]; +} + +/** + * The targets the reference values of a projected node were resolved through, + * by id – kept beside the node under a symbol, so it never reaches the + * writer, for the one reader that needs it after projection: the identity + * companion of an inline reference, which admits an entry’s id to a facet by + * the policy of the target that id belongs to. Only a reference naming + * several targets has anything to remember. + */ +const REFERENT_TARGETS: unique symbol = Symbol('referentTargets'); + +type WithReferentTargets = ProjectedNode & { + [REFERENT_TARGETS]?: Map; +}; + +function rememberTargets( + document: ProjectedNode, + targetsById: ReadonlyMap, +): void { + if (targetsById.size === 0) { + return; + } + const node = document as WithReferentTargets; + const remembered = (node[REFERENT_TARGETS] ??= new Map()); + for (const [id, target] of targetsById) { + remembered.set(id, target); + } +} + +/** + * Whether an id deserves a facet bucket, by the {@link FacetKeys facet policy} + * of the target it belongs to: admitted where that target declares none, and + * by its `only` where it does. `targetName` is the target the id was resolved + * or stored against; an id whose target is unknown falls back to the first + * declared, the same precedence every other reading of several targets uses. + */ +function admitsFacet( + policies: ReadonlyMap, + targets: readonly RootType[], + targetName: string | undefined, + id: string, +): boolean { + // Policies are inherited from targets, so where any exists a target does. + const policy = policies.get(targetName ?? targets[0].name); + return policy === undefined || policy.only(id); +} + /** * Project a text field. **Display** (when `output`) preserves *every* language * present – one label per language, or every label of it for an `array` field @@ -575,7 +677,7 @@ function foldedSearchValue(values: readonly string[]): string { * already-read raw values). * * A reference inheriting a {@link FacetKeys facet policy} from the type it - * names ({@link inheritedFacetKeys}) also writes the `${name}_facet` companion + * names ({@link inheritedFacetPolicies}) also writes the `${name}_facet` companion * the engine facets instead of the field: the subset of **what the field * stores** that the policy admits. So it is written after the field’s own * `transform` and the IRI filter, from the same `values` – for a keyed target @@ -599,6 +701,7 @@ function applyFacet( raw: readonly string[], field: KeywordField | ReferenceField, schema: SearchSchema | undefined, + targetsById?: ReadonlyMap, ): void { // A `reference` stores identity, so what it stores must be an IRI whatever // route the value arrived by. {@link iriString} guards the graph path, but a @@ -613,21 +716,27 @@ function applyFacet( const folded = dedupe(values.map((value) => fold(value))); const names = physicalFields(field, schema); const searchField = names.search[0]; - const policy = inheritedFacetKeys(field, schema); + const policies = inheritedFacetPolicies(field, schema); + const inherited = policies.size > 0; // `names.facet` is the companion exactly when a policy is inherited – - // physicalFields reads the same `inheritedFacetKeys`. + // physicalFields reads the same `inheritedFacetPolicies`. const facetField = names.facet as string; // The companion is a subset of what the field STORES – for a single-valued // field its first value, not the first admitted of all of them, or the facet // would count a value no filter on the field can reproduce. const stored = field.array === true ? values : values.slice(0, 1); - const admitted = policy === undefined ? [] : stored.filter(policy.only); + const targets = referencedTargetsOf(field, schema); + const admitted = !inherited + ? [] + : stored.filter((id) => + admitsFacet(policies, targets, targetsById?.get(id)?.name, id), + ); if (field.array === true) { setArray(document, field.name, stored); if (field.searchable) { setArray(document, searchField, folded); } - if (policy !== undefined) { + if (inherited) { setArray(document, facetField, admitted); } return; @@ -636,7 +745,7 @@ function applyFacet( if (field.searchable) { setString(document, searchField, folded[0]); } - if (policy !== undefined) { + if (inherited) { setString(document, facetField, admitted[0]); } } @@ -707,7 +816,7 @@ function applyInlineReference( document, valuesOf(node, alias), field, - referenceType, + [referenceType], schema, context, ); @@ -746,16 +855,27 @@ function applyIdentityCompanion( // keeps the first referent and drops the rest, and a companion holding an id // from a dropped one would match a filter whose hit shows no such entry. const stored = field.array === true ? referents : referents.slice(0, 1); - const ids = dedupe( - stored.flatMap((referent) => - (Array.isArray(referent[identity]) - ? referent[identity] - : [referent[identity]] - ) - .map(identityValue) - .filter((id): id is string => id !== undefined), - ), - ); + // Each id with the target it was resolved through, for the facet policy + // below: a stored endpoint says so itself ({@link TARGET_FIELD}), a bare id + // was remembered when the entry was projected ({@link rememberTargets}). + const targetById = new Map(); + for (const referent of stored) { + const remembered = (referent as WithReferentTargets)[REFERENT_TARGETS]; + const raw = referent[identity]; + for (const value of Array.isArray(raw) ? raw : [raw]) { + const id = identityValue(value); + if (id === undefined || targetById.has(id)) { + continue; + } + targetById.set( + id, + isObject(value) && typeof value[TARGET_FIELD] === 'string' + ? value[TARGET_FIELD] + : remembered?.get(id)?.name, + ); + } + } + const ids = [...targetById.keys()]; if (ids.length === 0) { return; } @@ -763,12 +883,18 @@ function applyIdentityCompanion( // Same rule as every other facetable reference: where the target declares a // facet policy, the facet reads a narrowed companion of its own, so an // excluded id is never seen by the engine rather than merely unlabelled. - const policy = inheritedFacetKeys(field, schema); - if (policy !== undefined) { + const policies = inheritedFacetPolicies(field, schema); + if (policies.size > 0) { + const targets = referencedTargetsOf( + identityFieldOf(field, schema) as SearchField, + schema, + ); setIdentity( document, names.facet as string, - ids.filter(policy.only), + ids.filter((id) => + admitsFacet(policies, targets, targetById.get(id), id), + ), field, nested, ); @@ -871,7 +997,7 @@ function applyNestedReferents( document: ProjectedNode, values: readonly unknown[], field: ReferenceField, - nestedType: SearchType, + nestedTypes: readonly SearchType[], schema: SearchSchema, context: ProjectionContext, ): readonly ProjectedNode[] { @@ -880,8 +1006,28 @@ function applyNestedReferents( // one entry per combination BEFORE it is projected (ADR 26). const referents = values .filter(isObject) - .flatMap((value) => tuplesOf(value, nestedType, field)) - .map((tuple) => projectFields(tuple, nestedType, schema, context, true)) + .flatMap((value) => { + // A lookup naming several targets projects each referent through the + // one it belongs to, and stores which – so an engine adapter can read + // an entry no collection answers for through the right declaration. + const nestedType = + nestedTypes.length === 1 + ? nestedTypes[0] + : targetOfReferent(value, nestedTypes as readonly RootType[]); + return tuplesOf(value, nestedType, field).map((tuple) => { + const projected = projectFields( + tuple, + nestedType, + schema, + context, + true, + ); + if (nestedTypes.length > 1 && Object.keys(projected).length > 0) { + projected[TARGET_FIELD] = nestedType.name; + } + return projected; + }); + }) // Fields, not identity, are what makes something a referent: a literal // value object under the alias (dirty source data), or a node this // reference type reads nothing from, projects nothing and is no referent. @@ -900,7 +1046,7 @@ function applyNestedReferents( // `id` is its key and stays part of what makes an entry distinct. const distinct = dedupeBy(referents, (referent) => JSON.stringify( - nestedType.class === undefined + nestedTypes[0].class === undefined ? { ...referent, id: undefined } : referent, ), diff --git a/packages/search/src/query.ts b/packages/search/src/query.ts index 43917281..3c6666c2 100644 --- a/packages/search/src/query.ts +++ b/packages/search/src/query.ts @@ -4,6 +4,8 @@ import { ID_FIELD, isoToUnixSeconds, nestedReferenceType, + referencedTargetsOf, + type SearchField, type SearchSchema, type SearchType, } from './schema.js'; @@ -535,7 +537,7 @@ export function validateQuery( issues.push({ part: 'facets', field: name, reason: 'not-facetable' }); } } - collectProjectionIssues(query.resolve, searchType, schema, issues); + collectProjectionIssues(query.resolve, [searchType], schema, issues); for (const sort of query.orderBy) { if ( sort.field !== 'relevance' && @@ -568,40 +570,61 @@ export function validateQuery( */ function collectProjectionIssues( projection: ReferenceProjection | undefined, - searchType: SearchType, + searchTypes: readonly SearchType[], schema: SearchSchema, issues: QueryIssue[], ): void { for (const [name, level] of Object.entries(projection ?? {})) { - const field = fieldNamed(searchType, name); - if (field === undefined) { + // A level is read against every type it may be reached through: a lookup + // naming several targets resolves a referent of any of them, so the level + // below is valid wherever ANY of those targets serves it – a field one + // target declares and another does not is exactly what a polymorphic + // selection asks for. + const fields = searchTypes + .map((searchType) => fieldNamed(searchType, name)) + .filter((field): field is SearchField => field !== undefined); + if (fields.length === 0) { issues.push({ part: 'resolve', field: name, reason: 'unknown-field' }); continue; } // Descending into an inline reference is free, so a nested level is valid // wherever the entries exist – what it is *for* is the lookup below it. - const nested = nestedReferenceType(schema, field); - if (nested !== undefined) { + // `searchSchema` holds the targets of one lookup to one declaration per + // name, so a name is nested or a lookup, never both; each is still read + // on its own here rather than one deciding for the other. + const nested: SearchType[] = []; + for (const field of fields) { + const referenceType = nestedReferenceType(schema, field); + if (referenceType !== undefined) { + nested.push(referenceType); + } + } + if (nested.length > 0) { collectProjectionIssues(level.resolve, nested, schema, issues); - continue; } - if (field.kind !== 'reference' || field.ref?.strategy !== 'lookup') { - issues.push({ part: 'resolve', field: name, reason: 'not-resolvable' }); + const lookups = fields.filter( + (field) => field.kind === 'reference' && field.ref?.strategy === 'lookup', + ); + if (lookups.length === 0) { + if (nested.length === 0) { + issues.push({ part: 'resolve', field: name, reason: 'not-resolvable' }); + } continue; } - const targetName = field.ref.target; - const target = [...schema.values()].find( - (rootType) => rootType.name === targetName, + const targets = lookups.flatMap((field) => + referencedTargetsOf(field, schema), ); - if (target === undefined) { + if (targets.length === 0) { // searchSchema rejects a lookup whose target it cannot resolve, so this // is a query built against a different schema than the engine serves. issues.push({ part: 'resolve', field: name, reason: 'not-resolvable' }); continue; } for (const wanted of level.fields ?? []) { - const targetField = fieldNamed(target, wanted); - if (targetField === undefined || targetField.output !== true) { + const served = targets.some( + (target) => fieldNamed(target, wanted)?.output === true, + ); + if (!served) { issues.push({ part: 'resolve', field: `${name}.${wanted}`, @@ -609,7 +632,7 @@ function collectProjectionIssues( }); } } - collectProjectionIssues(level.resolve, target, schema, issues); + collectProjectionIssues(level.resolve, targets, schema, issues); } } diff --git a/packages/search/src/schema.ts b/packages/search/src/schema.ts index bf184da9..fed4dca1 100644 --- a/packages/search/src/schema.ts +++ b/packages/search/src/schema.ts @@ -236,7 +236,19 @@ export type ReferenceStrategy = } | { readonly strategy: 'lookup'; - readonly target: string; + /** + * The Root Type whose collection the fields are read from – or + * **several**, where the referent may be any of them: a `creator` that + * is a `Person` or an `Organization`. Each value then resolves against + * every named collection and reports which one held it, so a surface can + * type it per value rather than claiming one kind for all of them. + * + * Order is precedence. A value that several collections hold – two Root + * Types whose `class` selections overlap – belongs to the first target + * declared, and so does a stored referent whose `rdf:type` matches none + * of them. List the most specific type first. + */ + readonly target: string | readonly string[]; /** * Also project the target’s **own fields from this document’s frame**, so * the reference stores what the referring document states about the @@ -287,7 +299,7 @@ export type ReferenceStrategy = * identifies the edge. Naming it is also what gives the companion a * **target**: the named field’s own `target` is the Root Type whose keys * these ids are, which is what a facet policy is inherited through - * ({@link inheritedFacetKeys}) and what types the filter at the surface. + * ({@link inheritedFacetPolicies}) and what types the filter at the surface. * * The companion holds ids only, so a facet over it is exact and * identity-keyed: an entry whose endpoint the graph named inline – a @@ -310,20 +322,22 @@ export interface ReferenceField extends SearchFieldBase, Searchable { readonly from?: ProjectionValue; /** * The `name` of the Root Type whose collection labels this reference’s facet - * buckets. Only an `idOnly` reference declares one: a `lookup` reads its - * labels from the `target` it already names, and an `inline` reference - * carries the referent’s own fields. The named type must declare an `output`, + * buckets – or several, in order of precedence, where the referent may be + * any of them (the same range a `lookup`’s `target` states). Only an + * `idOnly` reference declares one: a `lookup` reads its labels from the + * `target` it already names, and an `inline` reference carries the + * referent’s own fields. Each named type must declare an `output`, * `searchable` text field under its {@link SearchTypeBase.labelField} name * (`label` by default; validated by {@link searchSchema}), so an engine can * both reconstruct the label and search it (typeahead). */ - readonly labelSource?: string; + readonly labelSource?: string | readonly string[]; /** * Turn this reference into an **engine-level join**, so a query can filter * this type by a condition on the referent – `“every object published by * institution X”` in one round-trip instead of two. Valid only where the * reference names the type it resolves against – a `lookup`’s `target` or an - * `idOnly`’s {@link ReferenceField.labelSource} ({@link labelSourceNameOf}) – + * `idOnly`’s {@link ReferenceField.labelSource} ({@link labelSourceNamesOf}) – * which already asserts that this field’s values are ids of documents in that * type’s collection, exactly the fact a join needs. * @@ -474,7 +488,7 @@ export interface KeyField { * A {@link RootType}’s **facet policy**: which of its documents get a facet * bucket, as a predicate over the document key. Declared once, on the type, * and inherited by every facetable reference that *names* the type – a - * `lookup`’s `target`, an `idOnly`’s `labelSource` ({@link labelSourceNameOf}) + * `lookup`’s `target`, an `idOnly`’s `labelSource` ({@link labelSourceNamesOf}) * – because *which of a type’s ids deserve a bucket* is a fact about the type, * not about each field that points at it. A per-field policy would be one rule * declared N times, and forgetting one would silently reintroduce the buckets @@ -725,7 +739,7 @@ export function referenceTypeNamed( /** * The {@link RootType} a declaration names – a `lookup`’s `target`, an - * `idOnly`’s {@link ReferenceField.labelSource} ({@link labelSourceNameOf}), a + * `idOnly`’s {@link ReferenceField.labelSource} ({@link labelSourceNamesOf}), a * join edge – or `undefined` when the schema declares no Root Type by that * name. The one reading of *which type does this point at*, so the projection * (which re-keys a reference through its target’s {@link KeyField}), the @@ -880,35 +894,43 @@ function framingReach( // the cut as “reach 0, and nothing else to count” left the innermost // referent’s key one hop outside the frame, so it stored a node IRI that // matches nothing in the target’s collection. - const local = localLookupTypeOf(field, schema); - if (local !== undefined && !onPath.has(local.name)) { - furthest = Math.max(furthest, hops + framingReach(schema, local, onPath)); - continue; + // + // Several targets each contribute their own reach: a referent is framed + // once, and the frame has to hold whichever declaration it turns out to + // match, so the furthest of them decides. + // The field’s own traversal counts whatever it names. + furthest = Math.max(furthest, hops); + const local = localLookupTargetsOf(field, schema).filter( + (target) => !onPath.has(target.name), + ); + for (const target of local) { + furthest = Math.max( + furthest, + hops + framingReach(schema, target, onPath), + ); } // A keyed target’s key field is itself path-bearing, so its own traversal // counts too – the extraction reads it off the referent with that path. - const keyPath = keyedTargetKeyPath(field, schema); - furthest = Math.max( - furthest, - hops + (keyPath === undefined ? 0 : pathHopCount(keyPath)), - ); + // Counted for every target that is not locally expanded above, cut ones + // included: there the extraction falls back to exactly this hop. + for (const target of referencedTargetsOf(field, schema)) { + if (local.includes(target)) { + continue; + } + const keyPath = keyPathOf(target); + furthest = Math.max( + furthest, + hops + (keyPath === undefined ? 0 : pathHopCount(keyPath)), + ); + } } return furthest; } -/** The `path` of the key field of the keyed Root Type a reference names, when - * it names one that declares a {@link RootType.key}. */ -function keyedTargetKeyPath( - field: SearchField, - schema: SearchSchema, -): string | undefined { - if (field.kind !== 'reference') { - return undefined; - } - const targetName = labelSourceNameOf(field); - const target = - targetName === undefined ? undefined : rootTypeNamed(schema, targetName); - if (target?.key === undefined) { +/** The `path` of a Root Type’s key field, when it declares a + * {@link RootType.key}. */ +function keyPathOf(target: RootType): string | undefined { + if (target.key === undefined) { return undefined; } // `searchSchema` guarantees a key field that is declared and path-bearing – @@ -1058,7 +1080,7 @@ function assertIdentityCompanion( } if ( identityField.kind !== 'reference' || - labelSourceNameOf(identityField) === undefined + labelSourceNamesOf(identityField).length === 0 ) { throw new Error( `${where} names identity “${identity}”, which names no target: an identity companion holds ids of documents in a collection, so the field it harvests must be a reference declaring a “lookup” target or a label source.`, @@ -1329,15 +1351,41 @@ export function labelFieldOf(searchType: SearchType): TextField | undefined { } /** - * The Root Type a reference resolves labels from, by name: a `lookup`’s - * `target`, an `idOnly`’s {@link ReferenceField.labelSource}, or `undefined` - * when it resolves none. One reading for the two declarations, so a consumer - * never branches on the strategy to find the collection. + * The Root Types a reference resolves labels from, by name and in order of + * precedence: a `lookup`’s `target`, an `idOnly`’s + * {@link ReferenceField.labelSource}, or none when it resolves none. One + * reading for the two declarations, so a consumer never branches on the + * strategy to find the collection – and one **shape** for the one-target and + * the several-target declaration, so a consumer never branches on that either: + * a single name is the list of one. + */ +export function labelSourceNamesOf(field: ReferenceField): readonly string[] { + const declared = + field.ref?.strategy === 'lookup' ? field.ref.target : field.labelSource; + return declared === undefined + ? [] + : typeof declared === 'string' + ? [declared] + : declared; +} + +/** + * The Root Types a reference resolves against ({@link labelSourceNamesOf}), + * resolved in the given schema and in order of precedence. A name the schema + * does not declare as a Root Type is skipped – `searchSchema` rejects such a + * declaration, so this only happens for a field read against a foreign schema, + * where it resolves what it can. */ -export function labelSourceNameOf(field: ReferenceField): string | undefined { - return field.ref?.strategy === 'lookup' - ? field.ref.target - : field.labelSource; +export function referencedTargetsOf( + field: SearchField, + schema: SearchSchema | undefined, +): readonly RootType[] { + if (schema === undefined || field.kind !== 'reference') { + return []; + } + return labelSourceNamesOf(field) + .map((name) => rootTypeNamed(schema, name)) + .filter((target): target is RootType => target !== undefined); } /** @@ -1380,34 +1428,117 @@ function assertResolvableLabelSources( ); } } - const sourceName = - field.kind === 'reference' && field.ref?.strategy === 'lookup' - ? field.ref.target - : labelSource; - if (sourceName === undefined) { + if (field.kind !== 'reference') { continue; } - const source = rootTypeNamed(schema, sourceName); - if (source === undefined) { - // A name that IS declared, just not as a Root Type, is the confusing - // case: telling the author to declare a type they already declared - // would send them looking in the wrong place. Only a Root Type has a - // collection to resolve against, so name that instead. - throw new Error( - referenceTypeNamed(schema, sourceName) === undefined - ? `Reference “${searchType.name}.${field.name}” names unknown label source “${sourceName}”; declare a SearchType with that name.` - : `Reference “${searchType.name}.${field.name}” names label source “${sourceName}”, which is a Reference Type; a label source must be a Root Type, since a resolved label is read from that type’s own collection.`, - ); + const sources: RootType[] = []; + for (const sourceName of labelSourceNamesOf(field)) { + const source = rootTypeNamed(schema, sourceName); + if (source === undefined) { + // A name that IS declared, just not as a Root Type, is the confusing + // case: telling the author to declare a type they already declared + // would send them looking in the wrong place. Only a Root Type has a + // collection to resolve against, so name that instead. + throw new Error( + referenceTypeNamed(schema, sourceName) === undefined + ? `Reference “${searchType.name}.${field.name}” names unknown label source “${sourceName}”; declare a SearchType with that name.` + : `Reference “${searchType.name}.${field.name}” names label source “${sourceName}”, which is a Reference Type; a label source must be a Root Type, since a resolved label is read from that type’s own collection.`, + ); + } + if (labelFieldOf(source) === undefined) { + throw new Error( + `Reference “${searchType.name}.${field.name}” uses label source “${sourceName}”, which must declare an output, searchable text field “${labelFieldNameOf(source)}”.`, + ); + } + sources.push(source); } - if (labelFieldOf(source) === undefined) { + assertCompatibleTargets(searchType, field, sources); + } + } +} + +/** + * The targets of a reference naming **several** may not disagree about a + * field they both declare: a stored referent of either kind lands in one + * nested object, which an engine declares once, so `birthDate` cannot be a + * `date` on one target and a `keyword` on the other, nor a list on one and a + * single value on the other – nor an id on one and a nested document on the + * other, nor indexed on one and stored only on the other. Everything that + * decides a nested field’s physical shape has to agree: kind, arity, the + * Roles it opts into, and for a reference the strategy and what it points at. + * A field only one target declares is fine – the other simply never fills it + * – and so is a `text` field whose locales differ, since display stores every + * present language regardless. + */ +function assertCompatibleTargets( + searchType: SearchType, + field: ReferenceField, + targets: readonly RootType[], +): void { + const seen = new Map(); + for (const target of targets) { + for (const declared of target.fields) { + const earlier = seen.get(declared.name); + if (earlier === undefined) { + seen.set(declared.name, { target, field: declared }); + continue; + } + if (describeShape(earlier.field) !== describeShape(declared)) { throw new Error( - `Reference “${searchType.name}.${field.name}” uses label source “${sourceName}”, which must declare an output, searchable text field “${labelFieldNameOf(source)}”.`, + `Reference “${searchType.name}.${field.name}” names targets “${earlier.target.name}” and “${target.name}”, which both declare “${declared.name}” but not alike (${describeShape(earlier.field)} vs ${describeShape(declared)}); a referent of either kind is stored in one shape, so a field the targets share must be declared the same on both.`, ); } } } } +/** + * Everything that decides a field’s physical shape, spelled out for a message + * comparing two declarations: kind and arity, the Roles it carries, and for a + * reference its strategy, what it points at, and whether it stores a copy. + */ +function describeShape(field: SearchField): string { + const roles: string[] = ['filterable', 'facetable', 'sortable'].filter( + (role) => field[role as 'filterable' | 'facetable' | 'sortable'] === true, + ); + if (field.searchable !== undefined) { + roles.push('searchable'); + } + const ref = + field.kind === 'reference' && field.ref !== undefined + ? [ + field.ref.strategy, + ...labelSourceNamesOf(field), + ...(field.ref.strategy === 'inline' ? [field.ref.typeName] : []), + ...(field.ref.strategy === 'lookup' && field.ref.local === true + ? ['local'] + : []), + ] + : []; + return [ + field.array === true ? `${field.kind} list` : field.kind, + ...(ref.length > 0 ? [`→ ${ref.join(' ')}`] : []), + ...(roles.length > 0 ? [`(${roles.join(', ')})`] : []), + ].join(' '); +} + +/** + * The Root Type a stored referent was projected through, among the targets + * its lookup names: the one its discriminator names ({@link TARGET_FIELD}), + * or the first declared where it carries none – a single-target lookup stores + * none, and the first target is also what an unmatched referent was projected + * through. The one reading of a stored discriminator, so the projection + * (which prunes through it) and an engine adapter (which reconstructs through + * it) cannot read it differently. + */ +export function storedTargetOf( + entry: Readonly>, + targets: readonly RootType[], +): RootType { + const name = entry[TARGET_FIELD]; + return targets.find((target) => target.name === name) ?? targets[0]; +} + /** * One structural problem {@link validateSearchType} found: a field declares a * capability or property its `kind` cannot honour, or the declaration is @@ -1441,6 +1572,8 @@ export interface SearchTypeIssue { | 'joinable-not-allowed' | 'joinable-without-label-source' | 'joinable-with-inline-ref' + | 'joinable-with-several-targets' + | 'duplicate-target' | 'reserved-field-name' | 'key-field-unknown' | 'key-field-not-reference' @@ -1484,9 +1617,29 @@ export const ID_FIELD = 'id'; export const AND_KEY = 'and'; export const OR_KEY = 'or'; +/** + * The physical name under which a stored referent of a lookup naming + * **several** targets records which of them it belongs to: the `name` of the + * Root Type whose `class` its `rdf:type` matched first, or the first target + * declared where none did. Written by the projection, read by an engine + * adapter to reconstruct the referent through the right declaration when no + * collection answers for it – an unidentified referent, or an identified one + * whose document is not indexed. A single-target lookup stores none: there is + * nothing to tell apart. + * + * Reserved schema-wide ({@link validateSearchType} rejects a field of this + * name), so it can never shadow a declared field of a stored entry. + */ +export const TARGET_FIELD = '_target'; + /** The logical field names no {@link SearchType} may declare, each because a * surface already gives that name a meaning of its own. */ -const RESERVED_FIELD_NAMES: readonly string[] = [ID_FIELD, AND_KEY, OR_KEY]; +const RESERVED_FIELD_NAMES: readonly string[] = [ + ID_FIELD, + AND_KEY, + OR_KEY, + TARGET_FIELD, +]; /** Kinds that can feed full-text search (project a folded search field). */ const SEARCHABLE_KINDS: readonly FieldKind[] = ['text', 'keyword', 'reference']; @@ -1622,19 +1775,31 @@ export function validateSearchType( field.output === true && ((field.ref?.strategy === 'inline' && field.ref.typeName === undefined) || - (field.ref?.strategy === 'lookup' && field.ref.target === undefined)) + (field.ref?.strategy === 'lookup' && + labelSourceNamesOf(field as ReferenceField).length === 0)) ) { issue('missing-ref-type-name'); } // A join addresses the referent's collection – the one a lookup's // `target` or an idOnly's `labelSource` names. With neither, the flag // states an edge to nowhere. - if ( - field.joinable === true && - labelSourceNameOf(field as ReferenceField) === undefined - ) { + const targets = labelSourceNamesOf(field as ReferenceField); + if (field.joinable === true && targets.length === 0) { issue('joinable-without-label-source'); } + // An engine reference names ONE collection, and ids that live in + // several have no single collection to reference – so a reference whose + // referent may be any of several types can never be an engine edge. Its + // labels, facets and id filters all work from this document; a + // condition on the referent’s own fields does not (#717). + if (field.joinable === true && targets.length > 1) { + issue('joinable-with-several-targets'); + } + // Precedence between targets is declaration order, so a name declared + // twice would have to mean two different ranks. + if (new Set(targets).size !== targets.length) { + issue('duplicate-target'); + } // An inline reference is stored as a NESTED OBJECT, not as an id an // engine can point a reference field at, so the two cannot both hold: // the collection definition would emit the nesting and silently drop the @@ -1895,46 +2060,48 @@ export function identityFieldName(name: string): string { } /** - * The Root Type whose labels a reference’s **facet buckets** read – its own - * label source, or, for an inline reference, the one its - * {@link ReferenceStrategy.identity identity companion} points at. + * The Root Types whose labels a reference’s **facet buckets** read, by name and + * in order of precedence – its own label sources, or, for an inline reference, + * those its {@link ReferenceStrategy.identity identity companion} points at. + * Empty for a field that labels no bucket. * - * The same one-level-in reading {@link inheritedFacetKeys} makes, and for the - * same reason: the companion holds that field’s ids, so the type that names - * those ids is the type that can label them. Kept together with it so a facet - * cannot inherit a policy from one type and its labels from another – or, as - * happened first, inherit the policy and no labels at all. + * The same one-level-in reading {@link inheritedFacetPolicies} makes, and for + * the same reason: the companion holds that field’s ids, so the types that + * name those ids are the types that can label them. Kept together with it so a + * facet cannot inherit a policy from one type and its labels from another – + * or, as happened first, inherit the policy and no labels at all. */ -export function labelTargetNameOf( +export function labelTargetNamesOf( field: SearchField, schema: SearchSchema | undefined, -): string | undefined { +): readonly string[] { if (field.kind !== 'reference') { - return undefined; + return []; } - return labelSourceNameOf(identityFieldOf(field, schema) ?? field); + return labelSourceNamesOf(identityFieldOf(field, schema) ?? field); } /** - * The Root Type a {@link ReferenceStrategy.local local} lookup projects its - * referents through, or `undefined` for every other field. Such a reference - * stores nested documents shaped by the **target’s own declaration** – so it - * needs no reference type of its own, and reconstructs through the same path a - * resolved referent does. + * The Root Types a {@link ReferenceStrategy.local local} lookup projects its + * referents through, in order of precedence – or none for every other field. + * Such a reference stores nested documents shaped by the **target’s own + * declaration** – so it needs no reference type of its own, and reconstructs + * through the same path a resolved referent does. With several targets, each + * referent is projected through the one its `rdf:type` matches + * ({@link TARGET_FIELD}). */ -export function localLookupTypeOf( +export function localLookupTargetsOf( field: SearchField, schema: SearchSchema | undefined, -): RootType | undefined { +): readonly RootType[] { if ( - schema === undefined || field.kind !== 'reference' || field.ref?.strategy !== 'lookup' || field.ref.local !== true ) { - return undefined; + return []; } - return rootTypeNamed(schema, field.ref.target); + return referencedTargetsOf(field, schema); } /** @@ -1942,7 +2109,7 @@ export function localLookupTypeOf( * {@link ReferenceStrategy.identity identity companion} harvests, or * `undefined` where the field declares none (or the schema cannot resolve its * reference type). The one reading of *which nested field identifies the edge*, - * so the projection (which harvests it), {@link inheritedFacetKeys} (which + * so the projection (which harvests it), {@link inheritedFacetPolicies} (which * reads its target’s facet policy) and an adapter’s filter compiler cannot * resolve it differently. */ @@ -2318,7 +2485,7 @@ export function unixSecondsToIso(seconds: number): string { * The facet is the one member that depends on more than the declaration: a * facetable reference whose target declares a {@link FacetKeys facet policy} * facets a `${name}_facet` companion rather than itself, and only the `schema` - * can resolve that target ({@link inheritedFacetKeys}). Without one the field + * can resolve that target ({@link inheritedFacetPolicies}). Without one the field * facets on its own name – the same reading the projection makes without a * schema, where it cannot re-key a reference either. */ @@ -2348,7 +2515,7 @@ export function physicalFields( // otherwise a level further in than an engine can weld a condition to. const nestsAnObject = identityFieldOf(field, schema) !== undefined || - (localLookupTypeOf(field, schema) !== undefined && + (localLookupTargetsOf(field, schema).length > 0 && field.filterable === true); const identity = nestsAnObject ? identityFieldName(field.name) : undefined; const filtered = identity ?? field.name; @@ -2358,7 +2525,7 @@ export function physicalFields( facet: field.facetable !== true ? undefined - : inheritedFacetKeys(field, schema) === undefined + : inheritedFacetPolicies(field, schema).size === 0 ? filtered : `${filtered}_facet`, identity, @@ -2366,9 +2533,13 @@ export function physicalFields( } /** - * The {@link FacetKeys facet policy} a field inherits: the `facetKeys` of the - * Root Type it names ({@link labelSourceNameOf}), when the field is a facetable - * reference and the schema resolves that type. The boundary is *naming the + * The {@link FacetKeys facet policies} a field inherits, keyed by the `name` of + * the Root Type declaring each: the `facetKeys` of every type the field names + * ({@link labelSourceNamesOf}) that declares one, when the field is a facetable + * reference and the schema resolves those types. Empty where nothing is + * inherited. With several targets, an id is admitted to a bucket by the policy + * of the target it belongs to, and unconditionally where that target declares + * none. The boundary is *naming the * target* – the same line along which a reference is re-keyed and a join is * drawn – so a reference that names no type inherits nothing, whatever type * its values happen to point at; and a `derive`d reference, which produces its @@ -2384,21 +2555,24 @@ export function physicalFields( * rather than declaring it twice is what keeps *which of a type’s ids deserve * a bucket* a fact about the type. */ -export function inheritedFacetKeys( +export function inheritedFacetPolicies( field: SearchField, schema: SearchSchema | undefined, -): FacetKeys | undefined { +): ReadonlyMap { + const policies = new Map(); if ( schema === undefined || field.kind !== 'reference' || field.facetable !== true || field.derive !== undefined ) { - return undefined; + return policies; } const identity = identityFieldOf(field, schema); - const targetName = labelSourceNameOf(identity ?? field); - return targetName === undefined - ? undefined - : rootTypeNamed(schema, targetName)?.facetKeys; + for (const target of referencedTargetsOf(identity ?? field, schema)) { + if (target.facetKeys !== undefined) { + policies.set(target.name, target.facetKeys); + } + } + return policies; } diff --git a/packages/search/src/testing.ts b/packages/search/src/testing.ts index 723a933f..d6df1bda 100644 --- a/packages/search/src/testing.ts +++ b/packages/search/src/testing.ts @@ -11,7 +11,9 @@ import { facetableFields, filterableFields, ID_FIELD, - inheritedFacetKeys, + identityFieldOf, + inheritedFacetPolicies, + referencedTargetsOf, nestedReferenceType, outputFields, type RootType, @@ -269,8 +271,19 @@ export function describeSearchEngineContract( // second assertion bites. for (const searchType of types()) { for (const field of facetableFields(searchType)) { - const policy = inheritedFacetKeys(field, engine().schema); - if (policy === undefined) { + // A bucket keyed on an id of one target is admitted by that + // target’s policy, and a target declaring none admits every one of + // its ids. Which target a bucket belongs to is not visible here, so + // the contract is checked only where every target constrains: then + // a bucket failing every policy belongs to no admitting target. + const policies = [ + ...inheritedFacetPolicies(field, engine().schema).values(), + ]; + const targets = referencedTargetsOf( + identityFieldOf(field, engine().schema) ?? field, + engine().schema, + ); + if (policies.length === 0 || policies.length < targets.length) { continue; } const result = await engine().search(searchType, { @@ -280,7 +293,9 @@ export function describeSearchEngineContract( }); const buckets = result.facets[field.name] ?? []; for (const bucket of buckets) { - expect(policy.only(bucket.value)).toBe(true); + expect(policies.some((policy) => policy.only(bucket.value))).toBe( + true, + ); expect(bucket.label).toBeDefined(); } if (field.output !== true || field.filterable !== true) { @@ -288,7 +303,7 @@ export function describeSearchEngineContract( } const excluded = result.hits .flatMap((hit) => referenceIds(hit.document[field.name])) - .find((id) => !policy.only(id)); + .find((id) => !policies.some((policy) => policy.only(id))); if (excluded === undefined) { continue; } diff --git a/packages/search/test/polymorphic-lookup.test.ts b/packages/search/test/polymorphic-lookup.test.ts new file mode 100644 index 00000000..88d50ec1 --- /dev/null +++ b/packages/search/test/polymorphic-lookup.test.ts @@ -0,0 +1,641 @@ +import { describe, expect, it } from 'vitest'; +import { projectDocument, type SearchDocument } from '../src/project.js'; +import { + defineSearchType, + inheritedFacetPolicies, + inlineFramingDepth, + labelTargetNamesOf, + localLookupTargetsOf, + physicalFields, + referencedTargetsOf, + searchSchema, + storedTargetOf, + TARGET_FIELD, + validateSearchType, +} from '../src/schema.js'; +import { validateQuery } from '../src/query.js'; + +const SCHEMA_ORG = 'https://schema.org/'; +const alias = (type: string, field: string) => `urn:lde:${type}/${field}`; + +/** + * SCHEMA-AP-NDE ranges `creator` over Person OR Organization. The two differ + * in the one declaration that changes what a reference stores: Person is keyed + * on an authority IRI, Organization on nothing – and Person admits only its + * authority-keyed ids to a facet. + */ +const person = defineSearchType({ + name: 'Person', + class: `${SCHEMA_ORG}Person`, + labelField: 'name', + key: { field: 'sameAs' }, + facetKeys: { only: (id) => id.startsWith('https://rkd/') }, + fields: [ + { + name: 'name', + kind: 'text', + path: `${SCHEMA_ORG}name`, + locales: ['und'], + output: true, + searchable: { weight: 1 }, + }, + { + name: 'sameAs', + kind: 'reference', + path: `${SCHEMA_ORG}sameAs`, + array: true, + }, + { + name: 'birthDate', + kind: 'keyword', + path: `${SCHEMA_ORG}birthDate`, + output: true, + }, + ], +}); + +const organization = defineSearchType({ + name: 'Organization', + class: `${SCHEMA_ORG}Organization`, + labelField: 'name', + fields: [ + { + name: 'name', + kind: 'text', + path: `${SCHEMA_ORG}name`, + locales: ['und'], + output: true, + searchable: { weight: 1 }, + }, + { + name: 'location', + kind: 'keyword', + path: `${SCHEMA_ORG}location`, + output: true, + }, + ], +}); + +/** A plain lookup at the root, faceted: what the issue was filed against. */ +const work = defineSearchType({ + name: 'Work', + class: `${SCHEMA_ORG}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + path: `${SCHEMA_ORG}creator`, + array: true, + output: true, + filterable: true, + facetable: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }, + ], +}); + +/** The same range, one level in: an edge nesting a `local` lookup. */ +const creatorEdge = defineSearchType({ + name: 'CreatorEdge', + fields: [ + { + name: 'role', + kind: 'keyword', + path: `${SCHEMA_ORG}roleName`, + output: true, + }, + { + name: 'creator', + kind: 'reference', + path: `${SCHEMA_ORG}creator`, + output: true, + ref: { + strategy: 'lookup', + target: ['Person', 'Organization'], + local: true, + }, + }, + ], +}); + +const edgedWork = defineSearchType({ + name: 'EdgedWork', + class: `${SCHEMA_ORG}Painting`, + fields: [ + { + name: 'creator', + kind: 'reference', + path: `${SCHEMA_ORG}creator`, + array: true, + output: true, + filterable: true, + facetable: true, + ref: { strategy: 'inline', typeName: 'CreatorEdge', identity: 'creator' }, + }, + ], +}); + +const schema = searchSchema(work, edgedWork, person, organization, creatorEdge); + +const RKD = 'https://rkd/artists/1'; +const rembrandt = { + '@id': 'https://id.example/rembrandt', + '@type': [`${SCHEMA_ORG}Person`, `${SCHEMA_ORG}Thing`], + [alias('Person', 'name')]: [{ '@value': 'Rembrandt' }], + [alias('Person', 'sameAs')]: [{ '@id': RKD }], +}; +const sisters = { + '@id': 'https://id.example/zusters', + '@type': `${SCHEMA_ORG}Organization`, + [alias('Organization', 'name')]: [{ '@value': 'Zusters Benedictinessen' }], +}; +/** Typed as neither: the fallback case. */ +const untyped = { + '@id': 'https://id.example/unknown', + [alias('Person', 'name')]: [{ '@value': 'Onbekend' }], +}; + +describe('declaring several targets', () => { + it('reads them in order, through one shape for one and for many', () => { + expect(referencedTargetsOf(work.fields[0], schema)).toEqual([ + person, + organization, + ]); + expect(labelTargetNamesOf(edgedWork.fields[0], schema)).toEqual([ + 'Person', + 'Organization', + ]); + expect(localLookupTargetsOf(creatorEdge.fields[1], schema)).toEqual([ + person, + organization, + ]); + expect(localLookupTargetsOf(work.fields[0], schema)).toEqual([]); + }); + + it('inherits each target’s facet policy under its own name', () => { + const policies = inheritedFacetPolicies(work.fields[0], schema); + expect([...policies.keys()]).toEqual(['Person']); + // A policy anywhere among the targets earns the field a facet companion. + expect(physicalFields(work.fields[0], schema).facet).toBe('creator_facet'); + }); + + it('rejects a target named twice: order is precedence, and a rank cannot be held twice', () => { + expect( + validateSearchType({ + name: 'Work', + class: `${SCHEMA_ORG}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: ['Person', 'Person'] }, + }, + ], + }), + ).toContainEqual({ field: 'creator', reason: 'duplicate-target' }); + }); + + it('rejects an empty target list as naming no type at all', () => { + expect( + validateSearchType({ + name: 'Work', + class: `${SCHEMA_ORG}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: [] }, + }, + ], + }), + ).toContainEqual({ field: 'creator', reason: 'missing-ref-type-name' }); + }); + + it('refuses joinable: an engine reference names one collection', () => { + expect( + validateSearchType({ + name: 'Work', + class: `${SCHEMA_ORG}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + joinable: true, + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }, + ], + }), + ).toContainEqual({ + field: 'creator', + reason: 'joinable-with-several-targets', + }); + }); + + it('reserves the discriminator’s name', () => { + expect( + validateSearchType({ + name: 'Person', + class: `${SCHEMA_ORG}Person`, + fields: [{ name: TARGET_FIELD, kind: 'keyword', output: true }], + }), + ).toContainEqual({ field: TARGET_FIELD, reason: 'reserved-field-name' }); + }); + + it('resolves every named target, not just the first', () => { + expect(() => + searchSchema(person, { + name: 'Work', + class: `${SCHEMA_ORG}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + output: true, + ref: { strategy: 'lookup', target: ['Person', 'Nowhere'] }, + }, + ], + }), + ).toThrow(/names unknown label source “Nowhere”/); + }); + + it('rejects targets that declare one field differently', () => { + // A referent of either kind is stored in one nested object, so a field + // the targets share must be one shape. + const otherOrganization = defineSearchType({ + ...organization, + fields: [ + organization.fields[0], + { + name: 'birthDate', + kind: 'date', + path: `${SCHEMA_ORG}foundingDate`, + output: true, + }, + ], + }); + expect(() => searchSchema(work, person, otherOrganization)).toThrow( + /both declare “birthDate” but not alike \(keyword vs date\)/, + ); + // Arity counts too: one nested object cannot hold a value and a list + // under one name. + const listedOrganization = defineSearchType({ + ...organization, + fields: [ + organization.fields[0], + { + name: 'birthDate', + kind: 'keyword', + array: true, + path: `${SCHEMA_ORG}foundingDate`, + output: true, + }, + ], + }); + expect(() => searchSchema(work, person, listedOrganization)).toThrow( + /not alike \(keyword vs keyword list\)/, + ); + // So does everything else that decides the stored shape: a Role, and for + // a reference what it points at and whether it stores a copy. + const filteringOrganization = defineSearchType({ + ...organization, + fields: [ + organization.fields[0], + { + name: 'birthDate', + kind: 'keyword', + path: `${SCHEMA_ORG}foundingDate`, + output: true, + filterable: true, + }, + ], + }); + expect(() => searchSchema(work, person, filteringOrganization)).toThrow( + /not alike \(keyword vs keyword \(filterable\)\)/, + ); + const place = defineSearchType({ + name: 'Place', + class: `${SCHEMA_ORG}Place`, + fields: [ + { + name: 'label', + kind: 'text', + path: `${SCHEMA_ORG}name`, + locales: ['und'], + output: true, + searchable: { weight: 1 }, + }, + ], + }); + const address = (local: boolean) => ({ + name: 'address', + kind: 'reference' as const, + path: `${SCHEMA_ORG}address`, + output: true, + ref: { strategy: 'lookup' as const, target: 'Place', local }, + }); + expect(() => + searchSchema( + work, + place, + defineSearchType({ + ...person, + fields: [...person.fields, address(false)], + }), + defineSearchType({ + ...organization, + fields: [...organization.fields, address(true)], + }), + ), + ).toThrow( + /not alike \(reference → lookup Place vs reference → lookup Place local\)/, + ); + }); + + it('accepts an idOnly reference labelling its buckets from several sources', () => { + const labelled = defineSearchType({ + name: 'Work', + class: `${SCHEMA_ORG}CreativeWork`, + fields: [ + { + name: 'creator', + kind: 'reference', + path: `${SCHEMA_ORG}creator`, + facetable: true, + labelSource: ['Person', 'Organization'], + ref: { strategy: 'idOnly' }, + }, + ], + }); + const idOnly = searchSchema(labelled, person, organization); + expect(labelTargetNamesOf(labelled.fields[0], idOnly)).toEqual([ + 'Person', + 'Organization', + ]); + }); +}); + +describe('reading a stored discriminator', () => { + it('names the target stored, and falls back to the first where none is', () => { + expect( + storedTargetOf({ [TARGET_FIELD]: 'Organization' }, [ + person, + organization, + ]), + ).toBe(organization); + expect( + storedTargetOf({ name_und: 'Onbekend' }, [person, organization]), + ).toBe(person); + }); +}); + +describe('two targets nesting one name inline', () => { + it('must nest the same Reference Type', () => { + const addressOf = (typeName: string) => ({ + name: 'address', + kind: 'reference' as const, + path: `${SCHEMA_ORG}address`, + output: true, + ref: { strategy: 'inline' as const, typeName }, + }); + const postal = defineSearchType({ + name: 'PostalAddress', + fields: [ + { + name: 'street', + kind: 'keyword', + path: `${SCHEMA_ORG}streetAddress`, + output: true, + }, + ], + }); + const visiting = defineSearchType({ ...postal, name: 'VisitingAddress' }); + expect(() => + searchSchema( + work, + postal, + visiting, + defineSearchType({ + ...person, + fields: [...person.fields, addressOf('PostalAddress')], + }), + defineSearchType({ + ...organization, + fields: [...organization.fields, addressOf('VisitingAddress')], + }), + ), + ).toThrow( + /reference → inline PostalAddress vs reference → inline VisitingAddress/, + ); + }); +}); + +describe('framing depth over several targets', () => { + it('frames as far as the furthest target reaches', () => { + // Person's key field is one more hop than Organization's fields: the + // frame has to hold whichever declaration the referent turns out to match. + expect(inlineFramingDepth(schema, edgedWork)).toBe(2); + const shallow = searchSchema( + defineSearchType({ + ...edgedWork, + name: 'ShallowWork', + }), + defineSearchType({ + ...creatorEdge, + fields: [ + creatorEdge.fields[0], + { + ...creatorEdge.fields[1], + ref: { strategy: 'lookup', target: 'Organization', local: true }, + }, + ], + }), + organization, + ); + expect(inlineFramingDepth(shallow, shallow.get(edgedWork.class)!)).toBe(2); + }); +}); + +describe('projecting a plain lookup over several targets', () => { + const node = { + '@id': 'https://ex/work/1', + [alias('Work', 'creator')]: [rembrandt, sisters, untyped], + }; + const document = projectDocument(node, work, schema); + + it('re-keys each referent through the target its rdf:type matches', () => { + // The person through Person's key, the organization through nothing. + expect(document.creator).toEqual([ + RKD, + 'https://id.example/zusters', + 'https://id.example/unknown', + ]); + }); + + it('admits each id to a facet by the policy of its own target', () => { + // The organization has no policy to fail; the untyped referent fell back + // to the first target, whose policy it does not satisfy. + expect(document.creator_facet).toEqual([RKD, 'https://id.example/zusters']); + }); + + it('judges a transformed id by its target still', () => { + // A `transform` changes what the field stores; the policy is looked up by + // the stored value, so the organization keeps its unconditional admission + // and the person is still held to Person's policy. + const transformed = defineSearchType({ + ...work, + name: 'TransformedWork', + fields: [ + { + ...work.fields[0], + transform: (value: string) => value.replace('https://', 'http://'), + }, + ], + }); + const projected = projectDocument( + { + ...node, + [alias('TransformedWork', 'creator')]: node[alias('Work', 'creator')], + }, + transformed, + searchSchema(transformed, person, organization), + ); + expect(projected.creator_facet).toEqual(['http://id.example/zusters']); + }); +}); + +describe('projecting a local lookup over several targets', () => { + const node = { + '@id': 'https://ex/work/2', + [alias('EdgedWork', 'creator')]: [ + { + [alias('CreatorEdge', 'role')]: [{ '@value': 'etser' }], + [alias('CreatorEdge', 'creator')]: [rembrandt], + }, + { + [alias('CreatorEdge', 'role')]: [{ '@value': 'uitgever' }], + [alias('CreatorEdge', 'creator')]: [sisters], + }, + { + [alias('CreatorEdge', 'role')]: [{ '@value': 'drukker' }], + [alias('CreatorEdge', 'creator')]: [untyped], + }, + { + // Named inline as an organization, with no id: the case a collection + // can never answer for, and the one the discriminator exists for. + [alias('CreatorEdge', 'role')]: [{ '@value': 'opdrachtgever' }], + [alias('CreatorEdge', 'creator')]: [ + { + '@type': `${SCHEMA_ORG}Organization`, + [alias('Organization', 'name')]: [{ '@value': 'Gemeente' }], + [alias('Organization', 'location')]: [{ '@value': 'Maastricht' }], + }, + ], + }, + ], + }; + const document = projectDocument(node, edgedWork, schema); + const entries = document.creator as readonly SearchDocument[]; + const endpointOf = (index: number) => + entries[index].creator as SearchDocument; + + it('projects each endpoint through the target it matches, and says which', () => { + expect(endpointOf(0)).toMatchObject({ + id: RKD, + name_und: 'Rembrandt', + [TARGET_FIELD]: 'Person', + }); + expect(endpointOf(1)).toMatchObject({ + id: 'https://id.example/zusters', + name_und: 'Zusters Benedictinessen', + [TARGET_FIELD]: 'Organization', + }); + expect(endpointOf(3)).toMatchObject({ + name_und: 'Gemeente', + location: 'Maastricht', + [TARGET_FIELD]: 'Organization', + }); + expect(endpointOf(3)).not.toHaveProperty('id'); + }); + + it('falls back to the first target declared for a referent matching none', () => { + expect(endpointOf(2)).toMatchObject({ + id: 'https://id.example/unknown', + [TARGET_FIELD]: 'Person', + }); + }); + + it('admits each harvested id to the facet by the policy of its own target', () => { + expect(document.creator_id).toEqual([ + RKD, + 'https://id.example/zusters', + 'https://id.example/unknown', + ]); + expect(document.creator_id_facet).toEqual([ + RKD, + 'https://id.example/zusters', + ]); + }); + + it('stores no discriminator for a lookup naming one target', () => { + const single = searchSchema( + defineSearchType({ ...edgedWork, name: 'SingleWork' }), + defineSearchType({ + ...creatorEdge, + fields: [ + creatorEdge.fields[0], + { + ...creatorEdge.fields[1], + ref: { strategy: 'lookup', target: 'Person', local: true }, + }, + ], + }), + person, + ); + const singleWork = single.get(edgedWork.class)!; + const projected = projectDocument( + { + ...node, + [alias('SingleWork', 'creator')]: node[alias('EdgedWork', 'creator')], + }, + singleWork, + single, + ); + const [first] = projected.creator as readonly SearchDocument[]; + expect(first.creator).not.toHaveProperty(TARGET_FIELD); + }); +}); + +describe('validating a projection over several targets', () => { + const base = { + where: [], + orderBy: [], + limit: 10, + offset: 0, + facets: [], + locale: 'nl', + }; + + it('accepts a field any target serves, and rejects one none does', () => { + expect( + validateQuery( + { + ...base, + resolve: { creator: { fields: ['birthDate', 'location'] } }, + }, + work, + schema, + ), + ).toEqual([]); + expect( + validateQuery( + { ...base, resolve: { creator: { fields: ['deathDate'] } } }, + work, + schema, + ), + ).toEqual([ + { part: 'resolve', field: 'creator.deathDate', reason: 'unknown-field' }, + ]); + }); +}); diff --git a/packages/search/test/qualified-relation.test.ts b/packages/search/test/qualified-relation.test.ts index 69d3ca01..b6c5734d 100644 --- a/packages/search/test/qualified-relation.test.ts +++ b/packages/search/test/qualified-relation.test.ts @@ -3,7 +3,7 @@ import { projectDocument, type SearchDocument } from '../src/project.js'; import { defineSearchType, inlineFramingDepth, - labelTargetNameOf, + labelTargetNamesOf, physicalFields, searchSchema, } from '../src/schema.js'; @@ -1412,11 +1412,11 @@ describe('the label target a facet reads', () => { it('is the identity companion’s target, one level in', () => { // An inline reference names no label source of its own, so reading only // its own declaration would leave its facet buckets unlabelled. - expect(labelTargetNameOf(work.fields[0], schema)).toBe('Person'); + expect(labelTargetNamesOf(work.fields[0], schema)).toEqual(['Person']); }); it('is nothing for a field that references nothing', () => { - expect(labelTargetNameOf(creatorEdge.fields[0], schema)).toBeUndefined(); + expect(labelTargetNamesOf(creatorEdge.fields[0], schema)).toEqual([]); }); it('is nothing against a schema that does not declare the edge’s type', () => { @@ -1425,7 +1425,7 @@ describe('the label target a facet reads', () => { // reading every schema-less path in the projection makes. const foreign = searchSchema(person); - expect(labelTargetNameOf(work.fields[0], foreign)).toBeUndefined(); + expect(labelTargetNamesOf(work.fields[0], foreign)).toEqual([]); }); }); diff --git a/packages/search/test/schema.test.ts b/packages/search/test/schema.test.ts index 76d528ea..9d9b3f28 100644 --- a/packages/search/test/schema.test.ts +++ b/packages/search/test/schema.test.ts @@ -16,7 +16,7 @@ import { isoToUnixSeconds, isInternalField, isRangeFacet, - labelSourceNameOf, + labelSourceNamesOf, nestedFieldName, nestedReferenceType, outputFields, @@ -1564,26 +1564,43 @@ describe('searchSchema validation', () => { ); }); - it('reads the label source name off a lookup’s target and an idOnly’s labelSource', () => { + it('reads the label source names off a lookup’s target and an idOnly’s labelSource', () => { expect( - labelSourceNameOf({ + labelSourceNamesOf({ name: 'publisher', kind: 'reference', ref: { strategy: 'lookup', target: 'Organization' }, }), - ).toBe('Organization'); + ).toEqual(['Organization']); expect( - labelSourceNameOf({ + labelSourceNamesOf({ name: 'license', kind: 'reference', labelSource: 'Term', ref: { strategy: 'idOnly' }, }), - ).toBe('Term'); + ).toEqual(['Term']); + // Several targets, in the order declared: the one shape for one and + // for many, so no reader branches on which it was given. + expect( + labelSourceNamesOf({ + name: 'creator', + kind: 'reference', + ref: { strategy: 'lookup', target: ['Person', 'Organization'] }, + }), + ).toEqual(['Person', 'Organization']); + expect( + labelSourceNamesOf({ + name: 'creator', + kind: 'reference', + labelSource: ['Person', 'Organization'], + ref: { strategy: 'idOnly' }, + }), + ).toEqual(['Person', 'Organization']); // Resolves nothing: no target, no label source. expect( - labelSourceNameOf({ name: 'license', kind: 'reference' }), - ).toBeUndefined(); + labelSourceNamesOf({ name: 'license', kind: 'reference' }), + ).toEqual([]); }); it('rejects a labelSource on anything but an idOnly reference', () => { diff --git a/packages/search/vite.config.ts b/packages/search/vite.config.ts index 92874588..ec1f86b4 100644 --- a/packages/search/vite.config.ts +++ b/packages/search/vite.config.ts @@ -12,7 +12,7 @@ export default mergeConfig( thresholds: { functions: 100, lines: 100, - branches: 99.72, + branches: 99.74, statements: 100, }, }, From 64f4744ddf0a8e0aaa4b2ff0449ddbd965f05570 Mon Sep 17 00:00:00 2001 From: David de Boer Date: Thu, 10 Sep 2026 19:55:39 +0200 Subject: [PATCH 2/3] test(search): cover the polymorphic lookup where CI found it bare - Drop the GraphQL fixture that named one field at two referents: the schema now rejects that pairing, so the interface builder no longer re-checks it and the branch threshold is re-anchored. - Cover a single-target lookup skipping an id-only selection, an empty stored referent field, a single-valued polymorphic nesting and a cut descent that still declares its discriminator. --- .../coverage/coverage-summary.json | 45 +++++++ .../search-api-graphql/src/build-schema.ts | 19 +-- .../test/polymorphic-lookup.test.ts | 43 ++++-- packages/search-api-graphql/vite.config.ts | 5 +- .../test/polymorphic-lookup.test.ts | 122 ++++++++++++++++++ 5 files changed, 204 insertions(+), 30 deletions(-) create mode 100644 packages/search-api-graphql/coverage/coverage-summary.json diff --git a/packages/search-api-graphql/coverage/coverage-summary.json b/packages/search-api-graphql/coverage/coverage-summary.json new file mode 100644 index 00000000..b4972113 --- /dev/null +++ b/packages/search-api-graphql/coverage/coverage-summary.json @@ -0,0 +1,45 @@ +{ + "total": { + "lines": { "total": 457, "covered": 457, "skipped": 0, "pct": 100 }, + "statements": { "total": 468, "covered": 468, "skipped": 0, "pct": 100 }, + "functions": { "total": 119, "covered": 119, "skipped": 0, "pct": 100 }, + "branches": { "total": 330, "covered": 321, "skipped": 0, "pct": 97.27 }, + "branchesTrue": { "total": 0, "covered": 0, "skipped": 0, "pct": "Unknown" } + }, + "/Users/david/src/lde-issue-717-polymorphic-lookup/packages/search-api-graphql/src/build-schema.ts": { + "lines": { "total": 328, "covered": 328, "skipped": 0, "pct": 100 }, + "functions": { "total": 83, "covered": 83, "skipped": 0, "pct": 100 }, + "statements": { "total": 331, "covered": 331, "skipped": 0, "pct": 100 }, + "branches": { "total": 235, "covered": 229, "skipped": 0, "pct": 97.44 } + }, + "/Users/david/src/lde-issue-717-polymorphic-lookup/packages/search-api-graphql/src/facet-batch.ts": { + "lines": { "total": 37, "covered": 37, "skipped": 0, "pct": 100 }, + "functions": { "total": 14, "covered": 14, "skipped": 0, "pct": 100 }, + "statements": { "total": 42, "covered": 42, "skipped": 0, "pct": 100 }, + "branches": { "total": 20, "covered": 20, "skipped": 0, "pct": 100 } + }, + "/Users/david/src/lde-issue-717-polymorphic-lookup/packages/search-api-graphql/src/handler.ts": { + "lines": { "total": 16, "covered": 16, "skipped": 0, "pct": 100 }, + "functions": { "total": 5, "covered": 5, "skipped": 0, "pct": 100 }, + "statements": { "total": 17, "covered": 17, "skipped": 0, "pct": 100 }, + "branches": { "total": 19, "covered": 19, "skipped": 0, "pct": 100 } + }, + "/Users/david/src/lde-issue-717-polymorphic-lookup/packages/search-api-graphql/src/language.ts": { + "lines": { "total": 11, "covered": 11, "skipped": 0, "pct": 100 }, + "functions": { "total": 4, "covered": 4, "skipped": 0, "pct": 100 }, + "statements": { "total": 12, "covered": 12, "skipped": 0, "pct": 100 }, + "branches": { "total": 8, "covered": 7, "skipped": 0, "pct": 87.5 } + }, + "/Users/david/src/lde-issue-717-polymorphic-lookup/packages/search-api-graphql/src/print-sdl.ts": { + "lines": { "total": 11, "covered": 11, "skipped": 0, "pct": 100 }, + "functions": { "total": 2, "covered": 2, "skipped": 0, "pct": 100 }, + "statements": { "total": 11, "covered": 11, "skipped": 0, "pct": 100 }, + "branches": { "total": 7, "covered": 7, "skipped": 0, "pct": 100 } + }, + "/Users/david/src/lde-issue-717-polymorphic-lookup/packages/search-api-graphql/src/projection.ts": { + "lines": { "total": 54, "covered": 54, "skipped": 0, "pct": 100 }, + "functions": { "total": 11, "covered": 11, "skipped": 0, "pct": 100 }, + "statements": { "total": 55, "covered": 55, "skipped": 0, "pct": 100 }, + "branches": { "total": 41, "covered": 39, "skipped": 0, "pct": 95.12 } + } +} diff --git a/packages/search-api-graphql/src/build-schema.ts b/packages/search-api-graphql/src/build-schema.ts index 08ca9952..9f8738c9 100644 --- a/packages/search-api-graphql/src/build-schema.ts +++ b/packages/search-api-graphql/src/build-schema.ts @@ -656,22 +656,9 @@ export function buildGraphQLSchema( ) { return undefined; } - // A reference field is shared only where every member serves it as the - // same emitted type – same strategy, same referent – or the interface - // would offer one field of two types: an `idOnly` surfaces as an `IRI` - // where a lookup to the same name surfaces as an object. - if ( - declared.kind === 'reference' && - counterparts.some( - (counterpart) => - (counterpart as ReferenceField).ref?.strategy !== - declared.ref?.strategy || - referencedTypeName(counterpart as ReferenceField) !== - referencedTypeName(declared), - ) - ) { - return undefined; - } + // A shared reference field is served as one emitted type on every + // member: `searchSchema` holds the targets to the same strategy and the + // same referent for a name they share, so nothing is checked here. const required = !nullableId && counterparts.every((counterpart) => counterpart?.required === true); diff --git a/packages/search-api-graphql/test/polymorphic-lookup.test.ts b/packages/search-api-graphql/test/polymorphic-lookup.test.ts index 59341d77..4a9e5d10 100644 --- a/packages/search-api-graphql/test/polymorphic-lookup.test.ts +++ b/packages/search-api-graphql/test/polymorphic-lookup.test.ts @@ -49,13 +49,6 @@ const person: SearchType = { output: true, ref: { strategy: 'lookup', target: 'Place' }, }, - // Shared in name only: each member points it at a different type. - { - name: 'affiliation', - kind: 'reference', - output: true, - ref: { strategy: 'lookup', target: 'Organization' }, - }, ], }; const organization: SearchType = { @@ -78,12 +71,6 @@ const organization: SearchType = { output: true, ref: { strategy: 'lookup', target: 'Place' }, }, - { - name: 'affiliation', - kind: 'reference', - output: true, - ref: { strategy: 'lookup', target: 'Place' }, - }, ], }; const place: SearchType = { @@ -189,6 +176,36 @@ describe('serving a lookup over several targets', () => { } }); + it('refuses a declaration spelling the derived name itself', () => { + // A Reference Type named `PersonOrOrganization` would put its filter + // under the interface's filter name; refused rather than served wrongly. + const spelled: SearchType = { + name: 'PersonOrOrganization', + fields: [{ name: 'note', kind: 'keyword', output: true }], + }; + const nesting = (fields: readonly SearchField[]): SearchType => ({ + ...work, + fields, + }); + const note: SearchField = { + name: 'note', + kind: 'reference', + output: true, + ref: { strategy: 'inline', typeName: 'PersonOrOrganization' }, + }; + // Whichever registers first, the other is refused. + for (const fields of [ + [...work.fields, note], + [note, ...work.fields], + ]) { + expect(() => + buildGraphQLSchema( + searchSchema(nesting(fields), person, organization, place, spelled), + ), + ).toThrow(/collides with another type name/); + } + }); + it('shares a reference field only where every target points it at one type', () => { // Person.affiliation looks up an Organization, Organization.affiliation a // Place: two emitted types, so the interface offers neither. diff --git a/packages/search-api-graphql/vite.config.ts b/packages/search-api-graphql/vite.config.ts index 2708d450..dbea2aaa 100644 --- a/packages/search-api-graphql/vite.config.ts +++ b/packages/search-api-graphql/vite.config.ts @@ -29,7 +29,10 @@ export default mergeConfig( // Re-anchored for the selection-set projection: its defensive reads // (an unresolvable target, an absent field list) are reachable only // from a direct caller, since GraphQL validates the query first. - branches: 97.32, + // Re-anchored for the polymorphic lookup: the shared-field reading + // of its interface no longer re-checks what `searchSchema` already + // guarantees of a field two targets share, so that branch is gone. + branches: 97.27, statements: 100, }, }, diff --git a/packages/search-typesense/test/polymorphic-lookup.test.ts b/packages/search-typesense/test/polymorphic-lookup.test.ts index 023f5620..6619945e 100644 --- a/packages/search-typesense/test/polymorphic-lookup.test.ts +++ b/packages/search-typesense/test/polymorphic-lookup.test.ts @@ -372,6 +372,55 @@ describe('a stored referent of several possible kinds', () => { }); }); +describe('a lookup naming one target', () => { + it('reads nothing for a selection reduced to id', async () => { + const single = defineSearchType({ + ...work, + name: 'SingleWork', + class: 'https://example.org/SingleWork', + fields: [ + { + ...work.fields[1], + ref: { strategy: 'lookup', target: 'Person' }, + }, + ], + }); + const { fake, searches } = client({ + found: 1, + hits: [{ document: { id: 'https://w/3', creator: ['https://p/1'] } }], + }); + const engine = createTypesenseSearchEngine( + fake.client, + searchSchema(single, person), + { collections: { ...collections, SingleWork: 'singles' } }, + ); + + const result = await engine.search(single as never, { + ...base, + resolve: { creator: { fields: [] } }, + }); + + expect(searches).toHaveLength(0); + expect( + (result.hits[0].document as Record).creator, + ).toEqual([{ id: 'https://p/1', label: { nl: ['Frank Koel'] } }]); + }); +}); + +describe('a stored referent field with no entries', () => { + it('reconstructs nothing', async () => { + const { fake } = client({ + found: 1, + hits: [{ document: { id: 'https://w/4', creator: [] } }], + }); + const engine = createTypesenseSearchEngine(fake.client, schema, { + collections, + }); + const result = await engine.search(edgedWork as never, base); + expect(result.hits[0].document).toEqual({}); + }); +}); + describe('declaring the collection for a stored referent of several kinds', () => { it('declares the union of the targets’ fields once, plus the discriminator', () => { const definition = buildCollectionDefinition(edgedWork, { schema }); @@ -391,4 +440,77 @@ describe('declaring the collection for a stored referent of several kinds', () = ), ).toMatchObject({ type: 'string[]', index: false, optional: true }); }); + + it('declares a single-valued referent as one object with one discriminator', () => { + const single = defineSearchType({ + ...edgedWork, + name: 'Print', + class: 'https://example.org/Print', + fields: [ + { + name: 'creator', + kind: 'reference', + output: true, + ref: { + strategy: 'lookup', + target: ['Person', 'Organization'], + local: true, + }, + }, + ], + }); + const definition = buildCollectionDefinition(single, { + schema: searchSchema(single, person, organization), + }); + expect( + definition.fields?.find((field) => field.name === 'creator'), + ).toMatchObject({ type: 'object' }); + expect( + definition.fields?.find( + (field) => field.name === `creator.${TARGET_FIELD}`, + ), + ).toMatchObject({ type: 'string' }); + }); + + it('still declares the discriminator where the descent is cut', () => { + // Person reaches back to itself and to the root: both already on the + // path, so the nesting stops – but the projection stores the referent + // as an `{ id, _target }` object all the same. + const knowing = defineSearchType({ + ...person, + fields: [ + ...person.fields, + { + name: 'related', + kind: 'reference', + output: true, + ref: { + strategy: 'lookup', + target: ['Person', 'Painting'], + local: true, + }, + }, + ], + }); + const painting = defineSearchType({ + ...edgedWork, + labelField: 'title', + fields: [ + { + name: 'title', + kind: 'text', + locales: ['nl'], + output: true, + searchable: { weight: 1 }, + }, + ...edgedWork.fields, + ], + }); + const definition = buildCollectionDefinition(painting, { + schema: searchSchema(painting, knowing, organization, creatorEdge), + }); + const names = definition.fields?.map((field) => field.name) ?? []; + expect(names).toContain('creator.creator.related'); + expect(names).toContain(`creator.creator.related.${TARGET_FIELD}`); + }); }); From 801fda4528fe950160ba3c6486061ece5e3c246b Mon Sep 17 00:00:00 2001 From: David de Boer Date: Thu, 10 Sep 2026 20:00:17 +0200 Subject: [PATCH 3/3] test(search-typesense): reach the edge with no endpoint through the local lookup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The empty-endpoint case is read one level in, by the stored lookup, not by the inline entry above it – so the test states an edge without a creator rather than a work without edges. --- .../search-typesense/test/polymorphic-lookup.test.ts | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/packages/search-typesense/test/polymorphic-lookup.test.ts b/packages/search-typesense/test/polymorphic-lookup.test.ts index 6619945e..6f68dcef 100644 --- a/packages/search-typesense/test/polymorphic-lookup.test.ts +++ b/packages/search-typesense/test/polymorphic-lookup.test.ts @@ -407,17 +407,19 @@ describe('a lookup naming one target', () => { }); }); -describe('a stored referent field with no entries', () => { - it('reconstructs nothing', async () => { +describe('an edge stating no endpoint', () => { + it('reconstructs the entry without one', async () => { const { fake } = client({ found: 1, - hits: [{ document: { id: 'https://w/4', creator: [] } }], + hits: [ + { document: { id: 'https://w/4', creator: [{ role: 'drukker' }] } }, + ], }); const engine = createTypesenseSearchEngine(fake.client, schema, { collections, }); const result = await engine.search(edgedWork as never, base); - expect(result.hits[0].document).toEqual({}); + expect(result.hits[0].document).toEqual({ creator: [{ role: 'drukker' }] }); }); });