diff --git a/docs/api-coverage.md b/docs/api-coverage.md index 1cbc150..acce72d 100644 --- a/docs/api-coverage.md +++ b/docs/api-coverage.md @@ -29,9 +29,34 @@ A generic request method does not mean that every endpoint has a typed wrapper, | Backup and restore | `replication("backup" | "restore" | "restorestatus" | "deletebackup")`, `collections("BACKUP" | "RESTORE" | "LISTBACKUP" | "DELETEBACKUP" | "REQUESTSTATUS" | "DELETESTATUS")` | Typed core and SolrCloud backup/restore parameters and response models, including async status polling. Full core restore and SolrCloud collection backup/restore cycles are integration-tested on Solr 8.11.4/9.11.0/10.1.0 using local filesystem repositories and shared SolrCloud storage. See [backup and restore docs](backup-restore.md). | | Replication | `replication(command, params)`, `replicationFile(params)`; exported command-specific parameter and response models | V1 ReplicationHandler commands for index version, details, file list, leader/follower control (fetchindex/abortfetch across Solr 8/9/10), core backup/restore/status/delete, and raw file-content streaming. Integration coverage includes core backup/restore cycles on Solr 8.11.4/9.11.0/10.1.0. A disposable two-core leader/follower fetch using `fetchindex` and follower query read-back passed the full standalone/security matrix on Solr 8.11.4 / 9.11.0 / 10.1.0 (CI #640). Abort and polling lifecycle behavior remains unit-only. See Issue #95 and [replication docs](replication.md). | | Streaming Expressions | `stream()` | Sends `expr` with `wt=json` and parses the complete JSON response through the shared transport. It is not an incremental tuple-stream parser: no chunk-boundary protocol or memory-bound/backpressure guarantee is claimed. SolrCloud search is integration-tested on pinned Solr 8.11.4/9.11.0/10.1.0; unit tests cover JSON parsing, caller cancellation during body consumption, and cleanup after body-stream errors. See [Streaming Expressions](streaming.md) and Issue #93. | -| Export | `export()`, `exportStream()`; exported `SolrExportParams`, `SolrExportSort`, and `SolrExportFieldType` | Export requests require typed `fl` and `sort` parameters, support repeated filter queries, structured or raw sort expressions, batch sizing, JSON/JavaBin writers, and an explicit custom-parameter escape hatch. The raw Response preserves backpressure-aware streaming; `exportStream()` exposes its byte body directly. Sort fields must be single-valued docValues fields (int, long, float, double, or string); field lists additionally support date and boolean and may be multi-valued. Solr 10.1 `includeStoredFields` support is exposed but requires version-specific verification. Existing raw Export response tests pass on pinned Solr 8.11.4/9.11.0/10.1.0; typed parameter serialization and streamed chunk consumption have unit coverage. See [Export docs](export.md) and Issues #92–#94. | +| Export | `export()`, `exportStream()`; exported `SolrExportParams`, `SolrExportSort`, and `SolrExportFieldType` | Export requests require typed `fl` and `sort` parameters, support repeated filter queries, structured or raw sort expressions, batch sizing, JSON/JavaBin writers, and an explicit custom-parameter escape hatch. The raw Response preserves backpressure-aware streaming; `exportStream()` exposes its byte body directly. Sort fields must be single-valued docValues fields (int, long, float, double, or string); field lists additionally support date and boolean and may be multi-valued. Solr 10.1 `includeStoredFields=true` is runtime-tested on 10.1.0 with stored-only field read-back; no support claim is made for Solr 8/9. Existing raw Export response tests pass on pinned Solr 8.11.4/9.11.0/10.1.0; typed parameter serialization and streamed chunk consumption have unit coverage. See [Export docs](export.md) and Issues #92–#94. | | Advanced / Raw | `requestRaw(path, options)` with typed method/query/header/body/control options and a raw Fetch `Response` | Relative paths and same-origin absolute URLs use the shared transport. Cross-origin URLs, embedded credentials, and URL fragments are rejected before Fetch, preventing configured credentials from being sent to unrelated hosts. Generic request()/requestRaw() transport behavior has unit coverage only; arbitrary custom endpoint behavior remains unverified until endpoint-specific integration tests exist. See [raw requests](raw-requests.md) and Issue #98. | +## Deferred work and explicit scope boundaries + +The following gaps were reviewed for the TypeScript-first, balanced product scope. They are not implicit implementation commitments. + +### Accepted, focused verification work + +- **Export `includeStoredFields`** — verified on Solr 10.1.0 by [PR #261](https://github.com/vsimko/node-solr-lambda/pull/261) using stored-only field read-back. Solr 8/9 behavior remains unclaimed. +- **ReplicationHandler abort/async lifecycle** — one-shot `fetchindex` and follower read-back are verified, but abort/polling semantics are not; tracked by [#259](https://github.com/vsimko/node-solr-lambda/issues/259). + +### Deferred pending a concrete consumer requirement + +- **Solr-version-specific error payload variants** — error envelopes preserve unknown fields and expose the typed common shape. Add version-specific models only when a concrete incompatible payload is identified and can be tested. +- **Advanced authentication (for example OAuth or mTLS)** — deployment-specific credential acquisition and certificate lifecycle are outside the built-in Basic/Bearer transport contract. The custom Fetch/headers extension points remain available. +- **Retry telemetry** — retry policy is opt-in; logging/metrics integration belongs to caller observability policy. The transport deliberately avoids logging credentials or server-controlled bodies. +- **Additional validation adapters and operation-specific schema examples** — the generic schema interface and optional Zod peer cover the supported extension contract. Add another adapter only for a demonstrated ecosystem demand. +- **Runtime verification of Core Admin `RENAME`, `SWAP`, `MERGEINDEXES`, `SPLIT`, `REQUESTSTATUS`, and `REQUESTRECOVERY`** — these operations have isolated/destructive or operational side effects and remain type/serialization-covered only until a safe, deterministic disposable environment is justified. +- **Collections `MODIFYCOLLECTION`, `ADDROLE`, and `REMOVEROLE` runtime verification** — these mutate cluster-wide policy/configuration and are not run against shared CI state. SPLITSHARD/MIGRATE have the separate opt-in gate in [#232](https://github.com/vsimko/node-solr-lambda/issues/232). + +### Intentional non-goals + +- **Plugin-specific Schema similarity mutation and plugin-specific Metrics/Auth schemas** — arbitrary plugin contracts are not part of the closed typed surface. Common stable fields remain typed and extension points preserve unknown/custom fields. +- **Solr 10 V2 routes where the client explicitly implements V1 contracts** — V2 is a distinct API surface, not a silent compatibility extension. Add it only as a separately scoped feature with its own types and tests. + +These decisions do not change the compatibility matrix: unverified operations remain unverified until exact runtime evidence exists. + ## Request controls Typed client methods support per-request cancellation and timeout controls. Caller cancellation throws `SolrAbortError`; a configured timeout throws `SolrTimeoutError`. The timeout remains active while JSON response bodies are read and while raw `requestRaw()` or export response streams are consumed. See [error handling](errors.md). diff --git a/docs/api-inventory.json b/docs/api-inventory.json index def9bb1..446c685 100644 --- a/docs/api-inventory.json +++ b/docs/api-inventory.json @@ -213,7 +213,7 @@ "test/export.test.ts", "test/export-api-types.ts" ], - "notes": "Typed SolrExportParams requires fl and sort, supports filter query arrays, raw or structured sort expressions, batchSize, response writer selection, optional local expr, and customParams. export() returns the raw backpressure-aware Response. Sort fields must be single-valued docValues int/long/float/double/string fields; fl additionally supports date/boolean and multi-valued fields. includeStoredFields is documented for Solr 10.1 but remains version-specific. Existing live raw-response integration test covers Solr 8.11.4, 9.11.0, and 10.1.0; unit tests cover typed serialization." + "notes": "Typed SolrExportParams requires fl and sort, supports filter query arrays, raw or structured sort expressions, batchSize, response writer selection, optional local expr, and customParams. export() returns the raw backpressure-aware Response. Sort fields must be single-valued docValues int/long/float/double/string fields; fl additionally supports date/boolean and multi-valued fields. `includeStoredFields=true` is runtime-tested only on Solr 10.1.0 with a stored-only field read-back (PR #261 / CI #648); no Solr 8/9 claim is made. Existing live raw-response integration test covers Solr 8.11.4, 9.11.0, and 10.1.0; unit tests cover typed serialization." }, { "id": "config.get", diff --git a/docs/compatibility.md b/docs/compatibility.md index 9bf47d8..50b76ac 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -11,6 +11,10 @@ This page records the current client compatibility policy. It separates document Documentation evidence does not prove that every patch release behaves in the same way. +## Interpreting unverified operations + +An exposed action or parameter is not runtime-certified merely because it has a TypeScript model or serialization test. The API coverage page records the current triage of deferred and intentionally out-of-scope gaps. In particular, destructive Core Admin actions and cluster-wide Collections policy mutations remain unverified until isolated runtime evidence is added; SPLITSHARD/MIGRATE require the opt-in workflow in [#232](https://github.com/vsimko/node-solr-lambda/issues/232). Export `includeStoredFields=true` is verified only on Solr 10.1.0 by stored-only field read-back; no Solr 8/9 claim is made. ReplicationHandler abort/async lifecycle remains unverified under [#259](https://github.com/vsimko/node-solr-lambda/issues/259). + ## Reference baselines | Solr line | Documentation baseline | Documentation | @@ -41,7 +45,7 @@ The 8.11, 9.11, and 10.1 guides describe the API families below. The Replica Man | `metrics()` — `/solr/admin/metrics` | Documented | Documented | Documented | Tested: non-empty JVM registry/metric entries for JSON or a Prometheus metric declaration/sample on pinned 8.11.4 / 9.11.0 / 10.1.0; observed JSON on Solr 8/9 and Prometheus text on Solr 10.1 | | `security()` — authentication and authorization handlers | Documented | Documented | Documented | Tested: authenticated BasicAuth/RuleBasedAuthorization reads, anonymous 401 on both handlers, authentication `set-property` read-back, and malformed JSON rejection on pinned 8.11.4 / 9.11.0 / 10.1.0 images. Unit tests verify Basic/Bearer credentials are not exposed by client HTTP errors | | `stream()` — `/stream` | Documented | Documented | Documented | Tested: Streaming Expressions search against a disposable SolrCloud collection on pinned 8.11.4 / 9.11.0 / 10.1.0 images (CI #145) | -| `export()` — `/export` | Documented | Documented | Documented | Tested: raw response with docValues sort field on pinned 8.11.4 / 9.11.0 / 10.1.0 images | +| `export()` — `/export` | Documented | Documented | Documented | Tested: raw response with docValues sort field on pinned 8.11.4 / 9.11.0 / 10.1.0 images; `includeStoredFields=true` returns a stored-only field on pinned Solr 10.1.0 (PR #261). No `includeStoredFields` claim for Solr 8/9 | | `replication()` / `replicationFile()` — `/solr/{core}/replication` | Documented | Documented | Documented | Tested: indexversion, details, filelist (generation on newer versions / legacy indexversion on older versions), streamed filecontent, enablereplication/disablereplication, core backup, restorestatus, restore, and deletebackup on pinned 8.11.4 / 9.11.0 / 10.1.0 images. A disposable two-core leader/follower fetch using `fetchindex` with follower query read-back passed the full standalone/security matrix on Solr 8.11.4 / 9.11.0 / 10.1.0 (CI #640). `abortfetch`, `enablepoll`, and `disablepoll` retain unit serialization coverage only; cancellation of an in-progress transfer and polling lifecycle behavior remain unverified | | Core and collection backup/restore | `replication("backup" | "restore" | "restorestatus" | "deletebackup")`, `collections("BACKUP" | "RESTORE" | "LISTBACKUP" | "DELETEBACKUP" | "REQUESTSTATUS" | "DELETESTATUS")` | Documented | Documented | Documented | Tested: full core backup/restore/read-back and SolrCloud collection backup/list/restore/read-back/delete with asynchronous status polling on pinned 8.11.4 / 9.11.0 / 10.1.0 images using local filesystem repositories and shared SolrCloud storage | | `request()` / `requestRaw()` — generic caller-defined transport | Depends on the selected endpoint | Depends on the selected endpoint | Depends on the selected endpoint | Generic transport behavior has unit tests only; no endpoint-independent live Solr integration claim is made. Each custom endpoint remains unverified until tested individually | diff --git a/docs/export.md b/docs/export.md index 0886815..bdfe637 100644 --- a/docs/export.md +++ b/docs/export.md @@ -52,4 +52,4 @@ The field type union is descriptive metadata for application schemas; the client The supported Export writers are `json` and `javabin`; legacy `xsort` is accepted as an alias for JSON. Local `expr` processing must use `input()` as its source. Distributed export requires Streaming Expressions. -The existing live integration test exercises a raw Export response with a docValues sort field on Solr 8.11.4, 9.11.0, and 10.1.0. Typed parameter serialization and incremental stream consumption have unit coverage. `includeStoredFields` is documented by Solr 10.1; its behavior on Solr 8/9 is not claimed. +The live integration test exercises a raw Export response with a docValues sort field on Solr 8.11.4, 9.11.0, and 10.1.0. A separate Solr 10.1-only test creates a stored-only field and verifies that `includeStoredFields=true` returns its value (PR #261 / CI #648). This option is not claimed for Solr 8/9. Typed parameter serialization and incremental stream consumption have unit coverage. diff --git a/docs/roadmap.md b/docs/roadmap.md index 85cbd52..519f2a4 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -29,7 +29,7 @@ The scope is balanced: cover major Solr API areas and production readiness witho | Work | Tracking | | --- | --- | -| Classify residual documented API gaps as focused follow-up work, deferred scope, or explicit non-goals | [#256](https://github.com/vsimko/node-solr-lambda/issues/256) | +| Verify isolated ReplicationHandler abort/async lifecycle semantics, or explicitly retain unverified status | [#259](https://github.com/vsimko/node-solr-lambda/issues/259) | ### P1 — SolrCloud operation evidence @@ -43,7 +43,7 @@ v4.0 stable publication remains a separate release-owner decision and must not b ## Dependencies and execution -- #255 is the documentation baseline for #256; keep operation status grounded in exact tests and pinned versions. +- #255 established the documentation baseline. The #256 gap triage is delivered by the merged roadmap/API-coverage update; #258 is verified by PR #261, while #259 remains the focused ReplicationHandler lifecycle follow-up. Keep operation status grounded in exact tests and pinned versions. - #232 is independent of the documentation cleanup, but its results must be reflected in the Collections action matrix and API inventory before any compatibility claim changes. - #110 remains separate from feature implementation and depends on final documentation, release-candidate CI, external npm/GitHub trusted-publishing configuration, and release-owner approval. - Every implementation PR updates relevant API docs, compatibility evidence, and `api-inventory.json`; do not infer runtime support from type or serialization tests. @@ -62,6 +62,7 @@ Closed issues are retained as historical evidence, not active priorities: - Packaging, public API review, and operation inventory foundations: #104, #108, #184, and #185. - Feature and evidence follow-ups #228–#231 and #233–#238 are completed and merged. #238 (ReplicationHandler one-shot leader/follower fetch and read-back) was completed by merged PR #253. - The post-#253 documentation reconciliation is delivered by PR #257 (#255); it is not a continuing roadmap priority once merged. +- Solr 10.1 Export `includeStoredFields=true` stored-only field read-back is verified by PR #261 (#258); Solr 8/9 behavior remains unclaimed. - PR #253 also added the isolated opt-in SolrCloud test harness for SPLITSHARD/MIGRATE. Those actions remain unverified until the manual workflow has actually passed. ## Workflow diff --git a/test/integration/solr-export.test.ts b/test/integration/solr-export.test.ts index c93385a..397509b 100644 --- a/test/integration/solr-export.test.ts +++ b/test/integration/solr-export.test.ts @@ -25,3 +25,36 @@ integrationTest("Export endpoint returns a raw response for a docValues field", const body = await response.text(); assert.ok(body.includes("777"), "export stream should contain the docValues value"); }); + +integrationTest("Solr 10.1 Export includes stored-only fields when requested", async ({ + client, + createId, + addDocuments, + addSchemaField, + commit +}) => { + // includeStoredFields is a Solr 10.1 feature; do not infer support on Solr 8/9. + if (process.env.SOLR_VERSION !== "10.1.0") return; + + const id = createId("export_stored"); + const storedField = createId("export_stored_field"); + const sortField = createId("export_stored_sort"); + const storedValue = createId("stored_value"); + + await addSchemaField(storedField, "string", false); + await addSchemaField(sortField, "pint", true); + await addDocuments([{ id, [storedField]: storedValue, [sortField]: 1 }]); + await commit(); + + const response = await client.export({ + q: `id:${id}`, + fl: [storedField, sortField], + sort: `${sortField} asc`, + includeStoredFields: true, + wt: "json" + }); + + assert.equal(response.ok, true); + const body = await response.text(); + assert.ok(body.includes(storedValue), "includeStoredFields should return the stored-only field value"); +});