Skip to content

Improve /search validation consistency and missing-ID warnings - #187

Open
afoote-mitre wants to merge 5 commits into
devfrom
af/180-search-consistency
Open

afoote-mitre wants to merge 5 commits into
devfrom
af/180-search-consistency

Conversation

@afoote-mitre

@afoote-mitre afoote-mitre commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Address the first five items in #180 for POST /search, with one commit per item. These changes make errors, boolean inputs, date boundaries, and missing-ID reporting more predictable.

Partially addresses #180. Items 6 and 7 remain deferred for team discussion.

Why These Changes Were Needed

Change Why it was needed Primary files
Return a consistent single error object Router validation previously returned an array while middleware and server errors returned an object, even for the same request field. /search now consistently uses { "error": "CODE", "message": "Description" }, returning the first error when multiple validations fail. Equivalent trailing-slash and case-insensitive route variants receive the same shape. Clients that previously parsed router errors as arrays must use the documented object shape. /webSearch error handling is unchanged. routes/index.js, routes/index.test.js, api-docs/openapi.test.js, README.md, test/postman/buildCollection.js
Distinguish invalid page sizes from values above the maximum Non-integer values such as "abc" and 1.5 previously received a misleading maximum-limit error. Invalid types, fractional values, and nonpositive values now return RESULTS_PER_PAGE_INT; positive integers above 500 return MAX_RESULTS_LIMIT. Valid integer strings remain accepted. Arrays and booleans are rejected before coercion. routes/index.js, routes/index.test.js, README.md, test/postman/buildCollection.js
Normalize all boolean filters consistently knownExploited and disputed previously rejected string booleans accepted by the other three boolean filters. All five now accept JSON booleans and lowercase "true"/"false" strings, normalizing strings before downstream checks. Numbers, null, arrays, and other strings remain invalid. Omitted filters retain their existing behavior. routes/index.js, routes/index.test.js, routes/swagger.js, api-docs/openapi.test.js, README.md, test/postman/buildCollection.js
Accept useful date representations without weakening calendar validation Requiring milliseconds and a literal UTC suffix rejected valid date-only and explicit-offset inputs. A shared normalizer now accepts YYYY-MM-DD and timestamps with seconds, an explicit timezone, and up to three fractional-second digits. Date-only starts become midnight UTC; date-only ends include the full day through 23:59:59.999Z. Explicit timestamps retain their exact instant. Range ordering is checked after normalization; impossible dates, timezone-less timestamps, leap seconds, and finer precision remain invalid. This applies to publication, modification, SSVC, and KEV boundaries. utils/searchDates.js, utils/searchDates.test.js, routes/index.js, controllers/middleware.js, utils/errors.js, utils/errors.test.js, route and middleware tests, routes/swagger.js, README.md
Warn accurately about requested CVE IDs absent from the index Missing requested IDs previously disappeared without explanation. A bounded ID-only lookup now distinguishes absence from exclusion by other filters or pagination. Successful matching records and pagination metadata remain intact, and warnings remain before records. Failed verification produces a sanitized warning rather than a false missing-ID claim. utils/searchCveIdWarnings.js, utils/searchCveIdWarnings.test.js, utils/searchQueryBuilder.js, utils/searchQueryBuilder.test.js, controllers/searchController.js, routes/search.integration.test.js, routes/swagger.js, README.md, Postman source and collection

Missing-ID Warning Behavior

  • IDs already returned by the main search need no verification. Pre-1999 IDs retain the existing CVE_IDS_NOT_RETURNED warning and are not included in the additional lookup.
  • Other requested IDs absent from the returned page are normalized, deduplicated, and checked in at most one additional request, bounded by the existing 100-ID input limit.
  • The lookup uses exact ID matching and per-ID counts, independent of scope, other filters, and pagination. It retrieves no complete records and handles duplicate indexed documents without hiding another requested ID.
  • CVE_IDS_NOT_FOUND means an ID was absent from the searchable configured index at lookup time, not that it does not exist globally. Existing IDs excluded by filters or pagination do not receive this warning.
  • CVE_ID_AVAILABILITY_UNVERIFIED identifies IDs whose verification failed while preserving an otherwise successful search. Incomplete or prematurely terminated lookups are not interpreted as absence. The overall deadline and client cancellation still apply.
  • The additional request adds search work and can add latency. Index refresh delays and changes between the main search and lookup can affect visibility because the operations do not share a snapshot.

Date Input Examples

{
  "pubStartDate": "2024-01-01",
  "pubEndDate": "2024-12-31",
  "lastModStartDate": "2024-01-01T01:00:00+01:00"
}

The publication boundaries normalize to 2024-01-01T00:00:00.000Z and 2024-12-31T23:59:59.999Z. The modification boundary normalizes to 2024-01-01T00:00:00.000Z. All boundaries are inclusive.

Deferred Items

  • Item 6: Input-echoing error messages. Changes to severity errors that repeat caller input remain under discussion.
  • Item 7: Response headers. Rate-limit headers, cache policy, X-Powered-By, and operational use of User-Agent information remain under discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant