Repository navigation
Add immutable-field sorting to /search - #186
Open
afoote-mitre wants to merge 1 commit into
Open
afoote-mitre wants to merge 1 commit into
afoote-mitre wants to merge 1 commit into
Conversation
afoote-mitre
marked this pull request as draft
October 15, 2026 13:06
afoote-mitre
marked this pull request as ready for review
October 15, 2026 13:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add an optional
sortobject toPOST /search, allowing clients to retrieve matching records by CVE ID or publication date in either direction. This makes recently published records accessible without changing the existing default order or result limits.Partially addresses #178. Numeric CVE ID ordering and access beyond the existing result window remain deferred; this PR should not close the issue.
Why These Changes Were Needed
sortobjectcveIdordatePublished, withascordesc. The property is required when the object is supplied; an omitted order defaults to ascending. Unsupported properties, malformed objects, and extra keys return HTTP 400.utils/searchSort.js,utils/constants.js,utils/errors.js,routes/index.js,controllers/middleware.jsutils/searchQueryBuilder.jsdateUpdatedcan change between page requests, moving a record across an offset boundary and causing clients to miss it or receive it twice. Sorting is therefore limited to CVE ID and publication date. Publication-date sorts place missing dates last and break ties with CVE ID ascending.utils/searchSort.js,utils/searchQueryBuilder.jsutils/searchReadiness.jsREADME.md,routes/swagger.js,api-docs/openapi.json,test/postman/buildCollection.js,test/postman/CVE-Search-API.postman_collection.json, associated test filesSupported Sorting Options
sort.propertysort.order: "asc"sort.order: "desc"cveIddatePublishedProperty and order values are case-insensitive. Omitting
sortdefaults tocveIdascending; supplyingsortrequiresproperty, while an omittedorderdefaults toasc. Publication-date sorts place missing dates last in either direction and break ties with CVE ID ascending. Mutable fields such asdateUpdatedare not supported.Defaults and Missing Properties
sortfield{"sort":{"property":"cveId","order":"asc"}}."sort": {"property":"cveId"}orderdefaults toasc."sort": {"property":"datePublished"}orderdefaults toasc. Missing dates remain last and ties use CVE ID ascending."sort": {}SEARCH_SORT:propertyis required when the sort object is supplied."sort": {"order":"desc"}SEARCH_SORT: supplying only a direction does not default the missing property."sort": null, or an explicit null/emptypropertyororderSEARCH_SORT; defaults apply to omitted values, not explicitly invalid values.CVE ID sorting is currently alphabetical/textual, not numeric: in ascending order,
CVE-2024-10000precedesCVE-2024-9999. Publication-date sorting should be used for publication recency rather than inferring recency from CVE sequence numbers.Why Mutable Sort Fields Are Excluded
Separate page requests do not share a snapshot. If a client reads page 1 using
dateUpdateddescending and a record from page 2 is updated before the next request, that record can move onto the already-read page 1 and be missed. Records shifted in the other direction can be returned twice. An immutable CVE ID tie-breaker does not prevent movement when the primary sort field changes.For that reason, this PR accepts only immutable
cveIdanddatePublishedsort fields and rejects mutable fields such asdateUpdated. This reduces sort-key-driven movement but does not eliminate shifts caused by new records, deletions, or changed filter membership. Reconsidering mutable sorting requires a pagination design that preserves a shared snapshot across page requests. The internal, request-scoped PIT traversal in #156 does not provide that cross-request guarantee.Why Numeric CVE ID Sorting Is Deferred
Numeric CVE ID ordering must be applied before OpenSearch pagination and candidate limits to provide globally correct results. Sorting only the records retrieved by the API would reorder that page or bounded candidate subset, but could not account for matching records that OpenSearch did not return.
We are deferring numeric ordering until we have an indexed normalized sort key that orders IDs by numeric year and sequence. Once the key is available, the API can use it for sorting before pagination and candidate selection. Script-based sorting is not being pursued as part of this change because it would add request-time computation and require a separate performance assessment.
Until then,
cveIdcontinues to use textual ordering;datePublishedis available for publication-recency ordering.Example
Retrieve the most recently published critical matches first:
{ "cvssV3_1Severity": "CRITICAL", "sort": { "property": "datePublished", "order": "desc" } }