Add immutable-field sorting to /search

README.md

@@ -117,7 +117,7 @@ The Search API uses `PORT`, defaulting to `http://localhost:3000`. The tracked P
 
 An AWS load balancer can use `/api/ready-check` to stop routing search traffic to an unready task while the container orchestrator continues using `/api/health-check` for process liveness. Coordinate that probe selection with the infrastructure team before changing deployed health-check behavior.
 
-When the configured index is an alias spanning multiple indices, every reported mapped type of each required field must be compatible and searchable. CVE ID sorting also requires aggregation support. One compatible type cannot conceal a conflicting type in another index; readiness returns HTTP 503 rather than advertising that configuration as ready.
+When the configured index is an alias spanning multiple indices, every reported mapped type of each required field must be compatible and searchable. CVE ID and publication-date sorting also require aggregation support. One compatible type cannot conceal a conflicting type in another index; readiness returns HTTP 503 rather than advertising that configuration as ready.
 
 ## API Input Validation
 
@@ -145,7 +145,7 @@ Scope applies to filter matching as follows:
 | `cweId`, all CVSS severity and metric filters                                    | Problem types or metrics in the selected CNA and/or ADP containers                    |
 | All SSVC filters, `knownExploited`, KEV date filters, `referenceURL`, `disputed` | Structured metrics, references, or tags in the selected CNA and/or ADP containers     |
 | `cpeName`, `isVulnerable`, `virtualMatchString`, and version range filters       | CNA CPE applicability data; incompatible with `ADP_ONLY`                              |
-| `resultsPerPage`, `pageNumber`                                                   | Response controls; they do not select record data                                     |
+| `resultsPerPage`, `pageNumber`, `sort`                                           | Response controls; they do not select record data                                     |
 
 Missing data in selected containers does not make an otherwise compatible request invalid. A positive filter cannot match a record that lacks the requested data. Exclusion filters behave differently: `knownExploited: false` and `disputed: false` can match records with no selected KEV metric or disputed tag, including records without a selected container. Other supplied filters must still match. The API does not remove unselected containers from a returned record.
 
@@ -356,7 +356,20 @@ CVSS vector filters, SSVC filters, date-bounded positive `knownExploited` search
 
 The reported total and pagination are complete only within that bounded candidate set; qualifying records beyond the first 10,000 search candidates may not be represented. When a semantic search reaches that limit, the response sets `pagination.totalIsExact` to `false` and includes a `SEMANTIC_CANDIDATE_LIMIT_REACHED` warning. Semantic searches that remain below the limit and direct searches set `totalIsExact` to `true`. This preserves the existing bounded post-filter behavior until PIT-based pagination is implemented. A `knownExploited` filter without date boundaries uses a direct query and retains ordinary exact-total behavior.
 
-`/search` orders search results by CVE ID so repeated shallow-page requests use a deterministic sort. The response `pagination.pageSize` preserves the requested `resultsPerPage` value, while `pagination.resultsReturned` reports the number of records actually returned and may be smaller for a partial or out-of-range page. Current searches do not use a point-in-time snapshot, so index updates between separate page requests can shift results. Request-scoped PIT traversal would improve consistency within one request, but consistency across separate page requests would additionally require reusing the same snapshot.
+`/search` defaults to CVE ID ascending, using textual rather than numeric sequence ordering. For example, `CVE-2024-10000` sorts before `CVE-2024-9999`. Use the optional `sort` object to select either immutable `cveId` or `datePublished`, with `asc` or `desc`. The `property` is required when `sort` is supplied; omitted `order` defaults to `asc`. Values are case-insensitive. Publication-date sorts place missing dates last and use CVE ID ascending to break ties. Mutable `dateUpdated` sorting is not supported. Sort fields use base record metadata independently of `scope`.
+
+For the most recently published matching records first:
+
+```json
+{
+    "cvssV3_1Severity": "CRITICAL",
+    "sort": { "property": "datePublished", "order": "desc" }
+}
+```
+
+The selected order applies to direct searches and to candidate retrieval before semantic filtering. Semantic filtering preserves that order, and the full-record page fetch restores the selected CVE ID order. Changing the sort can therefore change which candidates fit within the semantic limit and the reported partial total; a descending sort does not remove either 10,000-record control. Narrowing filters, such as publication date ranges, can help divide a larger result set into accessible windows.
+
+The response `pagination.pageSize` preserves the requested `resultsPerPage` value, while `pagination.resultsReturned` reports the number of records actually returned and may be smaller for a partial or out-of-range page. Immutable sort values prevent records from moving because a sort key changed, but new records, deletions, or changed filter membership can still shift pages. Current searches do not use a point-in-time snapshot. Request-scoped PIT traversal would improve consistency within one request, but consistency across separate page requests would additionally require reusing the same snapshot.
 
 The API does not cache, coalesce, queue, rate-limit, or aggregate performance telemetry in process. Each request executes its own required OpenSearch operations. Public rate enforcement, admission control, shared caching, and durable metrics collection must be supplied by the deployment infrastructure.
 

api-docs/openapi.json

@@ -133,6 +133,10 @@
                   "summary": "All supported request body parameters",
                   "value": {
                     "scope": "FULL",
+                    "sort": {
+                      "property": "datePublished",
+                      "order": "desc"
+                    },
                     "keywordSearch": "remote code execution",
                     "keywordExactMatch": true,
                     "cveIds": "CVE-2024-1234,CVE-2024-5678",
@@ -659,6 +663,39 @@
             "default": "FULL",
             "description": "Controls which containers participate in filter matching. FULL searches base record fields, CNA data, and all ADP providers. CNA_ONLY searches base record fields and CNA data. ADP_ONLY searches base record fields and all ADP providers. Scope does not remove data from returned records. Values are case-insensitive. CPE applicability filters are not available with ADP_ONLY."
           },
+          "sort": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": [
+              "property"
+            ],
+            "properties": {
+              "property": {
+                "type": "string",
+                "enum": [
+                  "cveId",
+                  "datePublished"
+                ]
+              },
+              "order": {
+                "type": "string",
+                "enum": [
+                  "asc",
+                  "desc"
+                ],
+                "default": "asc"
+              }
+            },
+            "default": {
+              "property": "cveId",
+              "order": "asc"
+            },
+            "example": {
+              "property": "datePublished",
+              "order": "desc"
+            },
+            "description": "Orders matching records using immutable base-record fields, independently of scope. Defaults to CVE ID ascending when omitted. property is required and accepts cveId (textual, not numeric sequence ordering) or datePublished; order accepts asc or desc and defaults to asc. Values are case-insensitive. Publication-date sorts place missing dates last and use CVE ID ascending to break ties. The selected sort is applied before the 10,000-candidate limit for semantic searches, so changing it can change the bounded candidate set and reported partial total. Both 10,000-record controls still apply. New records, deletions, or changed filter membership can shift pages between requests; this option does not provide a shared snapshot. Mutable dateUpdated sorting is unsupported."
+          },
           "keywordSearch": {
             "type": "string",
             "maxLength": 512,

api-docs/openapi.test.js

@@ -94,6 +94,13 @@ describe('OpenAPI contract', () => {
         expect(schema.properties.isVulnerable.oneOf).toStrictEqual(schema.properties.noRejected.oneOf);
         expect(schema.properties.scope.enum).toStrictEqual(['FULL', 'CNA_ONLY', 'ADP_ONLY']);
         expect(schema.properties.scope.default).toBe('FULL');
+        expect(schema.properties.sort.type).toBe('object');
+        expect(schema.properties.sort.required).toStrictEqual(['property']);
+        expect(schema.properties.sort.additionalProperties).toBe(false);
+        expect(schema.properties.sort.properties.property.enum).toStrictEqual(['cveId', 'datePublished']);
+        expect(schema.properties.sort.properties.order.enum).toStrictEqual(['asc', 'desc']);
+        expect(schema.properties.sort.default).toStrictEqual({ property: 'cveId', order: 'asc' });
+        expect(schema.properties.sort.description).toContain('before the 10,000-candidate limit');
         expect(schema.properties.scope.description).toContain('does not remove data');
         expect(schema.properties.ssvcExploitation.enum).toStrictEqual(['NONE', 'POC', 'ACTIVE']);
         expect(schema.properties.ssvcAutomatable.enum).toStrictEqual(['YES', 'NO']);

controllers/middleware.js

@@ -3,6 +3,7 @@ const constants = require('../utils/constants');
 const cpe = require('../utils/cpe');
 const searchScope = require('../utils/searchScope');
 const ssvc = require('../utils/ssvc');
+const searchSort = require('../utils/searchSort');
 const error = new errors.errors();
 
 const cvssV2SeverityEnum = ['LOW', 'MEDIUM', 'HIGH'];
@@ -65,6 +66,7 @@ const versionRangeParams = ['versionStart', 'versionStartType', 'versionEnd', 'v
 const versionRangeValueParams = ['versionStart', 'versionEnd'];
 const versionRangeTypeParams = ['versionStartType', 'versionEndType'];
 const nonStringSearchParams = [
+    'sort',
     'keywordExactMatch',
     'cveIds',
     'isVulnerable',
@@ -180,6 +182,10 @@ function validateParams(req, res, next) {
             }
         }
 
+        if (!searchSort.normalizeSort(params.sort)) {
+            return res.status(400).json(error.searchSortError());
+        }
+
         if (!searchScope.isValidScope(params.scope)) {
             return res.status(400).json(error.scopeValueError());
         }

controllers/middleware.test.js

@@ -447,6 +447,7 @@ describe('middleware validation function', () => {
 
     test.each([
         ['keywordSearch', 'remote code'],
+        ['sort', { property: 'DATEPUBLISHED', order: 'DESC' }],
         ['keywordExactMatch', 'false'],
         ['cveIds', 'CVE-2024-0001,CVE-2024-0002'],
         ['noRejected', true],

routes/index.js

@@ -8,12 +8,14 @@ const featureFlags = require('../utils/featureFlags');
 const searchDeadline = require('../utils/searchDeadline');
 const searchProvider = require('../utils/searchProvider');
 const searchScope = require('../utils/searchScope');
+const searchSort = require('../utils/searchSort');
 
 const error = new errors.errors();
 const SEARCH_BODY_FIELDS = new Set(constants.SEARCH_QUERY_PARAMS);
 const SEARCH_STRING_BODY_FIELDS = constants.SEARCH_QUERY_PARAMS.filter(
     (field) =>
         ![
+            'sort',
             'keywordExactMatch',
             'cveIds',
             'isVulnerable',
@@ -182,6 +184,11 @@ function normalizeSearchParams(req, res, next) {
         if (typeof searchParams[param] === 'string') searchParams[param] = searchParams[param].trim().toLowerCase();
     });
 
+    if (searchParams.sort !== undefined) {
+        const sort = searchSort.normalizeSort(searchParams.sort);
+        if (sort) searchParams.sort = sort;
+    }
+
     req.searchParams = searchParams;
     next();
 }
@@ -211,6 +218,7 @@ router.post(
                     summary: "All supported request body parameters",
                     value: {
                         scope: "FULL",
+                        sort: { property: "datePublished", order: "desc" },
                         keywordSearch: "remote code execution",
                         keywordExactMatch: true,
                         cveIds: "CVE-2024-1234,CVE-2024-5678",

routes/index.test.js

@@ -106,6 +106,20 @@ describe('routes/index router', () => {
         });
     });
 
+    test('POST /search normalizes a sort object without mutating its JSON body', async () => {
+        const response = await request(app, {
+            method: 'POST',
+            path: '/search',
+            body: { sort: { property: ' DATEPUBLISHED ', order: ' DESC ' } }
+        });
+        expect(response.status).toBe(200);
+        expect(response.body.params.sort).toStrictEqual({ property: 'datePublished', order: 'desc' });
+        expect(mw.validateParams.mock.calls[0][0].body.sort).toStrictEqual({
+            property: ' DATEPUBLISHED ',
+            order: ' DESC '
+        });
+    });
+
     test('POST /search permits an omitted request body', async () => {
         const response = await request(app, {
             method: 'POST',

routes/search.integration.test.js

@@ -28,6 +28,126 @@ describe('search route integration', () => {
         mockSearchOpenSearch.mockReset();
     });
 
+    test.each(
+        [
+            null,
+            [],
+            'datePublished',
+            false,
+            1,
+            {},
+            { order: 'desc' },
+            { property: null },
+            { property: [] },
+            { property: '' },
+            { property: 'dateUpdated' },
+            { property: 'relevancy' },
+            { property: '_script' },
+            { property: 'cveMetadata.cveId.keyword' },
+            { property: 'cveId', order: null },
+            { property: 'cveId', order: [] },
+            { property: 'cveId', order: false },
+            { property: 'cveId', order: '' },
+            { property: 'cveId', order: 'newest' },
+            { property: 'cveId', extra: true }
+        ].map((value) => [value])
+    )('rejects malformed or unsupported /search sort %j before OpenSearch', async (sort) => {
+        const response = await request(createApp(), { method: 'POST', path: '/search', body: { sort } });
+        expect(response.status).toBe(400);
+        expect(response.body.error).toBe('SEARCH_SORT');
+        expect(mockSearchOpenSearch).not.toHaveBeenCalled();
+    });
+
+    test.each([
+        ['cveId', 'asc'],
+        ['cveId', 'desc'],
+        ['datePublished', 'asc'],
+        ['datePublished', 'desc']
+    ])('applies direct %s sorting in %s order with pagination', async (property, order) => {
+        mockSearchOpenSearch.mockResolvedValue(advancedResponse(0).data);
+        const response = await request(createApp(), {
+            method: 'POST',
+            path: '/search',
+            body: {
+                scope: 'ADP_ONLY',
+                sort: { property: property.toUpperCase(), order: order.toUpperCase() },
+                resultsPerPage: 2,
+                pageNumber: 2
+            }
+        });
+        expect(response.status).toBe(200);
+        const expectedSort =
+            property === 'cveId'
+                ? [{ 'cveMetadata.cveId.keyword': { order } }]
+                : [
+                      { 'cveMetadata.datePublished': { order, missing: '_last' } },
+                      { 'cveMetadata.cveId.keyword': { order: 'asc' } }
+                  ];
+        expect(mockSearchOpenSearch.mock.calls[0][0]).toMatchObject({ size: 2, from: 2, sort: expectedSort });
+    });
+
+    test.each(['cveId', 'datePublished'])(
+        'preserves %s candidate order through semantic filtering and full record fetches',
+        async (property) => {
+            const ids =
+                property === 'datePublished' ? ['CVE-2020-0002', 'CVE-2024-0001'] : ['CVE-2024-0001', 'CVE-2020-0002'];
+            const hit = (id, version) => ({
+                _id: id,
+                _source: {
+                    cveMetadata: { cveId: id },
+                    containers: {
+                        cna: {
+                            cpeApplicability: [
+                                {
+                                    nodes: [
+                                        {
+                                            cpeMatch: [
+                                                {
+                                                    vulnerable: true,
+                                                    criteria: `cpe:2.3:a:vendor:product:${version}:*:*:*:*:*:*:*`
+                                                }
+                                            ]
+                                        }
+                                    ]
+                                }
+                            ]
+                        }
+                    }
+                }
+            });
+            const matches = ids.map((id) => hit(id, '1.0'));
+            mockSearchOpenSearch
+                .mockResolvedValueOnce(advancedResponse(3, [matches[0], hit('CVE-2023-0003', '2.0'), matches[1]]).data)
+                .mockResolvedValueOnce(advancedResponse(2, [...matches].reverse()).data);
+            const response = await request(createApp(), {
+                method: 'POST',
+                path: '/search',
+                body: {
+                    cpeName: 'cpe:2.3:a:vendor:product:1.0:*:*:*:*:*:*:*',
+                    sort: { property, order: 'desc' },
+                    resultsPerPage: 2
+                }
+            });
+            expect(response.status).toBe(200);
+            expect(response.body.records.map((record) => record.cveMetadata.cveId)).toStrictEqual(ids);
+            expect(response.body.pagination.totalResults).toBe(2);
+            const candidate = mockSearchOpenSearch.mock.calls[0][0];
+            expect(candidate).toMatchObject({ size: constants.MAX_SEARCH_WINDOW, from: 0 });
+            expect(candidate.sort).toStrictEqual(
+                property === 'cveId'
+                    ? [{ 'cveMetadata.cveId.keyword': { order: 'desc' } }]
+                    : [
+                          { 'cveMetadata.datePublished': { order: 'desc', missing: '_last' } },
+                          { 'cveMetadata.cveId.keyword': { order: 'asc' } }
+                      ]
+            );
+            expect(candidate._source).not.toContain('cveMetadata.datePublished');
+            expect(mockSearchOpenSearch.mock.calls[1][0].query.bool.filter).toStrictEqual([
+                { terms: { 'cveMetadata.cveId.keyword': ids } }
+            ]);
+        }
+    );
+
     test('POST /search validates, builds the OpenSearch request, and formats pagination metadata', async () => {
         const record = {
             dataType: 'CVE_RECORD',