[apurvabhaleMS]

Description

SQL Server stores and compares only the first 256 characters of URI search values (UriSearchParam.Uri is varchar(256), with no overflow column). URIs that differ only after character 256 match each other in search, and nothing tells the caller. This PR keeps the current storage and search behavior, which is accepted as a known limitation, and makes it visible:

• Log on write. When a URI longer than 256 characters is indexed, the data store logs one warning per resource type and search parameter, with the count and the longest length. The URI value is never logged. This covers both MergeResources and UpdateResourceSearchParams (reindex).
• Warn on search. When a URI search value (equals, :above, :below, or chained) is 256 characters or longer, the search Bundle gets one warning / informational issue:

  URI parameter '_profile' exceeds 256 characters. Evaluation is limited to the first 256 characters, which may return broader search results.

• Reject colliding URI values in one resource (was HTTP 500). If a resource has two different values for the same URI search parameter that share the first 256 characters (for example two meta.profile entries), the index can't store both. Previously the write failed with HTTP 500 (SQL error 2627, duplicate key in @UriSearchParams), and an $import file containing one such resource failed entirely. Now:
  • create, update, conditional and bundle writes return 400 with an OperationOutcome naming the search parameter; in a batch only that entry fails;
  • $import reports each such resource as a per-line error and imports the rest;
  • only the version that becomes current is checked: history versions aren't indexed, so an older version with such values still imports;
  • $bulk-update counts a rejected resource as patch-failed, not updated, adds the message to the job issues, and audits it as a failed item;
  • $bulk-delete with _remove-references fails the page before deleting anything if a referencing resource is rejected, so no dangling references are left;
  • reindex no longer fails the whole job. Resources stored before a URI search parameter existed (for example a new custom search parameter) can have such values. Before, UpdateResourceSearchParams hit the same SQL 2627 and the reindex job failed with the SQL error in failureDetails, leaving the new parameter unusable. Now the data store leaves those values out for that resource and search parameter only, indexes everything else, and returns a warning. The orchestrator reports it as a warning issue, so the job stays Completed and the message shows in the status failureDetails (the same path the existing "no search parameters to reindex" information issue uses). At most 100 warnings are listed per job, followed by a count of the rest. Searches on that parameter don't return the resource.
• Bulk delete / bulk update keep working. CreateBulkDeleteHandler and CreateBulkUpdateHandler return 400 for any bundle issue raised by their validation search. They now skip the new warning, because the URI is valid, and log a warning that the filter can match more resources than intended. The warning is saved in the orchestrator job definition (new optional searchWarnings field, omitted from the JSON when empty) and returned as an information issue on the job status read. It's added after the status logic, so it doesn't change the returned status code.
• Writes (create, update, patch, conditional create/update/delete). When the client sends Prefer: return=OperationOutcome and the request used a URI value that is only partially compared (in the resource being written, or in the condition), the response is an OperationOutcome with the warning, keeping the status code, ETag, Last-Modified and Location headers. Conditional delete returns 200 with the OperationOutcome instead of 204. Without that Prefer header, or when there's nothing to warn about, responses are unchanged. Transaction entries never get it, because a transaction fails when any entry returns an OperationOutcome; batch entries get it in response.outcome. Only single-resource writes report it; import, bulk update and parallel bundles log only, because their request context can't be tied to one resource.
• 412 (condition matched several resources). Conditional create/update/delete still return 412 when the condition matches more than one resource (FHIR rule, unchanged). If the condition used a 256+ character URI, the 412 OperationOutcome now also includes the warning, which explains why it matched more than one.

The warning text and its match check live in Core (LongUriSearchValueIssue), so the handlers don't depend on SQL Server code. Cosmos DB is unaffected because it has no URI length limit.

Interface change: IFhirDataStore.BulkUpdateSearchParameterIndicesAsync now returns Task<IReadOnlyList<string>> (warnings) instead of Task. Cosmos DB returns an empty list. The new ReindexProcessingJobResult.Warnings / OmittedWarningCount fields are left out of the JSON when empty, so job results stay compatible with older versions in both directions.

Before and after

Scenario	Before	After
Search with a URI value of 255 characters or fewer	Results	Same
Search with a URI value of 256+ characters	Prefix matches, no signal	Same results plus a warning issue in the Bundle
Create / update / patch with a URI longer than 256 characters	201 / 200 with the resource	Same; with Prefer: return=OperationOutcome, an OperationOutcome with the warning (not in transactions)
Create a resource with two URIs sharing the first 256 characters	500, not saved	400 with a clear message, not saved
Batch with such an entry	That entry 500	That entry 400
$import of 1,000 resources, 30 with such URIs	Whole import fails (500), nothing imported	970 imported, 30 listed in the error file
Reindex for a new URI search parameter when a stored resource has two values sharing the first 256 characters	Reindex job Failed (SQL 2627 in failureDetails); parameter stays unusable	Job Completed; that resource isn't indexed for that parameter and is named in a warning in failureDetails
Conditional create with a 256+ character URI and Prefer: return=OperationOutcome	OperationOutcome	Same, plus the warning issue
Batch bundle: search entry with a 256+ character URI	Prefix matches	Same, plus the warning in that entry's searchset
$bulk-delete / $bulk-update filtered on a 256+ character URI	202	202, plus a warning in the server log; the status read includes the warning
Conditional update / delete with a 256+ character URI, 1 match	200 / 204	Same; with Prefer: return=OperationOutcome, 200 with an OperationOutcome containing the warning
Conditional create / update / delete with a 256+ character URI, 2+ matches	412	412 (unchanged), and the OperationOutcome also has the warning

Related issues

Addresses AB#210616.

Testing

• Unit (SQL Server): UriSearchParamListRowGeneratorTests covers truncation at 255/256/257/1000 characters, de-duplication of values that differ only past 256, the log entry without the value, skipping history rows, and collision detection (two values sharing the first 256 characters, a 256-character value and a longer one starting with it, identical values, values that differ earlier, short values, different search parameters). LongUriSearchValueFinderTests covers the boundary (255 vs. 256), :below, non-URI fields, nested/union/chained expressions and de-duplication. SqlServerSearchServiceLongUriWarningTests covers adding the warning once per request, never including the value, and skipping when there is no request context.

• Unit (Core): LongUriSearchValueIssueTests. CreateBulkDeleteHandlerTests and CreateBulkUpdateHandlerTests check that the job is still created with the warning saved in its definition, and that a real invalid parameter still returns 400. GetBulkDeleteHandlerTests and GetBulkUpdateHandlerTests check that the status includes the warning, and that a failed job still returns 500 with its error. The existing bulk orchestrator and processing job tests pass unchanged (150 tests).

• Unit (API): FhirResultExtensionsTests checks that additional issues are added only to an OperationOutcome response.

• E2E (SQL Server, new UriLengthSearchTests): uses _profile, so the same tests run on every FHIR version:

  • 256+ and exactly-256 search values return both resources that share the prefix, plus the warning;
  • a short value returns only its resource and no warning;
  • a resource with two colliding URIs returns 400 with the message and isn't stored; in a batch, only that entry returns 400;
  • $bulk-delete with a 256+ character _profile filter returns 202, deletes the matching resource, and the status read includes the warning;
  • conditional create with Prefer: return=OperationOutcome returns the warning;
  • a batch with a search entry and a conditional create entry returns the warning in both entries;
  • create with Prefer: return=OperationOutcome returns 201 with the warning and a Location header, and without it returns the resource;
  • conditional update and conditional delete matching one resource return the warning and update/delete it;
  • conditional update matching two resources by prefix returns 412 with the error and the warning;
  • a transaction creating a resource with a long URI and Prefer: return=OperationOutcome still succeeds, with no entry outcome.

  All 13 pass locally on Stu3, R4, R4B and R5. Unit tests cover the 412 filter (warning added once, not added to other errors) and FhirResult without an information message. R4 SQL create/update/conditional/patch/delete/batch/transaction E2E classes pass (the unauthorized-transaction test also fails on main).

• E2E (import, new tests in ImportTests): a file with 3 patients, one with colliding profiles, imports 2 and reports the third in the error file; a resource whose colliding values are only in a history version imports both versions with no errors.

• Unit (Core): BulkUpdateServiceTests checks that a resource rejected by the data store is counted as patch-failed, not updated. DeletionServiceTests checks that a rejected referencing resource stops a hard delete with _remove-references before anything is deleted (the test fails without the fix).

• Unit (reindex): UriSearchParamListRowGeneratorTests checks that only the colliding parameter's values are removed from the resource (other URI and token values stay), that the warning names the resource and parameter but not the URI, and that resources without collisions are untouched. ReindexProcessingJobTests checks that data-store warnings end up in a completed job result, the cap of 100, and that older results without warnings still deserialize. ReindexOrchestratorJobTests checks that warnings become warning issues plus an omitted-count summary, without failing the job.

• Manual reindex, local R4 server from main and from this branch: stored a Patient with two extension URIs sharing the first 256 characters, then added a custom uri search parameter on that extension and ran $reindex. On main the job ended Failed ("Violation of PRIMARY KEY constraint … @UriSearchParams … error 2627"), and the new parameter stayed "not supported". On this branch the job ended Completed, 2000/2000 resources, with failureDetails: "Resource Patient/… was not indexed for the URI search parameter 'ext-uri' because it has more than one value for it that are identical in their first 256 characters. Searches using 'ext-uri' will not return this resource." The parameter then worked for the other resources.

• Manual, against a local R4 server built from main and from this branch: a single create returned 500 before and 400 after. A batch of 10 creates (3 colliding) returned 500 for the 3 entries before and 400 after. An $import of 1,000 patients (30 colliding) failed entirely before (500, 0 imported); after, 970 were imported and the 30 were listed in the error file.

• The whole Microsoft.Health.Fhir.SqlServer.UnitTests suite passes, except CustomQueriesUnitTests.GivenCustomQueryClass_WithGivenWaitTime_QueryToDBWillOccurOnlyAfterWaitPeriod. That test fails only when run together with the other test in its class, passes on its own, and doesn't touch this change.

FHIR Team Checklist

• Labels: Bug, Azure Healthcare APIs, Schema Version unchanged, No-PaaS-breaking-change, No-ADR.
• CI is green before merge
• Review squash-merge requirements

Semver Change (docs)

Patch

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.69%. Comparing base (db8c37f) to head (9b9c171).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6013      +/-   ##
==========================================
+ Coverage   80.54%   80.69%   +0.14%     
==========================================
  Files         988      990       +2     
  Lines       38028    38193     +165     
  Branches     6061     6121      +60     
==========================================
+ Hits        30630    30818     +188     
+ Misses       5931     5906      -25     
- Partials     1467     1469       +2     

see 24 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

🟡 Changes recommended

Long-URI warnings can be duplicated across export checkpoints or emitted multiple times when searches encounter different affected parameters.

2 open issues

• Deduplicate persisted export warnings across checkpoints · New
• Merge long-URI warnings across searches · New

1 file not reviewed

• src/Microsoft.Health.Fhir.Core/Resources.Designer.cs: Generated file

What changed in this PR

Makes SQL Server’s 256-character URI search limit visible without changing its matching behavior, and prevents duplicate-key failures when URI values share the indexed prefix. Cosmos DB behavior remains unchanged.

Changes:

• Logs oversized URI values without exposing their contents and adds search Bundle warnings.
• Truncates URI index rows before deduplication and allows bulk operations to proceed with the new warning.
• Adds unit and SQL-backed end-to-end coverage for boundaries, collisions, warnings, and bulk-delete acceptance.

File	Description
test/​Microsoft.Health.Fhir.Shared.Tests.E2E/​Rest/​Search/​UriLengthSearchTests.cs	Tests prefix matches, warnings, collisions, and bulk delete.
test/​Microsoft.Health.Fhir.Shared.Tests.E2E/​Rest/​Search/​UriLengthSearchTestFixture.cs	Creates URI boundary and collision test data.
test/​Microsoft.Health.Fhir.Shared.Tests.E2E/​Microsoft.Health.Fhir.Shared.Tests.E2E.projitems	Registers the shared E2E tests.
src/​Microsoft.Health.Fhir.SqlServer/​Features/​Storage/​TvpRowGeneration/​Merge/​UriSearchParamListRowGenerator.cs	Truncates before deduplication and summarizes oversized values.
src/​Microsoft.Health.Fhir.SqlServer/​Features/​Storage/​SqlServerFhirDataStore.cs	Logs URI truncation during writes and reindexing.
src/​Microsoft.Health.Fhir.SqlServer/​Features/​Search/​SqlServerSearchService.cs	Adds long-URI search warnings.
src/​Microsoft.Health.Fhir.SqlServer/​Features/​Search/​Expressions/​Visitors/​LongUriSearchValueFinder.cs	Finds affected URI parameters in search expressions.
src/​Microsoft.Health.Fhir.SqlServer.UnitTests/​Features/​Storage/​TvpRowGeneration/​UriSearchParamListRowGeneratorTests.cs	Tests truncation, deduplication, and value-free logging.
src/​Microsoft.Health.Fhir.SqlServer.UnitTests/​Features/​Search/​SqlServerSearchServiceLongUriWarningTests.cs	Tests warning creation and duplicate suppression.
src/​Microsoft.Health.Fhir.SqlServer.UnitTests/​Features/​Search/​Expressions/​Visitors/​LongUriSearchValueFinderTests.cs	Tests boundaries and nested expression traversal.
src/​Microsoft.Health.Fhir.Core/​Resources.resx	Defines the warning message.
src/​Microsoft.Health.Fhir.Core/​Resources.Designer.cs	Exposes the warning resource accessor.
src/​Microsoft.Health.Fhir.Core/​Features/​Search/​LongUriSearchValueIssue.cs	Centralizes warning creation and recognition.
src/​Microsoft.Health.Fhir.Core/​Features/​Operations/​BulkUpdate/​Handlers/​CreateBulkUpdateHandler.cs	Accepts the warning while logging bulk-update risk.
src/​Microsoft.Health.Fhir.Core/​Features/​Operations/​BulkDelete/​Handlers/​CreateBulkDeleteHandler.cs	Accepts the warning while logging bulk-delete risk.
src/​Microsoft.Health.Fhir.Core.UnitTests/​Features/​Search/​LongUriSearchValueIssueTests.cs	Tests warning formatting and recognition.
src/​Microsoft.Health.Fhir.Core.UnitTests/​Features/​Operations/​BulkUpdate/​CreateBulkUpdateHandlerTests.cs	Tests job creation with the warning present.
src/​Microsoft.Health.Fhir.Core.UnitTests/​Features/​Operations/​BulkDelete/​CreateBulkDeleteHandlerTests.cs	Tests warning acceptance and invalid-parameter rejection.

_{Review effort: Balanced}