Skip to content

Reduce per-request allocations in StaticFileMiddleware - #70173

Open
davidfowl wants to merge 6 commits into
mainfrom
perf/staticfiles-request-path
Open

davidfowl wants to merge 6 commits into
mainfrom
perf/staticfiles-request-path

Conversation

@davidfowl

@davidfowl davidfowl commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Reduce per-request allocations in StaticFileMiddleware

  • You've read the Contributor Guide and Code of Conduct.
  • You've included unit or integration tests for your change, where applicable.
  • You've included inline docs for your change, where applicable.
  • There's an open issue for the PR that you are making.

Remove avoidable allocations from the UseStaticFiles request path.

Description

Every request that reaches StaticFileMiddleware calls FileExtensionContentTypeProvider.TryGetContentType. That call allocated an extension substring for the lookup. Each served or revalidated file also paid for a few small allocations in StaticFileContext. This PR removes them. All changes are internal.

  • FileExtensionContentTypeProvider
    • When Mappings is exactly a Dictionary<string, string> whose comparer supports span lookups, the constructor caches an AlternateLookup<ReadOnlySpan<char>>. This covers the default instance (OrdinalIgnoreCase) and user dictionaries with Ordinal or OrdinalIgnoreCase.
    • The extension is then looked up as a span, so no substring is allocated. Every other mapping type, including derived dictionaries, still uses IDictionary.TryGetValue(string).
    • The alternate lookup reads the same dictionary instance, so later changes to Mappings are still seen. The extension rules (IsNullOrWhiteSpace, then last . to the end) are unchanged.
  • StaticFileContext.LookupFileInfo
    • The ETag string is built with a single string.Create(InvariantCulture, $"\"{hash:x}\""), replacing '"' + Convert.ToString(hash, 16) + '"' (two strings).
    • The output is the same lowercase two's-complement hex. I checked this against Convert.ToString for 1M random values plus edge cases, and a new test covers negative hashes.
  • StaticFileContext.ApplyResponseHeadersAsync
    • Writes HeaderUtilities.FormatDate(lastModified) and etag.ToString() straight into Response.Headers, so it no longer allocates a typed ResponseHeaders wrapper.
    • These are exactly the values the typed setters wrote. The typed wrapper is still created lazily for Content-Range.
  • GetMaxPreconditionState: no longer takes a params PreconditionState[], which allocated an array per request.

Headers, status codes and log output are unchanged.

Benchmark

I used a local BenchmarkDotNet harness (not included). It runs StaticFileMiddleware with a PhysicalFileProvider on a DefaultHttpContext. Timings are on 2 cores, 3 runs each.

FileExtensionContentTypeProvider.TryGetContentType (default provider)

Path Before After
/css/site.css 25.6–25.9 ns, 32 B 18.3–19.0 ns, 0 B (−27%)
/lib/bootstrap/dist/js/bootstrap.bundle.min.js 26.2–26.7 ns, 32 B 16.2–16.3 ns, 0 B (−38%)
/unknown.foo 20.2–21.9 ns, 32 B 13.0–13.2 ns, 0 B (−36%)
/api/values (no extension) 3.69–3.72 ns, 0 B 3.68–3.78 ns, 0 B (unchanged)

StaticFileMiddleware.Invoke: time is dominated by the physical file stat, so these show only the allocation drop.

Request Before After
HEAD (200) 2765–2825 ns, 2344 B 2800–2875 ns, 2200 B (−144 B)
GET If-None-Match (304) 3013–3143 ns, 2752 B 3033–3063 ns, 2608 B (−144 B)
GET If-Modified-Since (304) 2960–2995 ns, 2560 B 2966–3047 ns, 2416 B (−144 B)
GET file not found 1502–1567 ns, 1592 B 1530–1533 ns, 1560 B (−32 B)
GET full body ~38 µs, ~20.6 KB ~39 µs, ~20.5 KB (I/O noise)

That is four fewer allocations per served or revalidated file, and one fewer for every request that has an extension.

Tests

  • Microsoft.AspNetCore.StaticFiles.Tests: 315 passed, 51 skipped.
  • Microsoft.AspNetCore.StaticFiles.FunctionalTests: 20 passed, 5 skipped.
  • Microsoft.AspNetCore.StaticAssets.Tests: 64/64.
  • New tests:
    • ApplyResponseHeaders_WritesExpectedETagAndLastModified checks the exact ETag against the old Convert.ToString(hash, 16) formula, including negative hashes, and checks Last-Modified, Accept-Ranges and Content-Type.
    • Content-type provider tests cover a case-sensitive custom mapping, changes to Mappings after construction, a trailing-dot key, a non-Dictionary mapping (SortedDictionary), and a derived dictionary that re-implements TryGetValue.

Benchmark (3 interleaved rounds)

New StaticFileBenchmark in src/Middleware/StaticFiles/perf/Microbenchmarks, added in this PR. It replaces the local harness described above. StaticFileMiddleware serves /css/site.css (1 KB) from an in-memory IFileProvider, so the numbers measure the changed CPU and allocation path without disk I/O.

  • TryGetContentType_Default (span lookup): 20.1–21.1 ns → 12.0–12.1 ns (about −40%), 32 B → 0 B.
  • StaticFileMiddleware.Invoke (Head, Get, GetIfNoneMatch → 304): −144 B per request in every case. Time is within noise: the PR was 2–6% faster in rounds 1–2 and 2–6% slower in round 3.
  • TryGetContentType_NonDictionaryMappings (a SortedDictionary mapping on the unchanged string fallback path): still 32 B. It is about 1 ns slower (12.7–13.2 ns → 13.6–14.9 ns) because of the added _useSpanLookup check. This affects only non-Dictionary custom mappings.
Round Method main Mean PR Mean main Allocated PR Allocated
1 TryGetContentType_Default 20.1 ns 12.0 ns 32 B 0 B
1 TryGetContentType_NonDictionaryMappings 13.2 ns 13.6 ns 32 B 32 B
1 Head 688.3 ns 665.7 ns 1,912 B 1,768 B
1 Get 914.7 ns 883.7 ns 2,000 B 1,856 B
1 GetIfNoneMatch 1.042 us 984.1 ns 2,288 B 2,144 B
2 TryGetContentType_Default 20.1 ns 12.0 ns 32 B 0 B
2 TryGetContentType_NonDictionaryMappings 12.7 ns 14.9 ns 32 B 32 B
2 Head 676.0 ns 656.0 ns 1,912 B 1,768 B
2 Get 911.5 ns 889.4 ns 2,000 B 1,856 B
2 GetIfNoneMatch 1.003 us 949.9 ns 2,288 B 2,144 B
3 TryGetContentType_Default 21.1 ns 12.1 ns 32 B 0 B
3 TryGetContentType_NonDictionaryMappings 12.7 ns 13.8 ns 32 B 32 B
3 Head 659.6 ns 700.8 ns 1,912 B 1,768 B
3 Get 871.4 ns 897.2 ns 2,000 B 1,856 B
3 GetIfNoneMatch 984.2 ns 1.003 us 2,288 B 2,144 B

Machine: AMD EPYC 9V74, 2 cores, Ubuntu 24.04 container; .NET SDK 12.0.100-alpha.1.26521.103, runtime 12.0.0 (12.0.26.52911); BenchmarkDotNet 0.13.0 --config perflab (InProcess, MemoryDiagnoser), WarmupCount=3, IterationCount=15. Order per round: main, then PR. "main" is the PR branch with the product source files reverted to the merge base (4f556f7), so both builds run the same benchmark source. Means and bytes come from the BDN JSON exporter.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

- Look up content types by extension span via Dictionary alternate lookup instead of allocating a substring.
- Build the ETag string with a single allocation.
- Write Last-Modified/ETag directly to the header dictionary instead of allocating ResponseHeaders.
- Avoid the params array when computing the precondition state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 31, 2026 08:00
@github-actions github-actions Bot added the area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlewares label Oct 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the static-file allocation optimizations.

No open issues 🎉

What changed in this PR

Reduces allocations in the static-file request path while preserving content-type lookup behavior and response headers.

Changes:

  • Uses span-based extension lookups for compatible dictionaries, retaining the string-based fallback for other mappings.
  • Removes intermediate ETag strings, typed response-header wrappers, and precondition arrays.
  • Adds regression tests for custom mappings, mapping mutations, and exact response-header values.
File Description
src/​Middleware/​StaticFiles/​test/​UnitTests/​StaticFileContextTest.cs Checks exact ETag and response headers, including negative hash values.
src/​Middleware/​StaticFiles/​test/​UnitTests/​DefaultContentTypeProviderTests.cs Covers mapping compatibility, mutations, and extension edge cases.
src/​Middleware/​StaticFiles/​src/​StaticFileContext.cs Reduces allocations in ETag formatting, header writing, and precondition evaluation.
src/​Middleware/​StaticFiles/​src/​FileExtensionContentTypeProvider.cs Adds allocation-free span lookups for compatible dictionaries.

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Retriggering CI with an empty commit. The failures are unrelated to this change (it only touches StaticFiles content-type lookup and header writes):

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Pushed empty commit a1d008f to retrigger CI. The only failures were in aspnetcore-components-e2e (Blazor E2E, Linux, Mono server):

Neither test depends on content-type mapping or ETag formatting. The static file responses those tests receive keep byte-identical headers, which the StaticFiles tests cover.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

…n PRs)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Pushed empty commit 38d1685 to retrigger CI. Everything except aspnetcore-components-e2e passed. These e2e failures, in the Mono client/WebAssembly and server legs:

  • VirtualizationTest.VariableHeight_HtmlTable_DoesNotProduceNestedTrElements
  • RedirectionTest.NavigationException_InAsyncContext_DoesNotBecomeUnobservedTaskException
  • BlazorWebJsInitializersTest.InitializersRunsModernCallbacksByDefaultWhenPresent (2 cases)

are the same set failing right now on unrelated PRs: #69752 (only touches the RequestDelegateGenerator) fails exactly these four, and #70068 fails the first two. That's 3 of the ~12 recent PRs I sampled. Static file content types are unchanged by this PR. FileExtensionContentTypeProvider returns the same mapping for every extension (covered by the existing provider tests plus the new ones), and .js, .mjs, .wasm and .dll resolve identically.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@PureWeen

PureWeen commented Nov 1, 2026

Copy link
Copy Markdown
Member

/review

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent

Copy link
Copy Markdown
Member Author

On 55c4a17, aspnetcore-ci passed. The only failure is in the aspnetcore-components-e2e Mono server leg (build 1667861), and it isn't caused by this PR.

Failing test: Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.ServerRoutingTest.NavigationLock_CanBlockExternalNavigation. Build Analysis gives it a 31.89% failure rate.
BrowserAssertFailedException: Expected: http://127.0.0.1:41537//myexternalpath3, Actual: http://127.0.0.1:41537/subdir/

Why it's unrelated

I didn't retrigger CI because main E2E is currently flaky. A maintainer can rerun the failed leg if needed.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlewares

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants