[davidfowl]

Avoid per-line MemoryStream in BufferedReadStream.ReadLine

• You've read the Contributor Guide and Code of Conduct.
• You've included unit or integration tests for your change, where applicable.
• You've included inline docs for your change, where applicable.
• There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.

Read buffered lines in place and trim multipart headers on spans

Description

BufferedReadStream.ReadLine/ReadLineAsync allocated a MemoryStream (plus its 200-byte buffer) for every line and copied the line into it one byte at a time. MultipartReader calls it for every boundary remainder, every section header and the blank line that ends the headers, so that is roughly four MemoryStreams per section.

This PR adds a fast path. When the terminating CRLF is already in the internal buffer, it is found with a vectorized IndexOf("\r\n"u8) and the line is decoded directly from the buffer. If there is no CRLF in the buffer, or the line would exceed lengthLimit, the code falls back to the existing loop. That means lines spanning buffers, unterminated lines and the Line length limit N exceeded. exception behave exactly as before.

MultipartReader.ReadHeadersAsync also used Substring(...).Trim(), which allocates a second string when the value has a leading space (the common Name: value shape). It now uses AsSpan(...).Trim().ToString().

No public API changes.

Benchmarks

Existing MultipartReaderBenchmark (src/Http/WebUtilities/perf/Microbenchmarks), run with the repo runner (--config perflab, InProcess, MemoryDiagnoser) on a 2-vCPU Linux VM. Base and PR binaries were run back to back for two rounds; the second round is shown here (round 1 was within noise of it). LargePayload=False:

Method	Boundary / Sections	Before	After	Alloc before → after
MultipartReaderParsing	6 / 1	1.613 μs	1.359 μs	7 → 5 KB
MultipartReaderParsing	6 / 2	2.951 μs	2.331 μs	9 → 7 KB
MultipartReaderParsing	6 / 3	3.877 μs	2.857 μs	12 → 8 KB
MultipartReaderParsing	28 / 3	3.935 μs	2.907 μs	12 → 8 KB
MultipartReaderParsing	70 / 3	4.404 μs	3.077 μs	12 → 8 KB
MultipartReaderParsingWithRead	6 / 3	3.749 μs	2.860 μs	11 → 8 KB
MultipartReaderParsingWithRead	70 / 3	3.834 μs	2.759 μs	12 → 8 KB

The LargePayload=True cases are dominated by the 10 MB body and are unchanged within noise.

Tests

• Added BufferedReadStreamTests covering multiple lines in one buffer (including a lone \r and a lone \n inside a line), empty lines, data left over after a line, the exact length-limit boundary for single-buffer lines (ASCII and multi-byte UTF-8), and a CRLF split across two buffer fills. I checked that the new tests also pass against the original implementation.
• Microsoft.AspNetCore.WebUtilities.Tests: 515 passed, 0 failed.

There is no existing issue; this came out of a perf survey of the request parsing paths.

Benchmark (3 interleaved rounds)

Existing MultipartReaderBenchmark.MultipartReaderParsing (src/Http/WebUtilities/perf/Microbenchmarks), all 18 parameter combinations.

Environment: AMD EPYC 9V74, 2 vCPU (container), Ubuntu 24.04; .NET SDK 12.0.100-alpha.1.26521.103, .NET 12.0.0 (12.0.26.52911); BenchmarkDotNet v0.13.0 --config perflab (InProcess, MemoryDiagnoser), WarmupCount=3, IterationCount=10. "main" is the merge base 4f556f73a6. Both builds use the same benchmark source, and runs were interleaved main, PR, main, PR, main, PR. Each cell shows mean / allocated per operation; allocation figures are exact bytes from the CSV exporter.

Case	main r1	PR r1	main r2	PR r2	main r3	PR r3	Δ time (median)	Alloc main → PR
Boundary=6, Sections=1, Large=False	1,303.0 ns / 6,760 B	1,051.0 ns / 5,536 B	1,153.0 ns / 6,760 B	1,006.0 ns / 5,536 B	1,138.0 ns / 6,760 B	1,205.0 ns / 5,536 B	-8.8%	6,760 B → 5,536 B
Boundary=6, Sections=1, Large=True	997.55 μs / 6,765 B	975.75 μs / 5,546 B	959.44 μs / 6,765 B	950.19 μs / 5,546 B	1,003.28 μs / 6,765 B	955.33 μs / 5,541 B	-4.2%	6,765 B → 5,546 B
Boundary=6, Sections=2, Large=False	2,674.0 ns / 9,352 B	1,694.0 ns / 6,816 B	2,086.0 ns / 9,352 B	1,706.0 ns / 6,816 B	2,177.0 ns / 9,352 B	1,595.0 ns / 6,816 B	-22.2%	9,352 B → 6,816 B
Boundary=6, Sections=2, Large=True	995.42 μs / 9,352 B	963.76 μs / 6,826 B	979.43 μs / 9,357 B	962.74 μs / 6,821 B	1,000.42 μs / 9,372 B	1,049.15 μs / 6,819 B	-3.2%	9,357 B → 6,821 B
Boundary=6, Sections=3, Large=False	2,869.0 ns / 11,944 B	2,391.0 ns / 8,096 B	3,011.0 ns / 11,944 B	2,136.0 ns / 8,096 B	2,683.0 ns / 11,944 B	2,282.0 ns / 8,096 B	-20.5%	11,944 B → 8,096 B
Boundary=6, Sections=3, Large=True	1,018.54 μs / 11,944 B	1,022.08 μs / 8,097 B	996.60 μs / 11,953 B	1,029.45 μs / 8,106 B	1,001.41 μs / 11,954 B	1,060.60 μs / 8,105 B	+2.8%	11,953 B → 8,105 B
Boundary=28, Sections=1, Large=False	1,190.0 ns / 6,816 B	1,106.0 ns / 5,592 B	1,196.0 ns / 6,816 B	1,065.0 ns / 5,592 B	1,179.0 ns / 6,816 B	1,102.0 ns / 5,592 B	-7.4%	6,816 B → 5,592 B
Boundary=28, Sections=1, Large=True	1,049.74 μs / 6,826 B	1,069.67 μs / 5,602 B	1,047.65 μs / 6,826 B	1,054.50 μs / 5,592 B	1,111.75 μs / 6,826 B	1,049.67 μs / 5,592 B	+0.5%	6,826 B → 5,592 B
Boundary=28, Sections=2, Large=False	2,237.0 ns / 9,408 B	1,708.0 ns / 6,872 B	2,125.0 ns / 9,408 B	1,611.0 ns / 6,872 B	2,158.0 ns / 9,408 B	1,635.0 ns / 6,872 B	-24.2%	9,408 B → 6,872 B
Boundary=28, Sections=2, Large=True	1,116.96 μs / 9,418 B	1,063.79 μs / 6,881 B	1,054.69 μs / 9,418 B	1,040.34 μs / 6,882 B	1,039.53 μs / 9,418 B	1,050.73 μs / 6,882 B	-0.4%	9,418 B → 6,882 B
Boundary=28, Sections=3, Large=False	2,926.0 ns / 12,000 B	2,598.0 ns / 8,152 B	3,061.0 ns / 12,000 B	2,334.0 ns / 8,152 B	2,853.0 ns / 12,000 B	2,429.0 ns / 8,152 B	-17.0%	12,000 B → 8,152 B
Boundary=28, Sections=3, Large=True	1,067.95 μs / 12,010 B	1,191.47 μs / 8,152 B	1,081.56 μs / 12,010 B	1,192.64 μs / 8,162 B	1,113.62 μs / 12,000 B	1,195.14 μs / 8,161 B	+10.3%	12,010 B → 8,161 B
Boundary=70, Sections=1, Large=False	1,274.0 ns / 6,952 B	1,123.0 ns / 5,728 B	1,291.0 ns / 6,952 B	1,052.0 ns / 5,728 B	1,274.0 ns / 6,952 B	1,086.0 ns / 5,728 B	-14.8%	6,952 B → 5,728 B
Boundary=70, Sections=1, Large=True	1,244.56 μs / 6,952 B	1,213.04 μs / 5,738 B	1,260.17 μs / 6,962 B	1,210.13 μs / 5,738 B	1,196.11 μs / 6,962 B	1,245.10 μs / 5,738 B	-2.5%	6,962 B → 5,738 B
Boundary=70, Sections=2, Large=False	2,274.0 ns / 9,544 B	1,689.0 ns / 7,008 B	2,090.0 ns / 9,544 B	1,653.0 ns / 7,008 B	2,155.0 ns / 9,544 B	2,007.0 ns / 7,008 B	-21.6%	9,544 B → 7,008 B
Boundary=70, Sections=2, Large=True	1,203.81 μs / 9,564 B	1,299.75 μs / 7,015 B	1,195.70 μs / 9,554 B	1,327.83 μs / 7,018 B	1,246.92 μs / 9,554 B	1,197.85 μs / 7,028 B	+8.0%	9,554 B → 7,018 B
Boundary=70, Sections=3, Large=False	2,865.0 ns / 12,136 B	2,401.0 ns / 8,288 B	3,207.0 ns / 12,136 B	2,185.0 ns / 8,288 B	2,864.0 ns / 12,136 B	2,286.0 ns / 8,288 B	-20.2%	12,136 B → 8,288 B
Boundary=70, Sections=3, Large=True	1,264.59 μs / 12,136 B	1,258.43 μs / 8,288 B	1,259.29 μs / 12,139 B	1,212.52 μs / 8,298 B	1,249.68 μs / 12,146 B	1,217.57 μs / 8,297 B	-3.3%	12,139 B → 8,297 B

• LargePayload=False (header and boundary line parsing dominates): the PR is faster in all 9 cases (−7% to −24% median) and allocates 1.2–3.8 KB less per parse (−18% to −32%), consistently in every round.
• LargePayload=True (a 10 MB section body dominates, and the changed line-reading code is a negligible share of the time): time is mixed, between −4% and +10%. Allocations still drop by the same 1.2–3.8 KB. The +10% case (Boundary=28, Sections=3) doesn't follow the boundary/section pattern of the neighboring cases. That suggests code layout or JIT noise rather than this change, but I haven't proven it.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[Copilot]

🟡 Changes recommended

The split-CRLF test may buffer the entire input because ArrayPool can return more than four bytes, allowing regressions to go undetected.

1 open issue

• Test may not exercise split-CRLF buffering due to oversized pooled array · New

What changed in this PR

Reduces allocations in WebUtilities multipart parsing by decoding fully buffered lines directly and trimming header values before allocating strings. No public API changes.

Changes:

• Adds a shared fast path for synchronous and asynchronous line reads, retaining the existing fallback.
• Uses span-based trimming for multipart header values.
• Adds tests for buffered lines, remaining data, length limits, and split CRLF handling.

File	Description
src/​Http/​WebUtilities/​test/​BufferedReadStreamTests.cs	Adds line-reading and boundary-condition tests.
src/​Http/​WebUtilities/​src/​MultipartReader.cs	Avoids an intermediate header-value string when trimming.
src/​Http/​WebUtilities/​src/​BufferedReadStream.cs	Decodes complete buffered lines without a temporary MemoryStream.

_{Review effort: Balanced}

[davidfowl]

The Platform tests Test: Windows Server x64 failure on 6bb5949 is infrastructure, not this change. NuGet restore failed while committing the assets file (NuGet.RestoreEx.targets(19,5): error : Cannot create a file when that file already exists. at NuGet.Common.FileUtility.Move → RestoreResult.CommitAssetsFileAsync), and eng\testing\linker\trimmingTests.targets(104,5) then reported MSB4181. No tests ran or failed. I pushed an empty commit (00d1aaa) to retrigger CI.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

The failures on 00d1aaa look unrelated to this change:

• aspnetcore-components-e2e: ServerRoutingTest.NavigationLock_CanBlockExternalNavigation (25.68% historical failure rate) and VirtualizationTest.VariableHeight_HtmlTable_DoesNotProduceNestedTrElements (9.45%, also failing on Fix token handling in Blazor WebAssembly AuthorizationMessageHandler #70106 and Avoid parsing Content-Type and allocating Items in HasFormContentType #70163) are browser navigation and rendering tests that don't touch BufferedReadStream.
• aspnetcore-ci: the Helix job monitor exited with code 1, but the run reports 0 failed tests (86,598 passed). It looks like a work item exited non-zero after its tests passed, not a test failure.

I pushed an empty commit (8086d3f) to retrigger CI. If the Helix failure happens again, I'll dig into the specific work item.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

Azure Pipelines hasn't started on the last two pushes to this PR (8086d3f and the earlier retrigger), even though pipelines are running for other PRs pushed in the same window. The previous run's failures look unrelated to this change:

• ServerRoutingTest.NavigationLock_CanBlockExternalNavigation (25.68% historical failure rate)
• VirtualizationTest.VariableHeight_HtmlTable_DoesNotProduceNestedTrElements (9.45% historical failure rate)
• A Helix job monitor error with 0 failed tests

Could a maintainer comment /azp run to start CI?

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

Heads-up: the two most recent "Retrigger CI" commits on this branch (8086d3f and 24b714d) were created with an empty tree, so each one deleted every file in the repo. That's why Azure Pipelines never started on them.

Fixed in 742c03a, which restores the tree of the last good commit (00d1aaa). The PR diff is back to the intended change only (+144/−5: BufferedReadStream.cs, MultipartReader.cs, BufferedReadStreamTests.cs), and this push also retriggers CI.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

Retriggered CI with an empty commit (d524329). The only failures were Blazor E2E tests unrelated to this WebUtilities change: NavigationLock_CanBlockExternalNavigation in the Mono server leg and the Helix run. Build Analysis lists a 26.76% historical failure rate for it.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

Retriggered CI with an empty commit (3a4a889). The previous run failed only the Blazor E2E test ServerRoutingTest.NavigationLock_CanBlockExternalNavigation (Build Analysis: 26.95% failure rate), unrelated to this WebUtilities change.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

Retriggered CI with an empty commit (bce2ab4). In aspnetcore-ci, the only failure was BlazorTemplates.Tests.BlazorWebTemplateTest.BlazorWebTemplate_ConditionalPasskeyRegistration on Windows platform tests (Build Analysis: 7.44% failure rate; it also failed on unrelated PR #70186). The aspnetcore-components-e2e failures are the Blazor Selenium flakes currently red on main. None of these relate to this WebUtilities change.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[davidfowl]

Status after the retrigger (bce2ab4): aspnetcore-ci is green. The only failure is in the separate aspnetcore-components-e2e pipeline (Blazor E2E on Linux, Mono server). I'm not retriggering again, for these reasons:

• The failing test is a Blazor navigation test (Expected: .../myexternalpath3, Actual: .../subdir/).
• That pipeline currently fails on main too: 4 of the last 5 main commits are red.
• This PR only touches src/Http/WebUtilities code and has no effect on Blazor navigation.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

[PureWeen]

/review

[github-actions]

✅ No documentation PR was created.

Confidence that documentation is needed: 3%

No docs needed: internal performance refactoring with tests only; no signals detected and no public or observable behavior change.

Documentation surfaces considered

• Conceptual article: Not required — Internal allocation optimization in BufferedReadStream.ReadLine and MultipartReader; no public API changes.
• Migration guidance: Not required — No user action needed when upgrading.
• Breaking change: Not required — No compatibility impact; behavior including line length limit exception is unchanged.