Skip to content

[ci-scan] Test failure: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130947

Closed

Description

@lewing

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1513558
Build error leg or test failing: Libraries Test Run release coreclr windows x64 Debug - System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
Pull request: #130816

Error Details

Flaky assertion in the AIA revocation test: the expected AIA fetch count (4) does not match the observed count, and the failure persists through all RetryHelper attempts (observed actual values 5, 7, 8). The sibling flake AiaAcceptsCertTypesAndIgnoresNonCertTypes is already tracked by #107364, but with a different method name, assertion type, and OS, so Build Analysis does not match this failure against it.

System.AggregateException : One or more errors occurred. (Assert.Equal() Failure: Values differ
Expected: 4
Actual:   5) (Assert.Equal() Failure: Values differ
Expected: 4
Actual:   7) (Assert.Equal() Failure: Values differ
Expected: 4
Actual:   8)
  ---- Assert.Equal() Failure: Values differ
  Expected: 4
  Actual:   8
  Stack Trace:
    /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(54,0): at System.RetryHelper.Execute(...)
    /_/src/libraries/System.Security.Cryptography/tests/X509Certificates/RevocationTests/AiaTests.cs(241,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits()

Console log: https://helixr1107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-130816-merge-82e95810a7454e0087/System.Security.Cryptography.Tests/1/console.e11e9c49.log?helixlogtype=result

Error Message

{
  "ErrorMessage": "System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits [FAIL]",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Note

This issue draft was prepared with GitHub Copilot assistance and reviewed by the submitting developer.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1513558
Error message validated: [System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits [FAIL]]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/17/2026 2:37:41 AM UTC

Report

Build Repository Test Pull Request
1523549 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1523323 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1523039 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1522962 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1521887 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1521811 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131223
1521645 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130932
1521051 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130622
1521006 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131182
1520962 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131068
1520634 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #123920
1519971 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130589
1519962 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131160
1519105 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130512
1519288 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1519214 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1519148 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131133
1519091 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1518362 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131007
1518068 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130622
1518037 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131084
1517809 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131080
1517533 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130888
1514659 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130877
1517264 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516915 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131016
1516834 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131045
1516784 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131029
1516704 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130207
3026382 dotnet-runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516707 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131041
1516698 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516689 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130761
1516594 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131033
1516568 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #126187
1516541 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516502 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516401 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #131018
1516213 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516111 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1516086 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1515911 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130935
1515486 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130834
1515690 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130994
1515246 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130983
3024312 dotnet-runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1514858 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130967
1514507 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130888
1514289 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1514221 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130956
1513558 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #130816
1513510 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
1512190 dotnet/runtime System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits #128799

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
4 52 53

Activity

  1. added
    blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
    Known Build ErrorUse this to report build issues in the .NET Helix tab
    on Jul 17, 2026
  2. dotnet-policy-service commented on Jul 17, 2026

    @dotnet-policy-service
    Contributor

    Tagging subscribers to this area: @bartonjs, @vcsjones, @dotnet/area-system-security
    See info in area-owners.md if you want to be subscribed.

  3. github-actions commented on Jul 17, 2026

    @github-actions
    Contributor

    Caution

    agentic threat detected
    Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

    Details

    The threat detection results could not be parsed.

    Review the workflow run logs for details.

    Workflow artifact: ci-fix
    Artifact kind: handoff
    Linked KBE: #130947

    Note

    AI/Copilot-generated root-cause hand-off (ci-failure-fix). No automated code change was produced: removing this failure requires either understanding platform AIA-download batching (product behavior) or relaxing an intentionally strict count assertion (a coverage/design judgement), neither of which is a small validated non-muting fix. Looping in owners to fix forward.

    Root cause (read-only analysis)

    AiaTests.AiaCompletionHasLimits asserts the exact number of chain elements produced as AIA fetching proceeds in fixed batches:

    Assert.Equal(4, chain.ChainElements.Count);  // actual: 5
    ...
    Assert.Equal(6, chain.ChainElements.Count);  // actual: 7
    ...
    Assert.Equal(8, chain.ChainElements.Count);  // actual: 8
    

    The expected sequence is 4, 6, 8 (the test asserts "we only download two certificates at a time"), but the failing run observed 5, 7, 8 — one extra certificate fetched at each intermediate stage. The whole assertion block is wrapped in RetryHelper.Execute, so the retries (actual values 5, 7, 8 across attempts) confirm the discrepancy is stable within the run rather than a transient hiccup.

    The test source itself flags this count as fragile black-box behavior:

    // This is a Windows black-box behavior that we're matching.
    // White-box suggests it's supposed to be 3, but maybe there's an off-by-one.
    // Windows also allow registry customization... but this test can't tolerate it.
    

    So the exact per-stage fetch count depends on platform AIA-download batching that is not guaranteed to be exactly two-at-a-time on every OS/build configuration. On the failing windows x64 Debug leg the chain builder resolved one additional certificate per stage.

    Why no in-bounds automated fix:

    • Changing the AIA fetch batching to deterministically match the asserted counts is product behavior in the chain-building/X509Chain layer — outside small-fix bounds and not something to alter to satisfy a test.
    • Relaxing the exact-count assertions (e.g. asserting bounds rather than equality) would weaken the invariant the test is deliberately pinning; whether that is acceptable is an owner judgement, not an automated mute.

    Evidence

    Suggested reviewers / area contacts

    • Area owners (area-System.Security): @bartonjs, @vcsjones (@dotnet/area-system-security)

    A fix would likely either pin the AIA download batching so the asserted counts hold across the CI OS/build matrix, or (with owner sign-off) adjust the assertion to tolerate the documented platform variance without losing the intent of the limit check.

    Generated by CI Outer-Loop Failure Fixer · 340.6 AIC · ⊞ 18.2K · ◷

  4. vcsjones commented on Jul 17, 2026

    @vcsjones
    Member

    This is likely a regression from #130456. Will take a look.

  5. vcsjones commented on Jul 17, 2026

    @vcsjones
    Member

    Hm, actually not likely that PR affected this as that PR is Linux and this failure is Windows.

  6. JulieLeeMSFT commented on Jul 17, 2026

    @JulieLeeMSFT
    Member

    Failed in (2):

    Console Log: Console Log
    Source: runtime-coreclr libraries-jitstress / net11.0-windows-Release-x64-jitstress1_tiered-Windows.10.Amd64.Open / System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits

    Failed tests:

    runtime-coreclr libraries-jitstress
    - net11.0-windows-Release-arm64-tailcallstress-Windows.11.Arm64.Open
    - net11.0-windows-Release-x64-jitstress1_tiered-Windows.10.Amd64.Open
    runtime-coreclr libraries-pgo
    - net11.0-windows-Release-arm64-defaultpgo-Windows.11.Arm64.Open
      - System.Security.Cryptography.X509Certificates.Tests.RevocationTests.AiaTests.AiaCompletionHasLimits
    
  7. bartonjs commented on Jul 17, 2026

    @bartonjs
    Member

    It seems like at least some of the machines in the test pool have an actual AIA limit of 3. So some have an off by one, some don't.

    So, it's caused by #130456, in that that's the PR that created the test.

  8. JulieLeeMSFT commented on Jul 22, 2026

    @JulieLeeMSFT
    Member

    @bartonjs, it's failing every day in both innerloop and outerloop. What's your plan? Should we disable the test?

  9. bartonjs commented on Jul 22, 2026

    @bartonjs
    Member

    I'll either fix it or disable it today.

  10. locked and limited conversation to collaborators on Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Known Build ErrorUse this to report build issues in the .NET Helix tabPriority:0Work that we can't release withoutarea-System.Securityblocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'blocking-clean-ci-optionalBlocking optional rolling runs

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions