[release/9.0] Fix KeyRingProvider thread pool starvation on cold start#66736
Open
DeagleGross wants to merge 2 commits into
Open
[release/9.0] Fix KeyRingProvider thread pool starvation on cold start#66736DeagleGross wants to merge 2 commits into
DeagleGross wants to merge 2 commits into
Conversation
* fix thread starvation
Contributor
|
Hi @DeagleGross. If this is not a tell-mode PR, please make sure to follow the instructions laid out in the servicing process document. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR backports a fix to the DataProtection KeyRingProvider cache-refresh logic to prevent thread-pool starvation on cold start when there is no cached key ring yet (startup / first requests).
Changes:
- Detect true cold-start (no cached key ring) and perform the initial key ring load synchronously on the calling thread while holding the lock, avoiding
TaskScheduler.Default+Task.Wait()starvation. - Preserve the existing “stale ring exists” behavior by continuing to dispatch refresh asynchronously so callers can return the stale ring without blocking.
- Add a regression unit test that asserts cold-start refresh work runs on the calling thread (invariant preventing starvation).
Show a summary per file
| File | Description |
|---|---|
| src/DataProtection/DataProtection/src/KeyManagement/KeyRingProvider.cs | Splits cold-start vs stale-refresh behavior so cold start loads inline under lock, avoiding thread-pool starvation. |
| src/DataProtection/DataProtection/test/Microsoft.AspNetCore.DataProtection.Tests/KeyManagement/KeyRingProviderTests.cs | Adds a regression test asserting cold-start refresh runs on the calling thread. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 0
halter73
reviewed
May 19, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
halter73
approved these changes
May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #66683 to release/9.0
KeyRingProvider.GetCurrentKeyRingCoreNewhandles two states with one mechanism:TaskScheduler.Default; every caller takes the early-return and immediately gets the stale ring. Nobody blocks.existingTask.Wait()— pinning a thread-pool thread on a task that needs a free thread-pool thread to run. With a constrained pool (e.g.ThreadPool.SetMaxThreads(16, …)and 16 concurrentProtectcalls — exactly the issue's repro), every worker is parked waiting for a worker that doesn't exist. The runtime's hill climber eventually injects extra threads (~118 s in the report) and the app recovers, but during the freeze nothing makes progress.Fix is to split up cold-start (no stale
cacheableKeyRingyet) and do the synchronous load on the first thread acquiring lock. Others will be waiting on lock as in the old implementation.Related #54675
Fixes #66380
Customer Impact
A customer running ASP.NET Core on .NET 10 reported (#66380) that their app freezes on startup when several authenticated requests arrive concurrently. The app handles the first ~10 requests and then completely freezes for ~118 seconds with every thread‑pool thread stuck inside
KeyRingProvider.GetCurrentKeyRingCoreNew, blocked inTask.Wait().Regression?
Risk
There is a workaround that customers can apply today without waiting for this fix:
Verification
Packaging changes reviewed?
When servicing release/2.3