[sophiatev]

We currently have a bug where we disregard active work being down and issue scale down of hosts that are actively executing work items. This can be very detrimental for long-running Activities in particular - the hosts executing them are scaled down, their Activity work items become visible again at which point hosts are provisioned again and the work items start again, etc. leading to oscillation and a constant start-stop-start pattern.

[Copilot]

🟡 Changes recommended

Both recommendation paths still ignore the retained work-item count, allowing scale-in while invisible activities are running, and the regression test does not cover this scenario.

1 open issue

• Prevent scale-in recommendations when invisible activities are running · New

What changed in this PR

This PR preserves in-flight activity messages in Azure Storage scaling metrics to address premature scale-down of workers executing long-running activities.

Changes:

• Retains work-item queue counts when all messages are invisible, while keeping latency at zero.
• Preserves existing control-queue behavior for timers that are not yet due.
• Adds regression coverage for queued, running, newly added, and completed activity messages.

File	Description
test/​DurableTask.AzureStorage.Tests/​AzureStorageScaleTests.cs	Tests queue metrics throughout the activity-message lifecycle.
src/​DurableTask.AzureStorage/​Monitoring/​DisconnectedPerformanceMonitor.cs	Includes invisible work-item messages in queue-length metrics.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

There are no active review issues for the update to account for in-flight activity messages in scaling decisions.

No open issues 🎉

1 resolved since last review

• Prevent scale-in recommendations when invisible activities are running

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Scale-in can still remove busy workers because running activities cannot migrate to unused concurrency slots on remaining workers, risking interruption and restart of long-running work.

1 open issue

• Prevent scale-in while workers have running activities · New

_{Review effort: Balanced}