Repository navigation
Honor non-retriable activity and sub-orchestration failures - #318
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the non-retriable failure-handling changes.
No open issues 🎉
What changed in this PR
Fixes #317 by making the shared orchestration executor honor non-retriable activity and sub-orchestration failures, including when an explicit retry policy is configured.
Changes:
- Immediately surface non-retriable failures without scheduling a retry, while preserving failure details and existing behavior for false or unset flags.
- Add parameterized coverage for activities, sub-orchestrations, replay, compensation, and continued retries.
- Document the fix and replay compatibility warning across all three packages.
| File | Description |
|---|---|
tests/durabletask/test_orchestration_executor.py |
Adds regression coverage for non-retriable failures and unchanged retry behavior. |
durabletask/worker.py |
Checks the non-retriable flag in both retryable failure-handling paths. |
durabletask-azuremanaged/CHANGELOG.md |
Documents the inherited fix and upgrade warning. |
CHANGELOG.md |
Documents the core behavior change and replay compatibility risk. |
azure-functions-durable/CHANGELOG.md |
Documents the inherited fix, including compatibility RetryOptions, and upgrade warning. |
Review effort: Balanced
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bernd Verst (berndverst)
left a comment
There was a problem hiding this comment.
Reviewed the latest commit, 6eac393. No blocking findings.
This accomplishes #317: both activity and sub-orchestration failures with isNonRetriable=true bypass delay calculation and reuse the terminal-failure path, preserving failure details and resuming user code for compensation. False/unset flags retain existing retry behavior. Both providers inherit the fix, including Functions RetryOptions. The regression matrix covers both task kinds, flag values, replay, caught/uncaught failures, and continued retries. The replay compatibility risk is correctly documented across all three changelogs.
The production change is idiomatic and Python 3.11-compatible. No meaningful performance regression is expected from inspection: only retryable failure handling gains a protobuf flag lookup and branch; successful execution is unchanged, and non-retriable failures avoid delay computation and retry work. This is a code-path assessment, not a benchmark result.
One non-blocking modernization suggestion: the new test's datetime.utcnow() at tests/durabletask/test_orchestration_executor.py:973 emits a DeprecationWarning on Python 3.12+. Prefer datetime.now(UTC).replace(tzinfo=None), importing UTC from datetime. UTC is available in Python 3.11, and stripping tzinfo preserves the naive-UTC representation expected by the existing protobuf timestamp assertions. No broader datetime refactor is needed here.
All 205 executor tests passed locally on Python 3.13 at this revision, including the 24 new cases. The original implementation commit also passed Python 3.11 PR CI; local 3.11 execution was blocked by package-download TLS failures. Additional local managed-provider tests were blocked during environment setup, so I am not claiming a local provider test pass.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Summary
Fixes #317.
Activity and sub-orchestration failures marked
failureDetails.isNonRetriable=truenow bypass explicit retry policies and immediately fail the pending task, preserving the original failure details. Orchestrations can catch the failure for compensation; otherwise, the orchestration fails. No retry timer or additional attempt is scheduled. False or unset flags retain existing retry behavior.The change is limited to the two failure-handling branches in the shared orchestration executor. Parameterized regression coverage exercises activities and sub-orchestrations, true/false/unset flags, new and replayed failures, caught and uncaught exceptions, and continued retries for retriable failures.
Compatibility
Existing histories that already retried non-retriable failures can replay differently. Drain affected instances on the previous SDK before upgrading.
Azure Managed and Azure Functions providers inherit the fix from the core SDK, including Functions compatibility
RetryOptions. All three package changelogs are updated. Package versions and dependency minimums remain unchanged; coordinate provider release minimums after the fixed core SDK is released.