Skip to content

Honor non-retriable activity and sub-orchestration failures - #318

Merged
andystaples merged 3 commits into
mainfrom
andystaples-non-retriable-activity-failures
Nov 2, 2026
Merged

andystaples merged 3 commits into
mainfrom
andystaples-non-retriable-activity-failures

Conversation

@andystaples

Copy link
Copy Markdown
Contributor

Summary

Fixes #317.

Activity and sub-orchestration failures marked failureDetails.isNonRetriable=true now bypass explicit retry policies and immediately fail the pending task, preserving the original failure details. Orchestrations can catch the failure for compensation; otherwise, the orchestration fails. No retry timer or additional attempt is scheduled. False or unset flags retain existing retry behavior.

The change is limited to the two failure-handling branches in the shared orchestration executor. Parameterized regression coverage exercises activities and sub-orchestrations, true/false/unset flags, new and replayed failures, caught and uncaught exceptions, and continued retries for retriable failures.

Compatibility

Existing histories that already retried non-retriable failures can replay differently. Drain affected instances on the previous SDK before upgrading.

Azure Managed and Azure Functions providers inherit the fix from the core SDK, including Functions compatibility RetryOptions. All three package changelogs are updated. Package versions and dependency minimums remain unchanged; coordinate provider release minimums after the fixed core SDK is released.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 18:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the non-retriable failure-handling changes.

No open issues 🎉

What changed in this PR

Fixes #317 by making the shared orchestration executor honor non-retriable activity and sub-orchestration failures, including when an explicit retry policy is configured.

Changes:

  • Immediately surface non-retriable failures without scheduling a retry, while preserving failure details and existing behavior for false or unset flags.
  • Add parameterized coverage for activities, sub-orchestrations, replay, compensation, and continued retries.
  • Document the fix and replay compatibility warning across all three packages.
File Description
tests/​durabletask/​test_orchestration_executor.py Adds regression coverage for non-retriable failures and unchanged retry behavior.
durabletask/​worker.py Checks the non-retriable flag in both retryable failure-handling paths.
durabletask-azuremanaged/​CHANGELOG.md Documents the inherited fix and upgrade warning.
CHANGELOG.md Documents the core behavior change and replay compatibility risk.
azure-functions-durable/​CHANGELOG.md Documents the inherited fix, including compatibility RetryOptions, and upgrade warning.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

@berndverst Bernd Verst (berndverst) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest commit, 6eac393. No blocking findings.

This accomplishes #317: both activity and sub-orchestration failures with isNonRetriable=true bypass delay calculation and reuse the terminal-failure path, preserving failure details and resuming user code for compensation. False/unset flags retain existing retry behavior. Both providers inherit the fix, including Functions RetryOptions. The regression matrix covers both task kinds, flag values, replay, caught/uncaught failures, and continued retries. The replay compatibility risk is correctly documented across all three changelogs.

The production change is idiomatic and Python 3.11-compatible. No meaningful performance regression is expected from inspection: only retryable failure handling gains a protobuf flag lookup and branch; successful execution is unchanged, and non-retriable failures avoid delay computation and retry work. This is a code-path assessment, not a benchmark result.

One non-blocking modernization suggestion: the new test's datetime.utcnow() at tests/durabletask/test_orchestration_executor.py:973 emits a DeprecationWarning on Python 3.12+. Prefer datetime.now(UTC).replace(tzinfo=None), importing UTC from datetime. UTC is available in Python 3.11, and stripping tzinfo preserves the naive-UTC representation expected by the existing protobuf timestamp assertions. No broader datetime refactor is needed here.

All 205 executor tests passed locally on Python 3.13 at this revision, including the 24 new cases. The original implementation commit also passed Python 3.11 PR CI; local 3.11 execution was blocked by package-download TLS failures. Additional local managed-provider tests were blocked during environment setup, so I am not claiming a local provider test pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@andystaples
andystaples merged commit fe2d7d7 into main Nov 2, 2026
25 checks passed
@andystaples
andystaples deleted the andystaples-non-retriable-activity-failures branch November 2, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Activity retry policies ignore failureDetails.isNonRetriable

3 participants