Skip to content

Drop Python 3.10 support and adopt Python 3.11 features - #302

Merged
Bernd Verst (berndverst) merged 8 commits into
mainfrom
berndverst-python-311-baseline
Oct 30, 2026
Merged

Bernd Verst (berndverst) merged 8 commits into
mainfrom
berndverst-python-311-baseline

Conversation

@berndverst

@berndverst Bernd Verst (berndverst) commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Require Python 3.11+ for durabletask and durabletask.azuremanaged; retain Python 3.13+ for azure-functions-durable.
  • Remove Python 3.10 package classifiers and CI/Nox sessions, and move the lowest-supported-version Pyright configuration to 3.11.
  • Update contributor guidance, development docs, example prerequisites, and each affected package's unreleased changelog.
  • Merge the latest main and reassess the newly introduced entity identity, dispatch recovery, generator cleanup, payload handling, and deterministic set/frozenset serialization code for additional modernization opportunities.

Python 3.11 feature assessment

Opportunity Decision
typing.Self Use for synchronous/asynchronous context managers so inherited clients, workers, locks, and storage helpers retain their concrete subclass types. Includes Azure Managed and Functions inheritance. Also use for the newly introduced immutable entity ID copy/deepcopy and pickle-reduction hooks.
ExceptionGroup Replace the private multi-callback error wrapper with the built-in group, preserving individual exceptions and tracebacks in memory and the first callback error as the explicit cause (__cause__ and serialized FailureDetails.inner_failure). Single callback failures still raise directly, and other observers are still notified in subscription order. The aggregate type/message remain observable changes; see replay compatibility below.
BaseExceptionGroup.subgroup() Replace the newly merged pre-3.11 version checks and manual traversal with native nested-group matching for cancellation and process-control exceptions. Propagate the original complete group, not a split subset.
Group-aware tracebacks and exception notes Render nested exception messages, tracebacks, and Python 3.11 add_note() diagnostics when serializing grouped failures, explicitly retaining standard-library limits of 15 children per group and 10 group levels. Truncation markers identify omitted diagnostics; the failure protobuf does not serialize a complete structured tree of group children. Keep ordinary non-group exception stack formatting unchanged. Covers core, Azure Managed, and synchronous/asynchronous Functions execution.
datetime.UTC Use the standard alias throughout hand-written package source and affected examples; UTC identity and timestamp behavior are unchanged.
Runtime-generic logging.LoggerAdapter Remove the pre-3.11 TYPE_CHECKING base-class alias.
Improved typing.get_type_hints() Remove the Python 3.10 nested-forward-reference evaluator and rely on standard-library resolution for dataclass reconstruction. Keep this native path for the newly merged set/frozenset support, with coverage for nested modern/legacy annotations, recursive frozenset dataclasses, and activity input/output discovery on Python 3.11.
datetime.fromisoformat() support for Z Remove the manual trailing-Z rewrite while preserving numeric offsets and naive timestamps.
Unified timeout exception Catch built-in TimeoutError instead of the now-aliased asyncio.TimeoutError.
Standard-library asyncio Remove the obsolete PyPI backport from package dependencies and development requirements; all supported runtimes include asyncio.
StrEnum Keep existing enums: conversion would change their string/formatting behavior rather than merely modernize implementation.
asyncio.TaskGroup / asyncio.timeout() Keep existing task ownership and wait_for calls: changing sibling cancellation, error propagation, or task lifetime is not justified by the version-floor change. Keep payload preflight and uploads ordered rather than introducing concurrent side effects.

Upstream behavior preserved

Retain the newly merged dispatch abandonment/uncertain-outcome recovery, generator cleanup and cancellation semantics, all-payload size preflight, and replay-visible entity-lock release ordering. In particular, ordinary exceptions during generator cleanup still preserve an already-prepared dispatch response; nested cancellation/process-control groups still propagate unchanged.

Preserve the newly merged deterministic set/frozenset encoder without changing canonical JSON ordering, Unicode object-name normalization, custom-hook call counts, or cycle handling. Set reconstruction remains opt-in, so recorded arrays retain their order and duplicates by default. The refresh does not add speculative source rewrites where the standard-library path already suffices.

Replay compatibility

Warning

The multi-callback aggregate exception type and message have changed. This is
not unconditionally replay-neutral: orchestrations branching on the previous
aggregate type or message can take a different path during replay. Drain
affected instances on the previous SDK or use a separate task hub for new
instances.

The first callback error remains the explicit cause and serialized inner
failure, preserving existing cause-based recovery decisions. A regression
replays legacy timer-cancellation history whose recovery activity is selected
by isinstance(error.__cause__, TaskCancelledError).

Release coordination

Package versions and inter-package dependency minimums are intentionally unchanged. Publish the core changes first, then coordinate provider versions and minimum core dependencies in a dedicated release PR. Functions continues to require Python 3.13.

Local limitation

The Azure Managed DTS-emulator session could not run because the local Docker/Podman runtime is stopped; emulator coverage remains for CI. Service-independent Azure Managed and Functions coverage is not affected.

Bernd Verst added 2 commits October 20, 2026 10:24
Raise the core and Azure Managed minimum to Python 3.11 while retaining Python 3.13 for Functions. Align CI, Nox, type checking, and documentation; use Self, UTC and ExceptionGroup, and remove obsolete compatibility shims.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
Resolve changelog conflicts by retaining both the Python 3.11 migration entries and the upstream orchestrator generator cleanup fixes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
@andystaples

Copy link
Copy Markdown
Contributor

If #307 is merged first, this PR will need to also update to remove it's >= 3.11 check. Likely will be found in the merge conflict, but wanted to surface anyway

Bernd Verst added 4 commits October 29, 2026 23:25
Preserve upstream payload preflight, entity identity, dispatch recovery, and task hub filtering improvements alongside the Python 3.11 baseline.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
Use native exception-group matching and preserve nested failure diagnostics and notes. Extend Self typing to immutable entity copy protocols, remove the obsolete asyncio backport dependency, and cover dispatch and provider compatibility.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
Retain native Python 3.11 type-hint resolution alongside deterministic set encoding and opt-in set reconstruction.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
Exercise modern and legacy nested set annotations, recursive frozenset dataclasses, and activity input/output discovery for both opt-in options without restoring the Python 3.10 forward-reference evaluator.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
@berndverst
Bernd Verst (berndverst) marked this pull request as ready for review October 30, 2026 20:28
Copilot AI balanced review requested due to automatic review settings October 30, 2026 20:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the Python baseline update and compatibility-code modernization.

No open issues 🎉

What changed in this PR

Raises the core SDK and Azure Managed provider minimum to Python 3.11 while retaining Python 3.13 for Azure Functions Durable. Uses native language features to simplify compatibility code and improve typing and grouped-failure diagnostics.

Changes:

  • Aligns package metadata, CI, Nox, type checking, and documentation with the supported Python versions.
  • Adopts Self, ExceptionGroup, datetime.UTC, and native type-hint and timestamp handling; removes the obsolete asyncio dependency.
  • Adds regression coverage for subclass typing, grouped failures, dispatch outcomes, and nested dataclass reconstruction.
File Description
tests/​durabletask/​typing/​context_managers.py Adds subclass-preserving context-manager and entity-copy type checks.
tests/​durabletask/​test_task_composition.py Verifies callback exception groups and preserved diagnostics.
tests/​durabletask/​test_serialization.py Tests native nested forward-reference resolution.
tests/​durabletask/​test_orchestration_dispatch_recovery.py Covers nested control-flow groups and dispatch outcomes.
tests/​durabletask/​test_failure_details.py Checks grouped tracebacks, notes, and ordinary stack formatting.
tests/​durabletask/​test_data_converter.py Tests activity set annotations with native type discovery.
tests/​durabletask/​scheduled/​test_models.py Covers UTC, offset, and naive timestamp parsing.
tests/​durabletask/​entities/​test_entity_id_parsing.py Verifies subclass copy and reduction behavior.
tests/​durabletask-azuremanaged/​test_azuremanaged_grpc_resiliency.py Tests grouped diagnostics through the Azure Managed worker.
tests/​azure-functions-durable/​typing/​import_package.py Checks inherited Functions context-manager return types.
tests/​azure-functions-durable/​test_worker_compat.py Tests grouped diagnostics in synchronous and asynchronous execution.
requirements.txt Removes the obsolete asyncio dependency.
README.md Documents package-specific Python requirements.
pyrightconfig.json Targets Python 3.11 and includes new typing checks.
pyproject.toml Raises the core Python minimum and removes asyncio.
noxfile.py Updates test sessions and baseline checks to Python 3.11.
examples/​scheduled_tasks.py Uses datetime.UTC for schedule creation.
examples/​large_payload/​README.md Updates the Python prerequisite.
examples/​history_export/​app.py Uses datetime.UTC for export timestamps.
examples/​distributed-tracing/​README.md Updates the Python prerequisite.
examples/​custom_data_converter/​src/​workflows.py Uses datetime.UTC in sample data.
durabletask/​worker.py Adopts native group matching, Self, UTC, and timeout aliases.
durabletask/​testing/​in_memory_backend.py Uses datetime.UTC throughout timestamp handling.
durabletask/​task.py Adopts built-in exception groups and runtime-generic logging.
durabletask/​serialization.py Removes the legacy forward-reference resolver.
durabletask/​scheduled/​schedule_entity.py Uses datetime.UTC for current timestamps.
durabletask/​scheduled/​models.py Relies on native trailing-Z timestamp parsing.
durabletask/​internal/​helpers.py Preserves grouped failure diagnostics and adopts UTC.
durabletask/​internal/​client_helpers.py Uses UTC for entity request timestamps.
durabletask/​history.py Uses UTC when converting history timestamps.
durabletask/​extensions/​history_export/​entity.py Uses UTC for export entity timestamps.
durabletask/​extensions/​history_export/​client.py Updates runtime timestamps and docstring examples to UTC.
durabletask/​extensions/​history_export/​azure_blob.py Preserves writer subclass types with Self.
durabletask/​extensions/​history_export/​activities.py Uses UTC when normalizing naive timestamps.
durabletask/​extensions/​history_export/​_internal.py Uses UTC in timestamp conversion helpers.
durabletask/​extensions/​azure_blob_payloads/​blob_payload_store.py Uses Self for synchronous and asynchronous context managers.
durabletask/​entities/​entity_metadata.py Uses UTC for entity metadata timestamps.
durabletask/​entities/​entity_lock.py Preserves lock subclass types with Self.
durabletask/​entities/​entity_instance_id.py Uses Self for copy and pickle-hook annotations.
durabletask/​client.py Preserves client subclass types in context managers.
durabletask-azuremanaged/​pyproject.toml Raises the provider’s Python minimum to 3.11.
durabletask-azuremanaged/​durabletask/​azuremanaged/​preview/​sandboxes/​client.py Preserves sandbox client subclass types with Self.
durabletask-azuremanaged/​durabletask/​azuremanaged/​internal/​access_token_manager.py Uses UTC for token expiry calculations.
durabletask-azuremanaged/​CHANGELOG.md Records Python requirements and inherited behavior changes.
docs/​development.md Updates supported versions and Nox command examples.
CHANGELOG.md Documents core compatibility, typing, and diagnostic changes.
azure-functions-durable/​CHANGELOG.md Documents inherited improvements while retaining Python 3.13.
azure-functions-durable/​azure/​durable_functions/​testing/​entity.py Uses UTC when decoding scheduled entity signals.
azure-functions-durable/​azure/​durable_functions/​internal/​history_export_compat.py Uses UTC for fallback export timestamps.
azure-functions-durable/​azure/​durable_functions/​internal/​compat/​orchestration_context.py Uses UTC for naive orchestration timestamps.
azure-functions-durable/​azure/​durable_functions/​internal/​compat/​history_projection.py Uses UTC in history timestamp formatting.
azure-functions-durable/​azure/​durable_functions/​internal/​compat/​durable_orchestration_status.py Uses UTC in status timestamp formatting.
azure-functions-durable/​azure/​durable_functions/​http/​builtin.py Uses UTC when normalizing retry timestamps.
.github/​workflows/​typecheck.yml Runs baseline type checks on Python 3.11.
.github/​workflows/​durabletask.yml Removes Python 3.10 from the core test matrix.
.github/​workflows/​durabletask-azuremanaged.yml Removes Python 3.10 from the provider test matrix.
.github/​copilot-instructions.md Updates contributor guidance for package-specific Python minimums.

Review effort: Balanced

@andystaples andystaples left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 3.11 migration looks well-scoped, and I do not think the narrow use of ExceptionGroup is inherently too disruptive for a minor release. Ordinary task failures and single callback failures keep their existing behavior. However, the dropped cause chain introduces a reproducible replay incompatibility that should be fixed before merging (inline below). The exception type/message changes remain observable even after that fix, so this should not be described as unconditionally replay-neutral. The rest of the version floors, CI/Nox, typing, and provider changes look consistent. One additional diagnostic-truncation suggestion below.

Comment thread durabletask/task.py Resolved
Comment thread durabletask/internal/helpers.py Resolved
Restore the first callback exception as the explicit cause so legacy cause-based recovery histories replay correctly and retain structured inner failures. Cover wide/deep groups with explicit traceback rendering limits, and document the remaining aggregate type/message replay compatibility change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
Address the automated code-quality finding without changing the test orchestrator's fall-through behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31
@berndverst
Bernd Verst (berndverst) merged commit 97427b2 into main Oct 30, 2026
25 checks passed
@berndverst
Bernd Verst (berndverst) deleted the berndverst-python-311-baseline branch October 30, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants