diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 5be6c6c6..5c6c4409 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -30,6 +30,12 @@ three packages: - When in doubt, write the changelog entry in terms of user impact (what users can now do or what behavior changed), not implementation mechanism (how it was implemented internally). +- Keep entries focused on the capability or fix, its relevant default, and + material compatibility or upgrade risks. Preserve actionable replay and + breaking-change warnings, but keep API tutorials, implementation + prescriptions, exhaustive edge cases, and workarounds in linked feature or + API documentation. Narrow an overly broad claim rather than expanding it + into a catalogue of exceptions. - Changelogs are not covered by the CI Markdown lint step. Review changes to them manually. - Use the current unindented changelog style: category labels such as `ADDED`, diff --git a/CHANGELOG.md b/CHANGELOG.md index d44516bf..09b6148d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,71 +7,58 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## Unreleased +## v1.13.0 + ADDED - Added `OrchestrationQuery.task_hub_names` to filter orchestration instances by one or more task hubs. +- Added deterministic JSON-array serialization for sets and frozensets. +Generic set reconstruction is opt-in through `JsonDataConverter`; enabling it +for existing orchestrations can change replay behavior. CHANGED - Dropped Python 3.10 support. Python 3.11 or later is now required. -- Removed the obsolete PyPI `asyncio` backport dependency; all supported -Python versions use the standard-library module. +- Removed the obsolete PyPI `asyncio` backport dependency. - Context managers now preserve subclass types in type checkers, including Azure Managed clients and workers. - Entity ID copy and pickle hooks now preserve subclass types in type checkers. - Multiple task completion callback failures now raise the built-in `ExceptionGroup`, preserving each exception and its traceback in memory and -the first callback error as the explicit cause and serialized inner failure. -A single callback failure is still raised directly. +the first callback error as the cause. A single callback failure is still +raised directly. > [!WARNING] > The aggregate exception type and message have changed. Orchestrations that > branch on the old aggregate type or message can take a different path during > replay; drain affected instances on the previous SDK or use a separate task -> hub for new instances. Branches on the aggregate's explicit first-error cause -> retain their previous behavior. +> hub for new instances. FIXED - Activity and sub-orchestration retry policies now honor `failureDetails.isNonRetriable`, surfacing the failure immediately without -scheduling a retry timer or another attempt. Existing histories that already -retried such failures can replay differently; drain affected instances on the -previous SDK before upgrading. +retrying. Existing histories that already retried such failures can replay +differently; drain affected instances on the previous SDK before upgrading. - Grouped failures now retain nested exception messages, tracebacks, and notes in failure details, including `ExceptionGroup` failures from task callbacks. -Traceback rendering is limited to 15 children per group and 10 group levels, -with truncation markers for omitted diagnostics. Failure details do not -serialize a complete structured tree of group children. -- Optimized set and frozenset serialization, particularly for deeply nested -frozensets and large singleton elements. JSON object names within sets are now -ordered consistently after conversion to strings, including numeric names. -Equivalent surrogate-pair and non-BMP object names retain the last value -rather than emitting duplicate JSON properties. -Set reconstruction remains opt-in. -- With set reconstruction enabled, annotated sets of typed tuples are now -recognized during input and activity-result discovery, and bare -`collections.abc.Set` annotations reconstruct JSON arrays as concrete sets. -- Implemented `__hash__` and `__repr__` for `EntityInstanceId`, making identity attributes immutable to guarantee hash stability when used in sets or dict keys. -- Large-payload externalization now checks every eligible payload's maximum -size before uploading or replacing any protobuf values, including carryover -events and entity payloads, for both synchronous and asynchronous calls. -Oversized messages remain unchanged and cause no uploads; their size error -now takes precedence over hypothetical earlier storage failures. +- Improved set and frozenset serialization performance and consistent JSON +object-key handling. +- `EntityInstanceId` is now hashable and has a diagnostic `repr`, allowing use +in sets and as dictionary keys. Entity IDs are now immutable. +- Oversized payloads now fail validation before any payload uploads or +replacements occur. - Orchestrations that exceed the configured external payload storage limit now fail without retrying and release their entity locks, allowing other callers to progress. Transient payload-store failures remain retryable. -- Previously unhandled non-cancellation `BaseException` exits, including result -serialization and generator teardown failures, now fail with a clear reason instead of leaving an accepted dispatch -unfinished. Cancellation and process-control signals retain their semantics. -The gRPC worker abandons work that exits before an outcome is sent and resets -the originating work-item stream after an uncertain completion or abandonment -RPC, allowing scheduler ownership recovery without retrying an uncertain outcome. +- Unexpected orchestration execution, result serialization, and teardown +failures no longer leave accepted work unfinished. Cancellation and +process-control signals retain their semantics. - Orchestrator generators are now closed at the end of each dispatch, including -while waiting for durable work, releasing resources held by async bridges. -`finally` blocks and context-manager exits can run before orchestration completion -and repeat across dispatches; see [cleanup semantics](docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks). +while waiting for durable work. `finally` blocks and context-manager exits can +run before orchestration completion and repeat across dispatches; see +[cleanup semantics](docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks). - Async cancellation raised by generator teardown no longer replaces dispatch responses or version-rejection abandonment. diff --git a/README.md b/README.md index d37c5d97..fb947a98 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,47 @@ The Azure Functions Durable provider requires Python 3.13 or later. ## Optional Features +### Task Hub Query Filters + +Set `OrchestrationQuery.task_hub_names` to filter orchestration queries by one or +more task hubs. The filter is available through synchronous and asynchronous +core and Azure Managed clients; filtering support depends on the backend. + +```python +from durabletask.client import OrchestrationQuery + +query = OrchestrationQuery(task_hub_names=["orders", "billing"]) +``` + +### Set Serialization + +The default `JsonDataConverter` serializes sets and frozensets as JSON arrays +ordered by their canonical JSON representation, not numeric order. For example, +`{2, 10}` serializes as `[10, 2]`. To reconstruct annotated sets and frozensets +instead of receiving lists, configure the same converter on clients and workers: + +```python +from durabletask.serialization import JsonDataConverter + +converter = JsonDataConverter(enable_generic_sets=True) +``` + +Pass it using the `data_converter` parameter on core or Azure Managed clients +and workers. `enable_set_coercion=True` is an alias for the same option. +Azure Functions uses its own shared codec rather than this converter. + +> [!WARNING] +> Set reconstruction is disabled by default to preserve the order and duplicates +> of recorded JSON arrays. Enabling it for existing orchestrations can change +> replay behavior. Iteration over reconstructed sets is not replay-safe; sort +> values explicitly before using their order to schedule durable work. + +### Entity IDs + +`EntityInstanceId` values are immutable, hashable, and usable as dictionary keys +or set elements. Construct a new ID to change an entity name or key; attribute +assignment and deletion raise `AttributeError`. + ### Large Payload Externalization Install the `azure-blob-payloads` extra to automatically offload oversized orchestration payloads to diff --git a/azure-functions-durable/CHANGELOG.md b/azure-functions-durable/CHANGELOG.md index 90636829..4b808272 100644 --- a/azure-functions-durable/CHANGELOG.md +++ b/azure-functions-durable/CHANGELOG.md @@ -7,49 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## Unreleased +## v2.0.0rc4 + ADDED +- Added inherited `OrchestrationQuery.task_hub_names` support to filter +orchestration instances by one or more task hubs. - Added `DFApp.configure_serialization(ensure_ascii=False)` to opt in to compact -Unicode JSON across durable clients, orchestration and entity workers, activity -bindings, and registered blueprints. The default remains ASCII-escaped JSON. -The opt-in uses the shared codec's `ensure_ascii` option available in -`azure-functions` v2.4.0b1; unsupported SDKs fail explicitly at configuration time. +Unicode JSON for durable payloads. The default remains ASCII-escaped JSON. CHANGED +- Updated the minimum `durabletask` dependency to v1.13.0. - Updated the minimum `azure-functions` dependency to v2.4.0b1. -- With the corresponding core SDK update, client and worker context managers -preserve Functions subclass types in type checkers. Python 3.13 remains the -minimum supported version for this package. -- With the corresponding core SDK update, multiple task completion callback -failures raise the built-in `ExceptionGroup`; a single failure is still raised -directly. The first callback error remains the explicit cause and serialized -inner failure. Branches on the previous aggregate exception type or message -can change during replay; see the [core upgrade warning](../CHANGELOG.md#unreleased). +- Client and worker context managers now preserve Functions subclass types in +type checkers. Python 3.13 remains the minimum supported version. +- Multiple task completion callback failures now raise the built-in +`ExceptionGroup`; a single failure is still raised directly. Branches on the +previous aggregate type or message can change during replay; see the +[core upgrade warning](../CHANGELOG.md#v1130). FIXED -- With the corresponding core SDK update, activity and sub-orchestration retry -policies, including compatibility `RetryOptions`, honor -`failureDetails.isNonRetriable` and surface the failure immediately without retrying. -Histories that already retried such failures can replay differently; drain -affected instances on the previous SDK before upgrading. -- With the corresponding core SDK update, grouped failures retain nested -exception messages, tracebacks, and notes in failure details. Tracebacks are -limited to 15 children per group and 10 group levels with truncation markers; -group children are not serialized as a complete structured tree. -- With the corresponding core `durabletask` SDK fix, oversized messages fail -large-payload validation before any uploads or payload replacements. Size -errors now take precedence over hypothetical earlier storage failures. -- With the corresponding core `durabletask` SDK fix, previously unhandled non-cancellation -`BaseException` exits from orchestration execution, result serialization, or teardown now produce a failed response -with the exception's type, message, and stack trace instead of losing the response. -- With the corresponding core `durabletask` SDK fix, orchestrator generators -are closed at the end of each dispatch, including while waiting for durable work. -See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks) -for repeated finalizers, discarded teardown actions/status, and yielding restrictions. -- With the corresponding core `durabletask` SDK fix, async cancellation raised by -generator teardown no longer replaces dispatch responses or version-rejection abandonment. +- `EntityInstanceId` and the compatibility `EntityId` are now immutable and +hashable, with a diagnostic `repr`. +- Activity and sub-orchestration retry policies, including compatibility +`RetryOptions`, now honor `failureDetails.isNonRetriable`. Histories that already +retried such failures can replay differently; drain affected instances on the +previous SDK before upgrading. +- Grouped failures now retain nested exception messages, tracebacks, and notes. +- Oversized payloads now fail validation before any uploads or replacements. +- Unexpected orchestration execution, result serialization, and teardown +failures now produce a failed response instead of losing the response. +- Orchestrator generators are now closed at each dispatch boundary, so +`finally` blocks and context-manager exits may run repeatedly before completion. +See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks). +- Async cancellation during generator teardown no longer replaces dispatch +responses or version-rejection abandonment. ## v2.0.0rc3 diff --git a/azure-functions-durable/README.md b/azure-functions-durable/README.md index 66caf2ce..9bf0d77b 100644 --- a/azure-functions-durable/README.md +++ b/azure-functions-durable/README.md @@ -32,6 +32,14 @@ Key capabilities include durable orchestrations and sub-orchestrations, durable timers, external events, durable entities, retries, versioning, durable HTTP calls (`context.call_http(...)`), recurring scheduled tasks, and history export. +`DurableFunctionsClient` and `SyncDurableFunctionsClient` inherit +`OrchestrationQuery.task_hub_names` for filtering orchestration queries by one +or more task hubs. Filtering support depends on the backend. + +`EntityInstanceId` and the compatibility `EntityId` are immutable and hashable, +so they can be used as dictionary keys or set elements. Construct a new ID +instead of assigning or deleting attributes on an existing ID. + ## Unicode JSON serialization By default, durable payloads preserve the shared `azure-functions` codec's diff --git a/azure-functions-durable/pyproject.toml b/azure-functions-durable/pyproject.toml index 5cdecffc..5fbc2696 100644 --- a/azure-functions-durable/pyproject.toml +++ b/azure-functions-durable/pyproject.toml @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta" [project] name = "azure-functions-durable" -version = "2.0.0rc3" +version = "2.0.0rc4" description = "Durable Task Python SDK provider implementation for Durable Azure Functions" keywords = [ "durable", @@ -30,7 +30,7 @@ requires-python = ">=3.13" license = {file = "LICENSE"} readme = "README.md" dependencies = [ - "durabletask[opentelemetry]>=1.12.0", + "durabletask[opentelemetry]>=1.13.0", "azure-identity>=1.19.0", "azure-functions>=2.4.0b1" ] diff --git a/durabletask-azuremanaged/CHANGELOG.md b/durabletask-azuremanaged/CHANGELOG.md index 43812433..261117be 100644 --- a/durabletask-azuremanaged/CHANGELOG.md +++ b/durabletask-azuremanaged/CHANGELOG.md @@ -7,44 +7,46 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## Unreleased +## v1.13.0 + +ADDED + +- Added inherited `OrchestrationQuery.task_hub_names` support to filter +orchestration instances by one or more task hubs. +- Added inherited deterministic set and frozenset serialization. +Generic set reconstruction is opt-in through `JsonDataConverter`; enabling it +for existing orchestrations can change replay behavior. + CHANGED +- Updated the minimum `durabletask` dependency to v1.13.0, including the +`azure-blob-payloads` extra. - Dropped Python 3.10 support. Python 3.11 or later is now required. -- With the corresponding core SDK update, client and worker context managers -preserve Azure Managed subclass types in type checkers. -- With the corresponding core SDK update, multiple task completion callback -failures raise the built-in `ExceptionGroup`; a single failure is still raised -directly. The first callback error remains the explicit cause and serialized -inner failure. Branches on the previous aggregate exception type or message -can change during replay; see the [core upgrade warning](../CHANGELOG.md#unreleased). +- Client and worker context managers now preserve Azure Managed subclass types +in type checkers. +- Multiple task completion callback failures now raise the built-in +`ExceptionGroup`; a single failure is still raised directly. Branches on the +previous aggregate type or message can change during replay; see the +[core upgrade warning](../CHANGELOG.md#v1130). FIXED -- With the corresponding core SDK update, activity and sub-orchestration retry -policies honor `failureDetails.isNonRetriable` and surface the failure immediately -without retrying. Histories that already retried such failures can replay -differently; drain affected instances on the previous SDK before upgrading. -- With the corresponding core SDK update, grouped failures retain nested -exception messages, tracebacks, and notes in failure details. Tracebacks are -limited to 15 children per group and 10 group levels with truncation markers; -group children are not serialized as a complete structured tree. -- With the corresponding core `durabletask` SDK fix, oversized messages fail -large-payload validation before any uploads or payload replacements. Size -errors now take precedence over hypothetical earlier storage failures. -- With the corresponding core `durabletask` SDK fix, orchestrations that exceed -the configured external payload storage limit fail without retrying and release -their entity locks. Transient payload-store failures remain retryable. -- With the corresponding core `durabletask` SDK fix, previously unhandled -non-cancellation `BaseException` exits, including result serialization failures, -now fail with a clear reason. Cancellation and process-control -exits recover dispatch ownership, and uncertain completion or abandonment RPCs -reset the originating work-item stream rather than retaining unfinished work. -- With the corresponding core `durabletask` SDK fix, orchestrator generators -are closed at the end of each dispatch, including while waiting for durable work. -See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks) -for repeated finalizers, discarded teardown actions/status, and yielding restrictions. -- With the corresponding core `durabletask` SDK fix, async cancellation raised by -generator teardown no longer replaces dispatch responses or version-rejection abandonment. +- `EntityInstanceId` is now immutable and hashable, with a diagnostic `repr`. +- Activity and sub-orchestration retry policies now honor +`failureDetails.isNonRetriable`. Histories that already retried such failures +can replay differently; drain affected instances on the previous SDK before +upgrading. +- Grouped failures now retain nested exception messages, tracebacks, and notes. +- Oversized payloads now fail validation before any uploads or replacements. +Orchestrations exceeding the external payload storage limit fail without +retrying and release entity locks; transient payload-store failures remain +retryable. +- Unexpected orchestration failures no longer leave accepted work unfinished. +- Orchestrator generators are now closed at each dispatch boundary, so +`finally` blocks and context-manager exits may run repeatedly before completion. +See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks). +- Async cancellation during generator teardown no longer replaces dispatch +responses or version-rejection abandonment. ## v1.12.0 diff --git a/durabletask-azuremanaged/pyproject.toml b/durabletask-azuremanaged/pyproject.toml index 4e72a0c8..7ece65dd 100644 --- a/durabletask-azuremanaged/pyproject.toml +++ b/durabletask-azuremanaged/pyproject.toml @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta" [project] name = "durabletask.azuremanaged" -version = "1.12.0" +version = "1.13.0" description = "Durable Task Python SDK provider implementation for the Azure Durable Task Scheduler" keywords = [ "durable", @@ -31,13 +31,13 @@ requires-python = ">=3.11" license = {file = "LICENSE"} readme = "README.md" dependencies = [ - "durabletask>=1.12.0", + "durabletask>=1.13.0", "azure-identity>=1.19.0" ] [project.optional-dependencies] azure-blob-payloads = [ - "durabletask[azure-blob-payloads]>=1.12.0" + "durabletask[azure-blob-payloads]>=1.13.0" ] [project.urls] diff --git a/pyproject.toml b/pyproject.toml index 4d3ca9f8..774e7571 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta" [project] name = "durabletask" -version = "1.12.0" +version = "1.13.0" description = "A Durable Task Client SDK for Python" keywords = [ "durable",