Add Python modernization and deprecation guardrails

.github/copilot-instructions.md

@@ -55,6 +55,20 @@ Examples:
 - Follow PEP 8 conventions.
 - Use `autopep8` for Python formatting.
 
+## Python Modernization
+
+- Use modern Python syntax, typing, and standard-library APIs compatible with
+  each package's minimum supported Python version.
+- Do not introduce APIs deprecated in any supported Python version. Nearby
+  legacy code is not precedent for new code.
+- Preserve public API compatibility and runtime annotation behavior when
+  modernizing.
+- Resolve deprecation warnings introduced by your changes. Any suppression
+  must be narrowly scoped and justified.
+- Ruff's `UP` rules complement Flake8 and `autopep8`; they do not detect every
+  deprecated API. Temporary rule exclusions in `pyproject.toml` track existing
+  cleanup work, not preferred patterns for new code.
+
 ## Public API Compatibility
 
 - Treat adding an abstract method or property to a public ABC as a breaking
@@ -91,7 +105,9 @@ priority over style.
 ## Python Linting
 
 This repository uses [flake8](https://flake8.pycqa.org/) for Python
-linting. Run it after making changes to verify there are no issues. Lint
+linting and Ruff's `UP` rules for modernization checks. Run `nox -s lint`
+for both checks. Ruff targets Python 3.11 for core/Managed and Python 3.13
+for Functions, including their tests and examples. Lint
 package source and its tests separately, matching CI:
 
 ```bash
@@ -103,6 +119,9 @@ python -m flake8 azure-functions-durable
 python -m flake8 tests/azure-functions-durable
 ```
 
+For a focused modernization check, run `python -m ruff check` with the changed
+paths. Run from the repository root to use the shared configuration.
+
 ## Markdown Style
 
 Use GitHub-style callouts for notes, warnings, and tips in Markdown files:
@@ -172,6 +191,15 @@ python -m pytest tests/azure-functions-durable \
   -m "not dts and not azurite and not functions_e2e"
 ```
 
+The root pytest configuration treats `DeprecationWarning` and
+`PendingDeprecationWarning` as errors. Nox keeps Python 3.13 as the strict
+deprecation gate for every package and reports these warnings without failing
+on the other supported versions. Keep the full compatibility matrix; do not
+raise package minimum versions just to satisfy modernization suggestions.
+Tests deliberately exercising deprecated APIs should assert their warnings
+with `pytest.warns`. Known third-party exceptions must be scoped by warning
+category, message, and originating module where practical, with a reason.
+
 Run Azure Functions Durable E2E tests through Nox, not directly from the root
 virtual environment. Nox creates an isolated Python 3.13 session environment,
 installs the local packages editable, and links it into each sample Function

dev-requirements.txt

@@ -2,3 +2,4 @@ grpcio-tools
 nox
 pymarkdownlnt
 pyright
+ruff

docs/development.md

@@ -29,6 +29,51 @@ Azure Functions unit tests cover Python 3.13--3.15; use
 Nox fails rather than silently skipping a session when its required Python
 interpreter is unavailable.
 
+### Modernization and deprecation checks
+
+`nox -s lint` runs Flake8 and Ruff's `UP` modernization rules; `autopep8`
+remains the formatter. Ruff uses Python 3.11 for core/Managed code and Python
+3.13 for Functions code, including their respective tests and examples.
+Generated protobuf files are excluded, including when passed explicitly.
+For a focused check, run from the repository root:
+
+```sh
+python -m ruff check durabletask/client.py
+```
+
+The temporary typing and UTC rule exclusions in `pyproject.toml` are tracked
+in [#322](https://github.com/microsoft/durabletask-python/issues/322) and
+[#323](https://github.com/microsoft/durabletask-python/issues/323).
+Behavior-sensitive generator and enum suggestions have file-specific
+exceptions tracked in
+[#324](https://github.com/microsoft/durabletask-python/issues/324).
+Excluded rules do not reject new occurrences until their cleanup is complete;
+they are not recommendations for new code.
+
+The root pytest configuration treats `DeprecationWarning` and
+`PendingDeprecationWarning` as errors and explicitly uses function-scoped event
+loops for async fixtures. Nox uses Python 3.13 as the representative
+strict deprecation gate for all three packages. On other supported versions,
+Nox reports these warnings without failing, preserving compatibility coverage
+without requiring APIs unavailable at the package's minimum version.
+Run the strict sessions with:
+
+```sh
+nox -s core_tests-3.13 azuremanaged_tests-3.13 functions_unit-3.13
+```
+
+Direct pytest runs from the repository root are strict on every interpreter.
+Tests intentionally exercising deprecated APIs should use `pytest.warns`.
+Any known third-party warning exception must be narrowly scoped by category,
+message, and originating module where practical, and include a reason.
+Two upstream Functions loose-codec warnings remain visible but non-fatal
+while legacy payload support is required; their removal is tracked in
+[#325](https://github.com/microsoft/durabletask-python/issues/325).
+Warnings raised in separately launched Functions workers are not governed by
+the parent pytest process's filters.
+
+### Services and focused validation
+
 Nox starts Azurite automatically for the core and Azure Functions tests. The
 Azure Managed tests start a disposable DTS emulator Docker container. Start
 Docker before running that session. Azure Functions end-to-end tests also

noxfile.py

@@ -137,6 +137,16 @@ def _pytest_arguments(
     return tuple(posargs) if has_target else (default_target, *posargs)
 
 
+def _pytest_warning_arguments(session: nox.Session) -> tuple[str, ...]:
+    """Keep Python 3.13 as the strict deprecation gate across packages."""
+    if session.python == "3.13":
+        return ()
+    return (
+        "-W", "default::DeprecationWarning",
+        "-W", "default::PendingDeprecationWarning",
+    )
+
+
 def _start_azurite(
     session: nox.Session,
     ports: Sequence[int],
@@ -280,8 +290,8 @@ def _stop_dts_emulator(container_name: str) -> None:
 
 @nox.session
 def lint(session: nox.Session) -> None:
-    """Run the Flake8 checks required by the GitHub Actions workflows."""
-    session.install("flake8")
+    """Run Flake8 and Ruff modernization checks without replacing formatting."""
+    session.install("flake8", "ruff")
     targets = session.posargs or (
         "durabletask",
         "durabletask-azuremanaged",
@@ -290,8 +300,10 @@ def lint(session: nox.Session) -> None:
         "tests/durabletask-azuremanaged",
         "tests/azure-functions-durable",
         "examples",
+        "noxfile.py",
     )
     session.run("flake8", *targets)
+    session.run("ruff", "check", *targets)
 
 
 @nox.session(python=["3.11"])
@@ -413,6 +425,7 @@ def core_tests(session: nox.Session) -> None:
         arguments = _pytest_arguments("tests/durabletask", session.posargs)
         session.run(
             "pytest",
+            *_pytest_warning_arguments(session),
             *arguments,
             "-m",
             "not dts",
@@ -441,7 +454,10 @@ def azuremanaged_tests(session: nox.Session) -> None:
             "tests/durabletask-azuremanaged",
             session.posargs,
         )
-        session.run("pytest", *arguments, "-m", "dts", "--verbose")
+        session.run(
+            "pytest", *_pytest_warning_arguments(session),
+            *arguments, "-m", "dts", "--verbose",
+        )
     finally:
         _stop_dts_emulator(container_name)
 
@@ -454,7 +470,7 @@ def functions_unit(session: nox.Session) -> None:
     session.install("pytest")
     arguments = _pytest_arguments("tests/azure-functions-durable", session.posargs)
     session.run(
-        "pytest", *arguments,
+        "pytest", *_pytest_warning_arguments(session), *arguments,
         "-m", "not dts and not azurite and not functions_e2e",
         "--verbose",
     )
@@ -508,7 +524,7 @@ def functions_e2e(session: nox.Session) -> None:
             session.posargs,
         )
         session.run(
-            "pytest", *arguments,
+            "pytest", *_pytest_warning_arguments(session), *arguments,
             "-m", "functions_e2e",
         )
     finally:

pyproject.toml

@@ -58,7 +58,42 @@ include = ["durabletask", "durabletask.*"]
 minversion = "6.0"
 testpaths = ["tests"]
 asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"
+filterwarnings = [
+    "error::DeprecationWarning",
+    "error::PendingDeprecationWarning",
+    # Legacy Functions payloads still require the upstream loose codec.
+    # Report only these known warnings without failing; see issue #325.
+    'default:azure\.functions df_loads was called without expected_type\.:DeprecationWarning:azure\.functions\._durable_functions$',
+    'default:azure\.functions Durable JSON codec reconstructed a custom object via the loose-mode object_hook path\.:DeprecationWarning:azure\.functions\._durable_functions$',
+]
 markers = [
     "azurite: tests that require Azurite (local Azure Storage emulator)",
     "functions_e2e: end-to-end tests that require the Azure Functions Core Tools (func) and Azurite",
 ]
+
+[tool.ruff]
+target-version = "py311"
+extend-exclude = ["*_pb2.py", "*_pb2_grpc.py", "*_pb2.pyi", "*_pb2_grpc.pyi", ".python_packages"]
+force-exclude = true
+
+[tool.ruff.per-file-target-version]
+"azure-functions-durable/**/*.py" = "py313"
+"tests/azure-functions-durable/**/*.py" = "py313"
+
+[tool.ruff.lint]
+select = ["UP"]
+ignore = [
+    # Typing migration requires runtime annotation compatibility review:
+    # https://github.com/microsoft/durabletask-python/issues/322
+    "UP006", "UP007", "UP035", "UP037", "UP040", "UP043", "UP045", "UP047",
+    # Repository-wide UTC alias cleanup:
+    # https://github.com/microsoft/durabletask-python/issues/323
+    "UP017",
+]
+
+[tool.ruff.lint.per-file-ignores]
+# These suggestions can change generator protocol or public enum behavior:
+# https://github.com/microsoft/durabletask-python/issues/324
+"durabletask/internal/orchestration_entity_context.py" = ["UP028"]
+"durabletask/scheduled/schedule_status.py" = ["UP042"]

tests/azure-functions-durable/test_client_compat.py

@@ -932,7 +932,8 @@ def test_entity_id_pickle_round_trip():
     with pytest.warns(DeprecationWarning):
         entity_id = df.EntityId("Counter", "one")
 
-    restored = pickle.loads(pickle.dumps(entity_id))
+    with pytest.warns(DeprecationWarning, match="EntityId is deprecated"):
+        restored = pickle.loads(pickle.dumps(entity_id))
     assert type(restored) is df.EntityId
     assert isinstance(restored, df.EntityId)
     assert restored.name == "counter"

tests/azure-functions-durable/test_worker_compat.py

@@ -629,10 +629,11 @@ def orchestrator(ctx: task.OrchestrationContext, _):
 def test_activity_retry_then_fan_out_uses_distinct_task_ids():
     """Regression test for Azure/azure-functions-durable-python#603."""
     def orchestrator(context):
-        options = df.RetryOptions(
-            first_retry_interval_in_milliseconds=100,
-            max_number_of_attempts=3,
-        )
+        with pytest.warns(DeprecationWarning, match="RetryOptions is deprecated"):
+            options = df.RetryOptions(
+                first_retry_interval_in_milliseconds=100,
+                max_number_of_attempts=3,
+            )
         yield context.call_activity_with_retry("flaky", options)
         tasks = [context.call_activity("square", value) for value in range(13)]
         return (yield context.task_all(tasks))

tests/durabletask-azuremanaged/test_dts_batch_actions.py

@@ -157,6 +157,8 @@ def emit(self, record):
             log_records.append(record)
 
     handler = ListHandler()
+    logger = logging.Logger("test.azuremanaged.client.pagination")
+    logger.addHandler(handler)
 
     with DurableTaskSchedulerWorker(host_address=endpoint, secure_channel=True,
                                     taskhub=taskhub_name, token_credential=None) as w:
@@ -165,7 +167,7 @@ def emit(self, record):
 
         c = DurableTaskSchedulerClient(host_address=endpoint, secure_channel=True,
                                        taskhub=taskhub_name, token_credential=None,
-                                       log_handler=handler)
+                                       logger=logger)
 
         # Create at least 3 orchestrations to test the limit
         ids = []

tests/durabletask/extensions/history_export/test_activities.py

@@ -352,7 +352,7 @@ def test_blob_name_matches_dotnet_hash_scheme():
     assert _dotnet_o_format(last_updated) == "2026-06-04T17:09:09.4209900+00:00"
 
     expected_hash = hashlib.sha256(
-        f"{_dotnet_o_format(last_updated)}|{instance_id}".encode("utf-8")
+        f"{_dotnet_o_format(last_updated)}|{instance_id}".encode()
     ).hexdigest()
 
     assert _blob_name_for(

tests/durabletask/test_batch_actions.py

@@ -238,14 +238,16 @@ def emit(self, record):
             log_records.append(record)
 
     handler = ListHandler()
+    logger = logging.Logger("test.client.pagination")
+    logger.addHandler(handler)
 
     worker = TaskHubGrpcWorker(host_address=HOST)
 
     worker.add_orchestrator(empty_orchestrator)
     worker.start()
 
     try:
-        with TaskHubGrpcClient(host_address=HOST, log_handler=handler) as c:
+        with TaskHubGrpcClient(host_address=HOST, logger=logger) as c:
             # Create at least 3 orchestrations to test the limit
             ids = []
             for i in range(3):

tests/durabletask/test_orchestration_executor.py

@@ -1044,7 +1044,7 @@ def orchestrator(ctx: task.OrchestrationContext, orchestrator_input):
     registry = worker._Registry()
     name = registry.add_orchestrator(orchestrator)
 
-    current_timestamp = datetime.utcnow()
+    current_timestamp = datetime.now(UTC).replace(tzinfo=None)
     # Simulate the task failing for the first time and confirm that a timer is scheduled for 1 second in the future
     old_events = [
         helpers.new_orchestrator_started_event(timestamp=current_timestamp),
@@ -1222,7 +1222,7 @@ def orchestrator(ctx: task.OrchestrationContext, orchestrator_input):
 
     registry = worker._Registry()
     name = registry.add_orchestrator(orchestrator)
-    current_timestamp = datetime.utcnow()
+    current_timestamp = datetime.now(UTC).replace(tzinfo=None)
     old_events = [
         helpers.new_orchestrator_started_event(timestamp=current_timestamp),
         helpers.new_execution_started_event(name, TEST_INSTANCE_ID, encoded_input=None),
@@ -1281,7 +1281,7 @@ def orchestrator(ctx: task.OrchestrationContext, orchestrator_input):
     registry = worker._Registry()
     name = registry.add_orchestrator(orchestrator)
 
-    current_timestamp = datetime.utcnow()
+    current_timestamp = datetime.now(UTC).replace(tzinfo=None)
 
     # Simulate the task failing for the first time — retry timer should be created at 1 second
     old_events = [
@@ -1370,7 +1370,7 @@ def orchestrator(ctx: task.OrchestrationContext, _):
     registry = worker._Registry()
     name = registry.add_orchestrator(orchestrator)
 
-    current_timestamp = datetime.utcnow()
+    current_timestamp = datetime.now(UTC).replace(tzinfo=None)
 
     # First failure — retry timer at 5 seconds (default backoff=1.0, so 5 * 1^0 = 5)
     old_events = [
@@ -1433,7 +1433,7 @@ def orchestrator(ctx: task.OrchestrationContext, orchestrator_input):
     name = registry.add_orchestrator(orchestrator)
     executor = worker._OrchestrationExecutor(registry, TEST_LOGGER, JsonDataConverter())
 
-    start = datetime.utcnow()
+    start = datetime.now(UTC).replace(tzinfo=None)
     t1 = start + timedelta(days=3)
     t2 = start + timedelta(days=6)
     t3 = start + timedelta(days=9)
@@ -1804,7 +1804,7 @@ def test_nondeterminism_expected_sub_orchestration_task_completion_wrong_task_ty
     This variation tests the case where the expected task type is wrong (e.g. the code schedules a timer task
     but the history contains a sub-orchestration completed task)."""
     def orchestrator(ctx: task.OrchestrationContext, _):
-        result = yield ctx.create_timer(datetime.utcnow())  # created timer but history expects sub-orchestration
+        result = yield ctx.create_timer(datetime.now(UTC).replace(tzinfo=None))  # created timer but history expects sub-orchestration
         return result
 
     registry = worker._Registry()
@@ -2084,7 +2084,7 @@ def orchestrator(ctx: task.OrchestrationContext, _):
 
     # Complete the timer task. The orchestration should move to the wait_for_external_event step, which
     # should then complete immediately because the event was buffered in the old event history.
-    timer_due_time = datetime.utcnow() + timedelta(days=1)
+    timer_due_time = datetime.now(UTC).replace(tzinfo=None) + timedelta(days=1)
     old_events = new_events + [helpers.new_timer_created_event(1, timer_due_time)]
     new_events = [helpers.new_timer_fired_event(1, timer_due_time)]
     executor = worker._OrchestrationExecutor(registry, TEST_LOGGER, JsonDataConverter())
@@ -2173,9 +2173,9 @@ def orchestrator(ctx: task.OrchestrationContext, input: int):
         helpers.new_event_raised_event("my_event", encoded_input="42"),
         helpers.new_event_raised_event("my_event", encoded_input="43"),
         helpers.new_event_raised_event("my_event", encoded_input="44"),
-        helpers.new_timer_created_event(1, datetime.utcnow() + timedelta(days=1))]
+        helpers.new_timer_created_event(1, datetime.now(UTC).replace(tzinfo=None) + timedelta(days=1))]
     new_events = [
-        helpers.new_timer_fired_event(1, datetime.utcnow() + timedelta(days=1))]
+        helpers.new_timer_fired_event(1, datetime.now(UTC).replace(tzinfo=None) + timedelta(days=1))]
 
     # Execute the orchestration. It should be in a running state waiting for the timer to fire
     executor = worker._OrchestrationExecutor(registry, TEST_LOGGER, JsonDataConverter())
@@ -3180,7 +3180,7 @@ def orchestrator(ctx: task.OrchestrationContext, _):
     orchestrator_name = registry.add_orchestrator(orchestrator)
     registry.add_activity(dummy_activity)
 
-    current_timestamp = datetime.utcnow()
+    current_timestamp = datetime.now(UTC).replace(tzinfo=None)
     # Simulate the task failing for the first time and confirm that a timer is scheduled for 1 second in the future
     old_events = [
         helpers.new_orchestrator_started_event(timestamp=current_timestamp),
@@ -3268,7 +3268,7 @@ def orchestrator(ctx: task.OrchestrationContext, _):
     orchestrator_name = registry.add_orchestrator(orchestrator)
     registry.add_activity(dummy_activity)
 
-    current_timestamp = datetime.utcnow()
+    current_timestamp = datetime.now(UTC).replace(tzinfo=None)
     # Simulate the task failing for the first time and confirm that a timer is scheduled for 2 seconds in the future
     old_events = [
         helpers.new_orchestrator_started_event(timestamp=current_timestamp),