Skip to content

feat: add per-invocation executor isolation - #735

Draft
zhongkechen wants to merge 3 commits into
fix/lmi-invocation-lifecyclefrom
feature/invocation-executor-isolation
Draft

zhongkechen wants to merge 3 commits into
fix/lmi-invocation-lifecyclefrom
feature/invocation-executor-isolation

Conversation

@zhongkechen

Copy link
Copy Markdown
Contributor

Issue Link, if available

Partial fix for #726, covering LMI cross-invocation executor isolation. Stacked on #733. Arbitrarily nested bounded-pool starvation remains a separate baseline concern.

Description

This adds an explicit, non-breaking way to give each Lambda invocation its own user-code executor while preserving the existing shared-executor contract.

  • Add DurableConfig.withInvocationExecutorFactory(...). The factory may be called concurrently on Lambda Managed Instances and must return a fresh, non-null, non-shutdown ExecutorService for each invocation.
  • Keep withExecutorService(...) unchanged: its executor remains caller-owned, shared by every invocation using the config, and is never shut down by the SDK. Configuring both options is rejected.
  • Acquire the invocation executor only after execution state has been initialized, route root/step/child/coordinator tasks through it, and reject an executor identity already leased to another active invocation.
  • Drain invocation-owned tasks first, then shut down and await the factory-created executor within the cleanup deadline established by fix: bound invocation lifecycle and cleanup #733. Use shutdownNow() only for the SDK-owned executor when tasks or termination remain; never apply it to a shared executor.
  • Preserve the invocation factory when LocalDurableTestRunner copies customer configuration, so every replay/resume gets a new executor and closes it before returning.
  • Update the LMI fixed2 fixture to use invocation-local fixed(2) pools. Keep the existing nested2 shared-fixed resource for baseline diagnostics but remove it from LMI pass/fail gating.
  • Document ownership, concurrency, factory freshness, and the distinction between LMI isolation and nested-capacity requirements.

This PR targets fix/lmi-invocation-lifecycle, the head branch of #733. It should be retargeted to main after #733 lands.

Compatibility

  • Existing withExecutorService() applications retain the same threads, sharing, and lifecycle behavior.
  • Isolation is opt-in; user code still includes root handlers, steps, and child contexts.
  • The SDK owns only executors returned by withInvocationExecutorFactory().
  • Per-invocation isolation prevents concurrent LMI roots from consuming one another's capacity. It does not claim that a fixed-size executor can support arbitrary synchronous nesting depth.

Testing

  • mvn test — all 10 reactor modules passed. The SDK ran 1,185 tests with 3 default-skipped opt-in tests; sdk-testing ran 15, integration tests ran 405, OTel ran 220, Insight ran 144, and examples ran 103 with 31 cloud tests disabled.
  • mvn -pl sdk,sdk-testing,lmi-tests spotless:check — passed.
  • mvn -pl lmi-tests -am -DskipTests package — LMI shaded fixture built successfully.
  • mvn -pl sdk -Dtest=LmiLifecycleRegressionTest -Dtest.lmi.regressions.enabled=true test — 3/3 passed, including two concurrent roots using isolated fixed(2) executors.
  • python3 -m unittest discover -s lmi-tests/tests — 41/41 passed.
  • Javadoc generation passed with the two pre-existing WaitForConditionOperation link warnings.

Cloud LMI execution was not run locally because it requires the configured persistent capacity provider. The updated fixed2 fixture is wired into #728's opt-in cloud workflow.

Checklist

  • Shared executor behavior remains backward compatible.
  • Factory-created executors are drained and shut down before response return.
  • Concurrent reuse of the same executor identity is rejected.
  • Keeps the PR in Draft state for stacked review.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant