Skip to content

fix(terraform): select module callers, not every stack [NOJIRA] - #345

Merged
tobias0106 merged 1 commit into
mainfrom
fix/terraform-discover-module-deps
Sep 15, 2026
Merged

tobias0106 merged 1 commit into
mainfrom
fix/terraform-discover-module-deps

Conversation

@tobias0106

Copy link
Copy Markdown
Contributor

Draft — the fan-out issue from #341, fixed for the case that actually hurts.

The problem

A changed file under modules/ is owned by no stack, so discovery falls back to selecting every stack. On terraform-core that means a one-line change to modules/route53 runs 11 plans and, on merge, 11 applies:

uses the module
accounts/{internal,monta,production,staging}/route53 yes 4
accounts/{internal,staging}/iam no selected anyway
accounts/{internal,staging}/s3 no selected anyway
accounts/{internal,production,staging}/vpc-endpoints no selected anyway

The seven extras are no-ops if they are clean. If one has drifted, an unrelated module PR is what applies that drift, under a review that never mentioned it.

Why not the nearest-ancestor fix

#341 proposed scoping fan-out to the changed file's nearest ancestor directory. That is right for terraform-db-access, where _shared-users-*.tf sits above the stacks it affects. It is wrong here: modules/route53 has no stacks underneath it, so it would select zero and silently skip the four stacks that do need replanning. Worse than fanning out.

What this does instead

Terraform already declares the edges — every source = "../../modules/route53" is a dependency. Discovery now reads them:

  1. For each stack, resolve its local source = "..." paths relative to the file that declares them.
  2. Take the transitive closure, so stack -> modules/a -> modules/b means a change to modules/b selects that stack.
  3. A change under modules/X/** selects every stack whose closure contains it.
  4. A change inside a stack resolves by the existing walk-up rule, untouched.
  5. A path that is neither a stack nor Terraform code — a repository root file, .github/** — still fans out to everything.

Nothing to declare, nothing to maintain, and it stays correct when a module is added or wired into a new stack. Registry and git sources are ignored; only paths inside the repository count.

Verified against all three repos

Run against the real trees, terraform-core:

changed path paths-filter today before this PR after
modules/route53/** 4 11 4
modules/vpc-endpoints/** 3 11 3
modules/iam-role/** 1 11 1
modules/s3-bucket/**, modules/vpc/** 0 11 0
accounts/internal/route53/zones/monta-me/records.tf 1 1 1
.github/**, root files 11 11

It reproduces the hand-maintained paths-filter mapping exactly, without the list.

monta-app/terraform: modules/cloudflare/tunnel/** now selects global/cloudflare alone instead of Grafana and both ClickHouse stacks — the second example named in #341.

The zero rows are a real finding, not a rounding error: modules/s3-bucket and modules/vpc are called by nothing. accounts/{staging,internal}/s3 contain only a backend.tf, a versions.tf and a comment naming the module they were meant to use.

Still to do for terraform-db-access

This does not cover _shared-users-*.tf in an environment root — those are not modules, and no stack references them. That repo still needs the nearest-ancestor rule, as its own change, alongside the tailscale and parallelism inputs it is waiting on.

Notes for review

The resolver is ~80 lines of Python written to $RUNNER_TEMP by a preceding step. It has to be inline: a reusable workflow checks out the caller's repository, so a script committed here would not be on disk when it runs. There is no test harness for these workflows, so the verification above was run against real checkouts of terraform-core and terraform, using the script extracted back out of the YAML.

🤖 Generated with Claude Code

A change under modules/ was owned by no stack, so discovery fell back to
selecting all of them. In terraform-core that turns a modules/route53
change into 11 plans and, on merge, 11 applies - the 7 unrelated stacks
apply whatever their plan happened to contain, which for a drifted stack
means applying that drift under a review that never mentioned it.

Terraform already declares these edges. Discovery now reads the local
source = "..." paths, takes the transitive closure, and selects the
stacks that actually call the changed module. Paths that are neither a
stack nor Terraform code still fan out.

terraform-core: route53 11 -> 4, vpc-endpoints 11 -> 3, iam-role 11 -> 1,
and the two modules nothing calls 11 -> 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tobias0106
tobias0106 marked this pull request as ready for review September 15, 2026 11:37
@tobias0106
tobias0106 requested a review from a team as a code owner September 15, 2026 11:37
@tobias0106
tobias0106 requested review from prasad-manu and removed request for a team September 15, 2026 11:37
@tobias0106
tobias0106 merged commit 559ec06 into main Sep 15, 2026
1 check passed
@tobias0106
tobias0106 deleted the fix/terraform-discover-module-deps branch September 15, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants