Repository navigation
Pin GitHub Actions to commit SHAs and add Dependabot cooldown/grouping - #10009
Merged
Merged
Conversation
Pin repository workflows, composite actions, and the generated Copilot workflow template to immutable commits. Configure Dependabot to delay GitHub Actions updates for seven days. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expand Dependabot coverage to nonstandard action manifests and pin the actions emitted by azd pipeline config. Regenerate the affected pipeline snapshots. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Load generated workflow action pins from a Dependabot-managed YAML manifest and validate action repositories and full commit SHAs before rendering. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… 7-day cooldown and combined minor/patch PR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…m with 7-day cooldown and combined minor/patch PR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hemarina
requested review from
Rick Winter (RickWinter),
Daniel Jurek (danieljurek),
Richard Park (richardpark-msft),
tg-msft and
Victor Vazquez (vhvb1989)
as code owners
September 11, 2026 20:30
|
Azure Pipelines: 22 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
…bot-npm-cooldown-and-groups
hemarina
marked this pull request as draft
September 11, 2026 20:35
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Add cli/azd/test/evals to the npm configuration.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds Dependabot npm coverage, seven-day cooldowns, and grouped updates for GitHub Actions and npm dependencies.
Changes:
- Adds npm configuration for five project directories.
- Groups GitHub Actions and npm minor/patch updates.
- Applies seven-day cooldowns.
File summaries
| File | Summary |
|---|---|
.github/dependabot.yml |
Adds Dependabot policies, but omits the active cli/azd/test/evals npm project from coverage. |
Review details
- Files reviewed: 34/34 changed files
- Comments generated: 1
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
hemarina
changed the title
claude-sonnet-4.6 is unavailable in this CI environment (session.create fails with 'Model "claude-sonnet-4.6" is not available.'), causing all vally eval jobs to error out. Point the CI-run eval specs at gpt-5.6-terra instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Addresses review feedback: this is an active npm project (own package.json/package-lock.json) installed by the vally CI workflows, so it should get the same 7-day cooldown and grouped minor/patch updates as the other npm directories. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved findings cover pipeline-manifest integration and validation, an outdated setup pin, a skipped preflight check, and undocumented eval-model changes.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (4)
.github/skills/azd-preflight/SKILL.md:41
- This exit criterion is not wired into the skill workflow:
references/workflow.mdreturns success immediately after the ninemage preflightchecks pass, and the new fix strategy is only loaded after a check fails. A manifest-only change with a clean preflight therefore skips the required action-pin validation/synchronization; add a conditional manifest check before the success path and after reruns.
- When the generated pipeline action manifest changes, `azure-dev.ymlt` and its snapshots are synchronized
cli/azd/extensions/azure.coding-agent/internal/cmd/templates/copilot-setup-steps.yml:17
- Azure/login v3.1.0 is the latest stable release as of 2026-09-10, but this line pins v3.0.2. That contradicts the PR's latest-stable claim; update this pin to the v3.1.0 commit and keep the generated/template copies consistent.
uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3.0.2
cli/azd/resources/pipeline/.github/workflows/azure-dev-actions.yml:13
generatePipelineDefinitionstill parses onlyazure-dev.ymlt(cli/azd/pkg/pipeline/pipeline_manager.go:1278-1284); it never reads this new manifest. A future Dependabot update will therefore leave generated workflows and snapshots on the old SHA, so wire the manifest into rendering/validation or add a CI drift check.
cli/azd/resources/pipeline/.github/workflows/azure-dev-actions.yml:13- This manifest is outside
.github/**, but the repository's actionlint workflow triggers on and sparse-checks out only.github, so this Dependabot source is never validated in CI. Add this path to the validation (or an explicit actionlint invocation) so a malformed manifest cannot silently break generated-action updates.
- Files reviewed: 39/39 changed files
- Comments generated: 2
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
- azd-preflight: wire the pipeline-action-pin-sync exit criterion into the actual
workflow by adding a conditional Test_ActionPinManifest_MatchesTemplate check
(Step 2), gating success on it (Step 3), applying the existing auto-fix strategy
when it fails (Step 4), and re-verifying after fixes (Step 5). Bump SKILL.md to
v1.4 per its own versioning convention.
- Add cli/azd/pkg/pipeline/pipeline_action_pins_test.go, a new Go test that parses
both the Dependabot-managed manifest (azure-dev-actions.yml) and the generated
pipeline template (azure-dev.ymlt) and fails if any manifest action pin isn't
mirrored verbatim in the template, catching future silent drift.
- Update azure/login in copilot-setup-steps.yml from the stale v3.0.2 pin to the
latest stable v3.1.0 (a641126d1b8aa4d1fa005f4f92df94a3a4c4c906).
- Extend .github/workflows/actionlint.yml so it actually validates the two
action-pinned files that live outside root .github/**: broaden the trigger
paths and sparse-checkout to include
cli/azd/resources/pipeline/.github/workflows/azure-dev-actions.yml and
cli/azd/extensions/azure.coding-agent/internal/cmd/templates/copilot-setup-steps.yml,
and lint them explicitly since actionlint's default project scan only looks at
.github/workflows/*.{yml,yaml}.
Keep only the azure/login pin bump (v3.0.2 -> v3.1.0) and the new Test_ActionPinManifest_MatchesTemplate drift check; drop the azd-preflight workflow wiring and the actionlint.yml path/checkout expansion as unnecessary scope for this fix.
The "use skills" commit (a7df027) had replaced working Go code that read cli/azd/resources/pipeline/.github/workflows/azure-dev-actions.yml at generation time and injected validated action pins into azure-dev.ymlt, with hardcoded literal SHAs in the template plus a documentation-only "skill" process asking an agent to manually copy pins over. That's exactly what the recent review correctly flagged: the manifest wasn't actually wired into generatePipelineDefinition, so future Dependabot bumps to the manifest wouldn't propagate. Revert to the code-based approach instead of layering more documentation on top of it: - cli/azd/pkg/pipeline/pipeline_manager.go: restore loadPipelineActionVersions/validatePipelineActionReference and have generatePipelineDefinition read+validate the manifest and populate {{.Actions.*}} at render time. - cli/azd/pkg/pipeline/pipeline_manager_test.go: restore Test_validatePipelineActionReference. - cli/azd/resources/pipeline/.github/azure-dev.ymlt: use {{.Actions.Checkout}}/{{.Actions.SetupAzd}}/{{.Actions.SetupTerraform}}/ {{.Actions.SetupDotNet}} again instead of hardcoded SHAs. - .github/skills/azd-preflight/SKILL.md and references/fix-strategies.md: drop the manifest-sync exit criterion and manual auto-fix strategy; no longer needed now that the sync is enforced in code. - Remove cli/azd/pkg/pipeline/pipeline_action_pins_test.go, the drift-detection test added as an interim fix; superseded by the restored runtime validation. All pipeline package and azure.coding-agent extension tests pass; the generated-pipeline snapshots are unchanged since the manifest and template SHAs already matched.
Revert cli/azd/pkg/pipeline/pipeline_manager.go, cli/azd/pkg/pipeline/pipeline_manager_test.go, and cli/azd/resources/pipeline/.github/azure-dev.ymlt to their upstream/main state, and delete cli/azd/resources/pipeline/.github/workflows/azure-dev-actions.yml. Regenerate the affected pipeline snapshots to match. Also drop the now-orphaned /cli/azd/resources/pipeline/.github/workflows directory entry from .github/dependabot.yml's github-actions ecosystem, since it no longer contains any workflow file to scan. This removes the generated-pipeline manifest/sync mechanism from scope; the checked-in azure-dev workflows in this repo remain pinned to commit SHAs.
eng/common/tsp-client and eng/common/spelling are vendored/synced from the azure-sdk-tools repo (per eng/common/README.md, changes made outside that repo get overwritten on the next sync), so Dependabot PRs against their package.json files would be pointless. Verified all remaining npm/github-actions directories have a real package.json or action.yml owned by this repo.
|
Azure Pipelines: Successfully started running 2 pipeline(s). 20 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
📋 Prioritization NoteThanks for the contribution! The linked issue isn't in the current milestone yet. |
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Four unresolved moderate findings concern Dependabot major-update handling, action-pin maintenance, and least-privilege permissions.
Review details
Suppressed comments (4)
.github/dependabot.yml:27
- The
update-types: ["minor", "patch"]group does not preserve the stated major-update behavior with Dependabot's current grouped-update implementation: once the group has an open PR, dependencies matched by the group but rejected byupdate-typesare marked handled, so major npm updates are silently skipped rather than opened separately (dependabot/dependabot-core#14202 remains open, and its fix PR #16113 is not merged). This contradicts the description's "majors stay separate" requirement; until the upstream fix is available, remove this group or use a configuration/automation path that explicitly verifies major updates are raised.
groups:
npm-minor-patch:
update-types: ["minor", "patch"]
.github/workflows/devcontainer-feature-test.yml:28
- This workflow has no explicit
permissionsblock, so the checkout jobs inherit repository defaults rather than the least-privilege scope required by the repository's CI convention (cli/azd/AGENTS.md:471-475). Add an explicitpermissions: contents: read(or equivalent job-level scopes) for this workflow.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
.github/workflows/event.yml:28
- Dependabot cannot currently advance a SHA-pinned GitHub Action when the upstream repository has no version tags, so the new weekly configuration will not update this
azure/azure-sdk-actionspin asmainchanges. This leaves the event workflow on an indefinitely stale action commit; add a maintained update path (for example, an upstream release/tag or a checked-in automation/verification that refreshes this SHA) rather than treating the root Dependabot entry as coverage for it.
- uses: azure/azure-sdk-actions@6a2d61414e790705ab24370421c0a5b04b65de53 # main
.github/workflows/lint-bicep.yml:16
- This reusable workflow has no explicit
permissionsblock, so its checkout job inherits repository or caller defaults rather than the least-privilege scope required by the repository's CI convention (cli/azd/AGENTS.md:471-475). Add an explicitpermissions: contents: read(or equivalent job-level scope) before this checkout.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- Files reviewed: 30/30 changed files
- Comments generated: 0 new
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
dependabot-core#14202: once a group (npm-minor-patch) matches a dependency, that dependency is marked handled regardless of update-types, so major updates that match the group pattern are silently skipped rather than opened as separate PRs -- the fix PR (#16113) for this upstream bug is not merged yet. Add an explicit npm-major group (patterns: ["*"], update-types: ["major"]) so every npm dependency is matched by exactly one of the two groups. This keeps minor/patch updates grouped into one PR and major updates grouped into a separate PR, without leaving any dependency in the ambiguous matched-but-unhandled state that triggers the bug.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Unresolved moderate findings remain for action-lock synchronization and explicit read-only permissions in two workflows.
Review details
Suppressed comments (3)
.github/dependabot.yml:7
actions-lock.jsonis another gh-aw action-pin manifest, but this configuration only watches YAML manifests. It is already out of sync:.github/aw/actions-lock.jsonstill mapsactions/checkoutto v6.0.2 andactions/setup-nodeto v6.4.0, while the generated.lock.ymlfiles use v7.0.1/v7.0.0. A latergh aw updatecan use that stale lock and reintroduce the older pins, and Dependabot will not repair the JSON; update the global lock and add a synchronization check or update workflow so the generated workflows and source manifest cannot diverge.
- package-ecosystem: "github-actions"
directories:
- "/"
- "/.github/actions/vally-eval"
- "/cli/azd/extensions/azure.coding-agent/internal/cmd/templates"
.github/workflows/devcontainer-feature-test.yml:28
- This workflow is missing the explicit least-privilege
permissionsblock required for modified workflows in this repository (for example,.github/workflows/schema-ci.yml:9-10). It checks out and executes PR-controlled devcontainer features while inheriting repository-default token permissions; addcontents: readbeforejobsso the workflow's token boundary is explicit.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
.github/workflows/lint-bicep.yml:16
- This workflow is missing the explicit least-privilege
permissionsblock required for modified workflows in this repository (for example,.github/workflows/schema-ci.yml:9-10). It checks out and executes PR-controlled Bicep code while inheriting repository-default token permissions; addcontents: readbeforejobsso the workflow's token boundary is explicit.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- Files reviewed: 30/30 changed files
- Comments generated: 0 new
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
…this PR devcontainer-feature-test.yml and lint-bicep.yml previously had no top-level `permissions:` block, so their checkout+execute jobs (which run PR-controlled devcontainer features / Bicep code) inherited repository-default token permissions instead of an explicit least-privilege scope. Both jobs only read the repo and run local scripts, so `contents: read` is sufficient. Matches the existing convention already used by schema-ci.yml (also touched by this PR).
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
.github/dependabot.yml groups all major npm updates contrary to the stated policy; the moderate finding has 2 votes.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
.github/dependabot.yml:31
- This
npm-majorgroup matches every major update and therefore combines all major dependency bumps into one Dependabot PR, contrary to the description that majors stay separate for review visibility. Remove this group so major updates remain ungrouped (and receive individual PRs).
- Files reviewed: 30/30 changed files
- Comments generated: 0 new
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
Replace the npm-minor-patch/npm-major split with one npm group (patterns: ["*"], no update-types filter) so every npm update - minor, patch, or major - lands in a single grouped PR. This avoids dependabot-core#14202 entirely (no ambiguity between competing groups with different update-types), at the cost of no longer separating major bumps from routine ones.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Dependabot grouping does not combine updates across the configured directories for either ecosystem.
Review details
Suppressed comments (2)
.github/dependabot.yml:14
- The wildcard group is not a single cross-directory group. With
directoriescontaining/and the composite-action path, Dependabot keeps the ordinary group handling directory-scoped; cross-directory grouping requiresgroup-by: dependency-name, which produces one PR per dependency rather than one PR for all dependencies. As written, a setup-node or upload-artifact update can still produce separate PRs for the root workflows andvally-eval, so the stated single-PR noise reduction is not achieved; please choose a supported grouping strategy or adjust the documented expectation.
groups:
github-actions:
patterns: ["*"]
.github/dependabot.yml:27
- The same grouping limitation applies to the npm block:
patterns: ["*"]groups updates within each manifest directory, not all four directories into one PR. Dependabot's cross-directory grouping mode isgroup-by: dependency-name, which changes the result to one PR per dependency, so the current config does not implement the description's single grouped npm PR; please choose a supported grouping strategy or revise that expectation.
groups:
npm:
patterns: ["*"]
- Files reviewed: 30/30 changed files
- Comments generated: 0 new
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
Richard Park (richardpark-msft)
approved these changes
Sep 12, 2026
hemarina
enabled auto-merge (squash)
September 12, 2026 00:31
Contributor
azure.coding-agent PR buildNote This is an unsigned development build. Install it only if you trust this PR. Install the extension: azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/10009/azure-coding-agent.zip"
|
Contributor
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix #9802
Summary
.github/workflows/**,.github/actions/vally-eval/action.yml, and the Copilot coding-agent setup-steps template to immutable commit SHAs, using each action's latest stable release where one is published. Release-version comments are kept beside each SHA for readability and future updates..github/dependabot.yml:github-actions: weekly checks across root workflows and the two nonstandard action/template directories that Dependabot's default.github/workflowsscan doesn't cover, a 7-day cooldown (cooldown.default-days: 7), and updates grouped per directory (groups.github-actions.patterns: ["*"]) — since grouping is scoped per-directory, not across the ecosystem, this yields up to 3 PRs (one per directory) rather than a single PR for all github-actions updates.npm(new ecosystem): covers this repo's own npm projects (.github/scripts,ext/vscode,ext/azuredevops/setupAzd,cli/azd/test/evals) with a weekly schedule, a 7-day cooldown, and all update types (minor, patch, and major alike) grouped per directory (groups.npm.patterns: ["*"], noupdate-typesfilter) — same per-directory scoping as above, so this yields up to 4 PRs (one per directory) rather than a single PR across the whole npm ecosystem. Using one unconditional group (noupdate-typessplit) avoids dependabot-core#14202 (still open): splitting minor/patch from major viaupdate-typeson separate groups can cause Dependabot to mark a dependency "handled" by whichever group matches it by pattern and then never check it against the other, silently dropping updates that fall outside that group'supdate-types.eng/common/**npm projects are intentionally excluded — they're synced fromazure-sdk-toolsand any local changes get overwritten.model:used by the 5 CI-run vally eval specs (eval-azd-deploy.yaml,eval-azd-qna.yaml,eval-foundry-authoring.yaml,eval-foundry-cli.yaml,eval-foundry-skill.yaml) fromclaude-sonnet-4.6togpt-5.6-terra, becauseclaude-sonnet-4.6is unavailable in this CI environment (see "Note on CI" below).Azure/azure-sdk-actionsdoes not publish releases or tags, so its currentmaincommit is pinned directly.Out of scope (dropped): an earlier iteration of this PR also introduced a generated-pipeline action-pin manifest (
cli/azd/resources/pipeline/.github/workflows/azure-dev-actions.yml) so Dependabot could track SHAs for theazd pipeline configoutput template. That mechanism added meaningful surface area for a secondary, less-visible pipeline and was removed to keep this PR focused; the generatedazure-dev.ymlttemplate is unchanged and still uses mutable version tags as before.Why?
Pinning actions to commit SHAs prevents supply-chain attacks where a tag could be moved to point to malicious code, per the GitHub Actions security hardening guide. A Dependabot cooldown window gives the community time to detect and report compromised releases before they're proposed as updates. Grouping updates within each directory (all github-actions per directory, all npm updates per directory) reduces PR noise compared to the default one-PR-per-dependency behavior, and using a single unconditional npm group sidesteps a known Dependabot grouping bug (see the
dependabot-core#14202note above).Validation
actionlintv1.7.12 on all changed workflows.go build ./...andgo test ./pkg/pipeline/...incli/azd.go test ./...incli/azd/extensions/azure.coding-agent.dependabot.ymldirectory (both ecosystems) contains a real, repo-owned manifest (action.yml/package.json) that isn't vendored/synced from elsewhere.Note on CI
Run vally evalschecks on this PR previously failed withRequest session.create failed with message: Model "claude-sonnet-4.6" is not available.— a Copilot model-availability issue in the eval environment. The eval spec changes in this PR address that directly by switching togpt-5.6-terra.