Repository navigation
Add Cloud Build recipe (cloudbuild-gcr.yaml) for gcr.io/cdapio/cdap-controller; retire previous nightly 'latest' - #175
Closed
HarshitJain09 wants to merge 2 commits into
Closed
HarshitJain09 wants to merge 2 commits into
HarshitJain09 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new Cloud Build configuration (cloudbuild-gcr.yaml) to automate building and pushing the cdap-controller Docker image, along with a mechanism to tag and retire superseded nightly images to stop vulnerability scanning on stale builds. The README.md has also been updated to document this process. The review feedback suggests optimizing the build steps by wrapping the gcloud image description queries in conditional checks, ensuring they only run when necessary (e.g., for nightly builds) and avoiding potential failures or redundant API calls when no previous image exists.
…ontroller; retire previous nightly 'latest' Moves the Cloud Build configuration for the gcr.io/cdapio/cdap-controller image out of the Cloud Source Repositories repo `cdap-oss-build` (End of Sale, no review gate) into this repository as `cloudbuild-gcr.yaml`, so the build recipe is reviewed alongside the Dockerfile it builds. The same file serves the nightly `latest` build and manual release builds (`_VERSION=<release>`). Behavioural change vs. the CSR recipe: after a new nightly `latest` is pushed, the digest that was previously `latest` is tagged `no-new-use-public-image-<sha256>`, so vulnerability scanning stops filing new findings against superseded nightly images. Guards: no-op when `_VERSION` is not `latest`, when there is no previous `latest`, or when the digest did not change; `_TAG_PREVIOUS=false` gives a dry run.
HarshitJain09
force-pushed
the
harshkjn/gcr-nightly-retire-previous
branch
from
November 3, 2026 05:46
4b4e983 to
d041c3c
Compare
* record-previous: only query the registry for the previous digest on nightly (_VERSION=latest) builds. Release builds never retire anything, so the lookup was a wasted call; the step now matches its own comment. * tag-previous-latest: check for a previous digest *before* describing the new `latest`. With `set -euo pipefail`, a transient lookup failure could previously fail a first-ever run that had nothing to retire. No behaviour change when a previous digest exists: the lookup still runs and a failure there still fails the build loudly (intended - the image is already pushed and a red build is the signal that the retire did not happen).
HarshitJain09
added a commit
to cdapio/cdap-build
that referenced
this pull request
Nov 3, 2026
tag-previous-latest now checks for a previous `latest` digest *before* describing the new one. With `set -euo pipefail`, a transient lookup failure could previously fail a first-ever run that had nothing to retire. Mirrors the same change in the cdap-operator recipe (cdapio/cdap-operator#175). No behaviour change when a previous digest exists: the lookup still runs and a failure there still fails the build loudly (intended - the image is already pushed and a red build is the signal that the retire did not happen).
Author
|
Closing — landed directly in the Cloud Build trigger configuration repository for now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Moves the Cloud Build recipe for the
gcr.io/cdapio/cdap-controllerimage out of the Cloud Source Repositories repocdap-oss-build(CSR is End-of-Sale, and that repo has no PR/review gate) into this repository ascloudbuild-gcr.yaml, so the build recipe is reviewed in the same place as theDockerfileit builds. Adds a short README section documenting the trigger ↔ file mapping.The same file serves both triggers in project
cdapio:cdap-controller-docker-nightly(dailylatestfromdevelop) andcdap-controller-docker(release builds, run with_VERSIONoverridden).Companion PR for the other image built from
cdap-oss-build: cdapio/cdap-build#578.Why
Nightly rebuilds of
latestleave the previous digest untagged-but-scanned, so vulnerability scanning keeps filing new findings against stale images. After this change the nightly build tags the previouslatestdigest withno-new-use-public-image-<sha256>right after the newlatestis pushed, which marks it as superseded for scanning.Why the recipe moves to GitHub instead of being edited in CSR
cdap-oss-build("a mirror of an external repository used to trigger Cloud Build") is to connect Cloud Build directly to the source repository rather than keep a wrapper repo in CSR (https://cloud.google.com/build/docs/automating-builds/github/connect-repo-github).cdapioalready has a GitHub-connected Cloud Build trigger (cdap-ui-tests->cdapio/cdap), so this extends an existing pattern rather than introducing a new integration.developbranch, as the Dockerfile it builds.cdap/cdap-controllerdigests since 2025-10-30 with no failures; this PR makes the nightly do it itself.Behaviour vs. the current CSR
cloudbuild-operator.yaml${_VERSION}digestrecord-previous), nightly builds onlygit clone … cdap-operator -b ${BRANCH_NAME}docker build -t gcr.io/cdapio/cdap-controller:${_VERSION} .cdap-operator/images:after all stepsdocker pushso the next step provably runs after the image exists (images:kept, harmless)latesttag-previous-latest), no-op unless_VERSION == latestSafety properties of the retire step
_VERSION≠latest) never retire anything, and make no registry lookups.latest(first run) → no-op, and no registry lookup is made._TAG_PREVIOUS=false→ prints what it would do, tags nothing.gcloud container images untag.Dry run (from a checkout of this branch)
Rollout plan (staged; every step reversible)
All CSR-backed Cloud Build triggers in project
cdapio, and where each moves:cdap-controller-docker-nightly(dailylatest)cloudbuild-operator.yamlcdap-operatorcloudbuild-gcr.yaml(this PR)cdap-docker(dailylatest)cloudbuild.yamlcdap-buildcloudbuild-gcr.yaml(cdapio/cdap-build#578)cdap-controller-docker(release, manual_VERSION)cloudbuild-operator.yamlcdap-maven-docker-build(weekly)cloudbuild-maven.yamldevelopwith_TAG_PREVIOUS=false; confirm it builds, pushes and prints the digest it would retire.cdap-controller-docker-nightly(one trigger field, same trigger ID, Cloud Scheduler untouched) from CSRcdap-oss-build/cloudbuild-operator.yamlto GitHubcdapio/cdap-operator@^develop$/cloudbuild-gcr.yaml; watch one nightly.cdap-dockerthe same way (see Add Cloud Build recipe (cloudbuild-gcr.yaml) for nightly gcr.io/cdapio/cdap; retire previous 'latest' cdap-build#578 for its Cloud Scheduler body change).cdap-maven-docker-build, then the release triggercdap-controller-docker(same file, same one-field change); delete the CSR files after a clean week.Worst case at any step is "no nightly today" (same as an ordinary build failure). The retire step only ever adds one tag to one digest; the live
latestis never touched and nothing is deleted. Rollback of a repoint is a one-field change back to CSR; rollback of a tag isgcloud container images untag. Before/after trigger exports will be recorded on the internal tracking bug.Review notes
record-previousnow only queries the registry for nightly (latest) builds, matching its own comment; and the retire step checks for a previous digest before querying the registry, so a transient lookup failure can no longer fail a first run that has nothing to retire. The second change is mirrored in Add Cloud Build recipe (cloudbuild-gcr.yaml) for nightly gcr.io/cdapio/cdap; retire previous 'latest' cdap-build#578.