Skip to content

feat: sign images with cosign and verify them on upgrade - #3401

Open
vpetersson-bot wants to merge 2 commits into
masterfrom
feat/cosign-image-signing
Open

vpetersson-bot wants to merge 2 commits into
masterfrom
feat/cosign-image-signing

Conversation

@vpetersson-bot

Copy link
Copy Markdown
Contributor

Description

Devices pull ghcr.io/screenly/anthias-* images and run them without checking where they came from. If someone pushed an image to those tags by hand, or from a compromised token, or from some other workflow, upgrade_containers.sh would start it with no questions asked.

This change signs the images in the release pipeline and checks them on the device before an upgrade starts them.

Signing (docker-build.yaml)

  • After the existing provenance attestations, install cosign v3.1.4 (sigstore/cosign-installer, pinned by SHA) and keyless-sign the server, viewer and redis digests for each board. The new .github/workflows/scripts/cosign-sign-images.sh does the signing, retrying transient registry errors the same way mirror-latest-tags.sh does.
  • Signatures are bound to the digest, so the latest-<board> tags that publish-latest points at those manifests are covered too.
  • GHCR fails the job if signing fails. Docker Hub is best-effort (continue-on-error), for the same rate-limit reason as its attestations.

Verification (device)

  • ansible/roles/system/tasks/cosign.yml installs the cosign v3.1.4 static binary for amd64, arm64 or armhf, pinned by sha256.
  • New bin/verify_image_signatures.sh takes the image list, looks up the digest the daemon actually pulled for each Anthias image, and runs cosign verify against the identity https://github.com/Screenly/Anthias/.github/workflows/docker-build.yaml@refs/heads/master with the GitHub Actions OIDC issuer. A fork, another workflow in this repo, or a branch build doesn't match. Images that aren't ours, such as the Caddy SSL sidecar, are skipped. If cosign is missing, verification fails.
  • upgrade_containers.sh runs it after compose pull (pull mode only) and before anything is started. On failure it brings back the viewer and celery containers the low-RAM path stopped, on their existing images, and exits non-zero. The device stays on its current release. ANTHIAS_SKIP_IMAGE_VERIFICATION=1 lets an operator who deliberately runs their own images opt out, with a warning.

Rollout: with this identity, cosign v3 also accepts the build-provenance attestation this workflow already pushes, because it is signed under the same identity. I checked current latest-pi2, latest-pi3, latest-pi4-64 and latest-x86 images with cosign v3.1.4 and they verify today. A different workflow identity was rejected. So devices that upgrade before the first signed build lands won't fail. Older tagged releases are unaffected, since install.sh fetches upgrade_containers.sh from the release ref.

Balena deployments don't use upgrade_containers.sh and aren't covered by this change.

Testing

  • tests/test_verify_image_signatures.py: 7 cases using fake docker/cosign shims. They cover signed, unsigned, the digest from the wrong repo, an image that was never pulled, third-party images skipped, cosign missing, and no arguments.
  • Ran cosign verify by hand against the live GHCR images, as described above.
  • ruff, ansible-lint (production profile), actionlint and shellcheck pass on the changed files.

Checklist

  • I have performed a self-review of my own code.
  • New and existing unit tests pass locally and on CI with my changes.
  • I have done an end-to-end test for Raspberry Pi devices.
  • I have tested my changes for x86 devices.
  • I added a documentation for the changes I have made (when necessary).

🤖 Generated with Claude Code

- Keyless-sign every pushed anthias-{server,viewer,redis} digest with
  cosign in docker-build.yaml (GHCR hard-fail, Docker Hub best-effort)
- Install a checksum-pinned cosign v3.1.4 on devices via ansible
- Add bin/verify_image_signatures.sh, which checks the pulled digest of
  each Anthias image against the docker-build.yaml@master identity
- upgrade_containers.sh refuses to start images that fail verification
  and leaves the device on its current release

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
Copilot AI lite review requested due to automatic review settings October 21, 2026 10:36
@vpetersson-bot
vpetersson-bot requested a review from a team as a code owner October 21, 2026 10:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

- Use [[ ]] for conditionals in the new signing/verification code
- Add an explicit return to pulled_digest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
Copilot AI lite review requested due to automatic review settings October 21, 2026 10:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants