feat: Support floating tag in product image selection

[Techassi]

Part of https://github.com/stackabletech/internal-issues/issues/169.

This PR adds support for auto updating product images. This is done by using a floating tag for product images instead of fully-qualified ones.

Significant changes are:

• Add a new stackableVersionStrategy field to the product image selection CRD interface:

  productImage:
    stackableVersionStrategy:
      default: Exact
      description: |-
        Configure the Stackable version strategy. Defaults to `Exact`.
  
        Currently, two variants are supported:
  
        - `Exact`, which uses the exact, fully-qualified, canonical version of a product image.
        - `LatestPatch`, referencing a floating tag which always points to the latest patch version
          in the current release line. The current release line is either automatically derived by
          the operator based on its own version, or can be overridden with `stackableVersion`.
  
        A potential newer image is only pulled when Pods are rotated or their containers are
        restarted. Pods are NOT rotated and containers are NOT restarted automatically when a new
        image is available. This behaviour makes this a passive update mechanism, rather than an
        active one.
  
        It should be noted that when this field is set to `LatestPatch`, the operator automatically
        uses `Always` as the pull policy for product images. If set to `Exact`, `IfNotPresent` is
        used. Explicitly setting `pullPolicy` takes precedence.
  
        ### Examples
  
        - The `stackableVersion` field is not set, the operator falls back to its own version, eg.
          26.7.0. If this field is set to `LatestPatch`, the `26.7` floating tag will be used for
          product images, else, `26.7.0` will be used.
        - The `stackableVersion` field is set to `26.3.0`. If this field is set to `LatestPatch`,
          the `26.3` floating tag will be used for product images, else, `26.3.0` will be used.
      enum:
      - Exact
      - LatestPatch
      type: string

• Remove the default of pullPolicy. This better aligns with the floating tag feature, because if enabled, the pull policy is automatically set to Always. The current default is (as far as I can see) a leftover from the past. Even the doc comment states that Always is the default, as we support floating tags (not true).

• The stackableVersion field is now a proper semantic version, instead of a plain string. This is needed, to correctly extract the floating tag if the auto update is enabled.

This PR also serves as a decision. Please vote with 👍 or 👎 or leave comments.

[sbernauer]

A few points on the CRD change:

• I like it in general, it's pretty simple to configure and understand for users.
• Naming things is hard 😓. I dislike autoUpdate because it's sounds like we are actively doing stuff and I would prefer it to be a bit more speaking. Ideas could we patchLevelUpdates or even more descritptive patchLevelFloatingTag - which makes it clear we are not actually updating anything ourselves, but just point at a floating tag. We might also to add a "real" update mechanism in the future (periodically checking for new versions and restarting), which could than get the autoUpdate (or whatever) name.
• We should clearly document that this will not automatically pull in updates, but only on Pod restarts.
  ** Also mention TLS cert rotation fixes thís implictely.

Comments on the implementations:

• For 0.0.0-dev images IMHO we want to default pullPolicy Always, as it's a giant floating tag.
  ** I had to revert a few test asserts Always -> IfNotDefault changes
• When a ":latest" image is used it's also a floating tag, so we should default to Always as well. That's what k8s seems to do as well.
  ** I also had to revert custom_without_tag and custom_with_colon_in_repo_and_without_tag test changes
• We should also warn in case the user configures a floating tag but a "wrong" pull policy.

I updated the tests and the refactored the implementation to turn them green again (one is left as an exercise for the reader). I also added some explanatory comments.
I send you the patch via Slack. Happy about comments!

[Techassi]

Naming things is hard

Agreed. Like discussed, I now went with useFloatingTag instead.

We should clearly document that this will not automatically pull in updates, but only on Pod restarts.

Yes, I think I sufficiently touched on this in my updated doc comment.

Also mention TLS cert rotation fixes thís implictely.

I would say that this is implicitly covered by my current doc comment. It basically says, that a container/pod needs to restart in order to have an effect.

All your comments on the implementation should be addressed in 83531c7.

[sbernauer]

I have to admin I prefer patchLevelFloatingTag as it makes it a bot more clear what the floatingTag is about.
Also we might want to have a different floating tag for some other functionality in the future (making things up: Only baseOS updates or product version bugfix bumps), in which case the abbreviated field name causes problems (which we don't have when we express what floating tag the user configures).

[sbernauer]

Any reason for the re-ordering (also in other places)?

[Techassi]

No not really, other than my OCD 🙈

I can revert if needed.

[sbernauer]

IIRC there even is a clippy lint that when constructing a struct you should use the order of the fields as it's defined in the struct.
That has product_version, stackable_version, use_floating_tag and repo, which I like :)

[Techassi]

@soenkeliebau mentioned that we might want to remove the term "floating tag" from the field name so that we don't explicitly mention the underlying mechanism (which could change down he line).

I agree that it makes sense, but naming things is hard. A few alternative names might be:

• useLatestPatchVersion
• autoUpdatePatchVersion
• useRollingVersion

[soenkeliebau]

Late to the party, sorry. @Techassi already represented my concerns above.

I have since considered this a bit more and can also see a problem with patchLevel since this could be confusing whether it refers to the product version or the stackable version as it sits at the same level in the CRD I believe.

CRD could look like this:

image:
  productVersion: 1.2.3
  useLatestPatchVersion: true

Which looks like it makes a lot of sense, but the parameters refer to entirely different things, even though they sound totally similar.

Also, K8s tends towards enums rather than bools, its even specified in their docs :)

Think twice about bool fields. Many ideas start as boolean but eventually trend towards a small set of mutually exclusive options. Plan for future expansions by describing the policy options explicitly as a string type alias (e.g. TerminationMessagePolicy).

If I may propose a totally new approach (sorry!), how about stackableVersionPolicy with two initial allowed values:

• exact
• latestPatch

and the default being exact?

That would give us this:

image:
  productVersion: 1.2.3
  stackableVersionPolicy: latestPatch

Maybe even stackableImageVersionPolicy but at some point it gets long ...

[Techassi]

Oh I like the idea with the enum. Two things:

• I like stackableVersionPolicy as the field name. Using stackableImageVersionPolicy is in my opinion overly verbose and also doubles information (it is already implied we are talking about the image by the parent key.
• I like both variants, however per Kubernetes conventions, they should use PascalCase:
  • Exact
  • LatestPatch

[soenkeliebau]

Indeed, sorry. I am simply unable to keep it straight in my head which case is to be used where :)

Edit to mention a random idea that just came to me:
Maybe we can even extend the CRD with a productVersionPolicy at some point..

[Techassi]

Okay, today we decided to go ahead with @soenkeliebau's enum proposal. The exact structure will look like this:

image:
  stackableVersionPolicy: Exact # or LatestPatch

The PR description will be updated accordingly as well.