feat(computing-unit): curate computing-unit images

[tanishqgandhi1908]

What changes were proposed in this PR?

Every computing unit runs the same image, fixed when the cluster is installed. ML Models needing a different Python version, a system package, or a library built from source cannot run, because a Python virtual environment only holds pip packages.

This lets an administrator register an image reference from a public registry, and a computing unit can then be started from it. Off by default (curatedImages.enabled: false) until the UI to manage these ships in #8470 and #8471.

How it works. Texera reads the image's manifest and config blob — a few kilobytes, never the layers — to check its start command runs computing-unit-master, which means it was built FROM the Texera computing-unit image, and to resolve the digest behind the reference. A misspelled, private or unsuitable image is refused in seconds, in front of the administrator, rather than when a user's unit will not start.

The row records owner/name@sha256:…, and that is what units run, so a tag its owner moves later cannot change what already ran. Nothing is copied and no registry is added — units pull the reference the same way the deployment's own image is already pulled.

Uniqueness is enforced by the database, not only checked in the service: two administrators registering the same link at the same moment both pass a read-then-write check and produce two rows for one image.

Non-root, for curated images only. A curated image was supplied by an administrator and reviewed by nobody, so a unit started from one runs as a non-root user with no privilege escalation and no capabilities. The deployment's own image is untouched — it is its operator's choice, and a deployment that has replaced it with an image needing root would break on upgrade.

Known limitation. The first unit on each node waits for the image to download — about 80 seconds for a 3 GB one — while later units on that node start at once. Pre-pulling ready images onto nodes is #8469.

Any related issues, documentation, discussions?

Closes #8468
Part of #8466

How was this PR tested?

Unit tests, chart rendering, and a deployment to minikube exercising both states.

sbt "ComputingUnitManagingService/testOnly org.apache.texera.service.resource.CuratedImageResourceSpec"
"ComputingUnitManagingService/testOnly org.apache.texera.service.util.KubernetesClientSpec"
"Config/testOnly org.apache.texera.common.config.KubernetesConfigSpec"
scalafmtCheckAll

CuratedImageResourceSpec 27 passed
KubernetesClientSpec 17 passed
KubernetesConfigSpec 6 passed
scalafmtCheckAll clean

helm template renders with the feature off and on; the manager Role gains jobs and pods/log only.

Deployed to minikube, feature off:

GET /api/cu-image 503 "Curated images are not enabled on this deployment."
POST /api/cu-image 503
create unit with iid 403 "Image 1 is not available..."
create unit without iid 200 deployment's own image, no security context

Feature on:
register a good image READY in 8s, pinned to @sha256:bdeadc3c...
duplicate reference 400 names the existing row
duplicate name 400
empty name 400
tag that does not exist FAILED, log names the tag and what to do instead
alpine (not a CU image) FAILED, "Its start command is: [/bin/sh]"

unit from a curated image
image tagandhi19/texera-cu-sklearn@sha256:bdeadc3c...
security {allowPrivilegeEscalation:false, capabilities:{drop:[ALL]}, runAsNonRoot:true, runAsUser:1001}
id uid=1001(texera)

delete the image while a unit runs on it
204, pod stays Running, unit still reports imageName 'Python ML'
a new unit from it 403

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @zyratlo, @aicam, @aglinxinyuan
  You can notify them by mentioning @zyratlo, @aicam, @aglinxinyuan in a comment.

[tanishqgandhi1908]

/request-review @aicam

[codecov-commenter]

Codecov Report

❌ Patch coverage is 29.57393% with 281 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.42%. Comparing base (ec3a9dd) to head (ab19e2f).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...texera/service/resource/CuratedImageResource.scala	22.85%	182 Missing and 7 partials ⚠️
...he/texera/service/util/ImageValidationClient.scala	41.26%	73 Missing and 1 partial ⚠️
...ache/texera/common/config/CuratedImageConfig.scala	0.00%	12 Missing ⚠️
.../apache/texera/service/util/KubernetesClient.scala	33.33%	5 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (29.57%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8475      +/-   ##
============================================
- Coverage     93.94%   93.42%   -0.52%     
- Complexity     4827     4828       +1     
============================================
  Files          1209     1212       +3     
  Lines         49705    50101     +396     
  Branches       6074     6123      +49     
============================================
+ Hits          46695    46809     +114     
- Misses         1522     1794     +272     
- Partials       1488     1498      +10     

Flag	Coverage Δ		*Carryforward flag
access-control-service	80.18% <ø> (ø)
agent-service	99.32% <ø> (ø)		Carriedforward from ec3a9dd
amber	89.86% <20.00%> (-0.07%)	⬇️
computing-unit-managing-service	53.78% <29.94%> (-21.27%)	⬇️
config-service	87.12% <ø> (ø)
file-service	83.65% <ø> (ø)
frontend	96.77% <ø> (ø)		Carriedforward from ec3a9dd
notebook-migration-service	83.73% <ø> (ø)
pyamber	98.47% <ø> (ø)		Carriedforward from ec3a9dd
workflow-compiling-service	77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

✅ No material benchmark regressions detected

🟢 2 better · 🔴 0 worse · ⚪ 13 noise (<±5%) · 0 without baseline

Compared against main ec3a9dd benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🟢	bs=10 sw=10 sl=64	376	0.23	24,895/32,767/32,767 us	🟢 -12.9% / 🔴 +101.7%
⚪	bs=100 sw=10 sl=64	795	0.485	124,332/138,746/138,746 us	⚪ within ±5% / 🔴 +27.7%
⚪	bs=1000 sw=10 sl=64	910	0.555	1,095,797/1,184,093/1,184,093 us	⚪ within ±5% / 🔴 +13.9%

Baseline details

Latest main ec3a9dd from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	376 tuples/sec	381 tuples/sec	754.83 tuples/sec	-1.3%	-50.2%
bs=10 sw=10 sl=64	MB/s	0.23 MB/s	0.233 MB/s	0.461 MB/s	-1.3%	-50.1%
bs=10 sw=10 sl=64	p50	24,895 us	24,567 us	12,950 us	+1.3%	+92.2%
bs=10 sw=10 sl=64	p95	32,767 us	37,619 us	16,243 us	-12.9%	+101.7%
bs=10 sw=10 sl=64	p99	32,767 us	37,619 us	19,574 us	-12.9%	+67.4%
bs=100 sw=10 sl=64	throughput	795 tuples/sec	817 tuples/sec	982.46 tuples/sec	-2.7%	-19.1%
bs=100 sw=10 sl=64	MB/s	0.485 MB/s	0.499 MB/s	0.6 MB/s	-2.8%	-19.1%
bs=100 sw=10 sl=64	p50	124,332 us	123,582 us	101,575 us	+0.6%	+22.4%
bs=100 sw=10 sl=64	p95	138,746 us	132,559 us	108,655 us	+4.7%	+27.7%
bs=100 sw=10 sl=64	p99	138,746 us	132,559 us	121,794 us	+4.7%	+13.9%
bs=1000 sw=10 sl=64	throughput	910 tuples/sec	922 tuples/sec	1,010 tuples/sec	-1.3%	-9.9%
bs=1000 sw=10 sl=64	MB/s	0.555 MB/s	0.563 MB/s	0.616 MB/s	-1.4%	-9.9%
bs=1000 sw=10 sl=64	p50	1,095,797 us	1,079,445 us	998,032 us	+1.5%	+9.8%
bs=1000 sw=10 sl=64	p95	1,184,093 us	1,181,094 us	1,039,429 us	+0.3%	+13.9%
bs=1000 sw=10 sl=64	p99	1,184,093 us	1,181,094 us	1,068,120 us	+0.3%	+10.9%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,531.44,200,128000,376,0.230,24895.41,32767.21,32767.21
1,100,10,64,20,2516.86,2000,1280000,795,0.485,124332.35,138746.40,138746.40
2,1000,10,64,20,21986.29,20000,12800000,910,0.555,1095797.00,1184093.07,1184093.07