Skip to content

chore(drivers): remove the Device Support package format from Core - #1432

Merged
frahlg merged 2 commits into
masterfrom
chore/remove-device-support-coupling
Sep 25, 2026
Merged

frahlg merged 2 commits into
masterfrom
chore/remove-device-support-coupling

Conversation

@frahlg

@frahlg frahlg commented Sep 25, 2026

Copy link
Copy Markdown
Member

Owner decision

Fredrik, 2026-09-25: FTW and its driver repository srcfl/device-drivers are one track. Zap and the Device Support register are a separate track. There must be no direct coupling between them in schemas, code paths or automation. Knowledge can move across by hand. srcfl/device-drivers has already removed its Device Support package pilot and the package format specs (srcfl/device-drivers#133).

This PR removes from Core the Device Support package format and everything that existed only for it.

Removed

  • driverrepo: sourceful.driver-index/v1 index, sourceful.driver-package/v1 package and both envelope formats. That covers refreshSourceful, cachedSourcefulManifest, sourcefulManifest, sourcefulPackageDriver, the package-contract validators, the Sourceful canonical JSON verifier, the package cache under cache/<repo>/packages/, and writing sourceful-package.envelope.json next to an install. The RuntimePolicy branch for installed packages is gone, including control-v2 policy construction. sourceful.go is now runtime_policy.go and holds only the channel's policy code.
  • ManifestDriver fields that only package adaptation filled: package_id, target, artifact_id, runtime_*, host_api_profile, package_key_id, package_envelope_*, commands, default_mode, lease_policy. The device-drivers beta/stable manifests use none of these; I checked the live drivers-beta manifest.
  • Catalog: package_id, runtime_abi and host_api_profile on drivers.CatalogEntry, which only a package could fill. The Diagnose report in web/diagnostics-modal.js looked for package_id, so channel installs showed as unknown. It now reports them as managed, with their channel and artifact SHA-256.
  • Config: DriverRepositoryFormatSourcefulIndexV1, its validation, and the per-driver control opt-in (DriverControlOptIn), which only a control-v2 package could satisfy.
  • CI: the device-support-contract job, which checked out a private repository with SOURCEFUL_CI_REPO_TOKEN and built packages with its driver_package.py. It is also gone from the aggregate job's needs and from its RESULTS list. go test + vet is the only required check on master (checked with gh api .../branches/master/protection/required_status_checks), so branch protection needs no change.
  • Test data and tests that exist only for the format: testdata/device-support-baseline.json, testdata/device-support-v1/**, TestDeviceSupportPythonContract (the test that read the FTW_DEVICE_SUPPORT_* variables), sourceful_test.go, installed_policy_format_test.go, the package cases in retained_package_format_test.go, the two package-entry cases in legacy_format_upgrade_test.go, and the control_v2_opt_in scenario in legacy_startup_test.go.
  • Docs: the sentences saying Device Support may consume the channel, and "FTW still understands sourceful.driver-index/v1" (docs/architecture.md, docs/device-repository.md, docs/writing-a-driver.md). VISION.md, docs/roadmap.md and SUPPORT.md had no such sentences.

Kept, and why

  • The ftw.manifest/v1 signed channel as it is. That includes directManifestRuntimePolicy, recordDirectManifestFormat, FTWSigned, the retained-format rules for ftw.manifest/v1, and the read-only policy with its permissions, auth POST path and config secrets. TestHistoricalDirectManifestAllowsVerifiedReinstallAndDefault moved into legacy_format_upgrade_test.go.
  • The com.sourceful.driver.<id> identity on read-only channel policies. drivers.RuntimePolicy.validate requires that form of a read-only artifact, so the channel path still needs it. Changing it would mean editing the control/runtime code in go/internal/drivers, which this PR leaves alone.
  • The control-v2 runtime in go/internal/drivers. It is not deleted here; see "Unreachable code" below.

Stored state

  • A config listing a repository with format: sourceful.driver-index/v1 still loads, from YAML and from stored settings. Loading drops the entry and startup logs one slog.Warn naming it. DropRetiredSettings, the existing retired-settings path used for Ask why, then deletes the entry from the stored settings. If that was the only repository listed, the site gets FTW's default signed channel, the same as a site that lists none. The write path stays strict: saving that format is refused as unsupported.
  • A driver config with control: loads without error. Before, control.enabled on anything other than a package stopped that driver from starting. The block is dropped at load, startup warns once (naming the drivers) that it has no effect, and DropRetiredSettings deletes it from stored settings.
  • A managed install recorded as sourceful.driver-index/v1, or an older unrecorded row whose directory still holds sourceful-package.envelope.json, is deactivated at startup through Manager.Deactivate. Startup logs a warning. driver-repository/effective is rebuilt from the remaining active selections, so the release's own driver runs. Such installs stay on disk but are no longer listed in InstalledVersions/AvailableVersions, cannot be activated, and cannot be a rollback target. RuntimePolicy refuses one that is somehow active.

Also in this PR, at the coordinator's request: the docs and code comments that still called the bundled drivers an "offline recovery" set now describe them as the release's own drivers, and describe the local → owner's selection → release resolution order (README, docs/architecture.md, docs/device-repository.md, docs/writing-a-driver.md, comments in api.go, config.go, manager.go).

Unreachable code, for the owner to decide

With packages gone, nothing produces a RuntimePolicy with IsControlV2() == true. This control-v2 code stays in the tree but no longer runs in production:

  • go/internal/drivers/control_v2.go: the control branch of RuntimePolicy.validate, the fields Commands, DefaultMode, Lease, SiteEnabled and MaxWrites, DriverCommandV1/DriverCommandResultV1, command/lease validation and parseLuaCommandResult.
  • go/internal/drivers/lua.go: CommandV2, DefaultModeV2, commandResult, and the v2 branches at the lifecycle and context sites (IsControlV2() checks).
  • go/internal/drivers/registry.go: controlV2Runtime, luaRuntime.CommandV2/DefaultModeV2, the v2 startup default, v2 lease expiry, dispatchV2Command, recordCommandResult and CommandResultSink.
  • go/internal/drivers/host.go: the v2 write scope (IsControlV2 check).
  • go/cmd/ftw/main.go reg.CommandResultSink → state.RecordDriverCommandResult and the driver_command_results table (go/internal/state).

Two other things now always take the same branch:

  • Nova inventory: driverinventory (sourceful.driver-inventory/v1) still has a managed source and package_id/package_channel that only a package could fill. Channel installs keep reporting legacy_repository, exactly as before. I left it unchanged because it is the Nova wire contract, and dropped the per-artifact AvailableVersions lookup in main.go that only fed it.
  • Zap automation: go/internal/integration/device_simulator_lua_test.go skips with "run through Zap tools/lua_lab/run_device_simulator_chain.sh". That is Zap automation driving an FTW test, and outside this PR's scope.

Test evidence

  • make verify (all Go suites, compose migration, container boundary, release workflow and native solver checks, go vet ./..., go build ./...): clean. 68 Go packages pass, none fail.
  • LC_ALL=C.UTF-8 npm test: 635 pass, 0 fail.
  • New tests:
    • TestLoadDropsDeviceSupportRepositoryAndControlOptIn and TestLoadKeepsTheDefaultChannelWhenOnlyADeviceSupportSourceWasListed (YAML load).
    • TestStoredDeviceSupportSettingsLoadAndAreDeleted (stored settings: load, startup check, deletion, idempotent second run).
    • TestDeviceRepositoryRejectsUnknownFormats (write path).
    • TestDeviceSupportPackageIsDeactivatedAtStartupAndTheBundledDriverRuns, for both the recorded-format and the envelope-only row. It asserts that the package is refused as-is, deactivated on the next start, gone from active/ and effective/, and that the path resolves to the release's copy and that copy runs in the registry. It also asserts the package is never offered or activated again. This test fails when the startup retirement step is disabled.
  • Browser check needed: the Diagnose modal's report text changed (managed instead of unknown for channel installs; no Package: line). A person needs to look at it in a browser before merge.
  • Overlapping open PRs: feat(prices): static / time-of-use tariff so planning works outside Europe #1051 and feat(modbus): share one TCP session per device and proxy it #1003 (config.go, docs/architecture.md) and Propose Jev as typed judgments for Ask why and household intents #1292 (docs/roadmap.md) already conflict with master in those files. git merge-tree shows this branch adds no new conflict hunks for them.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MuerPFZFG88kgu8sWVHeq7

FTW and srcfl/device-drivers are one track; Zap and the Device Support
register are another, with no coupling in schemas, code paths or
automation (owner decision 2026-09-25). device-drivers already dropped
its package pilot and specs (#133).

Remove the sourceful.driver-index/v1 index, package and envelope
support from driverrepo and config, the package-only ManifestDriver and
catalog fields, the driver control opt-in, the device-support-contract
CI job and its private-repo token, and the package fixtures and tests.
The ftw.manifest/v1 channel and its read-only runtime policy stay.

Stored state: a listed package repository is dropped at load with one
warning and deleted from stored settings; an active package install is
deactivated at startup so the release's own driver runs, and is never
offered again; a control opt-in is dropped with a warning instead of
stopping its driver.

Also update docs that still called the bundled drivers an offline
recovery set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MuerPFZFG88kgu8sWVHeq7
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-25T08:45:03.704335Z 438c0c2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 438c0c2cd7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/cmd/ftw/main.go
Comment on lines +3331 to 3335
out = append(out, driverinventory.RepositoryArtifact{
LogicalPath: installed.LogicalPath,
InstalledPath: installed.InstalledPath,
DriverID: installed.DriverID,
Version: installed.Version,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve managed provenance in driver inventory

For every active signed-channel driver, this now creates a RepositoryArtifact without PackageID or PackageChannel. driverinventory.inspectDriver only labels an artifact managed when both fields identify a package/channel; otherwise it emits source: "legacy_repository" (go/internal/driverinventory/inventory.go:192-198). Consequently, after retiring the package format, current FTW-channel installations are reported to Nova/fleet inventory as legacy repositories and lose their channel provenance. Update the inventory classification to recognize the remaining signed manifest format rather than depending on removed package fields.

Useful? React with 👍 / 👎.

@frahlg
frahlg merged commit 2328385 into master Sep 25, 2026
15 checks passed
@frahlg
frahlg deleted the chore/remove-device-support-coupling branch September 25, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant