Feat: registry (backends / transports / schedulers)

[Alexander-Auer]

feat: backend configuration and a closed registry

Part of #18.

Important

Depends on #27. Merge after #27 (PR 2 of 4; the full order is listed in #27).
Until #27 is merged, this diff also shows the earlier PRs. This PR's own change is one commit:
f90cdd4

A consumer can now say which backend it wants as plain data, and get a backend for it:

from intercede import registry

config = {"kind": "batch", "transport": "ssh", "scheduler": "slurm",
          "host": "batch.example.org", "user": "dirac",
          "known_hosts": "batch.example.org ssh-ed25519 AAAA..."}

async with registry.backend(config) as ce:
    await ce.submit(spec, copies)

Configuration: one flat public model

BackendConfig is the only public configuration class. The consumer uses the same field names for every backend, and never needs to know which classes interCEde uses inside:

kind	applies
arc, htcondor-ce	endpoint, auth (token, proxy)
cloud	endpoint, driver, vm_image, vm_flavor, driver_options, vm_networks, vm_ssh_key, vm_max_lifetime_s
batch + local	scheduler
batch + ssh	scheduler, host, user, known_hosts, port, jump_host, connect_timeout_s, auth (key, password or agent)
all but cloud	queue, container_launcher, max_output_bytes, max_output_files, transfer_timeout_s

The three backend families of IC-ADR-001 §4 are covered:

• monolithic: arc, htcondor-ce
• provider: cloud
• composed: batch, a transport combined with a scheduler

How it is checked. Internally, resolve() turns the flat model into the settings of exactly one backend (_backends/config.py), and the model calls it while it is being built. So mistakes are reported immediately, in the consumer's own field names:

host: does not apply to kind='arc'
user: required for kind='batch' with transport='ssh'; known_hosts: required for ...

Rules:

• Unknown fields and typos are refused, not ignored.
• There is nowhere to put a secret, as IC-ADR-003 requires. driver_options additionally refuses secret-looking keys.
• SSH requires known_hosts, because host key verification is not optional.
• An HTCondor-CE always needs a token.
• Limits must be positive.
• A container_launcher must contain an {image} placeholder.

Registry: closed, lazy, checked

• Closed tables. _BACKENDS, _TRANSPORTS and _SCHEDULERS map a name to "module:Class" and live in interCEde itself, as dictionaries rather than entry points, so nothing outside the library can add to them (IC-ADR-001 Rejected Ideas).
• Lazy imports. A class is imported only when a configuration asks for it, and cached. A test shows that import intercede imports no backend.
• Fails fast. Every registered class provides from_settings(...). The registry builds the object and checks it with isinstance against JobBackend, Transport or Scheduler. A missing module, a missing class, a missing from_settings, or an object that breaks its protocol is reported as a RegistryError at resolution, never at the first call.
• Composition. For batch, the transport is built from its own settings and the scheduler from the batch settings, then both are passed to the batch backend. So ssh+slurm and local+slurm share one class.
• registry.available() lists what a configuration can ask for in this version, for tooling.

Errors. ConfigurationError means the configuration is invalid. The new RegistryError means the configuration is valid, but this version does not ship what it asks for, or an entry is broken.

The tables currently contain only the internal fake, which the conformance suite now builds through the registry. Each real backend from issues #19 to #23 adds one line plus a from_settings. Until then, valid real configurations correctly report, for example, backend 'arc' is not available in this version of interCEde.

Differences from issue #18 and the ADR

• No entry points, no plugins and no override precedence. Issue [Feature]: Typed registry over entry points (backends / transports / schedulers) #18 predates the revised IC-ADR-001, which makes the backend set closed. The tables are plain dictionaries instead.
• One flat public configuration model instead of public per-family models. ADR-001 §5 sketches public per-kind models. This follows the flat shape of the ADR-003 examples ({"kind": "batch", "transport": "ssh", "scheduler": "slurm", "host": ...}) and keeps the per-family settings internal.
• Only kinds that are planned. lsf, sge, oar and torque are left out for now. Adding a kind later is non-breaking.

Tests

• tests/unit/test_config.py: every family, required and refused fields, auth values, limits, and the error messages
• tests/unit/test_registry.py: resolution per family, composition, broken entries, caching, available(), lazy imports, and that calls on the returned object reach the registered class
• tests/unit/_registry_dummies.py: stand-in classes the tests register

163 tests pass on Python 3.11 and 3.13, and pre-commit, including strict mypy, passes.