Skip to content

[debate] agent-spec v1: per-action mandate with enforced_by, replacing the autonomy label #10

Closed

Description

@RevenuePuzzles

What you're proposing

Replace agent-spec.schema.yaml v0.1.0 with a v1 that assigns authority action by action, each row requiring an enforcing mechanism, and drop the single per-agent autonomy label the current format requires.

What it affects

  • An aux.H* heuristic ID or definition
  • An aux.T* trust stage ID or definition
  • A tg.* gap ID or definition
  • A schema file in /schemas/
  • Public vocabulary (the words we use externally)
  • Other — aux-audit's rule set and its published output contract

Why now

The repository contradicts itself, in one file.

schemas/trust-architecture.yaml says:

Autonomy is not a slider you push to the right. It is a spectrum you choose per action, calibrated to evidence and stakes.

Enforcement lives in a mechanism, not in a prompt. A prompt saying "only update qualification information" is not the same as a tool that exposes only the qualification field. The first is a request; the second is a boundary.

schemas/agent-spec.schema.yaml requires:

autonomy:
  type: enum
  values: ["human-in-the-loop", "human-on-the-loop", "autonomous"]
  required: true

One slider position, for the whole agent. The reference input format requires precisely the abstraction the standard tells readers to avoid.

It is not cosmetic. aux-audit scores on it:

  • rules.ts — aux.H01 and aux.H03 take their base score from spec.autonomy (lines 55–70, 99–112)
  • guarantees is free text, described in the schema as "explicit contracts the agent makes with the user (e.g. 'will always ask before sending external email')" — a promise. Three rules read it. There is no field anywhere in the format for the mechanism that enforces it.

So the audit's two most prominent heuristics rest on a label the canon calls wrong, and its notion of a guarantee is the exact thing enforcement.rule says is not a control.

Second, smaller defect: the file is not JSON Schema, though it uses the reserved $schema key as a container for a bespoke notation ({ type: string, required: true }). Any JSON Schema tool reads it wrong, and check_schema fails on it.

This is the first thing a careful reader finds. The docs say "don't use autonomy labels"; the reference implementation requires one.

Proposal

Adopt a v1 built as real JSON Schema draft 2020-12, whose core is:

mandate:
  type: array
  description: Assigned authority, action by action. Never an autonomy score.
  items:
    required: [action, authority, enforced_by]
    properties:
      authority:
        enum: [autonomous, human_approval, human_only, prohibited]
      enforced_by:
        description: The mechanism, not the intention.
                     "System prompt" is not a valid answer for human_only or prohibited.

Plus human_control (observe / interrupt / approve / override / disable — a kill switch alone is not oversight) and shutdown with a required tested boolean, because an untested kill switch is a claim, not a control.

A working draft exists and is the basis of this proposal.

Two things to settle before adoption

  1. ID namespace. The draft's trust_stage enum is T01–T04. This repo's canon is aux.T01–aux.T04. Left unfixed this re-introduces the same fork that feat(schemas): merge the substance from the parallel schema set #9 had to unpick.
  2. Should the input carry the output at all? trust_stage in the draft is described as "the stage demonstrably earned" — but aux-audit computes the trust stage. Either the audit ignores the field, or it does something better: compares the claim against its own computation and reports the gap. "You declare T03; the spec supports T01" would be the most useful line the tool could print. That is a feature decision, not a schema detail.

Migration note

This is aux-audit v0.2, not a patch. Concretely:

  • spec.ts — rewritten. Real JSON Schema validation needs a validator; the package currently has exactly one runtime dependency (yaml), so adding ajv is its own call.
  • rules.ts — aux.H01 and aux.H03 stop asking "what is the autonomy label" and start asking "does every non-autonomous action name an enforcing mechanism". That is a better question and a different computation.
  • All fixtures and 34 tests.
  • repos/aux-audit/README.md publishes the v0.1 output contract; scores computed under v1 are not comparable to scores computed under v0.1, so the version has to move where anyone can see it.

Proposed sequencing, one PR:

  • schemas/agent-spec.v0.yaml — the current file, kept readable for anything already written against it
  • schemas/agent-spec.schema.yaml — v1
  • the CLI rewrite lands in the same PR. Renaming first would leave a window where the canonical filename describes a format the shipped CLI cannot read.

Alternatives considered

  • Keep v0.1, soften the docs. Rejected: the doctrine is the product. Weakening "define authority for specific actions" to match a schema is the wrong direction of fit.
  • Add mandate alongside autonomy, keep both. Rejected as a permanent state — two ways to say the same thing is how a standard rots — but it is a reasonable transitional shape if v1 needs to ship before the CLI rewrite is ready.
  • Do nothing before launch. Rejected. This is a self-contradiction in the flagship artifact, and it is more expensive to be shown it than to fix it.

Interim, regardless of this outcome

The repository should not ship a known self-contradiction silently. A deprecation header on agent-spec.schema.yaml naming both defects and pointing here is going up now — the same treatment evolution_stage: schema-undefined got, which is: name the defect where the reader meets it.

Activity

  1. RevenuePuzzles commented on Sep 4, 2026

    @RevenuePuzzles
    ContributorAuthor

    Measured: the migration is cheaper than this Issue estimated, and it has one trap

    A v1 package arrived — schema, migrate-v0-to-v1.py, MIGRATION.md, example-agent-spec.yaml, validate.py. Tested against a copy of main rather than reasoned about.

    Cost: about four lines, not a rewrite

    The Migration note above called this "aux-audit v0.2, not a patch". For the file swap, that was wrong.

    tests
    v1 dropped in as agent-spec.schema.yaml, no CLI change 5/34 pass
    + spec.ts repointed to agent-spec.v0.yaml 32/34 pass
    + the two tests that read the schema by name repointed 34/34 pass

    Total: one line in spec.ts:14, one entry in sync-schemas.mjs, two references in test/spec.test.ts. The v0 file's content is semantically identical to what is on main today — only comments differ.

    What remains genuinely v0.2 is the rule work: aux.H01 and aux.H03 reading mandate rows and enforced_by instead of a label. That is unchanged and still the real job.

    The trap: publishing v1 under the canonical name breaks the tool

    With v1 as agent-spec.schema.yaml and the CLI reading v0, a spec written to the canonical schema gets:

    $ aux-audit run example-agent-spec.yaml
    aux-audit: agent spec is not valid:
      - `version` is required and must be a non-empty string
      - `surface` is required
      - `autonomy` is required
      - `memory` is required
    exit 2
    

    Four errors demanding fields v1 deliberately removed, including autonomy — the exact field this Issue exists to eliminate. That trades an inconsistent document for a broken tool, which is worse: a reader can reason around the first.

    Proposed: land v1 canonical, with a shim

    Rather than defer v1 until v0.2 is ready, add version detection to spec.ts — roughly fifteen lines:

    • a spec carrying mandate or spec_version is v1
    • aux-audit v0.1 does not score v1, and says so:
      "This is an agent-spec v1 document. aux-audit 0.1.0 reads v0.1.0 — see MIGRATION.md. v1 scoring lands in 0.2.0."
    • exit 2, one accurate line, no false "required" errors

    Doctrine gets fixed at the canonical filename now, which is the launch-blocking half. The tool stops lying about why it refused. And v0 specs keep scoring exactly as they do today.

    Two corrections to the incoming package

    1. validate.py globs its own directory, so it must live at schemas/validate.py. My first run pointed it elsewhere and it skipped nearly everything — my error, not the tool's.
    2. Its claim of being tested "on the real repo contents" holds up: pointed correctly at main it fails, catching that brain-spec.schema.yaml used T01–T04 against this repo's aux.T01–aux.T04. That was my bug from feat(schemas): merge the substance from the parallel schema set #9. Fixed in fix(schemas): brain-spec used the wrong ID namespace; add a cross-schema check #12, which also adds the validator to CI.

    Still open here

    The two questions from this Issue are untouched by any of the above: whether the input format should carry trust_stage at all when the audit computes it, and — if it does — whether aux-audit should report the gap between claimed and supported stage. That second one is still the most useful line this tool could print.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions