Skip to content

reference.md

Latest commit

 

History

History
222 lines (173 loc) · 18.4 KB

reference.md

File metadata and controls

222 lines (173 loc) · 18.4 KB
Raw

Maestro reference

Details that the README leaves out to stay short.

Why Maestro instead of…

Tool What it is Why Maestro instead
Claude Code Dynamic Workflows The model generates the orchestration on the fly — different every run Maestro's flow is human-authored and identical every run: reviewable, diffable, ownable by a lead
Conductor / Temporal Server-side durable orchestration engines Maestro runs locally in your editor — no runtime, no infra, no ops
CrewAI / AutoGen / Swarm Agent frameworks that need their own model credentials Maestro uses your existing host + subscription; it holds no keys and calls no APIs itself

Maestro isn't trying to replace a durable production orchestrator or an autonomous agent. It owns the desk-side layer: the reviewed, repeatable flow a developer runs while building.

How the engine works

       you: /maestro or $maestro my-feature
                     │
       ┌─────────────▼──────────────┐    .maestro/engine/maestroctl.py (stdlib python3)
       │  LEAD AGENT (your session) │───► next → ONE action as JSON
       │  dispatches, never decides │◄─── complete / gate-record / input-record / fail
       └──┬─────────┬──────────┬────┘
          ▼         ▼          ▼
      subagents   scripts    you (gates + interview)
      (skill +    (validators,
       model per   stubs)
       step)
  • workflow.yaml — the graph: 6 node types (agent, interview, gate, script, parallel, subworkflow), per-node routes with tiny conditions, and back-edges for loops (an arrow to any earlier step; the engine cascade-resets downstream work and enforces a per-node visit cap so loops can't run away). Spec: docs/workflow-spec.md.
  • .maestro/runs/<slug>/state.yaml — the run ledger. Only the engine writes it. Resume, revise-cascades, gate history, parallel-join bookkeeping all live here.
  • The lead agent never interprets the graph. The deterministic resolver serves one fully-rendered action at a time; the LLM just dispatches it. That's what makes an LLM-driven orchestrator reliable — and it's all plain, tested Python (see CONTRIBUTING.md; no LLM in the loop).
  • Agent steps are instruction-first: write what the step should do; optionally pin a skill and model. Worker execution is the default; execution: lead keeps small context-continuous steps in the current session. Claude Code and Codex run independent worker steps with native subagents; Cursor uses the same engine inline and sequentially.

Install details

Installs: our skills/commands/agents into .claude/ or .cursor/, or repo-scoped Codex skills into .agents/skills/; the engine + workflows/ + ui/ + docs/ all live under a single .maestro/ parent. The engine is stdlib-only python3 — no execution CLI, config file, or additional package to install. When it finishes it prints host-specific next steps and points at ./maestro help. Upgrade by re-running the same command (it re-fetches, overwrites .maestro/engine//.maestro/ui/, and never overwrites a workflow you edited — see Upgrading in detail). (Private-repo fork? The piped curl can't authenticate — clone it and run ./install.sh from the checkout.)

Maestro init

The one-liner above installs the core pack. To finish setting the repo up, open your host and run /maestro-init (Claude Code/Cursor) or $maestro-init (Codex) once. It:

  1. detects your stack (detect-stack) and re-runs the installer with the matching --stack …, so only the core pack + the per-stack skills/agents your code actually uses get installed;
  2. builds the living-docs knowledge base (build-knowledge) — per-domain technical + functional docs and an architecture diagram under docs/, plus a per-repo docs/codebase-map.md (the standing grounding each HLD reads before choosing an architecture), which the design and review steps read as context.

The codebase map is commit-tracked and incremental: the engine (codebase_scan.py) records the git commit each docs/codebase-map.md reflects, so re-running (and the post-feature refresh in the archive phase) only re-examines the diff since that commit — not the whole repo. In an umbrella workspace each repo keeps its own map, committed and pushed in that repo.

/maestro-init                 # detect stack → install matching packs → build docs
$maestro-init                 # the same operation in Codex

It's idempotent — safe to re-run after a stack change (it refreshes, never duplicates). Newly installed skills/agents become active once your host reloads its config; then start a feature with /maestro <slug> or $maestro <slug>.

Prefer to pick stacks by hand? The pack ships per-stack reference skills + reviewers (Go, Java/Spring, Kotlin, Python/Django, React, Vue, Angular, Node, Rust, Flutter, Android, DB…); install only what your repo uses instead of running /maestro-init:

./maestro install codex --stack go,react,db   # core + those stacks only

Anything without a stack: tag is core and always installs; --stack all (or no flag) installs every stack.

Prerequisites in detail

  • Recommended: an umbrella workspace. For the best SDLC output, don't run Maestro inside one isolated service repo — stand up a parent/umbrella repo per project, clone the service repos you're changing (gitignored) underneath it, and install Maestro into that parent. The lead agent then sees the whole stack at once. Full setup: docs/umbrella-workspace.md. A single repo works too; the umbrella just gives better cross-repo results.
  • python3 ≥ 3.8 (stdlib only — nothing to pip install). Check: python3 --version.
  • git and, to install, bash + curl. Every repository selected for implementation must have at least one commit — each repo's implementation runs in its own isolated git worktree, which an uninitialized repo cannot provide (git init && git add -A && git commit once in each selected repo if needed).
  • An AI coding harness that supports skills — Claude Code (parallel subagents and Claude model aliases), Codex (native parallel subagents; invoke repo skills with $), or Cursor (inline sequential fallback).
  • Optional: node/npx, only for auto-installing the six external Superpowers helper skills. Without it the flow still runs (skills fall back to inline behavior).
  • OS: macOS and Linux are first-class. On Windows use WSL — the installer is bash and the engine's file lock uses fcntl (a msvcrt fallback exists, but WSL is the tested path).

The visual builder

./maestro ui serves the builder on a local port (default 8422, --port to change). Because it knows your repo, the Runs view and Open/Save work over HTTP in any browser — or open .maestro/ui/builder.html directly and it runs fully offline from file:// (the folder view needs Chrome/Edge).

  • Author visually — New blank canvas, Open any workflow in the repo, or Paste YAML. Drag nodes and arrows; an arrow pointing back is a loop (dashed, with its repeat-limit badge). Nodes are instruction-first (skill defaults to Auto), gates' options are their edges, and parallel branches drill in. Live validation, and Save (⌘S) exports YAML the engine accepts.
  • Runs (top-right) — a read-only live status board: pick a slug and the graph colours in as it progresses (green done, orange running, red failed, grey pending).

./maestro ui and ./maestro install are dev tooling only — the wrapper never drives a run or writes .maestro/**/state.yaml (that stays the Maestro skill + engine's job).

Engine commands

You almost never call the engine yourself. The lead agent issues its verbs — init, next, complete, gate-record, gate-input-record, fail, runs, … — for you as it drives the graph. Only two are worth running by hand, for inspection:

python3 .maestro/engine/maestroctl.py status --slug my-feature              # step table, gates, active steps
python3 .maestro/engine/maestroctl.py validate .maestro/workflows/my.yaml   # lint a workflow before running it

If a run gets stuck, reset --slug <slug> --step <id> --cascade and rebase --slug <slug> are your recovery levers — see Working as a team.

Pausing & resuming a run

A run is durable and does not have to finish in one turn. All progress lives in the engine ledger (.maestro/runs/<slug>/state.yaml), so you can close the session anytime and pick up later — resume with /maestro <slug> or $maestro <slug> and it continues from the exact step, re-running nothing.

Sometimes the lead agent ends its turn mid-run — after a long step, or because the harness ended the turn. That's normal and loses nothing; just resume. Whether the loop auto-continues past such a pause depends on your harness:

  • Claude Code — auto-continue with an opt-in Stop hook. Maestro ships .maestro/engine/stop_hook.py: while a run still has autonomous work pending it keeps the loop going, and it steps aside (lets the turn end) at every human gate and when the run finishes — the human stays in charge at gates. It's read-only and off by default. Enable it by adding to .claude/settings.json:

    {
  "hooks": {
    "Stop": [
      { "hooks": [ { "type": "command", "command": "python3 .maestro/engine/stop_hook.py" } ] }
    ]
  }
}

  • Cursor — resume manually. Cursor has no hook system, so the loop can't auto-continue; when it pauses, just re-run the /maestro <slug> command. Tip: run phase-by-phase (/maestro <slug> .maestro/workflows/design.yaml, then implementation) to keep each session short — a long single-session run is the most common cause of a mid-run stop.

  • Codex — resume from the durable ledger. Native subagents can run independent Maestro actions in parallel. If the lead turn pauses or you close the task, invoke $maestro <slug> again; the engine serves the exact pending action. If Codex subagents are disabled in your configuration, Maestro automatically uses the serial inline path.

Working as a team

Maestro is a shared, paved road, not a per-developer toy: keep the flows in a central, PR-reviewed repo owned by your leads/platform team (changes go through review, same as code), and governance falls out of the graph — route the "approve for release" gate to a lead while anyone runs the steps up to it, no extra RBAC layer needed.

The run ledger .maestro/runs/<slug>/state.yaml is written only by the engine and is git-tracked on purpose (so a run resumes on any machine). Team ownership follows ledger ownership:

  • One owner per slug at a time. Keep one coordinator on the parent feature slug. LLD work is automatically split into repository child slugs, so backend, frontend, and Flutter can push independently without inheriting or conflicting with another team's approval.
  • Commit the child run with its LLD. A repo approval updates only .maestro/runs/<feature>--lld--<repo>/ plus that repo's published LLD and receipt under the parent. Pull all approved child changes, then refresh the parent LLD status gate.
  • Resolving a state conflict: never hand-merge state.yaml. Take one side, then run python3 .maestro/engine/maestroctl.py status --slug <slug> to see where it stands and continue, or reset --slug <slug> --step <id> --cascade to redo from a known-good step.
  • Old-format runs: selecting one through /maestro offers a one-time upgrade. The engine backs up its state and artifacts, converts old PRD/HLD structure without reopening clear decisions, asks one baseline approval, and imports each old LLD as an independently validated, unapproved child draft. It never transfers an old approval merely because a Markdown file exists.
  • If you edit a workflow file mid-run, the engine halts on the next command with a hash mismatch — nobody silently diverges from the lead's flow. Accept the edit with maestroctl rebase --slug <slug> (it re-validates first) or start over with reset --slug <slug> --all.

Upgrading in detail

  • Upgrade: re-run the install one-liner (add --dry-run to preview). It overwrites .maestro/engine/, .maestro/ui/ and the installed skills; .maestro/index/ provenance and .maestro/runs/ are left alone. Workflows upgrade file by file, never over a local edit:

    Your .maestro/workflows/<name>.yaml Upgrade does
    missing adds it
    byte-identical to a stock copy (the last install or any past release) replaces it
    edited, upstream changed it keeps yours, writes upstream's as <name>.yaml.upstream
    edited, upstream unchanged keeps yours
    your own file name (e.g. my-sdlc.yaml) never touched

    Matching is strict: any change — even a comment, re-formatting or a re-save from the builder — counts as an edit, so only untouched stock copies are ever replaced (no backups needed). Reconcile each .upstream into your file — or mv it over yours to take upstream, after which the file auto-updates again — then delete it. Commit .maestro/workflow-manifest.json (the record of what the last upgrade offered). Every workflow is validated afterwards and failures are listed. Customising a copy under your own name avoids .upstream reconciliation entirely. A workflow that changes still halts an in-flight run until maestroctl rebase --slug <slug>. On the next /maestro resume, an old run is detected and upgraded once only after your confirmation.

  • Uninstall: delete the installed dirs (.claude/skills|commands|agents, .cursor/…, .agents/skills/, .maestro/engine/, .maestro/ui/). Keep .maestro/runs/<slug>/ — that's your work.

  • "workflow changed" halt: see Working as a team above (rebase or reset).

  • A step won't complete (exit 4): the engine refuses to advance without the declared artifact (non-empty) and output fields — re-run the step or maestroctl fail it.

  • status any time: python3 .maestro/engine/maestroctl.py status --slug <slug> prints the step table, visit counts, and gate history.

Customizing

The engine is generic; the SDLC pack is just one workflow. To make it yours:

  • Write your own workflow — a YAML file with nodes: (agent / interview / gate / script / parallel / subworkflow), routes, and back-edges for loops. The builder writes it for you. Spec: docs/workflow-spec.md. Run it with /maestro <slug> path/to/your.yaml or $maestro <slug> path/to/your.yaml.
  • Bring your own skills — an agent node names a skill; that can be one of ours, one you author (.claude/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md), or a third-party pack (Obra, Superpowers, …). Because the workflow node supplies the instruction, inputs, artifact path and output fields at runtime, a skill only has to describe how to do its job — so swapping one for another is a one-line skill: change, or omit skill: entirely and let the harness auto-pick from installed skills by description.
  • Change a shipped step's behaviour — edit its installed skill; the flow is untouched.
  • Models — per node or per workflow (defaults.model); values are host-specific and passed only when the host accepts the exact identifier. haiku / sonnet / opus work in Claude Code; use valid Codex model IDs in a Codex-specific workflow rather than relying on guessed alias translation.
  • Loop bounds — per node max_visits (+ on_exhausted), backstopped by defaults.max_visits (default 10).
  • The publish script in the example pack is a POC stub — wire it to your documentation/repository publishing convention. Implementation and QA do not fake-merge independent repos: the engine verifies and passes each repo's exact worktree, branch, and commit to QA.
  • Memory (improves over time). Run /build-knowledge or $build-knowledge once per workspace to seed .maestro/memory/knowledge/ from your codebase; the design and review steps read it, frozen at init. After a feature, the pre-merge archival phase distills lessons and consolidates them — a lesson becomes trusted (and injected into future runs) only once ≥3 runs corroborate it. See docs/memory.md.

File layout

After install, in your project repo:

.claude/  (or .cursor/ — set by the installer for those hosts)
  skills/     one SKILL.md per SDLC step + maestro (the lead agent) + bootstrap
              (detect-stack, maestro-init, build-knowledge, retrospect)
              + per-stack reference packs tagged stack:<x>
  agents/     core subagents (planner, implementer, reviewer, qa, analyst, general, discussant)
              + per-stack reviewers tagged stack:<x>
  commands/   /maestro, /maestro-init and /maestro-discuss slash-command shims (steps are invoked as skills)
.agents/skills/  Codex repo-scoped copy of the same flattened skill pack; invoke with `$`
.maestro/
  engine/     the deterministic engine (validate · init · next · complete · gate-record
              · interview-record · interview-record-batch · interview-brief · gate-input-record · fail · reset · rebase · status · todo · graph · note · runs)
              + ui_server.py + schemas
              + stop_hook.py (opt-in Claude Code auto-continue hook — see "Pausing & resuming")
              + codebase_scan.py (per-repo codebase-map commit tracking for incremental refresh)
  workflows/  the example pack: sdlc-main / workspace-sync / design / impl / qa,
              split into small stages: sdlc-main's design-review / delivery,
              design's design-import / -intake / -prd / -map / -hld / -lld /
              -contract (+ repo-lld per repository),
              plus archive / build-knowledge / retrospect — customize or replace
  ui/         builder.html (single-file visual editor)
  docs/       workflow-spec.md (the workflow spec)
  memory/     knowledge/ (seeded by /build-knowledge)
  index/      engine-owned living-doc commit provenance
  runs/<slug>/  everything for one feature: requirement/ + all artifacts + state.yaml
maestro       repo-local dev wrapper: `maestro ui` (serve the builder) + `maestro install` + `maestro help`