openswarm

Claude plans. Codex, OpenCode Go and DeepSeek write the code.

Benchmarks: openswarm vs oh-my-openagent (2026-10-17)

Compared with oh-my-openagent (OmO) v5.1.35 (0777686), whose model-core package resolves a delegated task to a model through per-category fallback chains. Scripts, raw results and methodology: bench/ and bench/e2e/. Where OmO comes out ahead, the table says so.

Real GitHub issues (SWE-bench Verified), 2026-10-18. Six hard instances from sympy, pytest and pylint; success means the official FAIL_TO_PASS and PASS_TO_PASS tests all pass. Every arm runs on the Pi coding agent with DeepSeek-V4.1-Flash (deepseek-flash), n=1 per cell, 20-minute cap; details and commands are in bench/swebench/.

Arm	Resolved (official tests)	Median wall-clock (IQR)	Median $ per task (DeepSeek list price)	Total $ (all runs)	n
Plain agent (baseline)	3/6 (50%)	124 s (110–175)	$0.0234	$0.16	6
superpowers v7.0.0	2/6 (33%)	154 s (136–340)	$0.0285	$0.31	6
OpenFlow 0.2.1	4/6 (67%)	230 s (200–266)	$0.0412	$0.23	6
openswarm (planner + Pi executor)	5/6 (83%)	228 s (187–254)	$0.0377	$0.21	6
oh-my-openagent v5.1.35	not run ¹	–	–	–	0

Caveats: n=1 on 6 instances, so one or two resolves are within noise. The two workflow tools took about 1.8x the baseline's median wall-clock and cost 76% (OpenFlow) / 61% (openswarm) more per task. The first published table undercounted pytest because of a shared temp-dir race in grading; it is now fixed and regraded (regrade notes). On the MiniMax-M3.1-Flash-Preview subset, baseline resolved 3/4 and superpowers 2/4; openswarm had no clean MiniMax run because the plan's usage cap was hit.

Raw per-run records and logs: bench/swebench/results/. Optimization backlog from these results: #3.

Synthetic end-to-end tasks. 8 small real tasks (4 JS, 4 Python), hidden acceptance tests applied only after the agent exits. For these synthetic tasks, every arm runs grok-4.7-high through the local cursor-agent (openswarm uses its new --backend cursor executor).

Arm	Success (hidden tests)	Median wall time (IQR)	Median tokens: total / input+output	n
Single agent (baseline)	24/24 (100%)	55 s (46–132)	105k / 28k	24
openswarm (planner + worktree executor + host verify)	24/24 (100%)	220 s (137–356)	207k / 64k	24
oh-my-openagent	not run ¹	–	–	0

On tasks this small the single agent already succeeds every time. openswarm's planner call and isolated, verified run cost about 4× the time and 2× the tokens. ¹ OmO runs inside OpenCode/senpi with its own provider logins. Cursor's subscription can't serve it, and paid APIs were out of scope for this benchmark. The estimated metered cost to run it on DeepSeek is about $1.3 (details).

Routing under quota pressure. A synthetic 80-task trace × 7 quota scenarios (560 decisions per router), with the same providers for both routers (Codex subscription, OpenCode Go, DeepSeek). Costs are computed from the price tables in bench/prices.json × estimated tokens. They are not bills.

Metric	openswarm	OmO	Codex-only baseline
First attempt sent to an already-exhausted quota	0.0%	23.4% (254 failed attempts)	28.6%
Decisions with no usable model (this provider set)	0.0%	19.8% ²	28.6%
Review tasks sent to a different model family	54.8%	71.4%	0%
Codex 5h window used per trace pass (computed)	0.48	0.85	1.78
Metered spend, all scenarios (computed)	$0.00	$1.54	$0.00
Routing decision latency, in-process median	0.008 ms	0.026 ms	–
Providers / model ids known to the router	3 backends / 27	26 / 36	–

openswarm reads quota windows before it routes, so it never spends an attempt on an exhausted plan. By design, OmO learns about an exhausted plan from the usage-limit error and then moves to the next rung. OmO keeps review work on a different model family more often, and it covers far more providers. ² 49 of OmO's 111 unrouted decisions are writing tasks. In OmO that chain only names Google/Anthropic models, which this provider set does not connect.

openswarm is a quota-aware dispatcher for coding agents. Your planner (Claude Code) breaks work into tasks and reviews the results. openswarm sends each task to the cheapest executor that can handle it, runs it in an isolated git worktree with a hard time budget, checks the result on the host, and reports back. Nothing runs forever.

_{The demo runs offline against a synthetic quota state (examples/demo-state.json), so you can reproduce it without any backend installed.}

Why

If you pay for several AI subscriptions, the scarce one is usually the planner. Claude Pro runs out long before a Codex or OpenCode Go plan does. Spending Claude tokens on typing out a refactor wastes the expensive model.

openswarm splits the work:

• Claude decomposes the task, writes acceptance criteria, reviews diffs and integrates.
• Executors do the typing:
  • Codex CLI is the main executor.
  • OpenCode Go handles cheap or mechanical work, non-GPT second opinions, overflow, and 1M context.
  • DeepSeek is the metered last resort.
  • Cursor CLI (cursor-agent) runs only when you pass --backend cursor, or when you add cursor to routing.default_order. It is not part of automatic routing.

Routing decisions use live quota data, not guesses.

Features

• Live model discovery. Reads the current Codex and OpenCode model catalogues and sorts them into tiers (fast, balanced, deep). Legacy models go last.
• Quota-aware routing. Looks at 5-hour, 7-day and 30-day windows, burn pace and reset times:
  • At the soft limit, easy work spills to other backends.
  • A hard limit skips the backend.
  • Quota that is about to expire unused gets spent at higher effort.
• Isolated execution. Each run gets its own git worktree and openswarm/<id> branch.
  • Dependency dirs (node_modules, .venv) are symlinked in and never committed.
  • Nothing is pushed.
• Hard budgets. --budget-minutes kills the executor's whole process tree. Watchers expire too, and you re-arm them on purpose.
• Host-side verification. --verify '<cmd>' runs your test command on the committed worktree.
  • Its exit code overrides the executor's self-report.
  • --verify-baseline ignores failures that already exist on the base commit.
• Automatic fallback. Rate limits and model-at-capacity errors trip a 30-minute breaker, and the task moves to the next backend in the same worktree.
• Risk alerts. openswarm watch streams progress and flags dangerous executor commands (push, rm -rf .., secret prints, sudo, publish). It also reports stalls and heartbeats.
• Planner-friendly.
  • SKILL.md teaches Claude Code the whole loop.
  • templates/openswarm-workflow.js fans tasks out in parallel.
  • openswarm report gives scheduled check-ins.
• Zero dependencies. One Python file using only the standard library.

Quickstart

Requirements: Python 3.9+, git, and at least one executor: codex CLI logged in, opencode with an OpenCode Go plan, or a DeepSeek API key.

# 1. Install as a Claude Code skill
git clone https://github.com/surlymochan/openswarm.git ~/.claude/skills/openswarm
alias openswarm=~/.claude/skills/openswarm/bin/openswarm

# 2. See what you have: plans, quotas, models, recommended concurrency
openswarm status --refresh

# 3. Ask where a task would go, without running anything
openswarm route --type impl --difficulty 3

# 4. Run a task in an isolated worktree with a 15-minute budget and a host-side test gate
cat > /tmp/task.md <<'TASK'
Add input validation to src/parse.ts: reject empty strings with a ParseError.
Acceptance: new unit test in test/parse.test.ts passes; existing tests still pass.
TASK
openswarm run --cwd ~/code/myrepo --type impl --difficulty 2 --budget-minutes 15 \
  --verify 'npm test' --task-file /tmp/task.md

# 5. Review and merge (openswarm never merges or pushes for you)
git -C ~/code/myrepo merge --no-ff openswarm/<run-id>

For longer work, detach and watch:

openswarm run --detach --cwd ~/code/myrepo --verify 'pytest -q' --task-file /tmp/task.md   # prints {"id": ...}
openswarm watch <id1> <id2>          # one line per actionable event; exits on DONE or after 28 min
openswarm report --since-minutes 60  # one-screen digest for scheduled check-ins
openswarm stop <id>                  # kill the executor; partial work is still committed to its branch

You can try routing with no backend installed:

OPENSWARM_HOME=$(mktemp -d) OPENSWARM_STATE_FILE=examples/demo-state.json bin/openswarm status

In Claude Code, just say things like "dispatch this to Codex", "swarm these five refactors", or "use openswarm". The skill takes it from there.

How routing works

Input	Effect
--type review/audit	Goes to a non-GPT family first (OpenCode Go) for an independent opinion
--type mechanical/docs/test at difficulty ≤ 2	OpenCode Go first, to save Codex
--difficulty 1..5	Sets tier and effort: fast/low, fast/medium, balanced/medium, balanced/high, deep/high
--context-tokens > 220k	Prefers 1M-context backends
A window ≥ 95% used	Skips the backend
A window ≥ 85% used	Spills difficulty ≤ 3 to other backends; harder tasks stay with effort −1
Burn pace > 1.25× even	Effort −1
Resets within 24h with < 60% used	Effort +1, so quota isn't wasted
DeepSeek balance below floor	Skips DeepSeek
--backend cursor	Forces the Cursor CLI. Cursor is never chosen on its own unless cursor is listed in routing.default_order

The full rules are in SKILL.md. Every threshold is in policy.json.

Architecture

        ┌──────────── planner (Claude Code + SKILL.md) ────────────┐
        │  decompose → acceptance criteria → review → integrate    │
        └───────┬──────────────────────────────────────▲───────────┘
                │ openswarm run / route / watch           │ result.json, diff, verify
        ┌───────▼──────────────────────────────────────┴───────────┐
        │ openswarm                                                   │
        │  probe ─► state.json (models, tiers, quotas, breakers)   │
        │  route ─► backend/model/effort + reasons + fallbacks     │
        │  run   ─► git worktree ─► executor ─► commit ─► verify   │
        │  watch ─► MSG / RISK / STALL / BUDGET / DONE             │
        └───────┬───────────────────┬───────────────────┬──────────┘
          codex exec          opencode run         opencode run
         (Codex plan)       (OpenCode Go plan)    (DeepSeek API)

Cursor CLI is a fourth executor, opt-in only (`--backend cursor`).

Path	What
bin/openswarm	The CLI: probe, status, route, run, watch, runs, report, stop, ingest-claude, ledger
SKILL.md	Claude Code skill: the planner's operating manual
policy.json	Defaults: thresholds, difficulty map, model caps, local data sources
backends/*.json	One manifest per executor: invocation, quota source, data boundary
templates/openswarm-workflow.js	Parallel fan-out template for Claude Code workflows
examples/demo-state.json	Synthetic state for offline demos
~/.claude/openswarm/ ($OPENSWARM_HOME)	Runtime data: state.json, ledger.jsonl, breakers.json, runs/<id>/

Configuration

Override any key in $OPENSWARM_HOME/policy.local.json. It is deep-merged over policy.json.

{
  "codex": {"plan": "plus"},
  "claude": {"plan": "pro"},
  "thresholds": {"window_soft": 0.8},
  "deepseek": {"keychain_items": ["openswarm-deepseek-api-key"]},
  "workspace": {"sibling_worktrees": false, "link_dirs": ["node_modules", ".venv"]}
}

Environment variable	Purpose
OPENSWARM_HOME	Runtime data dir (default ~/.claude/openswarm)
OPENSWARM_CODEX_PLAN, OPENSWARM_CLAUDE_PLAN	Declare your plans explicitly (recommended)
DEEPSEEK_API_KEY	DeepSeek key, used only for the balance and model-list API
OPENSWARM_OPENCODE_BIN, OPENSWARM_OPENCODE_DB	Non-standard OpenCode install locations
OPENSWARM_CURSOR_BIN	Cursor CLI binary. Otherwise cursor-agent on PATH, then ~/.local/bin/cursor-agent
OPENSWARM_STATE_FILE	Offline mode: route against a saved or synthetic state; nothing is probed
XSWARM_*	Legacy names from before the rename (formerly xswarm); still honoured, OPENSWARM_* wins

What openswarm reads on your machine

openswarm reads other tools' local files only to measure quota. It never uploads them. The status output starts with a Sources line showing what is enabled.

Source	Default	What is read	Switch
~/.codex/sessions/**/rollout-*.jsonl	on	Only rate_limits events (used %, reset time); conversation content is skipped	local_sources.codex_sessions
~/.codex/models_cache.json	on (fallback)	Model catalogue, when codex debug models is unavailable	none
~/.local/share/opencode/opencode.db	on	Read-only: model id and cost per assistant message	local_sources.opencode_db
~/.codex/auth.json	off	Only the chatgpt_plan_type claim, decoded locally. The token is never sent or stored	local_sources.codex_auth_plan
~/.claude.json	off	Only the plan/tier fields of oauthAccount	local_sources.claude_account_plan
macOS Keychain	off	Only items you list in deepseek.keychain_items	deepseek.keychain_items

Prompts and diffs go to the executor's model provider. Each backend's data boundary is listed in backends/*.json. Don't put secrets in task prompts.

Safety model

• Codex runs with -s workspace-write. OpenCode gets edit and bash permission scoped to the worktree with external_directory: deny.
• Executors never commit; openswarm commits on openswarm/<id>. Nothing is pushed or merged automatically.
• If a worktree can't be created, the run fails. It never falls back to editing your checkout in place.
• Codex ultra effort is never used, because it self-delegates and blows budgets.

Status and limitations

• Developed and used on macOS; Linux is covered by CI; Windows is untested.
• Model names, plan caps and the OpenCode Go price table in policy.json change often, so PRs are welcome.
• The Claude quota comes from Claude Code's statusline hook or openswarm ingest-claude. Without either, reviews default to light.

Contributing

See CONTRIBUTING.md. Run the tests (stdlib only, no backends needed):

python3 -m unittest discover -s tests -v

License

Apache-2.0