Add muse2api media provider for muse.ai rendering - #70
Merged
Merged
Conversation
Flow Kit -> Muse2APIProvider -> muse2api -> muse.ai. A new `muse2api` MediaProvider renders scene/reference images and i2v video through a muse2api gateway (https://github.com/crisng95/muse2api): - agent/services/muse2api_client.py: async HTTP client for /v1/images/generations, /v1/videos (+ task polling) and /v1/media, Bearer auth, OpenAI error bodies surfaced as "muse2api <code>". Input frames are sent inline as data URLs so the gateway can run on another host. - agent/sdk/services/muse2api_provider.py: translates ProviderJobs, downloads results into output/_shared/muse2api/ as file:// URLs with minted UUID media_ids (Flow-shaped results, pipeline unchanged). Chained end frames and r2v fail loudly unless MUSE2API_ALLOW_DEGRADED=1; edit/upscale/audio are not offered. - Registered in ProviderRegistry; available once MUSE2API_URL is set. - Config, README, fk-provider/fk-doctor and generation skills updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MfvQsdxJhWw15u6Zm51YJM
Spell out that the muse2api gateway is a separate service Flow Kit does not ship or start: clone crisng95/muse2api, configure the browser driver, start it, import a muse.ai account, verify /readyz, then point Flow Kit at it with MUSE2API_URL / MUSE2API_KEY. Adds a pointer from Quick Start and the env-var table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MfvQsdxJhWw15u6Zm51YJM
…locked") Two coroutines hitting get_db() before the connection existed both saw None and both connected; the second overwrote the global, so the first ran its PRAGMAs (including wal_checkpoint) on the second's connection while a statement was open on it. That failed intermittently with "sqlite3.OperationalError: database table is locked" in test_assistant_provider on CI (Python 3.10, also on main in run #33) and leaked a connection whose worker thread can keep the interpreter alive. Connect under a per-event-loop init lock (separate from _db_lock, which crud holds while calling get_db), set up a local, and publish it only when ready. Adds a regression test counting connect() calls under concurrent first callers; it fails without this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MfvQsdxJhWw15u6Zm51YJM
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new media provider that renders images and videos through a muse2api gateway, which exposes the muse.ai web app as an OpenAI-compatible HTTP API.
Summary
This PR introduces
Muse2APIProvider, a new media backend that complements the existingflowandassistantproviders. It allows Flow Kit to render media through a separate muse2api service that manages muse.ai accounts, failover, and browser automation.Key Changes
New HTTP client (
agent/services/muse2api_client.py): Thin async wrapper around muse2api's OpenAI-compatible endpointsPOST /v1/images/generations)POST /v1/videos,GET /v1/videos/{id})GET /v1/media/{name})no_account_available,upstream_quota_exhausted, etc.)New provider adapter (
agent/sdk/services/muse2api_provider.py): ImplementsMediaProviderinterfaceMUSE2API_ALLOW_DEGRADED=1output/_shared/muse2api/asfile://URLs with minted UUIDsConfiguration (
agent/config.py): New env vars for gateway URL, API key, models, timeouts, and concurrency limitsRegistry integration (
agent/sdk/services/registry.py): Registersmuse2apiprovider automatically; becomes available onceMUSE2API_URLis setComprehensive tests (
tests/unit/test_muse2api_provider.py): 200 lines coveringDocumentation: Updated README, provider guide, doctor guide, and skill docs to explain setup, capabilities, error codes, and troubleshooting. The README spells out that muse2api is a separate service that must be cloned from its own repo and started alongside Flow Kit.
Unrelated flaky-test fix (
agent/db/schema.py,tests/unit/test_db_connection.py):get_db()raced on its lazy connect — two first callers both opened a connection, the second overwrote the global, and the first ran its PRAGMAs on the second's connection mid-statement. That produced the intermittentsqlite3.OperationalError: database table is lockedintest_assistant_provider.pyon Python 3.10 (also seen onmain, run Closed: created in wrong repository #33) and leaked a connection. The connect now happens under a per-event-loop init lock and the connection is published only once set up. Regression test countsconnect()calls under concurrent first callers and fails without the fix.Implementation Details
file://URLs, so/fk-refresh-urlsis never neededMUSE2API_ALLOW_DEGRADED=1, which renders them as plain i2vhttps://claude.ai/code/session_01MfvQsdxJhWw15u6Zm51YJM