Feat/flow batchexecute migration

CLAUDE.md

@@ -7,14 +7,35 @@ Base URL: `http://127.0.0.1:8100`
 ```bash
 curl -s http://127.0.0.1:8100/health
 # Must return: {"extension_connected": true}
+
+curl -s http://127.0.0.1:8100/api/flow/status
+# Must return: {"transport": "batch", "flow_project_id": "<uuid>", ...}
 ```
 
+Also needed: **one signed-in `https://flow.google.com/` tab left open**. Only the
+page can sign a Flow request, so nothing works headless.
+
 ## How to work
 
 - Always use `/fk-*` skills — all rules and workflows live inside each skill
 - Never write scripts to loop API calls — use `POST /api/requests/batch`
 - `media_id` is always UUID format (`xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`), never `CAMS...` strings
-- **On any pipeline error** (request `FAILED`, stuck `PROCESSING`, `extension_connected: false`, HTTP 4xx/5xx from `:8100`, YouTube `HttpError`, error strings like `UNSAFE_GENERATION` / `not found` / `CAPTCHA` / `NO_FLOW_KEY`): invoke `/fk-doctor` before guessing a fix
+- **On any pipeline error** (request `FAILED`, stuck `PROCESSING`, `extension_connected: false`, HTTP 4xx/5xx from `:8100`, YouTube `HttpError`, error strings like `UNSAFE_GENERATION` / `not found` / `CAPTCHA` / `NO_AT_TOKEN` / `NO_FLOW_PROJECT` / `UNSUPPORTED_ON_BATCH_API`): invoke `/fk-doctor` before guessing a fix
+- `flow_key_present: false` is **normal** — the current transport has no bearer token
+
+## Since Flow moved (September 2026)
+
+Flow lives at `flow.google.com` and signs every call in the page. Consequences
+that change how you work:
+
+- **Projects are not created by Flow Kit any more.** Make one in the Flow UI and
+  pin its uuid as `FLOW_PROJECT_ID`, or pass `flow_project_id` to `POST /api/projects`.
+- **Four capabilities are unported** because their payloads were never captured:
+  4K upscale, r2v, start+end-frame chaining, and Omni Flash. They fail with
+  `UNSUPPORTED_ON_BATCH_API` rather than silently producing the wrong thing.
+  `FLOW_ALLOW_DEGRADED=1` drops chaining and r2v to plain i2v; upscale has no
+  fallback. To restore one properly, see `docs/CAPTURE.md`.
+- **A poll saying "Media not found." is not a failure.** Finished jobs report it.
 
 ## Skills
 
@@ -40,7 +61,7 @@ curl -s http://127.0.0.1:8100/health
 | `/fk-status` | Project status dashboard |
 | `/fk-switch-project` | Switch active project |
 | `/fk-fix-uuids` | Fix non-UUID media_ids |
-| `/fk-refresh-urls` | Refresh expired GCS URLs |
+| `/fk-refresh-urls` | Refresh expired signed media URLs |
 | `/fk-doctor` | Diagnose errors + prescribe fixes (Flow/extension/worker/YT) |
 | `/fk-add-material` | Set image material style |
 | `/fk-change-model` | Change video/image model |

README.md

@@ -50,7 +50,7 @@
 
 # FLOW KIT
 
-Standalone system to generate AI videos via Google Flow API. Uses a Chrome extension as browser bridge for authentication, reCAPTCHA solving, and API proxying.
+Standalone system to generate AI videos via Google Flow. Uses a Chrome extension as a browser bridge: it mints reCAPTCHA and runs Flow's batchexecute RPCs inside a signed-in `flow.google.com` tab, which is the only place they can be signed.
 
 ## Showcase
 
@@ -169,17 +169,29 @@ A local React dashboard (`dashboard/`) for monitoring and driving the pipeline 
 ## Architecture
 
 ```
-┌──────────────────┐     WebSocket      ┌──────────────────────┐
-│  Python Agent    │◄──────────────────►│  Chrome Extension     │
-│  (FastAPI+SQLite)│     localhost:9222  │  (MV3 Service Worker) │
-│                  │                    │                       │
-│  - REST API :8100│  ── commands ──►   │  - Token capture      │
-│  - Queue worker  │  ◄── results ──    │  - reCAPTCHA solve    │
-│  - Post-process  │                    │  - API proxy          │
-│  - SQLite DB     │                    │  (on labs.google)     │
-└──────────────────┘                    └──────────────────────┘
+┌──────────────────┐     WebSocket      ┌──────────────────────┐     ┌──────────────────┐
+│  Python Agent    │◄──────────────────►│  Chrome Extension     │────►│  flow.google.com │
+│  (FastAPI+SQLite)│    localhost:9222  │  (MV3 Service Worker) │     │  (signed-in tab) │
+│                  │                    │                       │     │                  │
+│  - REST API :8100│  ── envelopes ──►  │  - reCAPTCHA mint     │     │  batchexecute    │
+│  - Queue worker  │  ◄── responses ──  │  - runs the RPC in    │     │  cookie + `at`   │
+│  - Post-process  │                    │    the page's world   │     │                  │
+│  - SQLite DB     │                    │                       │     │                  │
+└──────────────────┘                    └──────────────────────┘     └──────────────────┘
 ```
 
+Flow signs every call with the session cookie plus a per-page `at` token, and a
+generate also carries a single-use reCAPTCHA. None of that can be replayed from
+outside the browser, so the agent builds the request and the **page** issues it.
+One signed-in Flow tab has to stay open; nothing here works headless.
+
+> **September 2026 — Flow moved.** It now lives at `flow.google.com` and the old
+> `aisandbox-pa.googleapis.com` REST API has no caller: the `Bearer ya29.…` it
+> needed stopped being minted. If you are upgrading from an older Flow Kit,
+> reload the extension (v0.3.0+) and pin `FLOW_PROJECT_ID` — see
+> [Configuration](#configuration). The legacy path is still there behind
+> `USE_BATCH_RPC=0`, but it is a post-mortem tool, not a fallback.
+
 ## Quick Start
 
 ### One-command setup
@@ -203,16 +215,49 @@ pip install -r requirements.txt
 
 ```bash
 # 1. Load Chrome extension: chrome://extensions → Developer mode → Load unpacked → extension/
-# 2. Open https://labs.google/fx/tools/flow and sign in
-# 3. Start agent
+# 2. Open https://flow.google.com/ and sign in — leave the tab open
+# 3. Create a project in the Flow UI and copy its uuid out of the URL
+export FLOW_PROJECT_ID=<that uuid>
+
+# 4. Start agent
 source venv/bin/activate   # if using setup.sh
 python -m agent.main
 
-# 4. Verify
+# 5. Verify
 curl http://127.0.0.1:8100/health
 # {"status":"ok","extension_connected":true}
+curl http://127.0.0.1:8100/api/flow/status
+# {"connected":true,"transport":"batch","flow_project_id":"…","flow_key_present":false}
 ```
 
+`flow_key_present: false` is expected — the current transport has no bearer
+token. Step 3 is not optional: Flow's project-creation endpoint went with the
+migration, so without a pinned project every request fails `NO_FLOW_PROJECT`.
+You can also pass `flow_project_id` per project on `POST /api/projects`.
+
+### Configuration
+
+| Env var | Default | What it does |
+|---------|---------|--------------|
+| `FLOW_PROJECT_ID` | — | The Flow project every RPC is scoped to. Required. |
+| `USE_BATCH_RPC` | `1` | `0` falls back to the pre-migration REST path (dead auth). |
+| `FLOW_ALLOW_DEGRADED` | `0` | `1` lets scene chaining and r2v fall back to plain i2v instead of failing. |
+| `DEFAULT_PAYGATE_TIER` | `PAYGATE_TIER_TWO` | Carried for the DB and dashboard; no longer selects a model. |
+
+### What does not work on the new API yet
+
+Three capabilities have no captured payload, so they fail with
+`UNSUPPORTED_ON_BATCH_API` rather than quietly producing the wrong thing:
+
+| Capability | Status | Workaround |
+|---|---|---|
+| 4K/1080p upscale (`/fk-pipeline` last step) | unported | none — keep the 1080p render |
+| Reference-to-video (r2v) | unported | `FLOW_ALLOW_DEGRADED=1` → i2v off the first reference |
+| Start+end-frame chaining (`/fk-gen-chain-videos`) | unported | `FLOW_ALLOW_DEGRADED=1` → i2v off the start frame |
+| Omni Flash (`model_family=omni_flash`) | unported | use `model_family=veo` |
+
+Restoring one starts with a capture, not a guess: [`docs/CAPTURE.md`](docs/CAPTURE.md).
+
 ## End-to-End Example: "Pippip the Fish Merchant"
 
 A chubby cat sells fish at a market. 3 scenes, vertical, Pixar 3D style.
@@ -751,7 +796,7 @@ These arrive in the response body as `data.error.details[].reason`. The worker a
 | Status | Source | Meaning | Handling |
 |--------|--------|---------|----------|
 | **400** | Flow API | Invalid payload, UNSAFE_GENERATION, entity not found (sometimes) | Route by `details.reason` — some are auto-recoverable, others terminal |
-| **401** | Flow API | Bearer token expired | Extension re-captures token from labs.google tab; request retries |
+| **401** | Flow API (legacy path only) | Bearer expired — on a post-migration profile it was never minted | Switch to the batch path (`USE_BATCH_RPC=1`) |
 | **403** | Extension (`background.js:432`) | `CAPTCHA_FAILED`, `NO_FLOW_TAB`, or `MODEL_ACCESS_DENIED` | CAPTCHA → retry loop; NO_FLOW_TAB → fail (user must open Flow); tier → fail |
 | **404** | Flow API | `media_id` not found (expired upload) | Same as "Requested entity was not found" — auto re-upload |
 | **429** | Flow API | Rate limited / quota | Back off + retry; if `USER_QUOTA_REACHED` appears, fail |
@@ -771,7 +816,10 @@ String patterns in `error_message` that the worker recognizes:
 | `Extension not connected` | Chrome extension offline or WS dropped | 503 returned; worker re-queues PENDING and waits |
 | `extension reconnected` / `extension disconnected` | WS bounce mid-request | Re-queue PENDING without incrementing `retry_count` |
 | `extension_switched` | User switched Flow tabs mid-generation | Re-queue PENDING |
-| `NO_FLOW_KEY` | Extension has no captured bearer token | User must open `labs.google/fx/tools/flow` and sign in |
+| `NO_FLOW_KEY` | Extension has no captured bearer token — **legacy path only**, expected on the batch path | Only meaningful with `USE_BATCH_RPC=0` |
+| `NO_AT_TOKEN` | Flow tab is signed out, on an interstitial, or still booting | Open `flow.google.com`, sign in, let the app load |
+| `NO_FLOW_PROJECT` | No Flow project to scope the RPC to | Pin `FLOW_PROJECT_ID` — **terminal, not retried** |
+| `UNSUPPORTED_ON_BATCH_API` | Upscale / r2v / chaining — payload never captured | See `docs/CAPTURE.md` — **terminal, not retried** |
 | `NO_FLOW_TAB` | No Google Flow tab available for reCAPTCHA | User must open a Flow tab |
 | `Failed to fetch` | Network drop inside extension service worker | Retry with backoff |
 | `timeout` / WS 60s no response | Extension hung mid-request | Re-queue PENDING |
@@ -803,7 +851,7 @@ From `youtube/upload.py` (HTTP errors from YouTube Data API v3):
 | Problem | Solution |
 |---------|----------|
 | Extension shows "Agent disconnected" | Start `python -m agent.main` |
-| Extension shows "No token" | Open `labs.google/fx/tools/flow` and sign in |
+| Extension shows "No token" | Expected on the batch path — there is no bearer token any more |
 | `CAPTCHA_FAILED: NO_FLOW_TAB` | Open a Google Flow tab |
 | 403 `MODEL_ACCESS_DENIED` | Tier mismatch — check `/api/flow/credits`, downgrade model in `models.json` |
 | 403 `PUBLIC_ERROR_UNUSUAL_ACTIVITY` / `reCAPTCHA evaluation failed` | Pause submits, clear cookies for `google.com` + `labs.google` in Chrome, sign back in, then resubmit with ≥1s gap and ≤5 concurrent. Switch network or wait 1–6 h if still blocked |

agent/api/flow.py

@@ -3,6 +3,7 @@
 from pydantic import BaseModel
 from typing import Literal, Optional
 
+from agent.config import USE_BATCH_RPC, FLOW_PROJECT_ID, FLOW_ALLOW_DEGRADED
 from agent.services.flow_client import get_flow_client
 from agent.services.omni_flash import (
     check_omni_flash_status,
@@ -97,10 +98,17 @@ class EditImageRequest(BaseModel):
 
 @router.get("/status")
 async def extension_status():
-    """Check if extension is connected."""
+    """Extension health, and which transport it is being asked to speak.
+
+    `flow_key_present` is a legacy-path signal: the batchexecute path has no
+    bearer token at all, so false is expected there rather than a fault.
+    """
     client = get_flow_client()
     return {
         "connected": client.connected,
+        "transport": "batch" if USE_BATCH_RPC else "legacy_rest",
+        "flow_project_id": FLOW_PROJECT_ID or None,
+        "allow_degraded": FLOW_ALLOW_DEGRADED,
         "flow_key_present": client._flow_key is not None,
     }
 

agent/api/models.py

@@ -36,6 +36,7 @@ def _reload_config(data: dict):
     config.UPSCALE_MODELS.update(data["upscale_models"])
     config.IMAGE_MODELS.clear()
     config.IMAGE_MODELS.update(data["image_models"])
+    config.DEFAULT_IMAGE_MODEL = data.get("default_image_model", "NANO_BANANA_PRO")
 
 
 @router.get("")
@@ -68,6 +69,9 @@ async def patch_models(body: dict):
     """
     current = _read_models()
 
+    if "default_image_model" in body:
+        current["default_image_model"] = body["default_image_model"]
+
     # Deep merge: only update keys that are provided.
     for section in (
         "video_models",

agent/api/projects.py

@@ -7,7 +7,7 @@
 from fastapi import APIRouter, HTTPException
 from pydantic import BaseModel
 
-from agent.config import BASE_DIR
+from agent.config import BASE_DIR, USE_BATCH_RPC
 from agent.models.project import Project, ProjectCreate, ProjectUpdate
 from agent.models.character import Character
 from agent.sdk.persistence.sqlite_repository import SQLiteRepository
@@ -125,6 +125,24 @@ async def _detect_user_tier(client) -> str:
         return "PAYGATE_TIER_ONE"
 
 
+def _read_flow_project_id(flow_result: dict) -> str:
+    """Pull the project uuid out of whichever transport answered.
+
+    The batch path answers `{"projectId": …}`; the legacy tRPC path buries it
+    under result/data/json/result.
+    """
+    data = flow_result.get("data") or {}
+    if isinstance(data, dict):
+        if isinstance(data.get("projectId"), str):
+            return data["projectId"]
+        try:
+            return data["result"]["data"]["json"]["result"]["projectId"]
+        except (KeyError, TypeError):
+            pass
+    logger.error("Unexpected Flow response: %s", flow_result)
+    raise HTTPException(502, "Could not read a Flow project id from the response")
+
+
 def _get_repo() -> SQLiteRepository:
     return SQLiteRepository()
 
@@ -154,25 +172,24 @@ async def create(body: ProjectCreate):
 
     detected_tier = await _detect_user_tier(client)
 
-    flow_result = await client.create_project(body.name, body.tool_name)
-    if flow_result.get("error"):
-        raise HTTPException(502, f"Flow API error: {flow_result['error']}")
-
-    try:
-        data = flow_result.get("data", {})
-        result = data["result"]["data"]["json"]["result"]
-        flow_project_id = result["projectId"]
-    except (KeyError, TypeError) as e:
-        logger.error("Unexpected Flow response: %s", flow_result)
-        raise HTTPException(502, f"Failed to parse Flow response: {e}")
-
-    logger.info("Flow project created: %s", flow_project_id)
+    # On the batch path Flow no longer creates projects for us — the uuid comes
+    # from the request or from FLOW_PROJECT_ID. The legacy path still mints one.
+    flow_project_id = client.flow_project_id(body.flow_project_id) if USE_BATCH_RPC else None
+    if flow_project_id:
+        logger.info("Flow project reused: %s", flow_project_id)
+    else:
+        flow_result = await client.create_project(body.name, body.tool_name)
+        if flow_result.get("error"):
+            raise HTTPException(502, f"Flow API error: {flow_result['error']}")
+        flow_project_id = _read_flow_project_id(flow_result)
+        logger.info("Flow project created: %s", flow_project_id)
 
     repo = _get_repo()
 
     # Step 2: Create local project with the Flow-assigned ID and detected tier
     create_data = body.model_dump(exclude_none=True)
     create_data.pop("tool_name", None)
+    create_data.pop("flow_project_id", None)
     create_data.pop("style", None)
     characters_input = create_data.pop("characters", None)
 

agent/config.py

@@ -16,10 +16,34 @@
 WS_PORT = int(os.environ.get("WS_PORT", "9222"))
 
 # ─── Google Flow API ────────────────────────────────────────
+# Legacy REST host. Flow moved to flow.google.com in September 2026 and stopped
+# minting the `Bearer ya29.…` this host needs, so these are only reachable with
+# USE_BATCH_RPC=0 on a browser profile that still has an old token.
 GOOGLE_FLOW_API = "https://aisandbox-pa.googleapis.com"
 GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY", "AIzaSyBtrm0o5ab1c-Ec8ZuLcGt3oJAA5VWt3pY")
 RECAPTCHA_SITE_KEY = os.environ.get("RECAPTCHA_SITE_KEY", "6LdsFiUsAAAAAIjVDZcuLhaHiDn5nnHVXVRQGeMV")
 
+# ─── Flow batchexecute (the current path) ───────────────────
+# Every call is signed in the page with the session cookie plus a per-page `at`
+# token, so the extension runs it inside a signed-in flow.google.com tab. Set
+# USE_BATCH_RPC=0 only to fall back to the dead REST path for a post-mortem.
+USE_BATCH_RPC = os.environ.get("USE_BATCH_RPC", "1") == "1"
+
+# The Flow project every RPC is scoped to. Project creation went with the old
+# labs.google tRPC endpoint, so a project is made once in the Flow UI and its
+# uuid pinned here; POST /api/projects falls back to it when no id is given.
+FLOW_PROJECT_ID = os.environ.get("FLOW_PROJECT_ID", "")
+
+# Capabilities whose payloads were never captured off the new UI (4K upscale,
+# reference-to-video, start+end-frame chaining) fail loudly by default. With
+# this on, the two that have a sane fallback degrade instead: chaining and r2v
+# both drop to plain i2v off the start frame. Upscale has no fallback.
+FLOW_ALLOW_DEGRADED = os.environ.get("FLOW_ALLOW_DEGRADED", "0") == "1"
+
+# The tier no longer picks a model — aspect is its own slot and the model names
+# are fixed — so it is only carried for the DB column and the dashboard.
+DEFAULT_PAYGATE_TIER = os.environ.get("DEFAULT_PAYGATE_TIER", "PAYGATE_TIER_TWO")
+
 # ─── Worker ──────────────────────────────────────────────────
 POLL_INTERVAL = int(os.environ.get("POLL_INTERVAL", "5"))
 VIDEO_POLL_INTERVAL = int(os.environ.get("VIDEO_POLL_INTERVAL", "10"))  # polling interval for video/upscale status
@@ -37,6 +61,9 @@
 VIDEO_MODELS = _MODELS["video_models"]
 UPSCALE_MODELS = _MODELS["upscale_models"]
 IMAGE_MODELS = _MODELS["image_models"]
+# Nickname from image_models. The batch path accepts GEM_PIX_2 (Nano Banana Pro)
+# and NARWHAL (Banana 2) and rejects everything else.
+DEFAULT_IMAGE_MODEL = _MODELS.get("default_image_model", "NANO_BANANA_PRO")
 
 # ─── API Endpoints ───────────────────────────────────────────
 ENDPOINTS = {

agent/models.json

@@ -56,5 +56,15 @@
   "image_models": {
     "NANO_BANANA_PRO": "GEM_PIX_2",
     "NANO_BANANA_2": "NARWHAL"
+  },
+  "default_image_model": "NANO_BANANA_PRO",
+  "batch_video_models": {
+    "_comment": "Wire names the flow.google.com batchexecute path accepts. video_models above are REST-era keys; resolve_video_model() folds them onto these (anything asking for 'ultra' lands on the ultra model).",
+    "accepted": [
+      "veo_3_1_i2v_lite_low_priority",
+      "veo_3_1_i2v_lite",
+      "veo_3_1_i2v_s_fast_ultra"
+    ],
+    "default": "veo_3_1_i2v_lite_low_priority"
   }
 }