Skip to content

CIMD: production rollout plan (ChatGPT, Codex, VS Code, Claude) #693

Closed
#696
Closed
CIMD: production rollout plan (ChatGPT, Codex, VS Code, Claude)#693
#696
Labels
enhancementNew feature or request

Description

@mashkovd
mashkovd
opened on Oct 18, 2026

Tracking issue for turning CIMD (#375, shipped dark in 4.100.0) on in production. CIMD stays disabled until every step below is done. None of this blocks Unified Identity (mctlhq/.github#91, closed as completed).

Why enabling it is not a one-line change

Findings from 2026-10-18:

  • OAUTH_CIMD_ENABLED was a global capability advertisement (now split, see below). With it on, discovery returns client_id_metadata_document_supported: true to every client. ChatGPT prefers CIMD whenever a server advertises it. A CIMD-capable client whose host is not in OAUTH_CIMD_ALLOWED_HOSTS, or whose document we refuse, then gets 400 with no DCR fallback. That includes our own claude.ai connector when it reconnects.
  • Real documents run through parseDocument on main:
Client Document Result on 4.101.1
ChatGPT https://chatgpt.com/oauth/client.json refused: token_endpoint_auth_method=private_key_jwt (#694)
Codex https://chatgpt.com/oauth/codex/client.json refused: one entry is http://localhost/callback (fixed by the Codex PR)
VS Code https://vscode.dev/oauth/client-metadata.json accepted (no-store, so the 30 s floor applies)
Claude https://claude.ai/oauth/mcp-oauth-client-metadata accepted
  • Cluster preflight PASSED. From inside a mctl-api pod (wget, Go-http-client/1.1 UA), all four documents return 200 with no cf-mitigated. Caveat: Go's TLS fingerprint differs from wget's, so Cloudflare heuristics are not fully ruled out until a real resolver fetch.

Steps

  • Codex compatibility (fix(cimd): drop unusable redirect entries instead of the document #695, merged 5dd4e73d, not released yet). Drop well-formed redirect entries outside the allowlist instead of refusing the document. Exact match against the accepted set only, with no localhost and no normalisation. A refused redirect is logged, clipped, together with the dropped entries. Still open: which redirect_uri spelling real Codex sends (see the comment below).
  • private_key_jwt for CIMD clients (ChatGPT). Separate security feature: CIMD: support private_key_jwt client authentication (ChatGPT) #694.
  • Split the flag into accepting and advertising (feat(oauth): split CIMD into OAUTH_CIMD_ACCEPT and OAUTH_CIMD_ADVERTISE #697, merged 25357249, not released yet). Shipped as OAUTH_CIMD_ACCEPT and OAUTH_CIMD_ADVERTISE: strict booleans, ADVERTISE requires ACCEPT, and the retired OAUTH_CIMD_ENABLED refuses startup when set. Proposed: OAUTH_CIMD_ACCEPT (the server resolves URL client_ids) and OAUTH_CIMD_ADVERTISE (discovery announces it).
    • Production could then run ACCEPT=true and ADVERTISE=false for manual protocol checks without inviting every client to switch.
    • Open question: ACCEPT alone still changes behaviour for URL-shaped client_ids (no fallback), so the "Before turning it on" check in docs/federation.md applies to ACCEPT, not only to ADVERTISE.
  • First e2e on a non-production OAuth endpoint (preview or staging), with both flags on and OAUTH_CIMD_ALLOWED_HOSTS=claude.ai,vscode.dev. Prove the chain client_kind=cimd → consent page → prn_ principal → oauth.grant_issued audit entry (client in parameters). Then repeat with ChatGPT and Codex once the two items above have landed. Needs a decision on which non-production endpoint can complete a ZITADEL sign-in.
  • Production rollout, as a separate owner-approved gitops PR:
    • the allowlist names every CIMD client actually in use, exact hosts only (at least chatgpt.com, claude.ai, vscode.dev);
    • ChatGPT is the first production smoke;
    • rollback is one gitops PR that removes the flag.

Refs #375, #687.

Activity

  1. mashkovd
    added
    enhancementNew feature or request
    on Oct 18, 2026

  2. mashkovd commented on Oct 18, 2026

    @mashkovd
    mashkovd
    on Oct 18, 2026
    MemberAuthor

    Codex e2e check, from the #695 review: once the document parses, Codex resolves to [http://127.0.0.1/callback] only. Before the production step, confirm on a real Codex authorize request which spelling it sends as redirect_uri. If it sends http://localhost:<port>/callback, the CIMD branch refuses with no fallback to the loopback rule, and the next decision is whether to accept localhost only when the document itself lists it.

  3. mashkovd
    closed this as completedin #696on Oct 18, 2026

  4. mashkovd commented on Oct 18, 2026

    @mashkovd
    mashkovd
    on Oct 18, 2026
    MemberAuthor

    #695 (Codex compatibility) and #697 (the ACCEPT/ADVERTISE split) shipped in mctl-api 4.103.0.

    • Release PR chore(main): release 4.103.0 #696, tag 4.103.0. gitops release-deploy run 39137363468, bump 6e6ac2c79. 4.102.0 was taken by feat: orchestrate explicit full customer account deletion #666, released separately.
    • Pods 2/2 ready on 4.103.0 since 14:47Z, 0 restarts, 0 ERROR lines. Boot passes the new flag validation; no OAUTH_CIMD_* env is set.
    • CIMD is off in production: discovery has no client_id_metadata_document_supported, and the ChatGPT client_id on /authorize gets 400. DCR registration and authorize work, and an existing bearer token validates through mctl_whoami.

    Next steps: #694 (private_key_jwt), then the non-production e2e.

  5. mashkovd commented on Oct 18, 2026

    @mashkovd
    mashkovd
    on Oct 18, 2026
    MemberAuthor

    ChatGPT end-to-end run against a local mctl-api: passed

    The run used the code now released as 4.105.0, against a local mctl-api reached through a cloudflared quick tunnel:

    Production was not touched: CIMD stays off there, and the api.mctl.ai metadata does not advertise it.

    Proven

    Step Evidence
    CIMD resolution ChatGPT's client_id is per app: https://chatgpt.com/oauth/<app-id>/client.json, token_endpoint_auth_method=private_key_jwt, RS256, jwks https://chatgpt.com/oauth/jwks.json
    Consent and ZITADEL sign-in oauth: ZITADEL sign-in ... result=consent_required → POST /oauth/consent 302 back to chatgpt.com/connector/oauth/... → audit oauth.grant_issued (client_kind: cimd) with the canonical prn_
    Code exchange POST /oauth/token 200 from openai-connectors-oauth/1.0; the refresh session row has auth_method=private_key_jwt and the canonical principal_id
    MCP call POST /mcp 200 from openai-mcp/1.0.0; mctl_whoami answered from the tunnel host (Teams: [], Accessible namespaces: [admins]), so the answer came from the local server and not from production
    Refresh The server restarted with a new signing key and OAUTH_TOKEN_TTL=2m. ChatGPT's next call got POST /mcp 401, then POST /oauth/token 200 (refresh grant, private_key_jwt), then a retried POST /mcp 200. In the database, the old row has rotated_at set, and the new row has parent_token_hash pointing at it with auth_method=private_key_jwt kept

    Found and fixed

    Harness-only, not an mctl-api bug

    • mcp-go's DNS-rebinding guard returns 403 invalid Host header when a non-loopback Host arrives on a loopback connection. That happened because the local proxy dialled 127.0.0.1. Pods receive on the pod IP, so production does not hit it.

    Remaining for this issue

    Enabling CIMD in production is a separate, owner-approved gitops PR with OAUTH_CIMD_ALLOWED_HOSTS=chatgpt.com.

  6. mashkovd commented on Oct 18, 2026

    @mashkovd
    mashkovd
    on Oct 18, 2026
    MemberAuthor

    Production rollout done

    CIMD went live on 2026-10-18 at 21:33Z through mctlhq/mctl-gitops#1991 (14b54891d) on mctl-api 4.105.0:

    • OAUTH_CIMD_ACCEPT=true;
    • OAUTH_CIMD_ADVERTISE=true;
    • OAUTH_CIMD_ALLOWED_HOSTS=chatgpt.com,claude.ai,vscode.dev.

    Discovery now returns client_id_metadata_document_supported: true.

    Production smoke

    Client Sign-in Audit oauth.grant_issued Refresh session MCP
    ChatGPT, new connector consent_required → consent client_kind: cimd, client_id: https://chatgpt.com/oauth/<app>/client.json auth_method=private_key_jwt, canonical prn_ connected
    Claude, new connector (published identity) consent_required → consent client_kind: cimd, client_id: https://claude.ai/oauth/mcp-oauth-client-metadata public client (PKCE), canonical prn_ mctl_whoami answered with the full team set
    ChatGPT, existing connector issued (no consent) client_kind: dcr its October DCR registration unchanged

    Reconnecting an existing connector keeps its stored DCR client_id. Only a newly created connector picks CIMD, so existing users are not moved.

    VS Code is on the allowlist but has not been exercised in production.

    Rollback: set OAUTH_CIMD_ADVERTISE to "false", and clients return to DCR on their next sign-in.

    This issue was closed early, at 14:44Z, by a closing keyword in #696, before the rollout steps were done. With this rollout the steps are now actually complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions