Skip to content

feat: verify UI and application services against real OpenAI without cloud - #24

Merged
justgithubaccount merged 1 commit into
mainfrom
codex/live-application-smoke
Oct 29, 2026
Merged

justgithubaccount merged 1 commit into
mainfrom
codex/live-application-smoke

Conversation

@justgithubaccount

@justgithubaccount justgithubaccount commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Outcome

Add and execute a no-cloud live application smoke using the existing OPENAI_API_KEY repository secret, as requested by the operator.

  • FastAPI v1.1.2, CrewAI v0.1.3, and reviewed UI feat/crewai-automation; exact audited revisions checked before exposing the key.
  • Real direct FastAPI user flow plus browser → Next proxy → FastAPI/CrewAI → OpenAI.
  • UI project creation, quick/project chats, saved request history/reload, CrewAI discovery and default/named tool-free runs.
  • OTLP collection on the runner; exported trace IDs must match the UI.
  • Six actual gpt-4o-mini calls maximum, 128 output tokens per call, bounded text input; failed attempts consume budget. No automatic paid retry expansion.
  • Only the runner-local test transport receives the real key. Application children get dummy credentials and loopback endpoints. Only sanitized summary is uploaded, not logs, DB, config or secrets.

No Timeweb/cloud resources, Terraform, Helm, cluster tools or real infrastructure flows are used. Temporary CrewAI flow uses the existing tool-free reporting agent on explicitly synthetic input. No backend/UI source changes are committed. No merge or deployment is authorized by this PR.

Verification

  • Five offline guard tests: PASS, including maximum request count, token normalization, invalid model/input, usage-only reporting and no retry on upstream errors.
  • All nine existing shell test scripts: PASS.
  • Ruff check/format, Python compilation, Node syntax and git diff whitespace checks: PASS.
  • Complete offline rehearsal with real pinned application source copies and only the OpenAI transport replaced: PASS, six calls, browser flows, OTLP trace matching and clean subprocess teardown. This is not live-provider evidence.
  • Fresh-context review of implementation tree: no Critical/Important/Minor findings. Reviewer independently repeated five guard tests and checked relevant source contracts. Bootstrap commit message adds [live-smoke]; reviewed file tree is unchanged.
  • Live Actions run 40866334399: SUCCESS at head 320981a9dca789dbd836c41473969b1283b4794f.
  • Downloaded the sanitized result artifact and verified result=PASS: six real gpt-4o-mini requests, all HTTP 200; 659 prompt and 215 completion tokens. Browser project creation/chat/history/reload, quick chat, nine-agent discovery and both default/named single-step reporting runs passed. No browser page errors.
  • 94 OTLP spans exported; the project-chat and both CrewAI trace IDs match the browser UI. No cluster tools executed and no cloud resources created.
  • Exact-head normal CI run 40866355418: SUCCESS.

Delivery limits

Public source repositories and external action refs were checked live. Ordinary PR CI has no provider key; the secret is step-scoped to the live check. The initial unmerged branch run requires the explicit commit-message opt-in; after merge, dispatch requires confirm_live=true.

Cloud/Kubernetes/ArgoCD/Loki/deployed health remains outside this check. Review is technical evidence, not operator merge approval. Main and unrelated worktrees are preserved.

Base: 20e7e8d502554bff3f0a80e4f0127924edd6b2b0; head: 320981a9dca789dbd836c41473969b1283b4794f; branch: codex/live-application-smoke. Live PR has no comments/reviews at handoff, is open and not merged.

Evidence limits from review: real provider validity/quota required the controlled run (now proven for these six requests only); cloud services were excluded; the test transport is not a security sandbox against hostile runner code; review is not merge/deployment authorization. No deferred code-review findings.

Drift surfaces: local_repo=NONE_DETECTED for the task worktree; routed_artifacts=NOT_CHECKED (not used); live_runtime=NOT_CHECKED for cloud deployment, with the disposable runner application/model flow verified as above. branch_merge_status: NOT_AUDITED.

@justgithubaccount
justgithubaccount merged commit c12b9d2 into main Oct 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant