Skip to content

Task/vercel handler resilience - #1548

Merged
greatest0fallt1me merged 14 commits into
CalloraOrg:mainfrom
emarkees:task/vercel-handler-resilience
Oct 21, 2026
Merged

greatest0fallt1me merged 14 commits into
CalloraOrg:mainfrom
emarkees:task/vercel-handler-resilience

Conversation

@emarkees

Copy link
Copy Markdown

fix(vercel): Recover from failed demo bootstraps in the Vercel handler and smoke-test the built bundle in CI

Closes #1516

Summary

Hardens the Vercel serverless deployment entrypoint against cold-start concurrency races, uninitialized / transient database connection failures, and SQLite demo migration syntax incompatibilities. Additionally fixes Vercel bundling artifacts, prevents the Express server from opening duplicate local listeners when imported by the serverless wrapper, and integrates a pre-deployment CI bundle smoke test.

Problem

  1. Cold-Start Connection Races: In serverless environments, cold starts could handle requests before Prisma or SQLite demo databases completed migration and seeding, resulting in unhandled rejections or crashes.
  2. Missing Self-Healing: Failed initializations were permanently cached or left unhandled without retry / exponential backoff, preventing subsequent requests from recovering.
  3. Demo Migration Dialect Conflict: The demo SQLite migration runner attempted to execute PostgreSQL-specific syntax (timestamptz, BYTEA, DEFAULT NOW(), gen_random_uuid()), breaking demo runs.
  4. Direct Execution False Positive: When scripts/vercel-entry.ts imported the compiled application (dist/index.js), require.main === module checks could falsely trigger app.listen() inside the serverless runtime.
  5. Bundle Dependencies: scripts/build-vercel.mjs stripped engines and critical dependencies from .vercel/output/functions/index.func/package.json, risking Node engine mismatch on Vercel.
  6. Billing Rate Limit Config: src/routes/index.ts had a nested property bug (config.creditsRateLimit.billingRateLimit) that threw at runtime.

Changes

1. Vercel Handler Resilience & Self-Healing (scripts/vercel-entry.ts)

  • Added ensureDatabaseReady() with bounded exponential backoff (up to 3 retries, configurable delay).
  • Added synchronization lock ensuring only one migration/seed cycle runs at a time per cold container.
  • If database initialization fails, returns a standard 503 Service Unavailable with SERVICE_INITIALIZING error envelope rather than crashing the Lambda.
  • Wrapped synchronous dispatch errors in proper JSON error responses.

2. Demo SQLite Migration & Seeding Fixes (src/db/demo.ts)

  • Added dialect classification for migration files (isPostgresMigration), safely skipping PostgreSQL migrations on SQLite while still tracking them in _prisma_migrations.
  • Sanitized SQL keywords (timestamptz -> TEXT, BYTEA -> BLOB, gen_random_uuid() -> hex string) for SQLite compatibility.
  • Made model catalog and pricing tier seeding idempotent (ON CONFLICT DO NOTHING / pre-existence checks) to prevent UNIQUE constraint failed errors on retries.

3. Server Direct Execution Guard (src/index.ts)

  • Factored out and exported checkDirectExecution(mainModule, currentModule, isVercel) helper.
  • Prevents app.listen() from firing when process.env.VERCEL === '1' or when imported by serverless handlers.

4. Prisma Fallback & Envelope Standardization (src/lib/prisma.ts, src/errors/errorEnvelopePolicy.ts)

  • Registered SERVICE_INITIALIZING under public error codes and allowed error types.
  • Fallback proxy intercepts Prisma queries in demo mode / uninitialized state with structured 503 error payloads instead of unhandled exceptions.

5. Build & CI Smoke Testing (scripts/build-vercel.mjs, scripts/smoke-vercel-bundle.mjs, .github/workflows/ci.yml)

  • Retained engines and runtime dependencies in Vercel function output package.json.
  • Added end-to-end smoke-vercel-bundle.mjs script that builds the Vercel output and tests handler cold-start against /healthz and demo endpoints.
  • Added bundle smoke test execution to GitHub Actions CI workflow.

6. OpenAPI & Route Fixes (docs/openapi.json, src/routes/index.ts)

  • Added missing SuccessEnvelope and BillingHistoryRow schemas under components.schemas to pass OpenAPI schema validation.
  • Fixed config.creditsRateLimit.billingRateLimit to config.billingRateLimit.

Commits

  • feat(errors): add SERVICE_INITIALIZING to public error codes
  • fix(server): export checkDirectExecution to prevent listening on Vercel bundle import
  • test(server): add unit tests for checkDirectExecution mode detection
  • fix(prisma): return 503 service unavailable envelope when in demo mode or client missing
  • fix(routes): correct billingRateLimit configuration property access
  • fix(demo): skip postgres-only migrations and make catalog seeding idempotent
  • test(demo): add unit tests for migration dialect detection and idempotent seeding
  • build(vercel): preserve engines and dependencies in deployment package.json
  • fix(vercel): add self-healing database initialization and sync error handling
  • test(vercel): add unit tests for vercel entrypoint cold-start and backoff
  • test(smoke): add smoke test script for bundled vercel deployment
  • ci: add vercel bundle smoke test step to build pipeline
  • docs(openapi): add SuccessEnvelope and BillingHistoryRow component schemas
  • docs: document self-healing vercel handler and ci smoke test

Verification

  • npm run error-codes:check passed cleanly.
  • npm run validate:openapi passed cleanly.
  • Unit tests passed (scripts/vercel-entry.test.ts, src/db/demo.test.ts, src/__tests__/directExecution.test.ts).
  • npm run build:vercel successfully built .vercel/output.
  • node scripts/smoke-vercel-bundle.mjs passed end-to-end against bundled handler.

@drips-wave

drips-wave Bot commented Oct 21, 2026

Copy link
Copy Markdown

@emarkees Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recover from failed demo bootstraps in the Vercel handler and smoke-test the built bundle in CI

2 participants