fix(db): adopt prisma migrate with baseline + drift CI guard (#233)

.changeset/prisma-migrate-history.md

@@ -0,0 +1,5 @@
+---
+'lens': minor
+---
+
+Adopt Prisma migrate: commit a baseline migration (`prisma/migrations/0_init`), make CI fail on drift between `schema.prisma` and the migrations, switch app startup and deploys to `prisma migrate deploy`, and remove the `db push` path.

.github/workflows/ci.yml

@@ -21,3 +21,58 @@ jobs:
       - run: npx tsc --noEmit
       - run: npm run build
       - run: npm test --if-present
+
+  db-migrations:
+    # #233 — schema changes must be reviewable migrations, not `db push`.
+    # This job is the authority on drift: it replays every committed migration
+    # against a real Postgres and fails if the result disagrees with
+    # prisma/schema.prisma. It also proves a new migration can be generated,
+    # applied and rolled back on a clean database.
+    name: Migration drift & deploy
+    runs-on: ubuntu-latest
+    needs: build
+    services:
+      postgres:
+        image: postgres:15-alpine
+        env:
+          POSTGRES_USER: lens
+          POSTGRES_PASSWORD: lens
+          POSTGRES_DB: lens
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U lens -d lens"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 10
+    env:
+      DATABASE_URL: postgresql://lens:lens@localhost:5432/lens
+      DIRECT_DATABASE_URL: postgresql://lens:lens@localhost:5432/lens
+      SHADOW_DATABASE_URL: postgresql://lens:lens@localhost:5432/lens_shadow
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+          cache: npm
+      - run: npm ci || npm install
+      - run: node node_modules/prisma/build/index.js generate
+
+      - name: Create empty shadow database
+        run: |
+          node -e '
+            const { Client } = require("pg");
+            (async () => {
+              const c = new Client({ connectionString: process.env.DATABASE_URL });
+              await c.connect();
+              await c.query("DROP DATABASE IF EXISTS lens_shadow");
+              await c.query("CREATE DATABASE lens_shadow");
+              await c.end();
+            })();
+          '
+
+      - name: Fail on drift between schema.prisma and prisma/migrations
+        run: bash scripts/db-drift-check.sh
+
+      - name: Prove generate / apply / rollback on a clean database
+        run: bash scripts/db-migration-cycle.sh

.github/workflows/load.yml

@@ -81,8 +81,8 @@ jobs:
       - name: Build
         run: npm run build
 
-      - name: Apply database schema
-        run: node node_modules/prisma/build/index.js db push --accept-data-loss
+      - name: Apply committed migrations
+        run: node node_modules/prisma/build/index.js migrate deploy
 
       - name: Start server
         run: |

CONTRIBUTING.md

@@ -46,7 +46,7 @@ git clone https://github.com/Miracle656/Lens.git
 cd Lens
 npm install
 cp .env.example .env   # fill in DATABASE_URL, REDIS_URL, WATCHED_PAIRS
-npx prisma db push
+npm run db:migrate     # apply committed migrations (prisma migrate dev) — do NOT `db push`
 npm run dev
 ```
 

README.md

@@ -380,8 +380,8 @@ npm install
 # 3. Copy env
 cp .env.example .env
 
-# 4. Push database schema
-npm run db:push
+# 4. Apply database migrations
+npm run db:deploy
 
 # 5. Seed the database with fixture data
 npm run seed

docs/backup-restore.md

@@ -85,7 +85,7 @@ before loading. Delete the decrypted `.dump.gz` when you are done.
 
 4. Point the matching Lens deployment at the new instance:
    - `DATABASE_URL` — pooled URL is fine for the app
-   - `DIRECT_DATABASE_URL` — direct URL (Prisma migrations / `db push`)
+   - `DIRECT_DATABASE_URL` — direct URL (Prisma `migrate deploy` / CLI)
 5. Check `/status`, then one `/prices/history` (or GraphQL equivalent) for a
    watched pair.
 6. Rollback is swapping those two env vars back to the previous instance.

docs/database-migrations.md

@@ -0,0 +1,130 @@
+# Database migrations
+
+Issue **#233**. Before this, `prisma/` contained only `schema.prisma` — there was
+no `migrations/` directory, and every schema change was applied with
+`prisma db push`. Lens now uses **`prisma migrate`**: schema changes are
+committed as reviewable SQL, and deployments apply them with `migrate deploy`.
+
+## Why not `db push`
+
+`prisma db push` diffs the *live* database against `schema.prisma` and applies
+whatever it decides, on the spot. That gives you none of the things a service
+that holds money-moving data needs:
+
+- **Nothing is reviewable.** A schema change is whatever the diff produced at
+  that instant, not a file a reviewer read.
+- **No reproducible deploy.** Two environments pushed at different times can
+  land on different shapes; nothing in the repo describes the live shape.
+- **No rollback.** There is no record of what changed, so there is nothing to
+  undo. With `--accept-data-loss` a push can silently drop a column.
+- **It has already bitten us.** A `db push` in the sibling service half-applied
+  and left the database on a shape no commit described. That is exactly the
+  failure `prisma migrate` exists to prevent.
+
+So: **do not run `prisma db push` against any environment.** It is no longer a
+`package.json` script. The `db push` call at app startup (`src/index.ts`) now
+runs `migrate deploy` instead.
+
+## What lives in the repo now
+
+```
+prisma/
+  schema.prisma                     # the source of truth for the data model
+  migrations/
+    migration_lock.toml             # provider = "postgresql" (do not hand-edit)
+    0_init/
+      migration.sql                 # baseline: recreates the production schema exactly
+```
+
+`0_init/migration.sql` is the **baseline** — the SQL that reproduces the current
+production schema from an empty database. It is generated, not hand-written:
+
+```bash
+prisma migrate diff \
+  --from-empty \
+  --to-schema-datamodel prisma/schema.prisma \
+  --script > prisma/migrations/0_init/migration.sql
+```
+
+## Daily workflow (local development)
+
+Edit `schema.prisma`, then generate a migration. Do **not** `db push`.
+
+```bash
+npm run db:migrate        # prisma migrate dev — creates and applies a new migration
+```
+
+`migrate dev` writes a new timestamped folder under `prisma/migrations/`.
+**Commit that folder.** It is now the reviewable record of the change.
+
+```bash
+npm run db:status         # which migrations are applied / pending
+npm run db:reset          # drop schema and re-apply all migrations (NEVER on prod)
+```
+
+## Deploying
+
+Deploys apply committed migrations only — they never diff the live database:
+
+```bash
+npm run db:deploy         # prisma migrate deploy
+```
+
+App startup (`src/index.ts`) runs `migrate deploy` for you, so a container that
+starts against a database with pending migrations applies them and only them,
+deterministically, or fails loudly if the database is ahead of the code.
+
+### Adopting the baseline on an EXISTING production database
+
+Production already matches `schema.prisma`, so the baseline's `CREATE TABLE`
+statements must **not** be re-run there — `migrate deploy` would collide with
+the tables that are already there. Instead, mark the baseline as already applied
+without executing it, which records it in Prisma's `_prisma_migrations` table:
+
+```bash
+npx prisma migrate resolve --applied 0_init
+```
+
+After that, `migrate deploy` sees `0_init` as applied and only runs future
+migrations. This one-time step is the standard Prisma path for adopting
+`migrate` on a database that already exists; it is why the baseline is committed
+separately rather than generated on deploy.
+
+## The CI drift guard
+
+The `db-migrations` job in `.github/workflows/ci.yml` brings up a Postgres
+service and runs:
+
+- `scripts/db-drift-check.sh` — deploys every committed migration, then
+  `migrate diff --from-migrations prisma/migrations --to-schema-datamodel
+  prisma/schema.prisma --exit-code` against a shadow database. It exits non-zero
+  — failing CI — whenever `schema.prisma` and `prisma/migrations` disagree. This
+  is the check that would have caught a half-applied `db push`.
+- `scripts/db-migration-cycle.sh` — proves a new migration can be generated,
+  applied and rolled back on a clean database, and that the drift guard actually
+  fires on an un-migrated schema change.
+
+The DB-free unit test `tests/prisma-migrations.test.ts` (part of `npm test`)
+additionally locks the **baseline** to `schema.prisma`: it fails while the repo
+has no migration history, and fails if you edit the schema without regenerating
+the baseline, so a missing migration is caught even without a database.
+
+Run the database-backed checks locally against the compose Postgres:
+
+```bash
+docker compose up -d postgres
+# create an empty shadow database alongside `lens`
+DATABASE_URL=postgresql://lens:lens@localhost:5432/lens \
+DIRECT_DATABASE_URL=postgresql://lens:lens@localhost:5432/lens \
+SHADOW_DATABASE_URL=postgresql://lens:lens@localhost:5432/lens_shadow \
+  npm run db:drift-check
+```
+
+## Rolling back a bad migration
+
+Prisma does not generate down-migrations. In a **non-production** environment,
+`npm run db:reset` drops the schema and re-applies every committed migration,
+which is the clean, reproducible way to back out of a half-applied change. In
+production, roll forward: commit a new migration that reverses the change and
+`migrate deploy` it. Either way the history in `prisma/migrations/` is the
+source of truth — the thing `db push` never gave us.

package.json

@@ -20,8 +20,13 @@
     "mcp:server": "tsx examples/mcp-server/run.ts",
     "mcp:agent": "tsx examples/mcp-server/agent.ts",
     "key:issue": "tsx scripts/issue-api-key.ts",
-    "db:push": "prisma db push",
     "db:generate": "prisma generate",
+    "db:migrate": "prisma migrate dev",
+    "db:deploy": "prisma migrate deploy",
+    "db:reset": "prisma migrate reset",
+    "db:status": "prisma migrate status",
+    "db:drift-check": "bash scripts/db-drift-check.sh",
+    "db:migration-cycle": "bash scripts/db-migration-cycle.sh",
     "db:migrate:partitioning": "tsx scripts/migrate-price-points-partitioning.ts",
     "seed": "tsx scripts/seed.ts"
   },