Skip to content

Implement Phase 10 API v2 policy migration - #3813

Merged
anth-volk merged 18 commits into
masterfrom
stage-10
Sep 4, 2026
Merged

Implement Phase 10 API v2 policy migration#3813
anth-volk merged 18 commits into
masterfrom
stage-10

Conversation

@anth-volk

@anth-volk anth-volk commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Fixes #3812

Summary

  • Add native UUID-based policy create, detail, and filtered collection operations under /v2/policies using Supabase/Postgres.
  • Add native user-policy association create, detail, collection, partial-update, and delete operations under /v2/user-policies.
  • Deduplicate immutable policy content, retain exact catalog-version references, and store durable mappings from API v1 identifiers to API v2 UUIDs.
  • Preserve user_policies.user_id as a UUID foreign key to users.id, and map each exact API v1 user identifier one-to-one to a v2 user UUID through legacy_user_mappings.
  • Mirror API v1 policy creates and saved-policy creates and updates immediately after their Cloud SQL transactions commit.
  • Store ordered saved-policy mirror events in Cloud SQL so synchronous retries apply committed mutations in source-revision order.
  • Add shared typed query-parameter definitions and repository guidance for all future route additions and query-contract changes.

Existing API v1 behavior retained

  • Policy and saved-policy reads remain on Flask and Cloud SQL.
  • API v1 responses retain integer identifiers and do not expose API v2 UUIDs.
  • Immutable core policies still have no update or delete operation.
  • No API v1 saved-policy delete operation is added.
  • Cloud SQL-only API v1 mutation mode remains available through DB_WRITE_POLICY=cloud_sql.

Intentional API additions and changes

  • New API v2 routes require country_id as a typed query parameter.
  • Policy create bodies contain immutable country, model, and parameter-value content without user-facing names or descriptions.
  • User-policy associations own optional names and descriptions and require an existing v2 user UUID.
  • Native association routes add no authentication or authorization; a supplied UUID identifies a database row but does not prove caller control.
  • Policy collection reads use exact model filtering rather than textual search.
  • Database timeout and availability failures return fixed 503 responses; other persistence failures return fixed 500 responses without exception text, SQL, parameters, policy content, or credentials.

Persistence and deployment preparation

  • Add separate reviewed Alembic revisions for the Cloud SQL saved-policy event schema and the Supabase policy, association, and mapping schema.
  • On first API v1 saved-policy mirroring for a legacy user ID, create a minimal v2 user and the one-to-one mapping in the same Supabase transaction as the association. Later requests reuse the mapped UUID.
  • Keep transition-created users' unavailable first_name, last_name, and email values null. This change does not add or infer Auth0 identifiers.
  • Add dormant-table qualification, catalog prerequisites, readiness checks, migration documentation, structured observability, and explicit application rollback instructions.
  • The schema revisions are committed as PR content and have not been applied to staging or production by this PR creation or update operation.

Verification

  • Focused Phase 10 unit and contract suite: 237 passed.
  • Disposable-Postgres Phase 10 lifecycle and integration suite: 21 passed.
  • Migration contract and generated-artifact suite: 49 passed.
  • Alembic reports no schema drift; upgrade, downgrade, and re-upgrade checks passed.
  • Ruff formatting and lint checks passed for changed files.
  • Migration contract export and repository quality checks passed.
  • Strict OpenSpec validation passed.
  • Changelog fragment included.

Remaining rollout work

  • Exercise the reviewed schema and initialized catalogs on a staging candidate.
  • Verify native API v2 operations, immediate API v1 mutation mirroring, retry behavior, and unchanged Cloud SQL reads.
  • Exercise rollback to Cloud SQL-only API v1 mutations while retaining committed API v2 records.

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.44444% with 228 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.18%. Comparing base (b06a1ec) to head (710bd3d).

Files with missing lines Patch % Lines
policyengine_api/fastapi_routes/dependencies.py 24.00% 11 Missing and 27 partials ⚠️
policyengine_api/services/user_policy_service.py 83.85% 14 Missing and 12 partials ⚠️
policyengine_api/services/v2/policies/services.py 74.11% 17 Missing and 5 partials ⚠️
policyengine_api/routes/policy_routes.py 79.20% 17 Missing and 4 partials ⚠️
...engine_api/services/v2/policies/transformations.py 84.25% 7 Missing and 10 partials ⚠️
...engine_api/services/v2/user_policies/validators.py 67.92% 9 Missing and 8 partials ⚠️
policyengine_api/services/user_policy_mirroring.py 88.00% 6 Missing and 3 partials ⚠️
...gine_api/data/v2/policy_migration_qualification.py 92.30% 7 Missing and 1 partial ⚠️
policyengine_api/query_parameters.py 90.69% 4 Missing and 4 partials ⚠️
policyengine_api/services/policy_mirroring.py 82.05% 5 Missing and 2 partials ⚠️
... and 18 more
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #3813       +/-   ##
===========================================
+ Coverage   36.56%   87.18%   +50.62%     
===========================================
  Files         103      150       +47     
  Lines        6291     8935     +2644     
  Branches     1061     1542      +481     
===========================================
+ Hits         2300     7790     +5490     
+ Misses       3809      703     -3106     
- Partials      182      442      +260     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@anth-volk
anth-volk marked this pull request as ready for review September 4, 2026 14:58
@anth-volk
anth-volk merged commit 3ab1edf into master Sep 4, 2026
14 checks passed
@anth-volk
anth-volk deleted the stage-10 branch September 4, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Phase 10 API v2 policy migration

1 participant