Skip to content

Fix flaky TestRateLimits; drop stale /v1 paths from the planning docs - #66

Merged
Adron merged 1 commit into
mainfrom
fix/flaky-ratelimit-stale-v1
Oct 22, 2026
Merged

Adron merged 1 commit into
mainfrom
fix/flaky-ratelimit-stale-v1

Conversation

@Adron

@Adron Adron commented Oct 22, 2026

Copy link
Copy Markdown
Member

Follow-up from reviewing #58, #60, and #61 against the header-versioned API.

Flaky TestRateLimits

It failed on #60's push run with got 401 "invalid_credentials", want 429 while the PR run of the same commit passed. The test sends 11 logins and expects the 11th to hit the 10-per-minute limit. Limits count per clock minute (now truncated to the minute), so a run that crossed a minute boundary reset its counter partway through.

  • testenv.Options gains Now, which replaces the API's clock.
  • TestRateLimits starts the API's clock at the top of a minute, offset from real time, so all 11 logins share one window however slow CI is. It also gets its own database, so repeated runs don't share counters.
  • Verified with go test -count=15 -run TestRateLimits: 15 of 15 pass. Before this change, repeats failed.

Stale /v1 in the docs

  • remaining_manual_steps.md now uses root paths throughout, and adds Substratal-Version: 1 to the sign-in curl and the app-registration steps. /health is exempt from the header.
  • plan-remaining.md: Phase 0 is checked at /health. I confirmed production returns 200 there and 404 at /v1/health (2026-10-22).
  • migrations/0008_scim_tokens.sql: the comment said it ran after 0005–0007, but those were renumbered 0009–0011 to follow it. This changes the comment only; the runner tracks versions, not checksums, so the applied migration is unaffected.

Left as-is on purpose: FCM's /v1/projects/… (Google's path), and versioning_test.go's check that /v1/… returns 404.

The full integration and contract suite and gen-check pass.

🤖 Generated with Claude Code

- TestRateLimits sent 11 logins and expected the 11th to hit the
  10-per-minute limit, but limits count per clock minute, so a run that
  crossed a minute boundary saw its counter reset and got 401. The API's
  clock in this test now starts at the top of a minute (testenv gains an
  Options.Now), so all 11 land in one window however slow CI is, and the
  test gets its own database so repeated runs don't share counters.
- remaining_manual_steps.md: root paths (/health, /auth/login,
  /applications, /roles, /api-keys, /webhooks) and the Substratal-Version
  header on the sign-in curl and the app-registration calls.
- plan-remaining.md: Phase 0 is checked at /health (200 on 2026-10-22;
  /v1/health is 404).
- migrations/0008_scim_tokens.sql: the comment said it came after
  0005–0007, which were renumbered 0009–0011 to follow it. Comment only;
  the runner tracks versions, not checksums.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

terraform plan (production)

This is exactly what merging will change in AWS.

Plan
random_password.mcp_session: Refreshing state... [id=none]
module.webhook_worker.aws_cloudwatch_log_group.this: Refreshing state... [id=/aws/lambda/substratal-prod-webhook-worker]
aws_apigatewayv2_api.main: Refreshing state... [id=rv9n7e3ww4]
aws_cloudwatch_log_group.api_access: Refreshing state... [id=/aws/apigateway/substratal-prod]
module.stripe_webhook.aws_cloudwatch_log_group.this: Refreshing state... [id=/aws/lambda/substratal-prod-stripe-webhook]
module.jobs.aws_cloudwatch_log_group.this: Refreshing state... [id=/aws/lambda/substratal-prod-jobs]
aws_kms_key.signing: Refreshing state... [id=0d2770cd-bbc8-4904-a432-2e9c9304a3ca]
data.aws_route53_zone.main: Reading...
aws_sesv2_email_identity.mail: Refreshing state... [id=mail.substratalapps.com]
aws_rds_cluster.main: Refreshing state... [id=substratal-prod-db]
aws_iam_role.scheduler: Refreshing state... [id=substratal-prod-scheduler]
aws_kms_key.data: Refreshing state... [id=b0cda5a8-651e-419f-b8ab-61772bce79da]
module.jobs.aws_iam_role.this: Refreshing state... [id=substratal-prod-jobs]
module.api.aws_cloudwatch_log_group.this: Refreshing state... [id=/aws/lambda/substratal-prod-api]
aws_sqs_queue.webhooks_dlq: Refreshing state... [id=https://sqs.us-east-1.amazonaws.com/196936048600/substratal-prod-webhooks-dlq]
data.aws_caller_identity.me: Reading...
module.stripe_webhook.aws_iam_role.this: Refreshing state... [id=substratal-prod-stripe-webhook]
data.aws_caller_identity.me: Read complete after 0s [id=196936048600]
aws_sns_topic.alarms: Refreshing state... [id=arn:aws:sns:us-east-1:196936048600:substratal-prod-alarms]
data.aws_servicequotas_service_quota.lambda_concurrency: Reading...
module.webhook_worker.aws_iam_role.this: Refreshing state... [id=substratal-prod-webhook-worker]
module.api.aws_iam_role.this: Refreshing state... [id=substratal-prod-api]
module.mcp.aws_iam_role.this: Refreshing state... [id=substratal-prod-mcp]
data.aws_route53_zone.main: Read complete after 0s [id=Z0783627X8ON1GH6I0XB]
aws_secretsmanager_secret.app: Refreshing state... [id=arn:aws:secretsmanager:us-east-1:196936048600:secret:substratal-prod/app-1hXTGg]
aws_acm_certificate.api: Refreshing state... [id=arn:aws:acm:us-east-1:196936048600:certificate/a8e2136f-64d3-456b-904a-05ea74eba9b3]
module.mcp.aws_cloudwatch_log_group.this: Refreshing state... [id=/aws/lambda/substratal-prod-mcp]
data.aws_servicequotas_service_quota.lambda_concurrency: Read complete after 1s [id=arn:aws:servicequotas:us-east-1::lambda/L-B99A9384]
aws_apigatewayv2_stage.default: Refreshing state... [id=$default]
aws_s3_bucket.audit_archive: Refreshing state... [id=substratal-prod-audit-archive-196936048600]
aws_kms_alias.signing: Refreshing state... [id=alias/substratal-prod-jwt-signing]
aws_rds_cluster_instance.main: Refreshing state... [id=substratal-prod-db-1]
aws_kms_alias.data: Refreshing state... [id=alias/substratal-prod-data]
aws_route53_record.dmarc: Refreshing state... [id=Z0783627X8ON1GH6I0XB__dmarc.mail.substratalapps.com_TXT]
aws_route53_record.dkim[0]: Refreshing state... [id=Z0783627X8ON1GH6I0XB_fmy2g7whqti56v5vn56es6izxo4yzayc._domainkey.mail.substratalapps.com_CNAME]
aws_route53_record.dkim[1]: Refreshing state... [id=Z0783627X8ON1GH6I0XB_fhl63kaowzjfd2splohuyaifnnansx5d._domainkey.mail.substratalapps.com_CNAME]
aws_route53_record.dkim[2]: Refreshing state... [id=Z0783627X8ON1GH6I0XB_rp4dt5rjjkpnhdyxud6l2spetqo5nom4._domainkey.mail.substratalapps.com_CNAME]
aws_sqs_queue.webhooks: Refreshing state... [id=https://sqs.us-east-1.amazonaws.com/196936048600/substratal-prod-webhooks]
aws_cloudwatch_metric_alarm.webhook_dlq: Refreshing state... [id=substratal-prod-webhooks-dlq]
aws_cloudwatch_metric_alarm.api_5xx: Refreshing state... [id=substratal-prod-api-5xx]
aws_cloudwatch_metric_alarm.security["destructive-blocked"]: Refreshing state... [id=substratal-prod-security-destructive-blocked]
aws_sns_topic_subscription.alarms[0]: Refreshing state... [id=arn:aws:sns:us-east-1:196936048600:substratal-prod-alarms:fffb7317-8153-4e66-9212-eded800c9c05]
aws_cloudwatch_metric_alarm.security["failed-logins"]: Refreshing state... [id=substratal-prod-security-failed-logins]
aws_cloudwatch_metric_alarm.security["refresh-reuse"]: Refreshing state... [id=substratal-prod-security-refresh-reuse]
aws_cloudwatch_metric_alarm.aurora_near_max: Refreshing state... [id=substratal-prod-aurora-acu-near-max]
aws_cloudwatch_log_metric_filter.security["failed-logins"]: Refreshing state... [id=substratal-prod-failed-logins]
aws_cloudwatch_log_metric_filter.security["refresh-reuse"]: Refreshing state... [id=substratal-prod-refresh-reuse]
aws_cloudwatch_log_metric_filter.security["destructive-blocked"]: Refreshing state... [id=substratal-prod-destructive-blocked]
module.jobs.aws_iam_role_policy.logs: Refreshing state... [id=substratal-prod-jobs:logs]
terraform_data.app_secret_init: Refreshing state... [id=5474e6c3-4843-79d8-9346-9fc3d3f8fa87]
module.mcp.aws_iam_role_policy.logs: Refreshing state... [id=substratal-prod-mcp:logs]
module.webhook_worker.aws_iam_role_policy.logs: Refreshing state... [id=substratal-prod-webhook-worker:logs]
module.stripe_webhook.aws_iam_role_policy.logs: Refreshing state... [id=substratal-prod-stripe-webhook:logs]
module.api.aws_iam_role_policy.logs: Refreshing state... [id=substratal-prod-api:logs]
aws_route53_record.cert_validation["api.substratalapps.com"]: Refreshing state... [id=Z0783627X8ON1GH6I0XB__45a2e898c02bdc2297dde6c5e5a798a8.api.substratalapps.com._CNAME]
module.api.aws_iam_role_policy.this[0]: Refreshing state... [id=substratal-prod-api:function]
module.mcp.aws_lambda_function.this: Refreshing state... [id=substratal-prod-mcp]
module.webhook_worker.aws_iam_role_policy.this[0]: Refreshing state... [id=substratal-prod-webhook-worker:function]
module.stripe_webhook.aws_iam_role_policy.this[0]: Refreshing state... [id=substratal-prod-stripe-webhook:function]
module.webhook_worker.aws_lambda_function.this: Refreshing state... [id=substratal-prod-webhook-worker]
module.stripe_webhook.aws_lambda_function.this: Refreshing state... [id=substratal-prod-stripe-webhook]
module.api.aws_lambda_function.this: Refreshing state... [id=substratal-prod-api]
aws_acm_certificate_validation.api: Refreshing state... [id=2026-10-20 20:44:42.323 +0000 UTC]
aws_apigatewayv2_domain_name.api: Refreshing state... [id=api.substratalapps.com]
aws_s3_bucket_server_side_encryption_configuration.audit_archive: Refreshing state... [id=substratal-prod-audit-archive-196936048600]
aws_s3_bucket_versioning.audit_archive: Refreshing state... [id=substratal-prod-audit-archive-196936048600]
aws_s3_bucket_public_access_block.audit_archive: Refreshing state... [id=substratal-prod-audit-archive-196936048600]
module.jobs.aws_iam_role_policy.this[0]: Refreshing state... [id=substratal-prod-jobs:function]
module.jobs.aws_lambda_function.this: Refreshing state... [id=substratal-prod-jobs]
aws_apigatewayv2_api_mapping.api: Refreshing state... [id=zdmjg2]
aws_route53_record.api: Refreshing state... [id=Z0783627X8ON1GH6I0XB_api.substratalapps.com_A]
module.webhook_worker.aws_lambda_alias.live: Refreshing state... [id=arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-webhook-worker:live]
module.mcp.aws_lambda_alias.live: Refreshing state... [id=arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-mcp:live]
module.api.aws_lambda_alias.live: Refreshing state... [id=arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-api:live]
aws_lambda_event_source_mapping.webhooks: Refreshing state... [id=bed6416f-6a72-474a-83e5-559c995cf8f3]
module.stripe_webhook.aws_lambda_alias.live: Refreshing state... [id=arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-stripe-webhook:live]
aws_cloudwatch_metric_alarm.lambda_throttles["api"]: Refreshing state... [id=substratal-prod-api-throttles]
aws_cloudwatch_metric_alarm.lambda_throttles["worker"]: Refreshing state... [id=substratal-prod-worker-throttles]
aws_lambda_permission.apigw["stripe"]: Refreshing state... [id=apigateway-stripe]
aws_lambda_permission.apigw["mcp"]: Refreshing state... [id=apigateway-mcp]
aws_lambda_permission.apigw["api"]: Refreshing state... [id=apigateway-api]
aws_apigatewayv2_integration.fn["api"]: Refreshing state... [id=3h3bk4c]
aws_apigatewayv2_integration.fn["mcp"]: Refreshing state... [id=8xk20ia]
aws_apigatewayv2_integration.fn["stripe"]: Refreshing state... [id=34l24l9]
aws_apigatewayv2_route.r["GET /mcp"]: Refreshing state... [id=s6ez97n]
aws_apigatewayv2_route.r["POST /internal/stripe/webhook"]: Refreshing state... [id=bczysio]
aws_apigatewayv2_route.r["GET /.well-known/{proxy+}"]: Refreshing state... [id=kb35wrh]
aws_apigatewayv2_route.r["DELETE /mcp"]: Refreshing state... [id=peme2oj]
aws_apigatewayv2_route.r["$default"]: Refreshing state... [id=cr82zzs]
aws_apigatewayv2_route.r["POST /mcp"]: Refreshing state... [id=a197cl8]
module.jobs.aws_lambda_alias.live: Refreshing state... [id=arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-jobs:live]
aws_iam_role_policy.scheduler: Refreshing state... [id=substratal-prod-scheduler:terraform-339ca2ff2a164d299b046f72e0]
aws_cloudwatch_metric_alarm.lambda_errors["jobs"]: Refreshing state... [id=substratal-prod-jobs-errors]
aws_cloudwatch_metric_alarm.lambda_errors["stripe"]: Refreshing state... [id=substratal-prod-stripe-errors]
aws_scheduler_schedule.job["webhook-dispatch"]: Refreshing state... [id=default/substratal-prod-webhook-dispatch]
aws_scheduler_schedule.job["entitlement-sweep"]: Refreshing state... [id=default/substratal-prod-entitlement-sweep]
aws_scheduler_schedule.job["seat-sync"]: Refreshing state... [id=default/substratal-prod-seat-sync]
aws_cloudwatch_metric_alarm.lambda_errors["mcp"]: Refreshing state... [id=substratal-prod-mcp-errors]
aws_scheduler_schedule.job["settings-projections"]: Refreshing state... [id=default/substratal-prod-settings-projections]
aws_scheduler_schedule.job["stripe-events"]: Refreshing state... [id=default/substratal-prod-stripe-events]
aws_scheduler_schedule.job["subscription-access"]: Refreshing state... [id=default/substratal-prod-subscription-access]
aws_scheduler_schedule.job["webhook-health"]: Refreshing state... [id=default/substratal-prod-webhook-health]
aws_scheduler_schedule.job["erasure-cascade"]: Refreshing state... [id=default/substratal-prod-erasure-cascade]
aws_scheduler_schedule.job["push-maintenance"]: Refreshing state... [id=default/substratal-prod-push-maintenance]
aws_scheduler_schedule.job["audit-archive"]: Refreshing state... [id=default/substratal-prod-audit-archive]
aws_scheduler_schedule.job["cleanup"]: Refreshing state... [id=default/substratal-prod-cleanup]
aws_cloudwatch_metric_alarm.lambda_errors["worker"]: Refreshing state... [id=substratal-prod-worker-errors]
aws_cloudwatch_metric_alarm.lambda_errors["api"]: Refreshing state... [id=substratal-prod-api-errors]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.api.aws_lambda_alias.live will be updated in-place
  ~ resource "aws_lambda_alias" "live" {
      ~ function_version = "9" -> (known after apply)
        id               = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-api:live"
        name             = "live"
        # (5 unchanged attributes hidden)
    }

  # module.api.aws_lambda_function.this will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "substratal-prod-api"
      ~ last_modified                  = "2026-10-22T06:34:56.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-api:9" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-api:9/invocations" -> (known after apply)
      ~ s3_key                         = "8d1a3f6ed856747a33e07b8fb48c66ce607337d9/api.zip" -> "a316d5201be81e7ab48d789bbea4f1c455ba3338/api.zip"
        tags                           = {}
      ~ version                        = "9" -> (known after apply)
        # (28 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.jobs.aws_lambda_alias.live will be updated in-place
  ~ resource "aws_lambda_alias" "live" {
      ~ function_version = "9" -> (known after apply)
        id               = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-jobs:live"
        name             = "live"
        # (5 unchanged attributes hidden)
    }

  # module.jobs.aws_lambda_function.this will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "substratal-prod-jobs"
      ~ last_modified                  = "2026-10-22T06:34:56.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-jobs:9" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-jobs:9/invocations" -> (known after apply)
      ~ s3_key                         = "8d1a3f6ed856747a33e07b8fb48c66ce607337d9/jobs.zip" -> "a316d5201be81e7ab48d789bbea4f1c455ba3338/jobs.zip"
        tags                           = {}
      ~ version                        = "9" -> (known after apply)
        # (28 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.mcp.aws_lambda_alias.live will be updated in-place
  ~ resource "aws_lambda_alias" "live" {
      ~ function_version = "10" -> (known after apply)
        id               = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-mcp:live"
        name             = "live"
        # (5 unchanged attributes hidden)
    }

  # module.mcp.aws_lambda_function.this will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "substratal-prod-mcp"
      ~ last_modified                  = "2026-10-22T06:34:56.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-mcp:10" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-mcp:10/invocations" -> (known after apply)
      ~ s3_key                         = "8d1a3f6ed856747a33e07b8fb48c66ce607337d9/mcp.zip" -> "a316d5201be81e7ab48d789bbea4f1c455ba3338/mcp.zip"
        tags                           = {}
      ~ version                        = "10" -> (known after apply)
        # (28 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.stripe_webhook.aws_lambda_alias.live will be updated in-place
  ~ resource "aws_lambda_alias" "live" {
      ~ function_version = "9" -> (known after apply)
        id               = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-stripe-webhook:live"
        name             = "live"
        # (5 unchanged attributes hidden)
    }

  # module.stripe_webhook.aws_lambda_function.this will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "substratal-prod-stripe-webhook"
      ~ last_modified                  = "2026-10-22T06:34:56.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-stripe-webhook:9" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-stripe-webhook:9/invocations" -> (known after apply)
      ~ s3_key                         = "8d1a3f6ed856747a33e07b8fb48c66ce607337d9/stripe-webhook.zip" -> "a316d5201be81e7ab48d789bbea4f1c455ba3338/stripe-webhook.zip"
        tags                           = {}
      ~ version                        = "9" -> (known after apply)
        # (28 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.webhook_worker.aws_lambda_alias.live will be updated in-place
  ~ resource "aws_lambda_alias" "live" {
      ~ function_version = "9" -> (known after apply)
        id               = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-webhook-worker:live"
        name             = "live"
        # (5 unchanged attributes hidden)
    }

  # module.webhook_worker.aws_lambda_function.this will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "substratal-prod-webhook-worker"
      ~ last_modified                  = "2026-10-22T06:34:56.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-webhook-worker:9" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:196936048600:function:substratal-prod-webhook-worker:9/invocations" -> (known after apply)
      ~ s3_key                         = "8d1a3f6ed856747a33e07b8fb48c66ce607337d9/webhook-worker.zip" -> "a316d5201be81e7ab48d789bbea4f1c455ba3338/webhook-worker.zip"
        tags                           = {}
      ~ version                        = "9" -> (known after apply)
        # (28 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 10 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

@Adron
Adron merged commit 9c3f30a into main Oct 22, 2026
4 checks passed
Adron added a commit that referenced this pull request Oct 22, 2026
Conflicts: keep remaining_manual_steps.md deleted (moved to [bespoke]
issues in #29); keep the Phase 0 line without the /v1 mention.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant