Skip to content

feat: faucet address-cap race, retention, logging, tests and entity tidy-ups (follow-ups to #829) - #830

Merged
fraseragain merged 8 commits into
developfrom
feat/faucet-hardening-low
Oct 13, 2026
Merged

fraseragain merged 8 commits into
developfrom
feat/faucet-hardening-low

Conversation

@fraseragain

@fraseragain fraseragain commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #829, which is stacked on #819. The base is feat/faucet-cost-and-reservations; retarget as the earlier PRs merge.

Closes #823, closes #824, closes #825, closes #826, closes #827, closes #828.

What changed

  • Serialise the faucet address-cap check so concurrent requests cannot exceed it #823 Address cap under the lock. The on-chain balance is read before the reservation and does not include transfers still on their way, so simultaneous requests could each be allowed the full room under FAUCET_ADDRESS_CAP_CHEQ. The cap check now also runs under the per-customer lock, subtracting requests made for the same address in the last FAUCET_BALANCE_SETTLE_SECONDS (default 20) from the room, so only as many requests as fit are accepted. The bootstrap top-up is not subject to it. Trade-off: for those 20 seconds a request that already reached the balance is counted twice, which can briefly refuse a legitimate follow-up only when the address is near its cap.
  • Add retention for the faucetRequest table #824 Retention. faucetRequest rows older than FAUCET_REQUEST_RETENTION_MONTHS (default 13 calendar months, 0 keeps everything) are deleted. Studio has no scheduler, so the cleanup runs in the background from normal reservations, at most once an hour per process, and never delays or fails the request that triggered it. A manual query is in the README.
  • Log and track faucet quota reservations, releases and rejections #825 Logging. Every reservation, completion, release, unknown outcome and rejection (quota, minimum interval, address cap, plus the early checks before Stripe), the sweep of abandoned reservations and the retention cleanup write one JSON log line with an event name and the customer, address, amount and relevant figures. The README lists the events and a support query. The event tracker is deliberately not used: its operations feed usage and billing reports.
  • Decide, test and document which auth methods may use the faucet endpoint #826 Auth methods. Session, API key and machine-to-machine credentials all remain allowed (as decided). A unit test pins that they all end in the same request:faucet:testnet scope check and that the route is not open to unauthenticated callers; the Swagger description and README state the supported credentials, including that an M2M application needs the scope granted in LogTo. No behaviour change.
  • FaucetRequestEntity tidy-ups: bigint transformer and database-side createdAt #827 Entity tidy-ups. amountNcheq is a bigint in the entity (via a column transformer), and createdAt is set by the database (DEFAULT now()) so all instances agree on when a request was made. New migration 1791500000002; the ledger test checks the schema still matches the entity.
  • Add handler-level faucet tests that run in CI #828 Handler tests. 21 handler-level tests call the real requestFaucetTokens and getBalances against a real Postgres with Stripe, the faucet, the account lookup and the on-chain balance faked. They cover session, API-key and M2M requests, the quota, minimum interval, address cap, failure handling, reservation status and the balances response. As agreed, CI is unchanged: they skip unless FAUCET_TEST_DATABASE_URL is set. They use Node's test runner (a dependency of the controller does not import under jest) and a small loader hook for the balance fake, and live in tests/integration, outside jest's tests/unit pattern.

How to test

  1. npm run test:unit (new pure tests: retention cutoff, address room, logging events, auth rule).
  2. With a disposable Postgres, the ledger and handler suites; the README has the exact commands. Ledger suite (jest): reserve/release, interval, statuses, address cap including simultaneous requests, retention, entity round trip and the schema-drift check. Handler suite (Node 20): the 21 end-to-end scenarios.
  3. I ran both against Postgres 16 on Node 20: 212 of 212 pass, including the 21 handler tests; local Node 26 runs 186 and skips the 26 database-backed ones. I also confirmed end to end that the retention cleanup is triggered by a normal request, deletes only rows past retention and runs at most once an hour, and that four simultaneous requests for an address with room for one are now refused three times.

Risks and rollout

  • Migration 1791500000002 is additive (a column default) and safe in any deploy order.
  • FAUCET_BALANCE_SETTLE_SECONDS can over-refuse for a few seconds near the cap; set it to 0 to turn the in-flight accounting off.
  • The first request after a deploy per process may delete old rows in the background; with the default 13 months and a table that only just appeared, nothing is deleted for over a year.

Architecture and best-practice review fixes

  • Merges the fixes from feat: allow API keys on the faucet endpoint with a monthly CHEQ quota #819 and feat: faucet cost, burst limiting and reservation status (follow-ups to #819) #829.
  • One clock for every decision. createdAt is set by the database, but month windows, the minimum interval, the pending timeout, the settle window and retention were compared against the application clock, so instances with different clocks could disagree about how old a request is. They now all use the database clock (clock_timestamp(), read after the lock is taken), the 429 Retry-After and balances response use the same window, and completedAt is set by the database. Tests fake the application clock hours ahead; I confirmed they fail against the previous behaviour (a fresh pending reservation was treated as expired, the month was wrong, and completedAt was 3 hours off).

Merged state verified on Node 20 against Postgres 16: 234 of 234 tests pass, including the 21 handler tests.

🤖 Generated with Claude Code

dubiouspip and others added 8 commits October 9, 2026 14:47
… createdAt

The entity now exposes amountNcheq as a bigint (via a column transformer) instead of a string, which removes the constructor guard, and createdAt is set by the database (DEFAULT now()) so all instances agree on when a request was made. Adds a migration for the column default; the ledger test checks the schema still matches the entity. Refs #827

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The on-chain balance is read before the reservation and does not include transfers still on their way, so simultaneous requests could each be allowed the full room under FAUCET_ADDRESS_CAP_CHEQ. The cap check now also runs under the per-customer lock, subtracting requests made for the same address in the last FAUCET_BALANCE_SETTLE_SECONDS (default 20) from the room, so only as many requests as fit are accepted. The bootstrap top-up is not subject to it. Refs #823

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
faucetRequest rows older than FAUCET_REQUEST_RETENTION_MONTHS (default 13 calendar months, 0 keeps everything) are deleted. Studio has no scheduler, so the cleanup runs in the background from normal reservations, at most once an hour per process, and never delays or fails the request that triggered it. The README documents a manual query as well. Refs #824

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Every reservation, completion, release, unknown outcome and rejection (quota, minimum interval, address cap, plus the early checks before Stripe), the sweep of abandoned reservations and the retention cleanup now write one JSON log line with an event name and the customer, address, amount and relevant figures, so a blocked or refunded request can be traced. The README lists the events and a support query. The event tracker is not used because its operations feed usage and billing reports. Refs #825

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ee auth methods

Session, API key and machine-to-machine credentials all end in the same scope check, so a unit test pins that the route requires request:faucet:testnet and is not open to unauthenticated callers. The Swagger description and README now state the supported credentials and that an M2M application needs the scope granted in LogTo. No behaviour change: all three methods remain allowed. Refs #826

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
21 tests call the real requestFaucetTokens and getBalances against a real Postgres with Stripe, the faucet, the account lookup and the on-chain balance faked. They cover session, API-key and machine-to-machine requests, the quota, minimum interval, address cap, failure handling, reservation status and the balances response. They use Node's test runner (the controller's dependencies do not load under jest) and a small loader hook for the balance fake, live outside tests/unit, and skip unless FAUCET_TEST_DATABASE_URL is set, so CI is unchanged. The README says how to run them. Refs #828

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
createdAt is set by the database (DEFAULT now()), but month windows, the minimum interval, the pending timeout, the address-cap settle window and retention were compared against the application clock, so instances with different clocks could disagree about how old a request is. They now all use the database clock: reserve reads clock_timestamp() after taking the lock, and the read paths read it once per call. getQuotaStatus returns the usage together with the window, so the 429 Retry-After and the balances response use the same clock, and completedAt is set by the database. Tests fake the application clock hours ahead and fail against the previous behaviour. Refs #827

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Base automatically changed from feat/faucet-cost-and-reservations to develop October 13, 2026 11:07
@fraseragain
fraseragain merged commit 766b1cd into develop Oct 13, 2026
12 checks passed
@fraseragain
fraseragain deleted the feat/faucet-hardening-low branch October 13, 2026 11:33
cheqd-bot Bot pushed a commit that referenced this pull request Oct 13, 2026
## [3.18.0-develop.3](3.18.0-develop.2...3.18.0-develop.3) (2026-10-13)

### Features

* delete faucet request history past a retention period ([038a079](038a079)), closes [#824](#824)
* faucet address-cap race, retention, logging, tests and entity tidy-ups (follow-ups to [#829](#829)) ([#830](#830)) ([766b1cd](766b1cd)), closes [#823](#823) [#824](#824) [#825](#825) [#826](#826) [#827](#827) [#823](#823) [#824](#824) [#825](#825) [#826](#826) [#827](#827) [#828](#828) [#819](#819)
* log faucet quota decisions as structured events ([b21176b](b21176b)), closes [#825](#825)

### Bug Fixes

* enforce the faucet address cap under the quota lock ([eed180b](eed180b)), closes [#823](#823)
* use the database clock for every faucet quota decision ([cb6c856](cb6c856)), closes [#827](#827)
@cheqd-bot

cheqd-bot Bot commented Oct 13, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 3.18.0-develop.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

fraseragain added a commit that referenced this pull request Oct 13, 2026
…e (follow-up to #830) (#839)

## What changed
Stacked on #830 (base is `feat/faucet-hardening-low`, so the diff is
only this change; merge after #819, #829 and #830).

- **Any address:** `POST /account/faucet` now accepts any valid
lowercase `cheqd` bech32 address, owned or not. Previously an address
the customer did not own returned `403`. The address is validated in
`faucetValidator` (before any subscription lookup): it must be a
lowercase `cheqd` bech32 account address (20 bytes, valid checksum),
otherwise `400`. With no address it still funds the customer's own
testnet account.
- **Initial top-up, once:** with `ENABLE_ACCOUNT_TOPUP=true`, the top-up
to `TESTNET_INITIAL_TOPUP_CHEQ` now runs only when the call creates the
customer's testnet account (`POST /account` and bootstrap), not on every
call.
- **Default stays 10,000 CHEQ:** `TESTNET_FAUCET_UPPER_CAP_CHEQ` is no
longer a fallback for `TESTNET_INITIAL_TOPUP_CHEQ`. It was the faucet
cap (often 100,000), so a deployment that only set the old name got a
100,000 top-up.
- **Bootstrap quota:** bootstrap now skips the top-up when the monthly
quota is exhausted (logs `faucet.bootstrap_topup_skipped`) instead of
returning `500`, matching `POST /account`.

## Why
Users should choose how much to top up and where, within a 100,000 CHEQ
monthly quota, in one or several requests.

## Behaviour unchanged
- Monthly quota (`FAUCET_MONTHLY_LIMIT_CHEQ`, default 100,000) is per
customer across all addresses; the 10k initial top-up counts against it.
- Per-address cap (`FAUCET_ADDRESS_CAP_CHEQ`, default 100,000) is
checked against the address's on-chain balance, so a funded address can
be topped up only to the cap.
- Minimum interval, reservations and plan/scope checks are as in #830.

## Edge cases
- A mainnet address has the same `cheqd1…` prefix, so it is accepted; it
receives testnet tokens only.
- If the one-off initial top-up fails, nothing retries it; the user uses
`POST /account/faucet`.
- For existing accounts, the bootstrap `testnetMinimumBalance` flag
stays `false` because no check runs.

## How to test
- Integration (Node 20, disposable Postgres):
`FAUCET_TEST_DATABASE_URL=... npx -y -p node@20 node --import tsx
--import ./tests/integration/hooks/register.mjs --test
tests/integration/faucet-handler.test.ts` — 35/35 pass, including cases
for a non-owned address, malformed/wrong-prefix/uppercase addresses, and
the initial top-up (new vs existing account, top up to 10,000, quota
exhausted, faucet failure, `ENABLE_ACCOUNT_TOPUP` off) on both `POST
/account` and bootstrap (`topupTestnet` is exported for these tests).
Removing the new-account gating or the quota skip makes 3 of them fail.
- Unit: `npx jest tests/unit/account` — 67 pass, 30 skipped (includes
`isValidFaucetAddress`).
- `tsc --noEmit` and Prettier clean.

## Risks
- Anyone with a faucet-enabled plan can fund any address, up to their
own monthly quota. Nothing caps total CHEQ received by one address
across customers beyond the 100,000 balance cap. A cross-customer
per-address limit could be added.
- `docker/Dockerfile` `TESTNET_MINIMUM_BALANCE` default is corrected
from 1000 to 10000 so it no longer overrides the code default. An
environment that sets 100000 explicitly still wins.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
cheqd-bot Bot pushed a commit that referenced this pull request Oct 13, 2026
## [3.18.0-develop.4](3.18.0-develop.3...3.18.0-develop.4) (2026-10-13)

### Features

* **faucet:** allow funding any valid cheqd testnet address ([7a95c5d](7a95c5d))
* **faucet:** fund any cheqd address; top up new testnet accounts once (follow-up to [#830](#830)) ([#839](#839)) ([1558ccf](1558ccf)), closes [#819](#819) [#829](#829)

### Bug Fixes

* **faucet:** keep the initial top-up at 10,000 CHEQ by default ([dc2c0c6](dc2c0c6))
* **faucet:** top up only newly created testnet accounts; skip on quota ([56d3f25](56d3f25))
* **swagger:** make the faucet description valid YAML so the endpoint is documented ([7f2f6ac](7f2f6ac))
@fraseragain fraseragain mentioned this pull request Oct 14, 2026
3 tasks
fraseragain added a commit that referenced this pull request Oct 14, 2026
**DRAFT. Do not merge until the checklist below is done.** Release of
`develop` to `main`: expected version **3.18.0** (minor, from the `feat`
commits).

## What's included
Faucet quota stack plus two follow-ups (44 commits on `develop` since
v3.17.0):
- **API keys on the faucet** (#819): `POST /account/faucet` works with
an API key (`request:faucet:testnet` scope). Per-customer **monthly
quota** `FAUCET_MONTHLY_LIMIT_CHEQ` (default 100,000 CHEQ, UTC calendar
month) and a per-address **cap** `FAUCET_ADDRESS_CAP_CHEQ` (defaults to
the monthly limit). `GET /account/balances` now returns a `faucet` block
(cap, quota used, reset date) that the portal reads.
- **Cost, burst limiting, reservations** (#829): cached Stripe lookup,
`FAUCET_MIN_INTERVAL_SECONDS` (10s) between requests, reservation
status.
- **Hardening** (#830): address-cap check under the per-customer lock,
request-history retention (13 months), structured logging, faucet call
timeout, handler-level tests.
- **Fund any address** (#839): any valid lowercase `cheqd` address, not
only owned ones. The initial top-up runs only when a testnet account is
created.
- **Default amount** (#841/#842): with no `amount`,
`FAUCET_DEFAULT_REQUEST_CHEQ` (default 10,000) is sent, limited by the
cap and quota left.
- **Swagger fix**: the faucet doc comment was invalid YAML, which
silently dropped `/account/faucet` from the generated docs. Fixed in
#839; the generated spec has all 59 paths.

## Deploy notes
- **Three new migrations** (a new `faucetRequest` table with its index
and FK, then `status`/`completedAt` columns and a `createdAt` default on
that same table; nothing touches existing tables): `1791500000000`,
`…001`, `…002`. The production "Database migrations" workflow runs `npm
run migrate` on push to `main` and applies them. Old code ignores them,
so a rollback of the image is safe.
- **Behaviour changes that need production env vars checked before
merge:**
- `TESTNET_FAUCET_UPPER_CAP_CHEQ` is no longer read. The initial top-up
now comes from `TESTNET_INITIAL_TOPUP_CHEQ` (default 10,000) and the
per-address cap from `FAUCET_ADDRESS_CAP_CHEQ` (default 100,000; it was
10,000).
- `TESTNET_MINIMUM_BALANCE` default in the Dockerfile changes from 1000
to 10000 (applies only if production does not set it).
- Intended production values: `TESTNET_INITIAL_TOPUP_CHEQ=10000`,
`FAUCET_ADDRESS_CAP_CHEQ=100000`. Remove
`TESTNET_FAUCET_UPPER_CAP_CHEQ`.
- No other new variables are required; all new `FAUCET_*` settings have
defaults (see `example.env`).

## Before merging (checklist)
- [ ] Production DigitalOcean app: env vars set as above; check whether
it still has a PRE_DEPLOY `database-migrations` job (staging's hung on
every deploy and was removed; see below)
- [ ] Manual staging pass: Balances page (via the portal), a faucet
request with a session and with an API key, a brand-new testnet account
gets 10,000 CHEQ
- [ ] An approving review (`main` requires 1)

## How to merge
- **Use a merge commit, not squash.** Semantic Release derives the
version from the conventional commits since the last tag; a squash would
collapse them into one `chore(release)` commit and release a patch
(3.17.1) instead of 3.18.0, and `main` would diverge from `develop`
again.
- On merge: Semantic Release (version, changelog, tag), the production
Docker image is built and pushed, and production migrations run. There
is no manual approval gate on the `production` environment.
- **Release the portal after this one**: its Balances page reads the new
`faucet` block from Studio.
- Afterwards open a `main` into `develop` back-merge PR (merge commit),
because Semantic Release commits `chore(release): X.Y.Z [skip ci]`
straight to `main`.

## Verification done on staging (`develop` at e748f7c)
- Deployed and serving the new code; live Swagger matches `develop` (59
paths).
- Handler-level integration tests: 37/37 pass against Postgres. Unit
tests: 209 pass.
- Unauthenticated sweep of 24 GET endpoints: no unexpected 5xx. Bad
credentials return 500 on every authenticated endpoint on both staging
and production (pre-existing, not part of this release).
- Not yet verified: authenticated flows and the portal UI (needs a
login).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
cheqd-bot Bot pushed a commit that referenced this pull request Oct 14, 2026
## [3.18.0](3.17.0...3.18.0) (2026-10-14)

### Features

* add a minimum interval between faucet requests per customer ([2d3ebf5](2d3ebf5)), closes [#821](#821)
* add monthly faucet quota service and helpers ([7eef4f7](7eef4f7)), closes [#818](#818)
* allow API keys on POST /account/faucet with monthly quota ([6ed02c8](6ed02c8)), closes [#818](#818)
* allow API keys on the faucet endpoint with a monthly CHEQ quota ([#819](#819)) ([17dcb04](17dcb04)), closes [cheqd/studio-portal#258](https://github.com/cheqd/studio-portal/issues/258)
* apply monthly quota as the per-request limit for explicit faucet amounts ([6979f42](6979f42)), closes [#818](#818)
* **db:** add faucetRequest table for monthly faucet quota ([ef43cb1](ef43cb1)), closes [#818](#818)
* debit the initial testnet top-up from the monthly faucet quota ([15c8d8c](15c8d8c)), closes [#818](#818)
* delete faucet request history past a retention period ([038a079](038a079)), closes [#824](#824)
* faucet address-cap race, retention, logging, tests and entity tidy-ups (follow-ups to [#829](#829)) ([#830](#830)) ([766b1cd](766b1cd)), closes [#823](#823) [#824](#824) [#825](#825) [#826](#826) [#827](#827) [#823](#823) [#824](#824) [#825](#825) [#826](#826) [#827](#827) [#828](#828) [#819](#819)
* faucet cost, burst limiting and reservation status (follow-ups to [#819](#819)) ([#829](#829)) ([e7d7b67](e7d7b67)), closes [#820](#820) [#821](#821) [#820](#820) [#821](#821) [#822](#822) [#828](#828)
* **faucet:** allow funding any valid cheqd testnet address ([7a95c5d](7a95c5d))
* **faucet:** fund any cheqd address; top up new testnet accounts once (follow-up to [#830](#830)) ([#839](#839)) ([1558ccf](1558ccf)), closes [#819](#819) [#829](#829)
* **faucet:** send a default amount when none is requested ([#841](#841)) ([7864611](7864611))
* **faucet:** send a default amount when none is requested ([#841](#841)) ([#842](#842)) ([18223ce](18223ce))
* log faucet quota decisions as structured events ([b21176b](b21176b)), closes [#825](#825)
* return faucet cap and monthly quota usage from GET /account/balances ([3ba590d](3ba590d)), closes [#818](#818)
* track whether each faucet quota reservation completed ([a371bec](a371bec)), closes [#822](#822)
* use a single 100k CHEQ cap for faucet balance and monthly quota ([4612b1f](4612b1f)), closes [#818](#818)

### Bug Fixes

* enforce the faucet address cap under the quota lock ([eed180b](eed180b)), closes [#823](#823)
* **faucet:** keep the initial top-up at 10,000 CHEQ by default ([dc2c0c6](dc2c0c6))
* **faucet:** top up only newly created testnet accounts; skip on quota ([56d3f25](56d3f25))
* **swagger:** make the faucet description valid YAML so the endpoint is documented ([7f2f6ac](7f2f6ac))
* time out the faucet call and warn on invalid faucet config ([9b14df1](9b14df1)), closes [#818](#818)
* use the database clock for every faucet quota decision ([cb6c856](cb6c856)), closes [#827](#827)

### Performance Improvements

* cache the Stripe subscription lookup and skip the balance read when quota is used up ([dd408e2](dd408e2)), closes [#820](#820)
@cheqd-bot

cheqd-bot Bot commented Oct 14, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 3.18.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants