Repository navigation
feat: ingest Blend lending pool state, rates, and derived utilisation - #453
Conversation
|
@onajidavid87-web Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Miracle656
left a comment
There was a problem hiding this comment.
Needs a rebase. I tried to resolve this myself and stopped, because the conflicts are not the mechanical kind and guessing at them in config for a money-adjacent feature is not a trade I want to make.
What conflicts: .env.example (4 hunks), src/config.ts (3), src/index.ts (3), src/app.ts (2), prisma/schema.prisma (1), plus the two generated OpenAPI files.
Why keep-both does not work here. The config.ts hunks cut through object literals rather than between them. main adds a publisher: { block for the SEP-40 publisher; this PR adds a blend: { block. Both open a property inside the same object, and both closing braces fall outside the conflict region — so concatenating the two sides leaves one block unclosed:
src/config.ts(89,1): error TS1131: Property or signature expected
src/config.ts(593,1): error TS1005: '}' expected
src/index.ts(308,3): error TS1005: '}' expected
Resolving that correctly means knowing which block each tail brace belongs to, thirteen times. You have that in your head; I would be inferring it.
One thing I did find while trying: keeping both sides of .env.example produces a duplicate MCP_MAX_PAYMENT_PRICE= — there is a terse copy and a documented one. Keep the documented block when you rebase.
The work itself is wanted, and the timing is good. Blend lending rates are what the earn surface quotes, and a rate we compute from pool state rather than take on trust is the difference between a number we can defend and one we repeat. The shape looks right from the file list — a real migration, retention for the rate history, metrics, and tests for the ingester, the utilisation maths, retention and the route.
After rebasing, three things this repo will check and that have caught PRs today:
console.*is forbidden in non-test source; usecreateLoggersrc/__tests__/envDrift.test.tsfails if a newprocess.envread is missing from.env.example— and it now coversscripts/andexamples/tootests/openapi.test.tsfails if a route is documented but not registered, or registered but not documented
Get npx tsc --noEmit and npx vitest run clean locally and I will take it.
…ingestion main already ships a lending_rates table (0_init baseline) and GET /rates/:asset reading it, so the Blend ingester now writes into that schema instead of creating its own table: - drop the branch's CREATE TABLE migration; add 20261023130000_extend_lending_rates_for_blend, which adds the reserve state columns (APR, utilisation, totals, asset address) as nullable columns and a (network, pool_id, asset, timestamp) index - insertLendingRates writes venue 'blend' and stores APYs in supply_rate / borrow_rate - keep main's GET /rates/:asset unchanged; add the GET /rates pool listing to main's rates.ts and drop the duplicate route from the spec - move the Blend ingester and rate retention job onto the structured logger, give the adapter its README type/description, regenerate openapi.json and the README tables Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…closes #212) A dedicated ingester reads Blend pool reserves over Soroban, derives utilisation and supply/borrow APR, and stores them in `lending_rates` rather than `price_points` — a lending rate is not a spot price, and the adapter's `derive` returns `[]` so nothing leaks into the volume or VWAP paths. `GET /rates/:asset` serves them, a retention job prunes old rows, and the new columns ship as a migration. What it got right, and why I am noting it: the schema change comes with `20261023130000_extend_lending_rates_for_blend/migration.sql` rather than a bare `schema.prisma` edit. Since `migrate deploy` replaced `db push` this morning, a schema edit alone reaches no live database. The migration is additive (every column nullable), leaves the baseline untouched, sorts after `0_init`, and its index leads with `network`. The first PR today to need that workflow got it wrong; this one did not. Three changes on top of the contribution. 1. **`classification` was missing.** #455 made it a required field on `VenueAdapter` about ten minutes earlier, so this did not compile on merge. Set to `'primary'`, with a comment recording that it is informational here: Blend emits no price points, so it never reaches the net/gross split — and it is 'primary' because it holds its own reserves, being the venue a DeFindex vault allocates *into* rather than one re-reporting another venue's trades. Also registered in `VENUE_CLASSIFICATIONS`, which #455's new drift test requires — that guard earned its place on the very next adapter. 2. **The ingester loop was unbounded.** Blend runs outside `startVenueAdapterLoop` (it writes `lending_rates`, so the generic derive/write cycle does not fit), which meant it also missed the per-cycle deadline #471 added this afternoon. A `while (true)` around `await ingestBlendPools(network)` is the exact #351 shape: a slow Soroban RPC stretches one sweep indefinitely, the loop never crashes, so `restartIngester` never fires and the venue goes quiet with nothing raised anywhere. Now wrapped in `runCycleWithDeadline`, the same treatment #471 gave the bespoke snapshot ingester, and `ingestBlendPools` takes the signal so an abandoned cycle stops at a pool boundary — one pool is one write unit, so nothing is left half-written. 3. Regenerated `clients/typescript/src/schema.d.ts` for the new `/rates` response shape; it is byte-compared in CI. Note for later: `blendAdapter` is exported but deliberately not in `venueAdapters`, so Blend does not appear in the README's generated venue table. That is defensible — the table describes price venues — but worth revisiting if more non-price venues arrive. 134 test files / 1,603 tests pass; tsc clean. Claude-Session: https://claude.ai/code/session_01USgemLt4Rnz4SGB1Srf3GB
|
Merged as Nearly 2,000 lines on a path that writes to the money database, and it arrived with five test files, a migration, env documentation and the retention job already wired into The schema change came with a migration. Lending rates go in their own table.
New env vars are in Three changes on top. 1.
|
The /rates endpoint (added by Divinity-Labs#453) references #/components/responses/BadRequest, but that response component was never defined. openapi-typescript aborts with 'Can't resolve $ref at #/paths/~1rates/get/responses/400', so 'npm run client:gen' — the first step of CI's Typecheck & build job — fails before tsc runs, turning the check red for every open PR. Define BadRequest with the same minimal shape every other 400 in the spec uses (description: Invalid parameters) and regenerate the committed artifacts: - openapi.json via openapi:gen - clients/typescript/src/schema.d.ts via client:gen, which had gone stale (missing the whole /rates path) precisely because generation was aborting, so the 'git diff --exit-code -- clients/typescript' gate could never pass either. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…Labs#453/Divinity-Labs#455 batch The second main-line merge (0742337) reproduced the exact defects of the first (d6e8256) and added one more, so this is the same repair re-applied on the new tip: - src/metrics.ts: the active_subscriptions gauge is again left unclosed with the main-side counters spliced into its options bag — the file does not parse. Closed it: gauge with registers: [register], followed by the full set of counters. - src/api/graphql.ts: both parents' import statements were kept again (metrics, mercurius, config) plus a new duplicate parseAssetStr / getActivePairs pair from this batch. Deduped to one statement each, keeping the richer variants. - .env.example: document BENCH_MODE / BENCH_SUBSCRIBERS_PER_PAIR / BENCH_EVENTS_PER_PAIR so the env-drift guard passes on this branch's own benchmark script. Verified on 0742337 + this commit: tsc clean, full suite 1606 passed / 1 skipped, check-dropped-tests reports no case missing against the queue line (4dc40a3). Generated with Codebuff 🤖 Co-Authored-By: Codebuff <noreply@codebuff.com>
Summary
Adds rate and pool state ingestion for Stellar's lending protocol (Blend):
lending_ratestable via Prisma migration (rather than forcing intoprice_points), with a 30-day retention prune job.calcUtilization), with edge cases (blend_pool_errors_totalmetric increment, preventing process crashes.mainnetortestnet) without cross-contamination.GET /ratesandGET /rates/:assetendpoints returningvenue: 'blend', documented inopenapi.yamland synced toopenapi.json.Related issue
Type of change
Checklist
npx tsc --noEmitpassesnpm run buildpassesCloses #212