[kentare]

What and why

The knowledge database (#125) needs work that outlives a process: ingestion runs that last several nights, conversions that take hours on a GPU host, and index builds that wait for a night window. The platform has no durable queue; background work runs inside one API process under in-memory locks and heartbeats. The review on #125 asked for this part to go in on its own first.

• ADR 0041 Durable jobs on pg-boss: a JobQueue port in @genai/jobs, implemented with pg-boss in packages/providers-pgboss. A job lives in the pgboss schema of the server whose state it follows and is sent in the same transaction; the worker runs jobs and the API only sends them.
• Only the worker's role can read or change the job tables: the worker runs as genai_worker, a member of genai_runtime with DML on pgboss. genai_runtime, the API's role, gets USAGE on the schema and EXECUTE on the console's SECURITY DEFINER read functions, which require withPlatformAccess, and no right on the tables. A queue the API sends to grants only what sending needs. Sending dead letters back goes through the worker.
• Rules for every queue: its contract in code; one attempt per call to an outside service, with timeouts from that service; idempotent handlers; keys and never content in job data, and the organisation taken from the record, never from the job; work longer than 24 hours split into jobs, with its progress in its record; deletion 7 days after a job ends and 14 days for one nobody takes; an admission rule for any queue that fetches, converts or calls a model; dead letters for operators, with an audited send-back (platform.jobs.redrive); schedule() in Europe/Oslo time.
• No queue of its own. ADR 0040, evaluating with Langfuse (Propose evaluating with Langfuse #100), needs none: its score wait stays in the eval run's lease, and its trace deletes are pending rows the nightly purge works through. The knowledge database's queues are in ADR 0042 (Propose the knowledge database #125, based on this branch).
• ADR 0022 points here: the pgboss tables have no row-level security, and the grants keep the API out of them. It also corrects its own claim that production connects with DATABASE_RUNTIME_URL: no environment sets it, and deployed apps switch role per connection. The README gets its row.

Tracked in #155, built in #180. Part of digdir/kunnskapsassistenten#151.

How it was tested

• Spike with pg-boss 12.38.0 in a pgvector/pgvector:0.8.2-pg17 container (tag evidence/knowledge-spikes): install with only CREATE, a job in a rolled-back transaction does not exist, one queued job per key, priority, three attempts and then the dead-letter queue, a killed worker's job taken over 30 s after the kill, a 10-minute job handed out once, and Node's default fetch giving up after 302 s.

• Read pg-boss 12.38.0's source: its defaults (expiry 15 minutes, deletion 7 days after a job ends, 14 days for a job nobody takes), and what starting and sending read and write (pgboss.version, pgboss.queue, pgboss.job_now(), an insert into the job table returning its id).

• The grants are built in Run durable jobs on pg-boss in the worker, behind a JobQueue port #180, with "genai_runtime cannot read, change or delete a job", "the console reads jobs as genai_runtime, only with platform access", "the worker sends and works jobs as genai_worker" and "the migration refuses a genai_worker that is a superuser or bypasses row-level security".

• Checked the Context against main: failStaleRuns and staleRunHours in apps/api/src/ingestion.ts, background-lease.ts and leader-lease.ts. Checked that nothing in infra/ or .github/ sets DATABASE_RUNTIME_URL.

• npx prettier --check docs/adr.

• npm run check passes (docs only)

Checklist

• Every new rule has a test whose name states it
• User-facing text is in the i18n catalogues, in bokmål and English, with no em-dashes
• Vendor SDKs stay in packages/providers-*
• ADRs, docs and docs/architecture/modules are updated if a contract, boundary or module changed
• No secrets, personal data or local paths

[nlundee]

This is what we asked for on #125, and it reads well. The claims about today's code hold on main (failStaleRuns skipping only the instance's own runs, the 6-hour sweep, the 2-minute lease, the in-process writer locks), and the retention, keys-only payloads and "one attempt, the queue retries" answer the points from the #125 review. Close to approve, one thing to settle first:

1. The definer function isn't the only way in (0041, Consequences, and the new line in 0022). genai_runtime gets DML on the pgboss tables, so the API can SELECT from pgboss.job directly, across organisations, without withPlatformAccess. Either grant the runtime only what send() needs, so the SECURITY DEFINER function really is the only read path, or say plainly that it's a convention and not enforced by the database. With keys-only payloads the impact is small, but 0022 should describe what the database actually enforces.

Smaller:

• #100 still says in its Why section that the wait for scores "is a durable job" (proposed in #125), which contradicts both this record and its own "No durable jobs yet". Worth fixing there so the two don't disagree. #100 and #117 also both use 0040.
• A handler should take the organisation from the record it loads, never from the job data. It follows from "keys, never content", but one sentence (and a test in the list) would make it a rule.
• The Context motivates work that lasts several nights, while a job can't run past 24 hours. One line saying long work is split into jobs under that limit, with progress in its record, would close the gap. #125 does this per queue already.
• The PR text says "Built in #155", the record says "Not built (issue 155)". I read #155 as the tracking issue, so maybe "Tracked in #155".

Fine to approve once the first point is settled.

[kentare]

Thanks. The first point is settled with grants (73871ab): the worker runs as genai_worker, the only role with rights on the pgboss tables, and genai_runtime gets the schema and the two read functions, nothing else. #180 builds it and tests that genai_runtime cannot read, change or delete a job. That also showed ADR 0022 claimed production uses DATABASE_RUNTIME_URL, which no environment sets; 3e2274f corrects it.

The organisation rule and the 24-hour split are in 0041 (the organisation test comes with the first real queue, #135), and the PR text now says "Tracked in #155". #100 merged as ADR 0040, where the durable job is only a condition for judges that block, so the two agree; #117 moves to 0043.

[nlundee]

All three points are settled, and better than I asked: grants instead of a convention, with genai_worker as the only role on the job tables and a test that genai_runtime can't read, change or delete a job. Good catch on DATABASE_RUNTIME_URL in ADR 0022 too. The organisation rule and the 24-hour split read well.

One correction on numbering: 0043 is taken by #185 (code runs in agentctl), so #117 is now 0044. I've already renumbered it there.

Approving.