[kentare]

What and why

Work that must outlive a process needs a durable queue, and genki has none. ADR 0041 (#179) chooses pg-boss. This PR builds the queue itself, with no real queue on it yet: the knowledge queues come with #135.

• JobQueue port in @genai/jobs (packages/jobs/src/job-queue.ts): send with a key, priority, start time and an optional transaction; work with heartbeats; schedule with a time zone; cancel; dead letters, send-back and counts. defineQueue states each queue's policy, expiry (at most 24 hours), heartbeat (at least 10 s), retries, dead-letter queue and payload schema, and a payload value with whitespace or longer than 128 characters is refused, so job data holds keys only. The console's reads are a port of their own, JobReads. An in-memory queue for tests and dev mode. MessageQueue stays.
• A contract suite in @genai/jobs/testing that the in-memory queue and pg-boss both run, so the in-memory queue does what pg-boss does with keys, dead letters and expiry.
• packages/providers-pgboss on pg-boss 12.38.0. The owner role installs and migrates schema pgboss, creates each queue with its <queue>.dead dead-letter queue and sets deletion on every queue (7 days after a job ends, 14 days for one nobody takes). Every other instance runs with migrate and createSchema off. A queue cannot change its policy, dead-letter queue or heartbeat in place.
• Only the worker's role can read or change the job tables (ADR 0041). The migration (packages/providers-postgres/src/schema.ts) creates genai_worker: no login, not a superuser, without BYPASSRLS, a member of genai_runtime, and it refuses to run if the role is a superuser or bypasses row-level security, as for genai_runtime. installJobQueue grants DML on the pgboss tables, sequences and functions to genai_worker only. genai_runtime, which the API uses, gets USAGE on the schema and EXECUTE on the two SECURITY DEFINER read functions, which refuse without app.platform_access and inside a tenant transaction, and nothing else; the install revokes any other right it had, and PUBLIC's default EXECUTE on pg-boss's functions. Since a role without table rights cannot start pg-boss, the console reads through PgBossJobReads.
• The worker (apps/worker/src/main.ts) switches its store pool to genai_worker (a new role option on createPostgresStores), installs and works platformQueues (packages/jobs/src/platform-queues.ts, empty for now) and stops the queue first on shutdown. The API keeps genai_runtime. npm run dev -w apps/worker runs the worker locally; dev:all does not start it.
• pg-boss and pgboss in the vendor rules of .dependency-cruiser.cjs, docs/architecture/modules/providers-pgboss.yaml (new), worker.yaml, order.yaml and the System Atlas page, a "Durable jobs" paragraph in docs/development.md, the worker's role in infra/README.md, and DATABASE_RUNTIME_URL per app in .env.example.

Part of #155. Stacked on #179: the base is docs/durable-jobs-adr until it merges.

Deploy: no infra change. Both apps log in as the app identity and switch role per connection, and the migration creates genai_worker and lets the app identity switch to it. It runs grant genai_runtime to genai_worker as the app identity, which needs admin option on genai_runtime; the identity has it if its own migration created genai_runtime, which is not checked in test or production yet. After the first deploy to test, select pg_has_role('genai_worker', 'genai_runtime', 'MEMBER') is true and select has_table_privilege('genai_runtime', 'pgboss.job', 'select') is false.

Not in this PR: the console screen for queue counts and dead letters, its routes and platform.jobs.redrive with a reason and audit (#181); the API sending jobs, since no queue it sends to exists yet; the tests "a handler run twice leaves the same result", "a client of an outside service makes one request per attempt" and "a handler acts for the organisation of the record it loads when the job's data names another", which need a real queue, handler and record.

How it was tested

• npm run check (461 test files, 4945 tests pass).

• npx vitest run packages/jobs packages/providers-pgboss: the contract suite runs against the in-memory queue and against pg-boss on postgres:16-alpine as genai_worker: "a queue refuses a payload its schema does not admit", "job payloads hold no content", "works a sent job once with its data", "a job sent in a rolled-back transaction does not exist", "a job that keeps failing ends in the dead-letter queue", "sending a dead letter back works the job again", "lists dead letters per organisation", "never works a cancelled job", "holds one queued job per key on a short queue", "holds one queued job per key inside a transaction", "sending a dead letter back keeps its key, and leaves it while another job holds the key", "takes jobs in priority order", "an attempt that runs past its expiry is aborted and retried". The transaction, send-back and expiry tests fail on the in-memory queue as first pushed (819b852) and pass on pg-boss.

• pg-boss on the platform server's roles: "only the owner installs pgboss, and the API and the worker run with migrate and createSchema off", "genai_runtime cannot read, change or delete a job" (select, update, delete and insert on pgboss.job and select on pgboss.queue give permission denied, starting pg-boss fails, and the only pgboss functions it may execute are the two read functions; it first grants genai_runtime DML by hand, so it also shows the install revokes it), "the worker sends and works jobs as genai_worker", "the console reads jobs as genai_runtime, only with platform access", "an ended job is deleted after 7 days" (reads deletion_seconds and retention_seconds for the queue and its dead-letter queue), "refuses to change a queue policy in place", "a killed worker's job is retried after its heartbeat" (ends the worker's pool mid-job), "a nightly schedule sends one job per occurrence with two instances".

• npx vitest run packages/providers-postgres/src/isolation.test.ts (owner with CREATEROLE, not a superuser): "runs the worker as genai_worker, a member of genai_runtime under the same row-level security" (on every tenant table the worker sees what genai_runtime sees, with and without a tenant), "the migration refuses a genai_worker that is a superuser or bypasses row-level security".

• Each grant and role rule switched off fails its test: the worker without DML (4 tests and the contract setup), readers keeping table rights (1), readers without EXECUTE (1), PUBLIC keeping EXECUTE (1), the role check (2), genai_worker outside genai_runtime (1), the store pool ignoring the role (1).

• In memory: "refuses a heartbeat shorter than pg-boss allows", "refuses a queue that expires before its heartbeat", "refuses an expiry longer than 24 hours", "refuses a queue name that reads as a dead-letter queue", "sends one job per occurrence in the time zone".

• Checked by hand at 1562837, before genai_worker, on a fresh local database from docker compose (pgvector pg17): npm run dev -w apps/worker started with pg-boss and 0 queues and created schema pgboss (version 46). A throwaway script, not in this PR, then installed four demo queues through installJobQueue and sent and worked jobs through PgBossJobQueue. Jobs for two organisations were worked, a job sent in a rolled-back transaction never appeared, a job delayed by 10 minutes waited, and a schedule sent one job per minute. A job that failed 3 times went to its .dead queue. counts and deadLetters under platform access listed it with its organisation and error, and after the failure was fixed, redrive sent it back and it completed on the first attempt. pg-boss's own dashboard (@pg-boss/dashboard 1.12.1) showed the same queues and jobs.

• npm run check passes

Checklist

• Every new rule has a test whose name states it
• User-facing text is in the i18n catalogues, in bokmål and English, with no em-dashes
• Vendor SDKs stay in packages/providers-*
• ADRs, docs and docs/architecture/modules are updated if a contract, boundary or module changed
• No secrets, personal data or local paths

[Copilot]

🟡 Changes recommended

Unresolved defects bypass keyed-job deduplication during transactions and redrives, leave handlers blocking the queue beyond their deadlines, and permit expiry values above the 24-hour limit.

4 open issues

• Reject attempt expiry values above 24 hours · New
• Include pending jobs in keyed queue policy checks · New
• Preserve job keys through dead-letter redrives · New
• Enforce attempt expiry and handle late completion · New

What changed in this PR

This PR adds durable-job infrastructure behind a JobQueue port, preparing the worker for future knowledge-processing queues. It introduces no production queues yet; API sending and the operator console remain deferred.

Changes:

• Adds queue contracts, an in-memory implementation and shared contract tests.
• Introduces a pg-boss provider with owner-role installation, runtime operation and guarded job-state reads.
• Integrates queue startup and shutdown into the worker, with dependency boundaries and documentation.

File	Description
tsconfig.json	Registers the pg-boss provider project.
packages/​providers-pgboss/​tsconfig.json	Configures provider compilation.
packages/​providers-pgboss/​src/​pgboss.test.ts	Tests contracts, permissions and durable lifecycle behavior.
packages/​providers-pgboss/​src/​index.ts	Implements pg-boss installation and the queue adapter.
packages/​providers-pgboss/​package.json	Declares provider exports and dependencies.
packages/​jobs/​src/​testing/​job-queue-contract.ts	Adds shared queue contract tests.
packages/​jobs/​src/​testing/​index.ts	Exports testing helpers.
packages/​jobs/​src/​platform-queues.ts	Introduces the empty platform queue registry.
packages/​jobs/​src/​job-queue.ts	Defines the port, validation and in-memory implementation.
packages/​jobs/​src/​job-queue.test.ts	Tests in-memory behavior and queue definitions.
packages/​jobs/​src/​index.ts	Exports queue APIs and the registry.
packages/​jobs/​package.json	Exposes the testing entry point.
package-lock.json	Locks provider and transitive dependencies.
docs/​development.md	Documents durable jobs and local worker startup.
docs/​architecture/​system-atlas.html	Adds the provider and worker connection.
docs/​architecture/​modules/​worker.yaml	Records the worker’s queue-provider dependency.
docs/​architecture/​modules/​providers-pgboss.yaml	Describes the provider’s responsibilities and connections.
docs/​architecture/​modules/​order.yaml	Places the provider in the architecture listing.
apps/​worker/​tsconfig.json	References the provider project.
apps/​worker/​src/​main.ts	Opens the queue and stops it before database shutdown.
apps/​worker/​package.json	Adds the provider dependency and development command.
.dependency-cruiser.cjs	Enforces pg-boss vendor boundaries.

_{Review effort: Balanced}