fix(plugin): discard shadow vectors when the registration is retired mid-embed

[Qiiks]

Problem

Cubic flagged two issues on #462 after it merged. Both are real; this PR fixes them.

P2 (a) — vectors written for a retired registration. processShadowQueueItem captures the shadow registration on entry, awaits the provider, then writes the returned vectors under the captured registration with no re-check:

const registration = shadowRegistrations.get(item.projectIdentity);
...
const embedded = await embedShadowItems(registration, ...);   // async gap
db.transaction(() => { /* writes vectors tagged registration.modelId */ })();

A provider call is a network round-trip (seconds), and on a cold Synapse model load it runs into tens of seconds. If unregisterProjectShadowEmbedding(projectIdentity) runs inside that window, the batch still commits — memory via saveEmbeddingIfHashMatches, commit via saveCommitEmbedding, chunk via replaceCompartmentChunkEmbeddings, all carrying a registration that no longer exists. If the same model id is re-armed later, those stale-lane rows are read as valid backfill and the real re-embed is skipped, so rows keep vectors from a lane that was deliberately retired. That is the same silent-staleness class the synapse:v1:pending vector-space warning exists to prevent.

The primary lane already guards this — embedItemsForProject re-reads the live registration after the await and returns null when generation or runtimeFingerprint changed. The shadow lane was missing the equivalent check.

P2 (b) — stale worker state for a retired registration. unregisterProjectShadowEmbedding clears shadowBackfillLastWriteOutcomes, but an item already in the worker loop could publish its outcome into that map afterwards. The stall detector reads that map, so a retired item's refusal could be attributed to a registration that replaced it — reporting a fresh lane as stalled_no_progress and leaving its remaining backfill unembedded.

Fix

(a) Re-check the live registration generation after the provider round-trip in all three scopes (memory, commit, chunk) and refuse the write when the registration was retired or replaced:

const embedded = await embedShadowItems(registration, ...);
const live = shadowRegistrations.get(item.projectIdentity);
if (!live || live.generation !== registration.generation) {
    return { writes: 0, refusalReason: "registration_retired_during_embed" };
}

generation is bumped by globalRegistrationGeneration on every new shadow registration, so this covers a replaced registration (different model/fingerprint re-armed mid-flight), not just a deleted one.

(b) Record the worker outcome only while the registration the item started under is still live.

Also adds registration_retired_during_embed to ShadowBackfillWriteRefusalReason plus its describeShadowBackfillWriteRefusal case, so the refusal surfaces through the existing stall reporting instead of the generic default text.

Test

discards shadow vectors when the registration is retired while the provider call is in flight retires the shadow from inside embedBatch — precisely inside the await window — then asserts loadAllEmbeddings(db, projectIdentity, modelId).size === 0.

Verified both directions:

• pre-fix: Expected: 0 / Received: 3 — the defect is real and the test bites.
• post-fix: pass.

tsc --noEmit clean; full file 41 pass / 0 fail; biome clean on the changed content.

Coverage note on P2 (b)

No test ships for the second half. A re-arm regression was written and then removed because it passed with and without the guard: in that path the replacement registration publishes its own outcome under the same key, so the stale publish is not the value the detector reads. Rather than ship a test that cannot fail, (b) is left as defense-in-depth for the interleaving that is not reachable through the test seam. Flagging that explicitly rather than implying coverage the suite does not have.

Confidence Score: 5/5

The PR appears safe to merge; the stale-vector race is guarded across all three scopes, and the previously identified outcome-state and coverage gaps are resolved.

Summary

This PR prevents asynchronous shadow embedding work from persisting vectors or worker outcomes after its registration has been retired or replaced.

• Re-checks the live registration generation after provider calls in memory, commit, and chunk scopes.
• Suppresses stale worker outcomes when the originating registration is no longer current.
• Adds a specific refusal reason and user-facing description for retirement during embedding.
• Adds regression coverage for all three embedding scopes, including positive controls for chunk candidate creation.

Diagram

sequenceDiagram
    participant W as Shadow worker
    participant R as Registration registry
    participant P as Embedding provider
    participant DB as Vector storage

    W->>R: Capture registration and generation
    W->>P: Embed candidate items
    R-->>R: Registration retired or replaced
    P-->>W: Return vectors
    W->>R: Re-read live generation
    alt Registration is still current
        W->>DB: Persist vectors
        W->>W: Publish worker outcome
    else Registration is retired or replaced
        W-->>W: Discard vectors
        W-->>W: Suppress stale outcome
    end

Loading

_{Reviews (4) · Last reviewed commit: "test(plugin): drop the dead second facto..."}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.