[gmaclennan]

Today the only sign that an attachment is partly synced to the cloud is a partner's connector failing downstream: the server logs to stdout and nothing else, so a 404, a 500 with UNKNOWN_ERROR or a 502 from the attachments route goes unnoticed until someone complains.

This adds optional Sentry reporting, on only when SENTRY_DSN is set (@sentry/node is imported on demand, so a deployment without a DSN does not pay for it). The attachments route in src/routes.js now sends one attachment.nonok event for every response it could not serve as a 200, tagged with the project, the first seven characters of the drive id, type, variant, the HTTP status, a coarse class (missing for an upstream 404, incomplete when core resets or truncates, error otherwise) and core's own error code where it sent one; the fetch cause (UND_ERR_SOCKET and the like) goes in the event detail and the pino line. The server also runs an integrity audit: src/attachments-audit.js lists a project's observations, fetches every variant of every photo and audio attachment from core's blob server, reads each to the end and counts ok / missing / incomplete / error. It is exposed as an authenticated GET /projects/:projectPublicId/attachments/audit and, when Sentry is on, runs once shortly after start and then every AUDIT_INTERVAL_MS (default one hour, 0 disables), capped at AUDIT_MAX_ATTACHMENTS newest attachments per project, sending one attachment.audit event per project with the counts in the event context. Each failure names the device that created the observation (createdBy), since that is the phone that has to sync again, and the result carries a devices map from those device ids to the names the project knows them by; both go to Sentry with the counts. The capture function is injected through the server options, so tests run without a DSN. Sentry's unhandled-rejection integration is set to strict so a DSN does not stop the process crashing the way it does today.

Scope and limits. Sentry is initialised after fastify and core are imported and its ESM loader hook is turned off, so there is no tracing or auto-instrumentation: only crashes, unhandled rejections and the two attachment events above are reported, and a 5xx that fastify turns into a response on another route is not. The dependency is @sentry/node 10.x rather than 11.x because 11 requires node 20.19 and the Dockerfile pins 20.17.0. And the audit reads every variant in full, so the default cap of 200 attachments is a guess at what a small VPS can comfortably read once an hour; once core exposes a blob status API the audit can stop downloading bytes.

Related changes. After digidem/comapeo-core#1350, a partial blob reaches this route as an upstream 404 with code BLOB_NOT_AVAILABLE_ERROR rather than a reset or truncation. The event already carries core's error code, and the audit's missing versus incomplete split should then key on that code rather than on how the body failed. The hourly audit stays useful either way, since it is the only server-side signal that media is missing.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@​sentry/​node@​10.76.2

View full report

[gmaclennan]

Let's separate this into 2 PRs: one for the audit endpoint, and the other for reporting missing attachments to Sentry. I'm not sure about the automated audit being sent to Sentry: servers can sit without sync for several days, so this seems like a lot of unnecessary noise and processing. Maybe open a 3rd draft PR for sentry audits, but perhaps rather than hourly, send after a sync, sent at most once per hour, and only if there is something missing.

[gmaclennan]

Split as asked, stacked in order:

• feat(attachments): add an attachment audit endpoint #88 the audit endpoint (on fix(attachments): return 502 when the blob server resets the connection #76). Each failure now also names the device that created the observation, with a map of device ids to names, from the comment thread here.
• feat(attachments): report attachments that could not be served to Sentry #89 Sentry reporting of attachments the route could not serve (on feat(attachments): add an attachment audit endpoint #88, because the route reuses the audit's failure classification).
• feat(attachments): audit attachments after each sync and report missing ones to Sentry #90, draft, the audit after a sync instead of the hourly timer: it watches each project's sync state, treats a sync as ended when a device that had data sync enabled disconnects or the cloud stops wanting data blocks, waits five seconds for stragglers, runs at most once per hour per project (a sync ending sooner is scheduled for the earliest allowed time, not dropped), and only sends to Sentry when something is missing. The counts are logged every run.

Three things I had to decide in #90 that you may want to change: it only runs when SENTRY_DSN is set, as the timer did; nothing runs at start, so a server restarted mid-sync waits for the next sync to end; and the five second debounce is a constructor option rather than an env var.

This PR can be closed once you are happy with the three.