[gmaclennan]

Today there is no way to ask a cloud server which of a project's attachments it can actually serve. A photo whose original is only partly synced looks fine in the observation list and only fails when a partner's connector tries to download it, and the only way to find all such photos is to download every one by hand.

This adds an authenticated GET /projects/:projectPublicId/attachments/audit. src/attachments-audit.js lists the project's observations newest first, fetches every variant of every photo and audio attachment from core's blob server, reads each to the end and counts ok / missing / incomplete / error, where missing is an upstream 404, incomplete a reset or truncation and error anything else. Each failure names the attachment and the device that created the observation (createdBy), since that is the phone that has to sync again, and the result carries a devices map from those device ids to the names the project knows them by. The number of attachments checked per call is capped by AUDIT_MAX_ATTACHMENTS (default 200), because the audit reads every variant in full.

Scope and limits. This is the endpoint only; nothing runs on a timer and nothing is reported anywhere. The 200 default is a guess at what a small VPS can read in one request, and the upstreamFailure and fetchFailure classifiers in the new module are exported so the attachments route can reuse them in the next PR. Once core exposes a blob status API the audit can stop downloading bytes.

Related changes. This replaces the audit endpoint part of #79, which is being split up after review. It stacks on #76 (fix/attachments-proxy-errors), whose test helper for a partly synced photo it moves into test/test-helpers.js for reuse. Sentry reporting of attachments that could not be served follows in #89 on top of this one, and #90 (draft) runs this audit after each sync.

[gmaclennan]

I wouldn't really like to keep this code in here, but as a temporary measure it might be useful for @rudokemper for diagnosing the issue and identifying which devices need to sync to fix it. Is the 200 attachment limit too tight? Is it actually fine to not limit it?

[rudokemper]

I do think it could be useful, although in my (limited) experience trying to resolve this a few times, geography has proven more useful than deviceId, as people can recognize the locations where there is missing data and quickly isolate who the data collector was. So that is why on Guardian Connector, we create a "missing attachments" GeoJSON that people can easily examine in QGIS. But there are also comapeo-cloud deployments not on Guardian Connector, so having an audit endpoint to resolve extant issues makes good sense.

For our use case, I don't think we need a limit, and it's probably preferable not to have one.