diff --git a/README.md b/README.md index 679b679..8bcdf41 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ [![CI](https://github.com/digidem/comapeo-cloud-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/digidem/comapeo-cloud-cli/actions/workflows/ci.yml) -A command-line interface for interacting with the Comapeo Cloud API. +A command-line interface for interacting with the CoMapeo Cloud API. ## Installation @@ -12,151 +12,250 @@ npm install -g comapeo-cloud-cli ## Configuration -The CLI requires the following environment variables: +The CLI reads two environment variables: -- `SERVER_URL`: The URL of your Comapeo Cloud server -- `SERVER_BEARER_TOKEN`: Your API bearer token +- `SERVER_URL`: the URL of your CoMapeo Cloud server +- `SERVER_BEARER_TOKEN`: your API bearer token -You can set these in your environment or create a `.env` file. For temporary overrides, use these command-line options with any command: +Set them in your environment or in a `.env` file in the current directory. +`COMAPEO_SERVER_URL` / `COMAPEO_SERVER_TOKEN` are accepted as aliases when the +un-prefixed names would collide with other tooling. -- `-s, --server-url ` - Override server URL -- `-t, --server-token ` - Override bearer token +Every command also accepts per-invocation overrides: + +- `-s, --server-url ` +- `-t, --server-token ` ```env -SERVER_URL=https://yourserver.com +SERVER_URL=https://yourserver.com/ SERVER_BEARER_TOKEN=your-token ``` -## Usage +Optional: set `COMAPEO_PROJECT` to a project id or name to stop passing `-p` to +every command. -### Server Information +## Output -Get server info: +List commands choose their format automatically: -```bash -comapeo-cloud -s https://yourserver.com -t your-token info -``` +| stdout | default format | +| -------------- | -------------- | +| a terminal | `table` | +| a pipe or file | `json` | + +So `comapeo-cloud list-observations -p ` is readable by a human, and +`comapeo-cloud list-observations -p \| jq '.[].docId'` keeps working in a +script with no extra flags. Override with `--format`: -### Projects +- `table` — aligned columns, truncated long values +- `json` — the full, untouched API records (default when piped) +- `ndjson` — one JSON record per line, the friendliest target for `jq` +- `csv` — every column, RFC 4180 quoting -List all projects: +Other global flags: `--columns ` (per command), `--timeout `, +`--no-color`, `-v/--verbose`, `-V/--version`. + +## Projects + +Project ids are 52-character strings that are painful to copy around, so `-p` +also accepts a **project name**: ```bash -comapeo-cloud -s https://yourserver.com -t your-token list-projects +comapeo-cloud list-projects # id + name table +comapeo-cloud list-projects --count # 10 +comapeo-cloud list-projects --search vigilancia # name substring + +comapeo-cloud list-observations -p "Expedição 2026" # resolves to the id ``` -### Observations +Exact (case-insensitive) names win; otherwise a substring must identify exactly +one project. An ambiguous name lists the candidates and exits `2` rather than +guessing. -List observations for a project: +## Observations ```bash -comapeo-cloud -s https://yourserver.com -t your-token list-observations -p "proj_abc123" +comapeo-cloud list-observations -p ``` -### Attachments +### Counting -Get an attachment: +`--count` prints a bare number — no labels, no colour — so it composes: ```bash -comapeo-cloud -s https://yourserver.com -t your-token get-attachment \ - -p "proj_abc123" \ - --drive-id "drive_123" \ - --type "photo" \ - --name "filename.jpg" \ - --variant "preview" +comapeo-cloud list-observations -p --count # 56 +comapeo-cloud list-observations -p --count --category agua # 17 +comapeo-cloud list-observations -p --count --has audio # 21 ``` -For audio files: +Break the number down with `--group-by` (tab-separated `keycount`): ```bash -comapeo-cloud -s https://yourserver.com -t your-token get-attachment \ - --project-id "proj_abc123" \ - --drive-id "drive_123" \ - --type "audio" \ - --name "recording.mp3" +comapeo-cloud list-observations -p --count --group-by category +comapeo-cloud list-observations -p --count --group-by day +comapeo-cloud list-observations -p --count --group-by tag:tipo-de-corpo-dagua ``` -### Remote Alerts +Valid groups: `category`, `preset`, `day`, `device`, `media`, `tag:`. -Create a remote detection alert: +> Presets are grouped by docId prefix (`preset:abb1b841`), not by name: the +> deployed servers do not expose a `/presets` route, so there is no preset name +> to resolve against. + +`--summary` prints a block instead of records: ```bash -comapeo-cloud -s https://yourserver.com -t your-token create-alert \ - --project-id "proj_abc123" \ - --start-date "2024-01-01T00:00:00Z" \ - --end-date "2024-01-02T00:00:00Z" \ - --source-id "src_123" \ - --alert-type "motion-detected" \ - --lon -67.890 \ - --lat 12.345 +$ comapeo-cloud list-observations -p --summary +matching 56 +date range 2026-10-26 → 2026-10-29 +devices 3 +media 112 photos, 21 audio +no attachments 2 +categories 9 ``` -List remote detection alerts for a project: +### Filtering + +All filtering happens client-side: the API's `/projects/:id/observations` +endpoint takes no query parameters and always returns every document. That is +cheap enough for field projects (well under a megabyte of JSON) and means every +filter below can be combined freely. + +| Flag | Meaning | +| ----------------------------------- | -------------------------------------------------------------------------- | +| `--category ` | match `tags.categoryId` | +| `--tag ` | match any tag; repeatable; `v1\|v2` for OR; arrays matched | +| `--preset ` | match a preset docId prefix | +| `--since ` / `--until ` | `createdAt` bounds; `YYYY-MM-DD` includes the whole day | +| `--bbox ` | observations inside a lon/lat box | +| `--has ` | has `photo`, `audio`, `video`, `attachment`, `location`, `notes`, `preset` | +| `--missing ` | inverse of `--has` | +| `--search ` | substring over notes and tag values | +| `--include-deleted` | include tombstones (excluded by default) | +| `--only-deleted` | show only tombstones | +| `--sort ` | `createdAt` (default), `updatedAt`, `category`, `notes`, `media`, `id` | +| `--reverse` | reverse the sort | +| `--limit ` / `--offset ` | paginate the printed page | +| `--columns ` | pick table/csv columns | + +**Deleted documents are excluded by default.** The API returns tombstones +mixed in with live records (`includeDeleted: true` server-side), and counting +them made "how many observations do we have?" wrong. `--count` now reports +live observations; use `--include-deleted` to see everything. + +Examples: ```bash -comapeo-cloud -s https://yourserver.com -t your-token list-alerts -``` +# everything tagged as a river, with audio, this week +comapeo-cloud list-observations -p \ + --tag tipo-de-corpo-dagua=rio --has audio --since 2026-10-26 -### Data Export +# what still has no media attached? +comapeo-cloud list-observations -p --missing attachment --format csv -Export data from a project as a zip containing attachments and geojoson file: - -```bash -comapeo-cloud -s https://yourserver.com -t your-token export-geojson \ - -p "proj_abc123" \ - -o "export.zip" +# readable table, newest first +comapeo-cloud list-observations -p --reverse --limit 20 --format table ``` -### Health Check - -Check server health: +## Attachments ```bash -comapeo-cloud healthcheck +comapeo-cloud get-attachment \ + -u "https://server/projects//attachments//photo/" \ + --variant thumbnail \ + -o photo.jpg ``` -## Development +Photo variants: `original`, `preview`, `thumbnail`. Audio only has `original`. +Pass `-o -` to stream the bytes to stdout. -1. Clone the repository -2. Install dependencies: +## Remote Alerts ```bash -bun install +comapeo-cloud list-alerts -p --count +comapeo-cloud list-alerts -p --since 2026-10-01 --alert-type motion-detected + +comapeo-cloud create-alert -p \ + --start-date 2024-01-01T00:00:00Z \ + --end-date 2024-01-02T00:00:00Z \ + --source-id src_123 \ + --alert-type motion-detected \ + --lon -67.890 --lat 12.345 ``` -3. Build the project: +## Data Export ```bash -bun run build +# ZIP: GeoJSON + media +comapeo-cloud export-geojson -p -o export.zip + +# plain GeoJSON, no downloads +comapeo-cloud export-geojson -p -o export.geojson --no-attachments + +# export just a filtered slice +comapeo-cloud export-geojson -p --category agua --since 2026-10-29 -o water.zip ``` -4. Run tests: +The output extension decides the behaviour: `.zip` bundles `comapeo_data.geojson` +plus `images/` and `audio/` folders; `.geojson` writes the GeoJSON alone and +never downloads media. `--no-attachments` forces the metadata-only path. + +`export-geojson` accepts the same filter flags as `list-observations`, so +whatever you can list you can also export. + +Properties in the exported GeoJSON are the observation's tags (`categoryId`, +`notes`, and any preset-specific field) plus `$created`, `$modified`, `$version`, +`$photos` and, when present, `$audio`. + +## Health Check ```bash -bun test +comapeo-cloud healthcheck # prints "ok" and exits 0 +comapeo-cloud info ``` -## Configuration Overrides +## Exit Codes + +| Code | Meaning | +| ---- | -------------------------------------------------------------- | +| `0` | success | +| `1` | API or runtime failure | +| `2` | usage error (bad flag, missing config, ambiguous project name) | +| `3` | not found (unknown project, no matching records) | -All commands support temporary server configuration via CLI flags: +## Development ```bash -comapeo-cloud -s https://alt.server.com -t alt-token [command] +bun install +bun run build # tsc -> dist/ +bun test # unit + CLI integration tests +bun run lint # prettier ``` -## Publishing +`bun test` is what CI runs; `npm test` is wired to the same command because some +dependencies (`chalk` 5) are ESM-only and do not load under jest's CJS runtime. + +### Layout -The package is automatically published to npm when a new release is created on GitHub. To create a new release: +``` +src/index.ts entry point +src/cli.ts program wiring and global options +src/commands/ one module per command group +src/lib/api.ts config, axios client, project resolution, error mapping +src/lib/filters.ts pure filter predicates (no I/O) +src/lib/group.ts grouping and summary maths +src/lib/format.ts table / json / ndjson / csv renderers +src/lib/columns.ts column definitions per data type +src/__tests__/ unit tests + a synthetic fixture (no real field data) +``` -1. Update the version in package.json -2. Create and push a new tag -3. Create a new release on GitHub +## Publishing -The GitHub Actions workflow will automatically: +The package is published to npm when a release is created on GitHub: -- Run tests -- Build the project -- Publish to npm +1. Update `version` in `package.json` +2. Create and push a tag +3. Create a GitHub release ## License diff --git a/package.json b/package.json index 14a635a..23d0474 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "comapeo-cloud-cli", - "version": "1.0.3", + "version": "1.1.0", "description": "CLI tool for interacting with Comapeo Cloud API", "type": "module", "main": "dist/index.js", @@ -9,7 +9,7 @@ }, "scripts": { "build": "tsc", - "test": "jest", + "test": "bun test", "prepare": "npm run build", "start": "node dist/index.js", "lint": "prettier --write ." diff --git a/src/__tests__/api.test.ts b/src/__tests__/api.test.ts new file mode 100644 index 0000000..6935668 --- /dev/null +++ b/src/__tests__/api.test.ts @@ -0,0 +1,169 @@ +import { AxiosError } from "axios"; +import { + describeError, + isProjectId, + matchProjects, + resolveServerConfig, +} from "../lib/api.js"; +import { CliError, EXIT } from "../lib/errors.js"; +import type { Project } from "../lib/types.js"; + +const REAL_ID = "ssgqifbqs8c14hpx8djuss4fkqoxkrga6xiqkpw5cscqqpqrjfko"; + +const projects: Project[] = [ + { projectId: "p1", name: "Vigilância wayamu" }, + { projectId: "p2", name: "Arumatari" }, + { projectId: "p3", name: "Arumatari" }, + { projectId: "p4", name: "Rio mapuera roowo kurunpenê komo" }, +]; + +describe("isProjectId", () => { + it("accepts a real 52-char id containing 1, 8 and 9", () => { + expect(isProjectId(REAL_ID)).toBe(true); + }); + + it("rejects names and truncated ids", () => { + expect(isProjectId("Expedição 2026")).toBe(false); + expect(isProjectId(REAL_ID.slice(0, 20))).toBe(false); + expect(isProjectId(`${REAL_ID} `)).toBe(true); // trailing space trimmed + }); +}); + +describe("matchProjects", () => { + it("prefers an exact case-insensitive name", () => { + const { matches, exact } = matchProjects(projects, "vigilância wayamu"); + expect(exact).toBe(true); + expect(matches.map((p) => p.projectId)).toEqual(["p1"]); + }); + + it("falls back to a substring match", () => { + const { matches, exact } = matchProjects(projects, "mapuera"); + expect(exact).toBe(false); + expect(matches.map((p) => p.projectId)).toEqual(["p4"]); + }); + + it("returns every duplicate so the caller can report ambiguity", () => { + expect(matchProjects(projects, "Arumatari").matches).toHaveLength(2); + }); + + it("returns nothing for an unknown name", () => { + expect(matchProjects(projects, "nope").matches).toHaveLength(0); + }); +}); + +describe("resolveServerConfig", () => { + const env = { ...process.env }; + + afterEach(() => { + process.env = { ...env }; + }); + + it("reads the historical env var names", () => { + process.env.SERVER_URL = "https://example.test/"; + process.env.SERVER_BEARER_TOKEN = "token"; + delete process.env.COMAPEO_SERVER_URL; + delete process.env.COMAPEO_SERVER_TOKEN; + expect(resolveServerConfig({})).toEqual({ + serverUrl: "https://example.test", + serverToken: "token", + timeoutMs: 30_000, + }); + }); + + it("accepts the COMAPEO_-prefixed aliases", () => { + delete process.env.SERVER_URL; + delete process.env.SERVER_BEARER_TOKEN; + process.env.COMAPEO_SERVER_URL = "https://alias.test"; + process.env.COMAPEO_SERVER_TOKEN = "alias-token"; + expect(resolveServerConfig({}).serverToken).toBe("alias-token"); + }); + + it("prefers flags over environment", () => { + process.env.SERVER_URL = "https://env.test"; + process.env.SERVER_BEARER_TOKEN = "env-token"; + expect( + resolveServerConfig({ + serverUrl: "https://flag.test", + serverToken: "flag-token", + }), + ).toMatchObject({ + serverUrl: "https://flag.test", + serverToken: "flag-token", + }); + }); + + it("strips trailing slashes from the URL", () => { + expect( + resolveServerConfig({ + serverUrl: "https://x.test///", + serverToken: "t", + }).serverUrl, + ).toBe("https://x.test"); + }); + + it("fails with a usage error naming the env var", () => { + delete process.env.SERVER_URL; + delete process.env.COMAPEO_SERVER_URL; + expect(() => resolveServerConfig({ serverToken: "t" })).toThrow( + /SERVER_URL must be provided/, + ); + }); + + it("validates the timeout", () => { + expect(() => + resolveServerConfig({ + serverUrl: "https://x.test", + serverToken: "t", + timeout: "soon", + }), + ).toThrow(/--timeout/); + }); +}); + +describe("describeError", () => { + it("passes CliError through untouched", () => { + const original = new CliError("boom", EXIT.USAGE); + expect(describeError(original)).toBe(original); + }); + + it("explains a 401 as an auth problem", () => { + const error = new AxiosError( + "Request failed", + "ERR_BAD_REQUEST", + undefined, + undefined, + { + status: 401, + data: { message: "Unauthorized" }, + statusText: "Unauthorized", + headers: {}, + config: {} as never, + }, + ); + expect(describeError(error).message).toMatch(/Authentication failed/); + }); + + it("explains a 404 with the path and a next step", () => { + const error = new AxiosError( + "Request failed", + "ERR_BAD_REQUEST", + { url: "/projects/x/observation" } as never, + undefined, + { + status: 404, + data: { message: "Route not found" }, + statusText: "Not Found", + headers: {}, + config: {} as never, + }, + ); + const described = describeError(error); + expect(described.message).toMatch(/Not found \(HTTP 404\)/); + expect(described.message).toMatch(/list-projects/); + expect(described.exitCode).toBe(EXIT.NOT_FOUND); + }); + + it("wraps a plain Error", () => { + expect(describeError(new Error("nope")).message).toBe("nope"); + }); +}); diff --git a/src/__tests__/cli.test.ts b/src/__tests__/cli.test.ts index 4ed9138..cfc9de4 100644 --- a/src/__tests__/cli.test.ts +++ b/src/__tests__/cli.test.ts @@ -1,13 +1,24 @@ import { spawn } from "node:child_process"; +import { readFileSync } from "node:fs"; import { join } from "node:path"; const CLI_PATH = join(process.cwd(), "dist", "index.js"); +const PACKAGE_VERSION = ( + JSON.parse(readFileSync(join(process.cwd(), "package.json"), "utf8")) as { + version: string; + } +).version; + +interface RunResult { + code: number | null; + output: string; +} describe("CLI Commands", () => { const runCommand = ( args: string[] = [], includeEnv = false, - ): Promise<{ code: number | null; output: string }> => { + ): Promise => { return new Promise((resolve) => { const env = { ...process.env, @@ -47,15 +58,86 @@ describe("CLI Commands", () => { expect(output).toContain("Options:"); }); - it("should show version number", async () => { + it("should show the version from package.json", async () => { const { output } = await runCommand(["--version"], true); expect(output).toMatch(/\d+\.\d+\.\d+/); + expect(output.trim()).toBe(PACKAGE_VERSION); }); - it("should require SERVER_URL and SERVER_BEARER_TOKEN for API commands", async () => { - const { output } = await runCommand(["list-projects"]); + it("should require SERVER_URL for API commands", async () => { + const { output, code } = await runCommand(["list-projects"]); expect(output).toContain( - "Error: SERVER_URL must be provided either as environment variable or --server-url option", + "SERVER_URL must be provided as an environment variable or with --server-url", + ); + expect(code).toBe(2); + }); + + it("should advertise the filtering and counting flags", async () => { + const { output } = await runCommand(["list-observations", "--help"], true); + for (const flag of [ + "--count", + "--group-by", + "--summary", + "--category", + "--tag", + "--since", + "--until", + "--bbox", + "--has", + "--missing", + "--include-deleted", + "--columns", + ]) { + expect(output).toContain(flag); + } + }); + + it("should reject an unknown output format", async () => { + const { output, code } = await runCommand( + ["list-projects", "--format", "yaml"], + true, + ); + expect(output).toContain('Unknown --format "yaml"'); + expect(code).toBe(2); + }); + + it("should require a project for list-observations", async () => { + const { output, code } = await runCommand(["list-observations"], true); + expect(output).toContain("A project is required"); + expect(code).toBe(2); + }); + + it("should reject an unknown --group-by value before any request", async () => { + const { output, code } = await runCommand( + [ + "list-observations", + "--project-id", + "x", + "--group-by", + "colour", + "--count", + ], + true, + ); + expect(output).toContain('Unknown --group-by "colour"'); + expect(code).toBe(2); + }); + + it("should reject an unknown column", async () => { + const { output, code } = await runCommand( + ["list-observations", "--project-id", "x", "--columns", "nope"], + true, + ); + expect(output).toContain('Unknown column "nope"'); + expect(code).toBe(2); + }); + + it("should reject a non-numeric --limit", async () => { + const { output, code } = await runCommand( + ["list-observations", "--project-id", "x", "--limit", "ten"], + true, ); + expect(output).toContain("--limit must be a non-negative integer"); + expect(code).toBe(2); }); }); diff --git a/src/__tests__/filters.test.ts b/src/__tests__/filters.test.ts new file mode 100644 index 0000000..fddf6f8 --- /dev/null +++ b/src/__tests__/filters.test.ts @@ -0,0 +1,225 @@ +import { + countMedia, + filterObservations, + mediaKind, + parseBbox, + parseDateBound, + parseTagExpression, + sortObservations, +} from "../lib/filters.js"; +import { observations } from "./fixtures/observations.js"; + +const ids = (list: { docId: string }[]): string[] => + list.map((observation) => observation.docId.slice(0, 2)); + +describe("media helpers", () => { + it("derives the media kind from the attachment URL", () => { + expect(mediaKind("https://x/projects/p/attachments/d/photo/abc")).toBe( + "photo", + ); + expect(mediaKind("https://x/projects/p/attachments/d/audio/abc")).toBe( + "audio", + ); + expect(mediaKind("https://x/projects/p/attachments/d/other/abc")).toBe( + "unknown", + ); + expect(mediaKind(undefined)).toBe("unknown"); + }); + + it("counts photos and audio separately", () => { + expect(countMedia(observations[1])).toEqual({ + photos: 1, + audio: 1, + video: 0, + total: 2, + }); + expect(countMedia(observations[3]).total).toBe(0); + }); +}); + +describe("parseDateBound", () => { + it("anchors a date-only bound to the start of the day", () => { + expect(parseDateBound("2026-01-02", "start")).toBe( + Date.parse("2026-01-02T00:00:00.000Z"), + ); + }); + + it("anchors an end bound to the end of the day, so the day is included", () => { + expect(parseDateBound("2026-01-02", "end")).toBe( + Date.parse("2026-01-02T23:59:59.999Z"), + ); + }); + + it("passes full timestamps through untouched", () => { + expect(parseDateBound("2026-01-02T10:00:00Z", "end")).toBe( + Date.parse("2026-01-02T10:00:00Z"), + ); + }); + + it("rejects garbage", () => { + expect(() => parseDateBound("last tuesday", "start")).toThrow( + /Cannot parse date/, + ); + }); +}); + +describe("parseBbox / parseTagExpression", () => { + it("parses west,south,east,north", () => { + expect(parseBbox("-57.4,-1.5,-57.1,-1.0")).toEqual({ + west: -57.4, + south: -1.5, + east: -57.1, + north: -1.0, + }); + }); + + it("rejects a bbox with the wrong number of parts", () => { + expect(() => parseBbox("-57.4,-1.5")).toThrow(/west,south,east,north/); + }); + + it("parses key=value and OR lists", () => { + expect(parseTagExpression("tipo=rio|igarape")).toEqual({ + key: "tipo", + values: ["rio", "igarape"], + }); + }); + + it("rejects a tag expression without =", () => { + expect(() => parseTagExpression("rio")).toThrow(/key=value/); + }); +}); + +describe("filterObservations", () => { + it("excludes deleted documents by default", () => { + expect(filterObservations(observations, {})).toHaveLength(5); + }); + + it("includes deleted documents on request", () => { + expect( + filterObservations(observations, { includeDeleted: true }), + ).toHaveLength(6); + }); + + it("can show only deleted documents", () => { + expect( + ids(filterObservations(observations, { onlyDeleted: true })), + ).toEqual(["1e"]); + }); + + it("filters by category, comma separated or repeated", () => { + expect( + filterObservations(observations, { category: ["agua"] }), + ).toHaveLength(2); + expect( + filterObservations(observations, { + category: ["agua", "coleta"], + }), + ).toHaveLength(3); + expect( + filterObservations(observations, { category: ["agua,coleta"] }), + ).toHaveLength(3); + }); + + it("matches scalar and array-valued tags, including OR lists", () => { + expect( + filterObservations(observations, { tag: ["tipo-de-corpo-dagua=rio"] }), + ).toHaveLength(1); + expect( + filterObservations(observations, { + tag: ["tipo-de-corpo-dagua=rio|igarape"], + }), + ).toHaveLength(2); + expect( + filterObservations(observations, { + tag: ["condicao-de-placas-e-limites=boa-condicao"], + }), + ).toHaveLength(1); + }); + + it("requires every --tag expression to match", () => { + expect( + filterObservations(observations, { + tag: ["tipo-de-corpo-dagua=rio", "categoryId=coleta"], + }), + ).toHaveLength(0); + }); + + it("filters by media presence and absence", () => { + expect(filterObservations(observations, { has: ["audio"] })).toHaveLength( + 1, + ); + expect( + filterObservations(observations, { missing: ["attachment"] }), + ).toHaveLength(2); + }); + + it("filters by location presence", () => { + expect( + filterObservations(observations, { missing: ["location"] }), + ).toHaveLength(1); + }); + + it("filters by inclusive date range", () => { + expect( + filterObservations(observations, { + since: "2026-01-02", + until: "2026-01-03", + }), + ).toHaveLength(2); + }); + + it("filters by bounding box", () => { + expect( + filterObservations(observations, { bbox: "-57.25,-1.35,-57.05,-1.05" }), + ).toHaveLength(2); + }); + + it("searches notes and tag values", () => { + expect( + filterObservations(observations, { search: "igarapé" }), + ).toHaveLength(1); + expect(filterObservations(observations, { search: "placa" })).toHaveLength( + 1, + ); + }); + + it("filters by preset docId prefix", () => { + expect( + filterObservations(observations, { preset: "presetA" }), + ).toHaveLength(2); + }); + + it("rejects an unknown --has token before touching the data", () => { + expect(() => + filterObservations(observations, { has: ["hologram"] }), + ).toThrow(/Unknown --has\/--missing value/); + }); + + it("combines filters conjunctively", () => { + expect( + filterObservations(observations, { + category: ["agua"], + has: ["audio"], + }), + ).toHaveLength(1); + }); +}); + +describe("sortObservations", () => { + it("sorts by createdAt ascending by default", () => { + expect( + ids(sortObservations(observations, "createdAt")).slice(0, 2), + ).toEqual(["1a", "1b"]); + }); + + it("reverses on request", () => { + expect( + ids(sortObservations(observations, "createdAt", true)).slice(0, 2), + ).toEqual(["1f", "1e"]); + }); + + it("sorts media-rich observations last when sorting by media", () => { + const sorted = sortObservations(observations, "media"); + expect(countMedia(sorted[sorted.length - 1]).total).toBe(2); + }); +}); diff --git a/src/__tests__/fixtures/observations.ts b/src/__tests__/fixtures/observations.ts new file mode 100644 index 0000000..faa8c8e --- /dev/null +++ b/src/__tests__/fixtures/observations.ts @@ -0,0 +1,129 @@ +import type { Observation } from "../../lib/types.js"; + +/** + * Synthetic fixture with the same shape as a live `/observations` payload. + * + * Deliberately not real field data: the projects this CLI was built against + * hold observations from indigenous territories, which should not be + * committed to a public repository. Numbers asserted in the tests are + * derived from this fixture only. + */ +const docId = (seed: string): string => seed.padEnd(64, "0").slice(0, 64); + +const base = { + schemaName: "observation", + versionId: "aa/1", + originalVersionId: "aa/1", + links: [], + metadata: {}, +}; + +export const observations: Observation[] = [ + { + ...base, + docId: docId("1a"), + createdAt: "2026-01-01T10:00:00.000Z", + updatedAt: "2026-01-01T10:00:00.000Z", + deleted: false, + lat: -1.1, + lon: -57.1, + attachments: [ + { + url: "https://example.test/projects/p1/attachments/drive1/photo/aaa", + }, + ], + tags: { + categoryId: "agua", + notes: "Rio principal", + "tipo-de-corpo-dagua": "rio", + }, + presetRef: { docId: docId("presetA"), versionId: "p/1", url: "u" }, + createdBy: "deviceA", + }, + { + ...base, + docId: docId("1b"), + createdAt: "2026-01-02T11:00:00.000Z", + updatedAt: "2026-01-02T11:00:00.000Z", + deleted: false, + lat: -1.2, + lon: -57.2, + attachments: [ + { + url: "https://example.test/projects/p1/attachments/drive1/photo/bbb", + }, + { + url: "https://example.test/projects/p1/attachments/drive1/audio/ccc", + }, + ], + tags: { + categoryId: "agua", + notes: "Igarapé", + "tipo-de-corpo-dagua": "igarape", + }, + presetRef: { docId: docId("presetA"), versionId: "p/1", url: "u" }, + createdBy: "deviceA", + }, + { + ...base, + docId: docId("1c"), + createdAt: "2026-01-03T12:00:00.000Z", + updatedAt: "2026-01-03T12:00:00.000Z", + deleted: false, + lat: -1.3, + lon: -57.3, + attachments: [ + { + url: "https://example.test/projects/p1/attachments/drive1/photo/ddd", + }, + { + url: "https://example.test/projects/p1/attachments/drive1/photo/eee", + }, + ], + tags: { + categoryId: "placas-e-limites", + notes: "Placa 01", + "tipo-de-placas-e-limites": "placa", + "condicao-de-placas-e-limites": ["boa-condicao"], + }, + createdBy: "deviceB", + }, + { + ...base, + docId: docId("1d"), + createdAt: "2026-01-04T13:00:00.000Z", + updatedAt: "2026-01-04T13:00:00.000Z", + deleted: false, + attachments: [], + tags: { categoryId: "aldeia-antiga", notes: "Aldeia antiga" }, + createdBy: "deviceB", + }, + { + ...base, + docId: docId("1e"), + createdAt: "2026-01-05T14:00:00.000Z", + updatedAt: "2026-01-05T14:00:00.000Z", + deleted: true, + lat: -1.5, + lon: -57.5, + attachments: [ + { + url: "https://example.test/projects/p1/attachments/drive1/photo/fff", + }, + ], + tags: { categoryId: "agua", "tipo-de-corpo-dagua": "rio" }, + createdBy: "deviceA", + }, + { + ...base, + docId: docId("1f"), + createdAt: "2026-01-06T15:00:00.000Z", + updatedAt: "2026-01-06T15:00:00.000Z", + deleted: false, + lat: -1.6, + lon: -57.6, + attachments: [], + tags: { categoryId: "coleta" }, + createdBy: "deviceA", + }, +]; diff --git a/src/__tests__/format.test.ts b/src/__tests__/format.test.ts new file mode 100644 index 0000000..6affd6c --- /dev/null +++ b/src/__tests__/format.test.ts @@ -0,0 +1,148 @@ +import { + renderCount, + renderCsv, + renderList, + renderTable, + renderSummary, + resolveFormat, + type Column, +} from "../lib/format.js"; + +const columns: Column[] = [ + { key: "id", header: "id" }, + { key: "notes", header: "notes", maxWidth: 6 }, +]; +const rows = [ + { id: "abc", notes: "short" }, + { id: "def", notes: "far too long to fit" }, +]; + +describe("resolveFormat", () => { + const originalIsTTY = process.stdout.isTTY; + + afterEach(() => { + Object.defineProperty(process.stdout, "isTTY", { + value: originalIsTTY, + configurable: true, + }); + }); + + it("defaults to json when stdout is not a TTY", () => { + Object.defineProperty(process.stdout, "isTTY", { + value: undefined, + configurable: true, + }); + expect(resolveFormat()).toBe("json"); + }); + + it("defaults to table on a TTY", () => { + Object.defineProperty(process.stdout, "isTTY", { + value: true, + configurable: true, + }); + expect(resolveFormat()).toBe("table"); + }); + + it("honours an explicit format regardless of TTY", () => { + Object.defineProperty(process.stdout, "isTTY", { + value: true, + configurable: true, + }); + expect(resolveFormat("csv")).toBe("csv"); + }); + + it("rejects an unknown format", () => { + expect(() => resolveFormat("yaml")).toThrow(/Unknown --format/); + }); +}); + +describe("renderTable", () => { + it("prints a header, a rule and one line per row", () => { + const output = renderTable(columns, rows); + const lines = output.split("\n"); + expect(lines).toHaveLength(4); + expect(lines[0]).toContain("id"); + expect(lines[1]).toContain("─"); + expect(lines[3]).toContain("def"); + }); + + it("truncates columns that declare a maxWidth", () => { + expect(renderTable(columns, rows)).toContain("far t…"); + }); + + it("says so when nothing matched", () => { + expect(renderTable(columns, [])).toContain("no matches"); + }); +}); + +describe("renderCsv", () => { + it("quotes values containing commas and quotes", () => { + const output = renderCsv( + [{ key: "notes", header: "notes" }], + [{ notes: 'a,b"c' }], + ); + expect(output).toBe('notes\n"a,b""c"'); + }); +}); + +describe("renderList", () => { + it("emits untouched records for json", () => { + const records = [{ docId: "x", extra: true }]; + expect(JSON.parse(renderList("json", columns, [], records))).toEqual( + records, + ); + }); + + it("emits one record per line for ndjson", () => { + const records = [{ docId: "x" }, { docId: "y" }]; + expect(renderList("ndjson", columns, [], records).split("\n")).toHaveLength( + 2, + ); + }); + + it("projects through columns for table and csv", () => { + expect(renderList("csv", columns, rows, rows)).toContain("abc"); + expect(renderList("table", columns, rows, rows)).toContain("abc"); + }); +}); + +describe("renderCount", () => { + it("prints a bare number when not grouped", () => { + expect(renderCount(56, undefined, "table")).toBe("56"); + }); + + it("prints tab separated grouped counts", () => { + expect( + renderCount( + 5, + [ + ["agua", 3], + ["coleta", 2], + ], + "table", + ), + ).toBe("agua\t3\ncoleta\t2"); + }); + + it("prints an object for json", () => { + expect(JSON.parse(renderCount(5, [["agua", 3]], "json"))).toEqual({ + agua: 3, + }); + }); + + it("prints csv for csv", () => { + expect(renderCount(5, [["agua", 3]], "csv")).toBe("value,count\nagua,3"); + }); +}); + +describe("renderSummary", () => { + it("aligns labels", () => { + const output = renderSummary([ + { label: "a", value: "1" }, + { label: "long label", value: "2" }, + ]); + const lines = output.split("\n"); + expect(lines).toHaveLength(2); + expect(lines[1]).toContain("long label"); + }); +}); diff --git a/src/__tests__/group.test.ts b/src/__tests__/group.test.ts new file mode 100644 index 0000000..562736d --- /dev/null +++ b/src/__tests__/group.test.ts @@ -0,0 +1,79 @@ +import { + assertGroupKey, + groupCounts, + groupValue, + summarize, +} from "../lib/group.js"; +import { observations } from "./fixtures/observations.js"; + +describe("groupCounts", () => { + it("counts by category, most frequent first", () => { + expect(groupCounts(observations, "category")).toEqual([ + ["agua", 3], + ["aldeia-antiga", 1], + ["coleta", 1], + ["placas-e-limites", 1], + ]); + }); + + it("counts by day", () => { + const counts = groupCounts(observations, "day"); + expect(counts).toHaveLength(6); + expect(counts[0]).toEqual(["2026-01-01", 1]); + }); + + it("counts by media combination", () => { + expect(groupCounts(observations, "media")).toEqual([ + ["photo", 3], + ["none", 2], + ["photo+audio", 1], + ]); + }); + + it("counts by arbitrary tag", () => { + expect(groupCounts(observations, "tag:tipo-de-corpo-dagua")).toEqual([ + ["(none)", 3], + ["rio", 2], + ["igarape", 1], + ]); + }); + + it("groups presets by docId prefix because preset names are unavailable", () => { + expect(groupValue(observations[0], "preset")).toBe("preset:presetA0"); + }); +}); + +describe("assertGroupKey", () => { + it("accepts the documented values", () => { + expect(assertGroupKey("category")).toBe("category"); + expect(assertGroupKey("tag:notes")).toBe("tag:notes"); + }); + + it("rejects anything else", () => { + expect(() => assertGroupKey("colour")).toThrow(/Unknown --group-by/); + expect(() => assertGroupKey("tag:")).toThrow(/Unknown --group-by/); + }); +}); + +describe("summarize", () => { + it("reports the same totals the assertions above rely on", () => { + const summary = summarize(observations); + expect(summary).toMatchObject({ + total: 6, + active: 5, + deleted: 1, + withoutLocation: 1, + withoutAttachments: 2, + photos: 5, + audio: 1, + devices: 2, + categories: 4, + }); + }); + + it("reports the created date range", () => { + const summary = summarize(observations); + expect(summary.firstCreatedAt).toBe("2026-01-01T10:00:00.000Z"); + expect(summary.lastCreatedAt).toBe("2026-01-06T15:00:00.000Z"); + }); +}); diff --git a/src/cli.ts b/src/cli.ts new file mode 100644 index 0000000..66cabe3 --- /dev/null +++ b/src/cli.ts @@ -0,0 +1,47 @@ +import chalk from "chalk"; +import { Command } from "commander"; +import { registerAlertCommands } from "./commands/alerts.js"; +import { registerGetAttachmentCommand } from "./commands/attachments.js"; +import { registerExportCommand } from "./commands/export.js"; +import { registerServerCommands } from "./commands/info.js"; +import { registerListObservationsCommand } from "./commands/observations.js"; +import { registerListProjectsCommand } from "./commands/projects.js"; +import { getVersion } from "./lib/version.js"; + +export function buildProgram(): Command { + const program = new Command(); + + program + .name("comapeo-cloud") + .description("CLI tool for interacting with the CoMapeo Cloud API") + .version(getVersion(), "-V, --version", "output the version number") + .option("-s, --server-url ", "Server URL (overrides SERVER_URL)") + .option( + "-t, --server-token ", + "Bearer token (overrides SERVER_BEARER_TOKEN)", + ) + .option( + "--format ", + "Output format: table|json|ndjson|csv (default: table on a TTY, json when piped)", + ) + .option("--timeout ", "HTTP timeout in milliseconds", "30000") + .option("--no-color", "Disable coloured output") + .option("-v, --verbose", "Verbose progress on stderr") + .showHelpAfterError(); + + program.hook("preAction", (_thisCommand, actionCommand) => { + const options = actionCommand.optsWithGlobals() as { color?: boolean }; + if (options.color === false) { + chalk.level = 0; + } + }); + + registerServerCommands(program); + registerListProjectsCommand(program); + registerListObservationsCommand(program); + registerExportCommand(program); + registerGetAttachmentCommand(program); + registerAlertCommands(program); + + return program; +} diff --git a/src/commands/alerts.ts b/src/commands/alerts.ts new file mode 100644 index 0000000..c5251e7 --- /dev/null +++ b/src/commands/alerts.ts @@ -0,0 +1,188 @@ +import type { Command } from "commander"; +import { handleError, requireProjectId } from "../lib/api.js"; +import { ALERT_COLUMNS, alertRow, selectColumns } from "../lib/columns.js"; +import { createContext } from "../lib/context.js"; +import { CliError, EXIT } from "../lib/errors.js"; +import { parseDateBound } from "../lib/filters.js"; +import { renderCount, renderList, write } from "../lib/format.js"; +import type { ApiList, RemoteAlert } from "../lib/types.js"; +import { collect, paginate } from "./shared.js"; + +interface ListAlertsOptions { + projectId?: string; + count?: boolean; + since?: string; + until?: string; + alertType?: string[]; + sourceId?: string[]; + includeDeleted?: boolean; + onlyDeleted?: boolean; + limit?: string; + offset?: string; + columns?: string; +} + +interface CreateAlertOptions { + projectId?: string; + startDate: string; + endDate: string; + sourceId: string; + alertType: string; + lon: string; + lat: string; +} + +const parseCoordinate = (value: string, name: string, limit: number) => { + const parsed = Number.parseFloat(value); + if (!Number.isFinite(parsed) || Math.abs(parsed) > limit) { + throw new CliError( + `--${name} must be a number between -${limit} and ${limit}, got "${value}".`, + EXIT.USAGE, + ); + } + return parsed; +}; + +export function registerAlertCommands(program: Command): void { + program + .command("list-alerts") + .description("List remote detection alerts for a project") + .option( + "-p, --project-id ", + "Project public id or name (or $COMAPEO_PROJECT)", + ) + .option("--count", "Print only the number of matching alerts") + .option("--since ", "Only alerts detected on/after this date") + .option("--until ", "Only alerts detected on/before this date") + .option( + "--alert-type ", + "Filter by metadata.alert_type (repeatable)", + collect, + [], + ) + .option("--source-id ", "Filter by source id (repeatable)", collect, []) + .option("--include-deleted", "Include deleted alerts") + .option("--only-deleted", "Show only deleted alerts") + .option("--limit ", "Print at most n alerts") + .option("--offset ", "Skip the first n alerts") + .option("--columns ", "Columns for table/csv output") + .action(async (options: ListAlertsOptions, cmd: Command) => { + try { + const context = createContext(cmd); + const projectId = await requireProjectId( + context.client, + options.projectId, + ); + + const response = await context.client.get>( + `/projects/${projectId}/remoteDetectionAlerts`, + ); + const all = response.data?.data ?? []; + + const since = options.since + ? parseDateBound(options.since, "start") + : undefined; + const until = options.until + ? parseDateBound(options.until, "end") + : undefined; + const alertTypes = new Set( + (options.alertType ?? []).map((value) => value.toLowerCase()), + ); + const sourceIds = new Set(options.sourceId ?? []); + + const matching = all.filter((alert) => { + if (options.onlyDeleted) { + if (!alert.deleted) return false; + } else if (!options.includeDeleted && alert.deleted) { + return false; + } + + if (since !== undefined || until !== undefined) { + const detected = Date.parse(alert.detectionDateStart ?? ""); + if (Number.isNaN(detected)) return false; + if (since !== undefined && detected < since) return false; + if (until !== undefined && detected > until) return false; + } + + if (alertTypes.size > 0) { + const type = String(alert.metadata?.alert_type ?? "").toLowerCase(); + if (!alertTypes.has(type)) return false; + } + + if (sourceIds.size > 0 && !sourceIds.has(alert.sourceId)) + return false; + + return true; + }); + + if (options.count) { + write(renderCount(matching.length, undefined, context.format)); + return; + } + + const page = paginate(matching, options); + const columns = selectColumns( + ALERT_COLUMNS, + options.columns, + ALERT_COLUMNS.map((column) => column.key), + ); + + write(renderList(context.format, columns, page.map(alertRow), page)); + } catch (error) { + handleError(error); + } + }); + + program + .command("create-alert") + .description("Create a remote detection alert") + .option( + "-p, --project-id ", + "Project public id or name (or $COMAPEO_PROJECT)", + ) + .requiredOption("--start-date ", "Detection start (ISO timestamp)") + .requiredOption("--end-date ", "Detection end (ISO timestamp)") + .requiredOption("--source-id ", "Source id") + .requiredOption("--alert-type ", "Alert type") + .requiredOption("--lon ", "Longitude") + .requiredOption("--lat ", "Latitude") + .action(async (options: CreateAlertOptions, cmd: Command) => { + try { + const context = createContext(cmd); + const projectId = await requireProjectId( + context.client, + options.projectId, + ); + + const lon = parseCoordinate(options.lon, "lon", 180); + const lat = parseCoordinate(options.lat, "lat", 90); + + const start = parseDateBound(options.startDate, "start"); + const end = parseDateBound(options.endDate, "end"); + if (end < start) { + throw new CliError( + "--end-date must be after --start-date.", + EXIT.USAGE, + ); + } + + const response = await context.client.post( + `/projects/${projectId}/remoteDetectionAlerts`, + { + detectionDateStart: options.startDate, + detectionDateEnd: options.endDate, + sourceId: options.sourceId, + metadata: { alert_type: options.alertType }, + geometry: { type: "Point", coordinates: [lon, lat] }, + }, + ); + + write("remote alert created"); + if (context.verbose) { + write(JSON.stringify(response.data ?? {}, null, 2)); + } + } catch (error) { + handleError(error); + } + }); +} diff --git a/src/commands/attachments.ts b/src/commands/attachments.ts new file mode 100644 index 0000000..5ccc222 --- /dev/null +++ b/src/commands/attachments.ts @@ -0,0 +1,90 @@ +import { writeFile } from "node:fs/promises"; +import type { Command } from "commander"; +import { handleError } from "../lib/api.js"; +import { createContext } from "../lib/context.js"; +import { CliError, EXIT } from "../lib/errors.js"; +import { write } from "../lib/format.js"; + +const PHOTO_VARIANTS = ["original", "preview", "thumbnail"]; + +export const getUrlComponents = (url: string) => { + const match = url.match( + /\/projects\/([^/]+)\/attachments\/([^/]+)\/([^/]+)\/([^/]+)$/, + ); + if (!match) { + throw new CliError( + `Invalid attachment URL: ${url}\nExpected .../projects//attachments///.`, + EXIT.USAGE, + ); + } + const [, projectId, driveId, type, name] = match; + return { projectId, driveId, type, name }; +}; + +const outputName = (name: string, kind: string): string => { + const extension = kind === "photo" ? ".jpg" : kind === "audio" ? ".mp3" : ""; + return `${name}${extension}`; +}; + +interface GetAttachmentOptions { + url: string; + variant?: string; + output?: string; +} + +export function registerGetAttachmentCommand(program: Command): void { + program + .command("get-attachment") + .description("Download a single attachment (photo or audio)") + .requiredOption("-u, --url ", "Full attachment URL") + .option( + "--variant ", + "Photo variant: original|preview|thumbnail (audio only supports original)", + ) + .option("-o, --output ", 'Output filename, or "-" for stdout') + .action(async (options: GetAttachmentOptions, cmd: Command) => { + try { + const context = createContext(cmd); + const { projectId, driveId, type, name } = getUrlComponents( + options.url, + ); + + const kind = type.toLowerCase(); + const variant = options.variant?.toLowerCase(); + + if (variant) { + if (kind === "photo" && !PHOTO_VARIANTS.includes(variant)) { + throw new CliError( + `Unknown photo variant "${options.variant}". Use one of: ${PHOTO_VARIANTS.join(", ")}.`, + EXIT.USAGE, + ); + } + if (kind !== "photo" && variant !== "original") { + throw new CliError( + `Variants are only supported for photos; "${kind}" only has the original file.`, + EXIT.USAGE, + ); + } + } + + const query = variant ? `?variant=${variant}` : ""; + const response = await context.client.get( + `/projects/${projectId}/attachments/${driveId}/${kind}/${name}${query}`, + { responseType: "arraybuffer" }, + ); + const data = Buffer.from(response.data as ArrayBuffer); + + if (options.output === "-") { + process.stdout.write(data); + return; + } + + const filename = + options.output ?? outputName(name, kind === "photo" ? "photo" : kind); + await writeFile(filename, data); + write(`saved ${filename} (${data.length} bytes)`); + } catch (error) { + handleError(error); + } + }); +} diff --git a/src/commands/export.ts b/src/commands/export.ts new file mode 100644 index 0000000..d261448 --- /dev/null +++ b/src/commands/export.ts @@ -0,0 +1,299 @@ +import { + mkdir, + mkdtemp, + readFile, + readdir, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { basename, extname, join } from "node:path"; +import chalk from "chalk"; +import type { Command } from "commander"; +import JSZip from "jszip"; +import { handleError, requireProjectId } from "../lib/api.js"; +import { createContext } from "../lib/context.js"; +import { CliError, EXIT } from "../lib/errors.js"; +import { filterObservations, mediaKind } from "../lib/filters.js"; +import { write } from "../lib/format.js"; +import { fetchObservations } from "../lib/observations.js"; +import type { Observation } from "../lib/types.js"; +import { + addFilterOptions, + toFilterOptions, + type FilterCliOptions, +} from "./shared.js"; + +const DEFAULT_OUTPUT = "comapeo_export.zip"; +const DOWNLOAD_CONCURRENCY = 6; + +interface ExportOptions extends FilterCliOptions { + projectId?: string; + output?: string; + attachments?: boolean; + limit?: string; +} + +interface GeoJsonFeature { + type: "Feature"; + geometry: { type: "Point"; coordinates: [number, number] } | null; + id: string; + properties: Record; +} + +/** Attachment path shape: .../projects//attachments/// */ +const parseAttachmentUrl = (url: string) => { + const match = url.match(/\/attachments\/([^/]+)\/([^/]+)\/([^/]+)$/); + if (!match) return undefined; + const [, driveId, kind, name] = match; + return { driveId, kind, name }; +}; + +const extensionFor = (kind: string): string => { + if (kind === "photo") return ".jpg"; + if (kind === "audio") return ".mp3"; + return extname(kind) || ""; +}; + +/** Run async work with a small concurrency cap instead of a 112-wide Promise.all. */ +async function mapLimit( + items: T[], + limit: number, + task: (item: T, index: number) => Promise, +): Promise { + const results = new Array(items.length); + let cursor = 0; + + const workers = Array.from( + { length: Math.min(limit, items.length) }, + async () => { + for (;;) { + const index = cursor; + cursor += 1; + if (index >= items.length) return; + results[index] = await task(items[index], index); + } + }, + ); + + await Promise.all(workers); + return results; +} + +export function registerExportCommand(program: Command): void { + const command = program + .command("export-geojson") + .description( + "Export project observations as GeoJSON, optionally zipped with media", + ) + .option( + "-p, --project-id ", + "Project public id or name (or $COMAPEO_PROJECT)", + ) + .option( + "-o, --output ", + `Output file. ".zip" (default ${DEFAULT_OUTPUT}) bundles media, ".geojson" writes plain GeoJSON`, + ) + .option("--no-attachments", "Skip media downloads (metadata only)"); + + addFilterOptions(command).action( + async (options: ExportOptions, cmd: Command) => { + try { + const context = createContext(cmd); + const projectId = await requireProjectId( + context.client, + options.projectId, + ); + + const output = options.output ?? DEFAULT_OUTPUT; + const extension = extname(output).toLowerCase(); + if (extension !== ".zip" && extension !== ".geojson") { + throw new CliError( + `--output must end in .zip (bundle with media) or .geojson (plain GeoJSON), got "${output}".`, + EXIT.USAGE, + ); + } + + const all = await fetchObservations(context.client, projectId); + const matching = filterObservations(all, toFilterOptions(options)); + if (matching.length === 0) { + throw new CliError( + "No observations match those filters; nothing to export.", + EXIT.NOT_FOUND, + ); + } + + const wantsMedia = + extension === ".zip" && options.attachments !== false; + const tmpRoot = wantsMedia + ? await mkdtemp(join(tmpdir(), "comapeo-export-")) + : undefined; + + try { + const mediaDir = tmpRoot; + if (tmpRoot) { + await mkdir(join(tmpRoot, "images"), { recursive: true }); + await mkdir(join(tmpRoot, "audio"), { recursive: true }); + } + + const usedNames = new Set(); + const download = async ( + url: string, + kind: string, + ): Promise => { + if (!mediaDir) return null; + const parts = parseAttachmentUrl(url); + if (!parts) return null; + + const folder = kind === "audio" ? "audio" : "images"; + const base = basename(parts.name) + extensionFor(kind); + let candidate = base; + let suffix = 2; + while (usedNames.has(`${folder}/${candidate}`)) { + candidate = `${basename(parts.name)}-${suffix}${extensionFor(kind)}`; + suffix += 1; + } + usedNames.add(`${folder}/${candidate}`); + + try { + const response = await context.client.get( + `/projects/${projectId}/attachments/${parts.driveId}/${parts.kind}/${parts.name}`, + { responseType: "arraybuffer" }, + ); + await writeFile( + join(mediaDir, folder, candidate), + Buffer.from(response.data as ArrayBuffer), + ); + return `${folder}/${candidate}`; + } catch (error) { + if (context.verbose) { + const message = + error instanceof Error ? error.message : String(error); + console.error( + chalk.yellow(`failed to download ${url}: ${message}`), + ); + } + return null; + } + }; + + const features = await mapLimit( + matching, + DOWNLOAD_CONCURRENCY, + async (observation: Observation): Promise => { + const photos: string[] = []; + const audio: string[] = []; + + for (const attachment of observation.attachments ?? []) { + if (!wantsMedia || !attachment.url) continue; + const kind = mediaKind(attachment.url); + if (kind !== "photo" && kind !== "audio") continue; + const saved = await download(attachment.url, kind); + if (!saved) continue; + if (kind === "photo") photos.push(saved); + else audio.push(saved); + } + + if (context.verbose) { + console.error( + chalk.dim( + `${observation.docId.slice(0, 8)} ${photos.length} photo(s), ${audio.length} audio`, + ), + ); + } + + // Tags were previously dropped here entirely: the interface + // declared categoryId/name/notes but the object never set + // them, so every export lost its categories and field data. + const properties: Record = { + ...observation.tags, + $created: observation.createdAt, + $modified: observation.updatedAt, + $version: `${observation.docId}@1`, + $photos: photos, + }; + if (audio.length > 0) properties.$audio = audio; + + const hasLocation = + typeof observation.lat === "number" && + typeof observation.lon === "number"; + + return { + type: "Feature", + geometry: hasLocation + ? { + type: "Point", + coordinates: [ + observation.lon as number, + observation.lat as number, + ], + } + : null, + id: observation.docId, + properties, + }; + }, + ); + + const featureCollection = { + type: "FeatureCollection" as const, + features, + }; + + if (!tmpRoot) { + await writeFile(output, JSON.stringify(featureCollection, null, 2)); + const mediaCount = features.reduce( + (sum, feature) => + sum + + (feature.properties.$photos as string[]).length + + ((feature.properties.$audio as string[] | undefined)?.length ?? + 0), + 0, + ); + write( + `exported ${features.length} observations (${mediaCount} media) -> ${output}`, + ); + return; + } + + await writeFile( + join(tmpRoot, "comapeo_data.geojson"), + JSON.stringify(featureCollection, null, 2), + ); + + const zip = new JSZip(); + zip.file( + "comapeo_data.geojson", + await readFile(join(tmpRoot, "comapeo_data.geojson")), + ); + + let mediaCount = 0; + for (const folder of ["images", "audio"]) { + const folderPath = join(tmpRoot, folder); + const files = await readdir(folderPath); + for (const file of files) { + mediaCount += 1; + zip.file( + `${folder}/${file}`, + await readFile(join(folderPath, file)), + ); + } + } + + const archive = await zip.generateAsync({ type: "nodebuffer" }); + await writeFile(output, archive); + + write( + `exported ${features.length} observations (${mediaCount} media, ${archive.length} bytes) -> ${output}`, + ); + } finally { + if (tmpRoot) { + await rm(tmpRoot, { recursive: true, force: true }); + } + } + } catch (error) { + handleError(error); + } + }, + ); +} diff --git a/src/commands/info.ts b/src/commands/info.ts new file mode 100644 index 0000000..6675fee --- /dev/null +++ b/src/commands/info.ts @@ -0,0 +1,34 @@ +import type { Command } from "commander"; +import { handleError } from "../lib/api.js"; +import { createContext } from "../lib/context.js"; +import { write } from "../lib/format.js"; +import type { ApiItem } from "../lib/types.js"; + +export function registerServerCommands(program: Command): void { + program + .command("info") + .description("Get server information") + .action(async (_options: unknown, cmd: Command) => { + try { + const context = createContext(cmd); + const response = + await context.client.get>>("/info"); + write(JSON.stringify(response.data, null, 2)); + } catch (error) { + handleError(error); + } + }); + + program + .command("healthcheck") + .description("Check the health of the server") + .action(async (_options: unknown, cmd: Command) => { + try { + const context = createContext(cmd); + await context.client.get("/healthcheck"); + write("ok"); + } catch (error) { + handleError(error); + } + }); +} diff --git a/src/commands/observations.ts b/src/commands/observations.ts new file mode 100644 index 0000000..1cdce69 --- /dev/null +++ b/src/commands/observations.ts @@ -0,0 +1,193 @@ +import chalk from "chalk"; +import type { Command } from "commander"; +import { handleError, requireProjectId } from "../lib/api.js"; +import { + ALL_OBSERVATION_COLUMNS, + DEFAULT_OBSERVATION_COLUMNS, + OBSERVATION_COLUMNS, + observationRow, + selectColumns, +} from "../lib/columns.js"; +import { createContext } from "../lib/context.js"; +import { buildPredicate, sortObservations } from "../lib/filters.js"; +import { + renderCount, + renderList, + renderSummary, + write, + type SummaryLine, +} from "../lib/format.js"; +import { assertGroupKey, groupCounts, summarize } from "../lib/group.js"; +import { assertSortKey, fetchObservations } from "../lib/observations.js"; +import { + addFilterOptions, + assertPagination, + paginate, + toFilterOptions, + type FilterCliOptions, +} from "./shared.js"; + +interface ObservationCliOptions extends FilterCliOptions { + projectId?: string; + count?: boolean; + groupBy?: string; + summary?: boolean; + sort?: string; + reverse?: boolean; + limit?: string; + offset?: string; + columns?: string; +} + +const summaryLines = (summary: ReturnType): SummaryLine[] => { + const lines: SummaryLine[] = [ + { label: "matching", value: String(summary.total) }, + ]; + + if (summary.deleted > 0) { + lines.push({ + label: "active/deleted", + value: `${summary.active} / ${summary.deleted}`, + }); + } + + lines.push({ + label: "date range", + value: + summary.firstCreatedAt && summary.lastCreatedAt + ? `${summary.firstCreatedAt.slice(0, 10)} → ${summary.lastCreatedAt.slice(0, 10)}` + : "n/a", + }); + + lines.push({ label: "devices", value: String(summary.devices) }); + lines.push({ + label: "media", + value: `${summary.photos} photos, ${summary.audio} audio`, + }); + + if (summary.withoutAttachments > 0) { + lines.push({ + label: "no attachments", + value: String(summary.withoutAttachments), + }); + } + + if (summary.withoutLocation > 0) { + lines.push({ + label: "no location", + value: String(summary.withoutLocation), + }); + } + + lines.push({ label: "categories", value: String(summary.categories) }); + + return lines; +}; + +export function registerListObservationsCommand(program: Command): void { + const command = program + .command("list-observations") + .description( + "List observations for a project. Filtering and counting happen client-side.", + ) + .option( + "-p, --project-id ", + "Project public id or name (or $COMAPEO_PROJECT)", + ) + .option("--count", "Print only the number of matching observations") + .option( + "--group-by ", + "With --count/--summary: group by category|preset|day|device|media|tag:", + ) + .option( + "--summary", + "Print a summary block instead of individual observations", + ) + .option( + "--sort ", + "Sort by createdAt|updatedAt|category|notes|media|id (default: createdAt)", + ) + .option("--reverse", "Reverse the sort order") + .option("--limit ", "Print at most n observations") + .option("--offset ", "Skip the first n observations") + .option( + "--columns ", + `Columns for table/csv output (default: ${DEFAULT_OBSERVATION_COLUMNS.join(",")})`, + ); + + addFilterOptions(command).action( + async (options: ObservationCliOptions, cmd: Command) => { + try { + const context = createContext(cmd); + + // Validate every flag before touching the network, so a typo is a + // usage error rather than a round trip that fails afterwards. + const predicate = buildPredicate(toFilterOptions(options)); + const groupKey = options.groupBy + ? assertGroupKey(options.groupBy) + : undefined; + const sortKey = assertSortKey(options.sort ?? "createdAt"); + const columns = selectColumns( + OBSERVATION_COLUMNS, + options.columns, + context.format === "csv" + ? ALL_OBSERVATION_COLUMNS + : DEFAULT_OBSERVATION_COLUMNS, + ); + assertPagination(options); + + const projectId = await requireProjectId( + context.client, + options.projectId, + ); + const all = await fetchObservations(context.client, projectId); + const matching = all.filter(predicate); + + const counts = groupKey ? groupCounts(matching, groupKey) : undefined; + + if (options.count) { + write(renderCount(matching.length, counts, context.format)); + return; + } + + if (options.summary) { + const summary = summarize(matching); + if (context.format === "json" || context.format === "ndjson") { + write( + JSON.stringify( + { + projectId, + summary, + ...(counts ? { groups: Object.fromEntries(counts) } : {}), + }, + null, + 2, + ), + ); + return; + } + + const blocks = [renderSummary(summaryLines(summary))]; + if (counts) { + blocks.push( + "", + chalk.dim(`by ${groupKey}`), + renderCount(matching.length, counts, "table"), + ); + } + write(blocks.join("\n")); + return; + } + + const sorted = sortObservations(matching, sortKey, options.reverse); + const page = paginate(sorted, options); + + write( + renderList(context.format, columns, page.map(observationRow), page), + ); + } catch (error) { + handleError(error); + } + }, + ); +} diff --git a/src/commands/projects.ts b/src/commands/projects.ts new file mode 100644 index 0000000..1f8b03c --- /dev/null +++ b/src/commands/projects.ts @@ -0,0 +1,59 @@ +import type { Command } from "commander"; +import { handleError } from "../lib/api.js"; +import { PROJECT_COLUMNS, projectRow, selectColumns } from "../lib/columns.js"; +import { createContext } from "../lib/context.js"; +import { renderCount, renderList, write } from "../lib/format.js"; +import type { ApiList, Project } from "../lib/types.js"; + +interface ListProjectsOptions { + count?: boolean; + search?: string; + columns?: string; +} + +export function registerListProjectsCommand(program: Command): void { + program + .command("list-projects") + .description("List all projects") + .option("--count", "Print only the number of projects") + .option("--search ", "Filter by project name (case-insensitive)") + .option( + "--columns ", + "Columns for table/csv output (projectId,name)", + ) + .action(async (options: ListProjectsOptions, cmd: Command) => { + try { + const context = createContext(cmd); + const response = + await context.client.get>("/projects"); + const all = response.data?.data ?? []; + + const needle = options.search?.trim().toLowerCase(); + const projects = needle + ? all.filter((project) => project.name.toLowerCase().includes(needle)) + : all; + + if (options.count) { + write(renderCount(projects.length, undefined, context.format)); + return; + } + + const columns = selectColumns( + PROJECT_COLUMNS, + options.columns, + PROJECT_COLUMNS.map((column) => column.key), + ); + + write( + renderList( + context.format, + columns, + projects.map(projectRow), + projects, + ), + ); + } catch (error) { + handleError(error); + } + }); +} diff --git a/src/commands/shared.ts b/src/commands/shared.ts new file mode 100644 index 0000000..4e7480b --- /dev/null +++ b/src/commands/shared.ts @@ -0,0 +1,116 @@ +import type { Command } from "commander"; +import { CliError, EXIT } from "../lib/errors.js"; +import type { FilterOptions } from "../lib/filters.js"; + +/** Repeatable option collector: `--tag a=b --tag c=d` -> ["a=b", "c=d"]. */ +export const collect = (value: string, previous: string[] = []): string[] => [ + ...previous, + value, +]; + +export interface FilterCliOptions { + category?: string[]; + tag?: string[]; + preset?: string; + since?: string; + until?: string; + bbox?: string; + has?: string[]; + missing?: string[]; + search?: string; + includeDeleted?: boolean; + onlyDeleted?: boolean; +} + +/** + * The filter flag set shared by `list-observations` and `export-geojson`, + * so whatever you can list you can also export. + */ +export function addFilterOptions(command: Command): Command { + return command + .option( + "--category ", + "Filter by tags.categoryId (repeatable, comma separated)", + collect, + [], + ) + .option( + "--tag ", + "Filter by any tag; repeatable, use v1|v2 for OR", + collect, + [], + ) + .option("--preset ", "Filter by preset docId prefix") + .option( + "--since ", + "Only records created on/after this date (YYYY-MM-DD or ISO)", + ) + .option("--until ", "Only records created on/before this date") + .option( + "--bbox ", + "Only observations inside a lon/lat bounding box", + ) + .option( + "--has ", + "Only with photo|audio|video|attachment|location|notes|preset (repeatable)", + collect, + [], + ) + .option( + "--missing ", + "Only without photo|audio|video|attachment|location|notes|preset (repeatable)", + collect, + [], + ) + .option("--search ", "Substring match over notes and tag values") + .option("--include-deleted", "Include deleted (tombstoned) documents") + .option("--only-deleted", "Show only deleted documents"); +} + +export function toFilterOptions(options: FilterCliOptions): FilterOptions { + return { + category: options.category, + tag: options.tag, + preset: options.preset, + since: options.since, + until: options.until, + bbox: options.bbox, + has: options.has, + missing: options.missing, + search: options.search, + includeDeleted: options.includeDeleted, + onlyDeleted: options.onlyDeleted, + }; +} + +/** Validate `--limit` / `--offset` without needing the data. */ +export function assertPagination(options: { + limit?: string; + offset?: string; +}): void { + for (const [name, raw] of [ + ["limit", options.limit], + ["offset", options.offset], + ] as const) { + if (raw === undefined) continue; + const parsed = Number.parseInt(raw, 10); + if (!Number.isFinite(parsed) || parsed < 0 || `${parsed}` !== raw.trim()) { + throw new CliError( + `--${name} must be a non-negative integer, got "${raw}".`, + EXIT.USAGE, + ); + } + } +} + +/** `--limit` / `--offset` applied after filtering and sorting. */ +export function paginate( + items: T[], + options: { limit?: string; offset?: string }, +): T[] { + assertPagination(options); + const offset = options.offset ? Number.parseInt(options.offset, 10) : 0; + const limit = options.limit ? Number.parseInt(options.limit, 10) : undefined; + const end = limit === undefined ? undefined : offset + limit; + return items.slice(offset, end); +} diff --git a/src/helpers.ts b/src/helpers.ts index d6957b0..542a8b1 100644 --- a/src/helpers.ts +++ b/src/helpers.ts @@ -1,63 +1,19 @@ -import { AxiosError } from "axios"; -import chalk from "chalk"; -import { config } from "dotenv"; -import axios from "axios"; - -config(); - -export function validateEnv( - serverUrl?: string, - serverToken?: string, -): { SERVER_URL: string; SERVER_BEARER_TOKEN: string } { - const SERVER_URL = serverUrl || process.env.SERVER_URL; - const SERVER_BEARER_TOKEN = serverToken || process.env.SERVER_BEARER_TOKEN; - - if (!SERVER_URL) { - console.error( - chalk.red( - "Error: SERVER_URL must be provided either as environment variable or --server-url option", - ), - ); - process.exit(1); - } - - if (!SERVER_BEARER_TOKEN) { - console.error( - chalk.red( - "Error: SERVER_BEARER_TOKEN must be provided either as environment variable or --server-token option", - ), - ); - process.exit(1); - } - - return { SERVER_URL, SERVER_BEARER_TOKEN }; -} - -export function getApiClient(options: { - serverUrl?: string; - serverToken?: string; -}) { - const { SERVER_URL, SERVER_BEARER_TOKEN } = validateEnv( - options.serverUrl, - options.serverToken, - ); - - return axios.create({ - baseURL: SERVER_URL, - headers: { - Authorization: `Bearer ${SERVER_BEARER_TOKEN}`, - "Content-Type": "application/json", - }, - }); -} - -export function handleError(error: unknown) { - if (error instanceof AxiosError) { - console.error(chalk.red("API Error:")); - console.error(chalk.red(error.response?.data?.message || error.message)); - } else { - console.error(chalk.red("An unexpected error occurred:")); - console.error(chalk.red(String(error))); - } - process.exit(1); -} +/** + * Deprecated re-export shim. + * + * `helpers.ts` used to hold validateEnv/getApiClient/handleError. The + * implementation now lives in `src/lib/api.ts`; this file stays so any + * local script importing the old path keeps compiling. + */ +export { + createApiClient, + describeError, + getApiClient, + handleError, + isProjectId, + matchProjects, + resolveProjectId, + resolveServerConfig, +} from "./lib/api.js"; + +export { EXIT, CliError } from "./lib/errors.js"; diff --git a/src/index.ts b/src/index.ts index 71b4f10..0dcc82c 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,462 +1,6 @@ #!/usr/bin/env node -import { Command } from "commander"; -import { writeFile } from "node:fs/promises"; -import { mkdir, readdir, readFile, rm } from "node:fs/promises"; -import JSZip from "jszip"; -import chalk from "chalk"; -import { validateEnv, getApiClient, handleError } from "./helpers.js"; +import { buildProgram } from "./cli.js"; -export const program = new Command(); +export const program = buildProgram(); -interface AlertOptions { - projectId: string; - startDate: string; - endDate: string; - sourceId: string; - alertType: string; - lon: string; - lat: string; - serverUrl?: string; - serverToken?: string; -} - -interface ServerInfoOptions { - serverUrl?: string; - serverToken?: string; -} - -// Shared utility functions -const getOutputFilename = ( - name: string, - type: string, - providedOutput?: string, -): string => { - const extension = type === "photo" ? ".jpg" : ".mp3"; - return providedOutput || `${name}${extension}`; -}; - -const downloadAttachment = async ( - projectId: string, - driveId: string, - type: string, - name: string, - variant: string, - serverUrl?: string, - serverToken?: string, -): Promise => { - const response = await getApiClient({ - serverUrl, - serverToken, - }).get( - `/projects/${projectId}/attachments/${driveId}/${type}/${name}${variant}`, - { responseType: "arraybuffer" }, - ); - return Buffer.from(response.data); -}; - -const getUrlComponents = (url: string) => { - const urlMatch = url.match( - /\/projects\/([^\/]+)\/attachments\/([^\/]+)\/([^\/]+)\/([^\/]+)$/, - ); - if (!urlMatch) { - throw new Error("Invalid attachment URL format"); - } - const [, projectId, driveId, type, name] = urlMatch; - return { projectId, driveId, type, name }; -}; - -program - .name("comapeo-cloud") - .description("CLI tool for interacting with Comapeo Cloud API") - .version("1.0.0") - .option("-s, --server-url ", "Server URL (overrides SERVER_URL env var)") - .option( - "-t, --server-token ", - "Server bearer token (overrides SERVER_BEARER_TOKEN env var)", - ); - -// Server Info Command -program - .command("info") - .description("Get server information") - .action(async (options: ServerInfoOptions) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).get("/info"); - console.log(chalk.green("Server information:")); - console.log(JSON.stringify(response.data, null, 2)); - } catch (error) { - handleError(error); - } - }); - -// List Projects Command -program - .command("list-projects") - .description("List all projects") - .action(async (options: { serverUrl?: string; serverToken?: string }) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).get("/projects"); - console.log(chalk.green("Projects:")); - console.log(JSON.stringify(response.data, null, 2)); - } catch (error) { - handleError(error); - } - }); - -// List Observations Command -program - .command("list-observations") - .description("List observations for a project") - .requiredOption("-p, --project-id ", "Project public ID") - .action( - async (options: { - projectId: string; - serverUrl?: string; - serverToken?: string; - }) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).get(`/projects/${options.projectId}/observations`); - console.log(chalk.green("Observations:")); - console.log(JSON.stringify(response.data, null, 2)); - } catch (error) { - handleError(error); - } - }, - ); - -// Export GeoJSON Command -program - .command("export-geojson") - .description("Export project observations as GeoJSON") - .requiredOption("-p, --project-id ", "Project public ID") - .option("-o, --output ", "Output file path") - .action( - async (options: { - projectId: string; - output?: string; - serverUrl?: string; - serverToken?: string; - }) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - interface ObservationData { - docId: string; - createdAt: string; - updatedAt: string; - deleted: boolean; - attachments: Array<{ - driveId: string; - type: string; - name: string; - url: string; - }>; - tags: Record; - lat: number; - lon: number; - } - - interface GeoJSONFeature { - type: "Feature"; - geometry: { - type: "Point"; - coordinates: [number, number]; - }; - id: string; - properties: { - categoryId?: string; - name?: string; - notes?: string; - $created: string; - $modified: string; - $version: string; - $photos: string[]; - }; - } - - console.log(chalk.blue("Fetching observations...")); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).get<{ data: ObservationData[] }>( - `/projects/${options.projectId}/observations`, - ); - console.log( - chalk.blue(`Found ${response.data.data.length} observations`), - ); - - // Create temporary directory for zip contents - console.log(chalk.blue("Creating temporary directory...")); - const tmpDir = "tmp_export"; - await mkdir(tmpDir, { recursive: true }); - await mkdir(`${tmpDir}/images`, { recursive: true }); - - const geojson = { - type: "FeatureCollection" as const, - features: await Promise.all( - response.data.data.map(async (obs): Promise => { - // Download attachments - console.log(chalk.blue(`Processing observation ${obs.docId}...`)); - const photoFiles = await Promise.all( - obs.attachments - // .filter(att => att.type === "photo") - .map(async (att) => { - console.log("processing", att); - const { projectId, driveId, type, name } = getUrlComponents( - att.url, - ); - const filename = getOutputFilename(name, type); - try { - console.log( - chalk.blue(`Downloading photo ${filename}...`), - ); - const photoData = await downloadAttachment( - options.projectId, - driveId, - type, - name, - "", - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - await writeFile( - `${tmpDir}/images/${filename}`, - photoData, - ); - console.log( - chalk.green(`Successfully downloaded ${filename}`), - ); - return filename; - } catch (error) { - console.error( - chalk.red(`Failed to download attachment ${filename}`), - ); - return null; - } - }), - ); - - return { - type: "Feature", - geometry: { - type: "Point", - coordinates: [obs.lon, obs.lat], - }, - id: obs.docId, - properties: { - $created: obs.createdAt, - $modified: obs.updatedAt, - $version: `${obs.docId}@1`, - $photos: photoFiles.filter((f): f is string => f !== null), - }, - }; - }), - ), - }; - - // Write GeoJSON file - console.log(chalk.blue("Writing GeoJSON file...")); - await writeFile( - `${tmpDir}/comapeo_data.geojson`, - JSON.stringify(geojson, null, 2), - ); - - // Create zip file - console.log(chalk.blue("Creating ZIP archive...")); - const outputPath = options.output || "comapeo_export.zip"; - const zip = new JSZip(); - - // Add GeoJSON file - zip.file( - "comapeo_data.geojson", - await readFile(`${tmpDir}/comapeo_data.geojson`), - ); - - // Add images folder - const imageFiles = await readdir(`${tmpDir}/images`); - for (const file of imageFiles) { - zip.file( - `images/${file}`, - await readFile(`${tmpDir}/images/${file}`), - ); - } - - // Generate zip file - const zipContent = await zip.generateAsync({ type: "nodebuffer" }); - await writeFile(outputPath, zipContent); - - // Clean up temporary directory - console.log(chalk.blue("Cleaning up temporary files...")); - await rm(tmpDir, { recursive: true, force: true }); - - console.log( - chalk.green(`Export completed successfully: ${outputPath}`), - ); - } catch (error) { - handleError(error); - } - }, - ); - -// Get Attachment Command -program - .command("get-attachment") - .description("Get an attachment") - .requiredOption("-u, --url ", "Full attachment URL") - .option( - "--variant ", - "Variant (original, preview, or thumbnail for photos; original for audio)", - ) - .option("-o, --output ", "Output filename") - .action( - async (options: { - url: string; - variant?: string; - output?: string; - serverUrl?: string; - serverToken?: string; - }) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const { projectId, driveId, type, name } = getUrlComponents( - options.url, - ); - const variant = options.variant ? `?variant=${options.variant}` : ""; - - const data = await downloadAttachment( - projectId, - driveId, - type, - name, - variant, - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - - const filename = getOutputFilename(name, type, options.output); - await writeFile(filename, data); - console.log(chalk.green(`Attachment saved as: ${filename}`)); - } catch (error) { - handleError(error); - } - }, - ); - -// Create Remote Alert Command -program - .command("create-alert") - .description("Create a remote detection alert") - .requiredOption("-p, --project-id ", "Project public ID") - .requiredOption("--start-date ", "Detection start date (ISO timestamp)") - .requiredOption("--end-date ", "Detection end date (ISO timestamp)") - .requiredOption("--source-id ", "Source ID") - .requiredOption("--alert-type ", "Alert type") - .requiredOption("--lon ", "Longitude") - .requiredOption("--lat ", "Latitude") - .action(async (options: AlertOptions) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).post(`/projects/${options.projectId}/remoteDetectionAlerts`, { - detectionDateStart: options.startDate, - detectionDateEnd: options.endDate, - sourceId: options.sourceId, - metadata: { - alert_type: options.alertType, - }, - geometry: { - type: "Point", - coordinates: [ - Number.parseFloat(options.lon), - Number.parseFloat(options.lat), - ], - }, - }); - console.log(chalk.green("Remote alert created successfully:")); - console.log(JSON.stringify(response.data, null, 2)); - } catch (error) { - handleError(error); - } - }); - -// List Alerts Command -program - .command("list-alerts") - .description("List remote detection alerts for a project") - .requiredOption("-p, --project-id ", "Project public ID") - .action( - async (options: { - projectId: string; - serverUrl?: string; - serverToken?: string; - }) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).get(`/projects/${options.projectId}/remoteDetectionAlerts`); - console.log(chalk.green("Remote detection alerts:")); - console.log(JSON.stringify(response.data, null, 2)); - } catch (error) { - handleError(error); - } - }, - ); - -// Healthcheck Command -program - .command("healthcheck") - .description("Check the health of the server") - .action(async (options: { serverUrl?: string; serverToken?: string }) => { - try { - validateEnv( - options.serverUrl || program.opts().serverUrl, - options.serverToken || program.opts().serverToken, - ); - const response = await getApiClient({ - serverUrl: options.serverUrl || program.opts().serverUrl, - serverToken: options.serverToken || program.opts().serverToken, - }).get("/healthcheck"); - console.log(chalk.green("Server is healthy:")); - console.log(JSON.stringify(response.data, null, 2)); - } catch (error) { - handleError(error); - } - }); - -program.parse(); +await program.parseAsync(process.argv); diff --git a/src/lib/api.ts b/src/lib/api.ts new file mode 100644 index 0000000..fd5de9c --- /dev/null +++ b/src/lib/api.ts @@ -0,0 +1,240 @@ +import axios, { AxiosError, type AxiosInstance } from "axios"; +import chalk from "chalk"; +import { config } from "dotenv"; +import { CliError, EXIT } from "./errors.js"; +import type { ApiList, Project } from "./types.js"; + +config(); + +/** + * Project public ids are 52-character lowercase alphanumeric strings, e.g. + * `ssgqifbqs8c14hpx8djuss4fkqoxkrga6xiqkpw5cscqqpqrjfko`. + * + * The alphabet is [a-z0-9], not RFC4648 base32: real ids contain 1, 8 and + * 9 (`...c14hpx8...`), which base32 would reject. + */ +const PROJECT_ID_PATTERN = /^[a-z0-9]{52}$/; + +const DEFAULT_TIMEOUT_MS = 30_000; + +export interface ServerConfig { + serverUrl: string; + serverToken: string; + timeoutMs: number; +} + +export interface ServerOptionInput { + serverUrl?: string; + serverToken?: string; + timeout?: string | number; +} + +const pickEnv = (...names: string[]): string | undefined => { + for (const name of names) { + const value = process.env[name]; + if (value && value.trim() !== "") return value; + } + return undefined; +}; + +/** + * Resolve server configuration from flags, then environment. + * + * The historical names are SERVER_URL / SERVER_BEARER_TOKEN. The + * COMAPEO_-prefixed aliases exist because the un-prefixed names are + * generic enough to collide with other tooling in a shared shell. + */ +export function resolveServerConfig(options: ServerOptionInput): ServerConfig { + const serverUrl = + options.serverUrl ?? pickEnv("SERVER_URL", "COMAPEO_SERVER_URL"); + const serverToken = + options.serverToken ?? + pickEnv("SERVER_BEARER_TOKEN", "COMAPEO_SERVER_TOKEN"); + + if (!serverUrl) { + throw new CliError( + [ + "SERVER_URL must be provided as an environment variable or with --server-url.", + " export SERVER_URL=https://your-server.example/", + " comapeo-cloud -s https://your-server.example/ ...", + ].join("\n"), + EXIT.USAGE, + ); + } + + if (!serverToken) { + throw new CliError( + [ + "SERVER_BEARER_TOKEN must be provided as an environment variable or with --server-token.", + " export SERVER_BEARER_TOKEN=your-token", + " comapeo-cloud -t your-token ...", + ].join("\n"), + EXIT.USAGE, + ); + } + + const timeoutMs = + options.timeout === undefined + ? DEFAULT_TIMEOUT_MS + : Number.parseInt(String(options.timeout), 10); + + if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) { + throw new CliError( + `--timeout must be a positive number of milliseconds, got "${options.timeout}".`, + EXIT.USAGE, + ); + } + + return { + serverUrl: serverUrl.replace(/\/+$/, ""), + serverToken, + timeoutMs, + }; +} + +export function createApiClient(config: ServerConfig): AxiosInstance { + return axios.create({ + baseURL: config.serverUrl, + timeout: config.timeoutMs, + headers: { + Authorization: `Bearer ${config.serverToken}`, + "Content-Type": "application/json", + }, + }); +} + +/** Convenience: validate config and build a client in one step. */ +export function getApiClient(options: ServerOptionInput): AxiosInstance { + return createApiClient(resolveServerConfig(options)); +} + +export function isProjectId(value: string): boolean { + return PROJECT_ID_PATTERN.test(value.trim()); +} + +/** + * Pure project matching, extracted so it can be unit tested without a + * server. Precedence: exact case-insensitive name, then substring. + */ +export function matchProjects( + projects: Project[], + input: string, +): { matches: Project[]; exact: boolean } { + const needle = input.trim().toLowerCase(); + const exact = projects.filter((p) => p.name.trim().toLowerCase() === needle); + if (exact.length > 0) return { matches: exact, exact: true }; + return { + matches: projects.filter((p) => p.name.toLowerCase().includes(needle)), + exact: false, + }; +} + +/** + * Accept either a project public id or a project name. + * + * Ids are passed straight through (no round trip). Names are resolved + * against /projects; an ambiguous name lists the candidates rather than + * silently picking one. + */ +export async function resolveProjectId( + client: AxiosInstance, + input: string, +): Promise { + const value = input.trim(); + if (isProjectId(value)) return value; + + const response = await client.get>("/projects"); + const projects = response.data.data ?? []; + const { matches, exact } = matchProjects(projects, value); + + if (matches.length === 1) return matches[0].projectId; + + if (matches.length === 0) { + throw new CliError( + `No project matches "${input}". Run "comapeo-cloud list-projects" to see the available projects.`, + EXIT.NOT_FOUND, + ); + } + + const listed = matches.map((p) => ` ${p.projectId} ${p.name}`).join("\n"); + throw new CliError( + `Project name "${input}" is ambiguous (${matches.length} ${exact ? "exact" : "partial"} matches):\n${listed}\nPass a project id, or a longer fragment of the name.`, + EXIT.USAGE, + ); +} + +/** Project id from a flag or $COMAPEO_PROJECT, resolved to a public id. */ +export async function requireProjectId( + client: AxiosInstance, + fromFlag?: string, +): Promise { + const input = fromFlag ?? pickEnv("COMAPEO_PROJECT"); + if (!input) { + throw new CliError( + "A project is required: pass -p/--project-id , or set COMAPEO_PROJECT.", + EXIT.USAGE, + ); + } + return resolveProjectId(client, input); +} + +/** + * Turn axios/network failures into messages that say what to do next, + * rather than echoing the raw HTTP status. + */ +export function describeError(error: unknown, config?: ServerConfig): CliError { + if (error instanceof CliError) return error; + + if (error instanceof AxiosError) { + const status = error.response?.status; + const apiMessage = + (error.response?.data as { message?: string } | undefined)?.message ?? + error.message; + + if (status === 401 || status === 403) { + return new CliError( + `Authentication failed (HTTP ${status}) at ${config?.serverUrl ?? "the server"}.\n ${apiMessage}\nCheck SERVER_BEARER_TOKEN / --server-token.`, + EXIT.FAILURE, + ); + } + + if (status === 404) { + return new CliError( + `Not found (HTTP 404): ${error.config?.url ?? "unknown path"}.\n ${apiMessage}\nCheck the project id (run "comapeo-cloud list-projects") and that the server build exposes that endpoint.`, + EXIT.NOT_FOUND, + ); + } + + if (error.code === "ECONNABORTED") { + return new CliError( + `Request timed out after ${config?.timeoutMs ?? DEFAULT_TIMEOUT_MS}ms. Retry with a larger --timeout.`, + EXIT.FAILURE, + ); + } + + if (error.code === "ECONNREFUSED" || error.code === "ENOTFOUND") { + return new CliError( + `Cannot reach ${config?.serverUrl ?? "the server"} (${error.code}). Check the URL and your network.`, + EXIT.FAILURE, + ); + } + + return new CliError( + `API error${status ? ` (HTTP ${status})` : ""}: ${apiMessage}`, + EXIT.FAILURE, + ); + } + + if (error instanceof Error) return new CliError(error.message, EXIT.FAILURE); + + return new CliError(String(error), EXIT.FAILURE); +} + +export function handleError(error: unknown, config?: ServerConfig): void { + const cliError = describeError(error, config); + console.error(chalk.red(`Error: ${cliError.message}`)); + // Set the code instead of calling process.exit(): exiting mid-write + // truncates piped stdout/stderr, which is exactly how `| jq` and CI + // log capture lose the final line. + process.exitCode = cliError.exitCode; +} diff --git a/src/lib/columns.ts b/src/lib/columns.ts new file mode 100644 index 0000000..83931ac --- /dev/null +++ b/src/lib/columns.ts @@ -0,0 +1,133 @@ +import { CliError, EXIT } from "./errors.js"; +import { countMedia } from "./filters.js"; +import type { Column, Row } from "./format.js"; +import type { Observation, Project, RemoteAlert } from "./types.js"; + +const dash = (value: unknown): string => { + if (value === undefined || value === null || value === "") return "-"; + return String(value); +}; + +const mediaCell = (observation: Observation): string => { + const media = countMedia(observation); + if (media.total === 0) return "-"; + const parts: string[] = []; + if (media.photos > 0) parts.push(`${media.photos}p`); + if (media.audio > 0) parts.push(`${media.audio}a`); + if (media.video > 0) parts.push(`${media.video}v`); + return parts.join("/"); +}; + +export const OBSERVATION_COLUMNS: Column[] = [ + { key: "id", header: "id" }, + { key: "created", header: "created" }, + { key: "updated", header: "updated" }, + { key: "category", header: "category" }, + { key: "notes", header: "notes", maxWidth: 32 }, + { key: "media", header: "media", align: "right" }, + { key: "photos", header: "photos", align: "right" }, + { key: "audio", header: "audio", align: "right" }, + { key: "lat", header: "lat", align: "right" }, + { key: "lon", header: "lon", align: "right" }, + { key: "preset", header: "preset" }, + { key: "device", header: "device" }, + { key: "deleted", header: "deleted" }, +]; + +export const DEFAULT_OBSERVATION_COLUMNS = [ + "id", + "created", + "category", + "notes", + "media", + "lat", + "lon", +]; + +export const ALL_OBSERVATION_COLUMNS = OBSERVATION_COLUMNS.map( + (column) => column.key, +); + +export function observationRow(observation: Observation): Row { + const media = countMedia(observation); + return { + id: observation.docId.slice(0, 8), + created: (observation.createdAt ?? "").replace("T", " ").slice(0, 19), + updated: (observation.updatedAt ?? "").replace("T", " ").slice(0, 19), + category: dash(observation.tags?.categoryId), + notes: dash(observation.tags?.notes), + media: mediaCell(observation), + photos: media.photos, + audio: media.audio, + lat: observation.lat === undefined ? "-" : observation.lat.toFixed(6), + lon: observation.lon === undefined ? "-" : observation.lon.toFixed(6), + preset: observation.presetRef?.docId.slice(0, 8) ?? "-", + device: observation.createdBy?.slice(0, 8) ?? "-", + deleted: observation.deleted ? "yes" : "no", + }; +} + +export const PROJECT_COLUMNS: Column[] = [ + { key: "projectId", header: "projectId" }, + { key: "name", header: "name" }, +]; + +export function projectRow(project: Project): Row { + return { projectId: project.projectId, name: project.name }; +} + +export const ALERT_COLUMNS: Column[] = [ + { key: "id", header: "id" }, + { key: "start", header: "detectionStart" }, + { key: "end", header: "detectionEnd" }, + { key: "alertType", header: "alertType" }, + { key: "sourceId", header: "sourceId" }, + { key: "lon", header: "lon", align: "right" }, + { key: "lat", header: "lat", align: "right" }, +]; + +export function alertRow(alert: RemoteAlert): Row { + const coordinates = alert.geometry?.coordinates ?? []; + return { + id: alert.docId.slice(0, 8), + start: (alert.detectionDateStart ?? "").replace("T", " ").slice(0, 19), + end: (alert.detectionDateEnd ?? "").replace("T", " ").slice(0, 19), + alertType: dash(alert.metadata?.alert_type), + sourceId: dash(alert.sourceId), + lon: typeof coordinates[0] === "number" ? coordinates[0].toFixed(6) : "-", + lat: typeof coordinates[1] === "number" ? coordinates[1].toFixed(6) : "-", + }; +} + +/** + * Resolve `--columns` against a known column set. Unknown names are a + * usage error listing the valid ones, rather than silently empty cells. + */ +export function selectColumns( + available: Column[], + requested: string | undefined, + fallback: string[], +): Column[] { + const keys = requested + ? requested + .split(",") + .map((key) => key.trim().toLowerCase()) + .filter((key) => key !== "") + : fallback; + + const byKey = new Map(available.map((column) => [column.key, column])); + const selected: Column[] = []; + + for (const key of keys) { + const column = byKey.get(key); + if (!column) { + throw new CliError( + `Unknown column "${key}". Available: ${available.map((c) => c.key).join(", ")}.`, + EXIT.USAGE, + ); + } + selected.push(column); + } + + return selected; +} diff --git a/src/lib/context.ts b/src/lib/context.ts new file mode 100644 index 0000000..96340e7 --- /dev/null +++ b/src/lib/context.ts @@ -0,0 +1,40 @@ +import type { AxiosInstance } from "axios"; +import type { Command } from "commander"; +import { + createApiClient, + resolveServerConfig, + type ServerConfig, +} from "./api.js"; +import { resolveFormat, type OutputFormat } from "./format.js"; + +export interface GlobalCliOptions { + serverUrl?: string; + serverToken?: string; + timeout?: string; + format?: string; + color?: boolean; + verbose?: boolean; +} + +export function globalOptions(command: Command): GlobalCliOptions { + return command.optsWithGlobals() as GlobalCliOptions; +} + +export interface Context { + config: ServerConfig; + client: AxiosInstance; + format: OutputFormat; + verbose: boolean; +} + +/** Everything a command needs, resolved from flags + environment once. */ +export function createContext(command: Command): Context { + const options = globalOptions(command); + const config = resolveServerConfig(options); + return { + config, + client: createApiClient(config), + format: resolveFormat(options.format), + verbose: options.verbose === true, + }; +} diff --git a/src/lib/errors.ts b/src/lib/errors.ts new file mode 100644 index 0000000..cf99dda --- /dev/null +++ b/src/lib/errors.ts @@ -0,0 +1,25 @@ +/** + * Exit codes used across the CLI. + * + * 0 success, 1 runtime/API failure, 2 usage error, 3 nothing matched / + * not found. Keeping them distinct lets scripts branch on the failure + * kind without parsing stderr. + */ +export const EXIT = { + OK: 0, + FAILURE: 1, + USAGE: 2, + NOT_FOUND: 3, +} as const; + +export type ExitCode = (typeof EXIT)[keyof typeof EXIT]; + +export class CliError extends Error { + readonly exitCode: ExitCode; + + constructor(message: string, exitCode: ExitCode = EXIT.FAILURE) { + super(message); + this.name = "CliError"; + this.exitCode = exitCode; + } +} diff --git a/src/lib/filters.ts b/src/lib/filters.ts new file mode 100644 index 0000000..4c14af7 --- /dev/null +++ b/src/lib/filters.ts @@ -0,0 +1,341 @@ +import { CliError, EXIT } from "./errors.js"; +import type { MediaKind, Observation, TagValue } from "./types.js"; + +/** + * All filtering in this CLI happens client-side: the server's + * `/projects/:id/observations` route accepts no query parameters and + * always returns every document, tombstones included. These functions + * are therefore pure and take a plain observation, which keeps them + * testable without a network or a fixtures server. + */ + +export interface FilterOptions { + category?: string[]; + tag?: string[]; + preset?: string; + since?: string; + until?: string; + bbox?: string; + has?: string[]; + missing?: string[]; + search?: string; + includeDeleted?: boolean; + onlyDeleted?: boolean; +} + +/** `.../attachments///` -> kind */ +export function mediaKind(url: string | undefined): MediaKind { + if (!url) return "unknown"; + const match = url.match(/\/attachments\/[^/]+\/([^/]+)\/[^/]+$/); + if (!match) return "unknown"; + const kind = match[1].toLowerCase(); + if (kind === "photo" || kind === "audio" || kind === "video") return kind; + return "unknown"; +} + +export function countMedia(observation: Observation): { + photos: number; + audio: number; + video: number; + total: number; +} { + let photos = 0; + let audio = 0; + let video = 0; + for (const attachment of observation.attachments ?? []) { + const kind = mediaKind(attachment.url); + if (kind === "photo") photos += 1; + else if (kind === "audio") audio += 1; + else if (kind === "video") video += 1; + } + return { + photos, + audio, + video, + total: (observation.attachments ?? []).length, + }; +} + +/** + * Split `12.5` / `2026-10-29` / `2026-10-29T13:00:00Z` into epoch ms. + * Date-only values are anchored to the start or end of the UTC day + * depending on the bound, so `--until 2026-10-29` includes all of the + * 29th rather than excluding everything after midnight. + */ +export function parseDateBound(value: string, bound: "start" | "end"): number { + const trimmed = value.trim(); + const dateOnly = /^\d{4}-\d{2}-\d{2}$/.test(trimmed); + const iso = dateOnly + ? `${trimmed}T${bound === "start" ? "00:00:00.000" : "23:59:59.999"}Z` + : trimmed; + const parsed = Date.parse(iso); + if (Number.isNaN(parsed)) { + throw new CliError( + `Cannot parse date "${value}". Use YYYY-MM-DD or a full ISO timestamp.`, + EXIT.USAGE, + ); + } + return parsed; +} + +export interface BoundingBox { + west: number; + south: number; + east: number; + north: number; +} + +/** Parse `west,south,east,north` (the GeoJSON / Overpass bbox order). */ +export function parseBbox(value: string): BoundingBox { + const parts = value.split(",").map((part) => Number.parseFloat(part.trim())); + if (parts.length !== 4 || parts.some((n) => !Number.isFinite(n))) { + throw new CliError( + `--bbox expects "west,south,east,north" in degrees, got "${value}".`, + EXIT.USAGE, + ); + } + const [west, south, east, north] = parts; + return { west, south, east, north }; +} + +/** + * Parse a repeatable `--tag key=value` expression. + * Values may be a `|`-separated set (`rio|igarape`) and are compared + * against both scalar tags and array-valued tags. + */ +export interface TagExpression { + key: string; + values: string[]; +} + +export function parseTagExpression(value: string): TagExpression { + const separator = value.indexOf("="); + if (separator <= 0) { + throw new CliError( + `--tag expects "key=value", got "${value}". Example: --tag tipo-de-corpo-dagua=rio`, + EXIT.USAGE, + ); + } + const key = value.slice(0, separator).trim(); + const raw = value.slice(separator + 1).trim(); + const values = raw + .split("|") + .map((part) => part.trim().toLowerCase()) + .filter((part) => part !== ""); + if (key === "" || values.length === 0) { + throw new CliError( + `--tag expects "key=value", got "${value}".`, + EXIT.USAGE, + ); + } + return { key, values }; +} + +export const flattenTagValue = (value: TagValue | undefined): string[] => { + if (value === undefined || value === null) return []; + if (Array.isArray(value)) + return value.map((item) => String(item).toLowerCase()); + return [String(value).toLowerCase()]; +}; + +const tagValueText = (value: TagValue | undefined): string => + flattenTagValue(value).join(" "); + +const matchesTagExpression = ( + observation: Observation, + expression: TagExpression, +): boolean => { + const present = flattenTagValue(observation.tags?.[expression.key]); + if (present.length === 0) return false; + return expression.values.some((wanted) => present.includes(wanted)); +}; + +const HAS_TOKENS = [ + "photo", + "photos", + "audio", + "video", + "attachment", + "attachments", + "media", + "location", + "coords", + "notes", + "preset", +] as const; + +const assertHasToken = (token: string): void => { + if (!(HAS_TOKENS as readonly string[]).includes(token)) { + throw new CliError( + `Unknown --has/--missing value "${token}". Use photo, audio, video, attachment, location, notes or preset.`, + EXIT.USAGE, + ); + } +}; + +const matchesHas = (observation: Observation, token: string): boolean => { + assertHasToken(token); + const media = countMedia(observation); + switch (token) { + case "photo": + case "photos": + return media.photos > 0; + case "audio": + return media.audio > 0; + case "video": + return media.video > 0; + case "attachment": + case "attachments": + case "media": + return media.total > 0; + case "location": + case "coords": + return ( + typeof observation.lat === "number" && + typeof observation.lon === "number" + ); + case "notes": + return String(observation.tags?.notes ?? "").trim() !== ""; + case "preset": + return Boolean(observation.presetRef?.docId); + default: + return false; + } +}; + +/** Build the predicate for a set of CLI filter options. */ +export function buildPredicate( + options: FilterOptions, +): (observation: Observation) => boolean { + const parsedTags = (options.tag ?? []).map(parseTagExpression); + const since = options.since + ? parseDateBound(options.since, "start") + : undefined; + const until = options.until + ? parseDateBound(options.until, "end") + : undefined; + const bbox = options.bbox ? parseBbox(options.bbox) : undefined; + const categories = new Set( + (options.category ?? []) + .flatMap((entry) => entry.split(",")) + .map((entry) => entry.trim().toLowerCase()) + .filter((entry) => entry !== ""), + ); + const has = (options.has ?? []).map((entry) => entry.trim().toLowerCase()); + const missing = (options.missing ?? []).map((entry) => + entry.trim().toLowerCase(), + ); + const preset = options.preset?.trim(); + const search = options.search?.trim().toLowerCase(); + + // Validate up-front so a typo fails on the first call, not on the + // first observation that happens to reach the branch. + for (const token of [...has, ...missing]) { + assertHasToken(token); + } + + return (observation: Observation): boolean => { + if (options.onlyDeleted) { + if (!observation.deleted) return false; + } else if (!options.includeDeleted && observation.deleted) { + return false; + } + + if (categories.size > 0) { + const category = String(observation.tags?.categoryId ?? "").toLowerCase(); + if (!categories.has(category)) return false; + } + + for (const expression of parsedTags) { + if (!matchesTagExpression(observation, expression)) return false; + } + + if (preset) { + const docId = observation.presetRef?.docId ?? ""; + if (!docId.toLowerCase().startsWith(preset.toLowerCase())) return false; + } + + if (since !== undefined || until !== undefined) { + const created = Date.parse(observation.createdAt); + if (Number.isNaN(created)) return false; + if (since !== undefined && created < since) return false; + if (until !== undefined && created > until) return false; + } + + if (bbox) { + const { lat, lon } = observation; + if (typeof lat !== "number" || typeof lon !== "number") return false; + if (lon < bbox.west || lon > bbox.east) return false; + if (lat < bbox.south || lat > bbox.north) return false; + } + + for (const token of has) { + if (!matchesHas(observation, token)) return false; + } + + for (const token of missing) { + if (matchesHas(observation, token)) return false; + } + + if (search) { + const haystack = [ + String(observation.tags?.notes ?? ""), + ...Object.entries(observation.tags ?? {}).flatMap(([key, value]) => [ + key, + tagValueText(value), + ]), + observation.presetRef?.docId ?? "", + ] + .join(" ") + .toLowerCase(); + if (!haystack.includes(search)) return false; + } + + return true; + }; +} + +export type SortKey = + | "createdAt" + | "updatedAt" + | "category" + | "notes" + | "media" + | "id"; + +const sortValue = (observation: Observation, key: SortKey): string => { + switch (key) { + case "createdAt": + return observation.createdAt ?? ""; + case "updatedAt": + return observation.updatedAt ?? ""; + case "category": + return String(observation.tags?.categoryId ?? ""); + case "notes": + return String(observation.tags?.notes ?? ""); + case "media": { + const media = countMedia(observation); + return String(media.total).padStart(6, "0"); + } + case "id": + return observation.docId ?? ""; + } +}; + +export function sortObservations( + observations: Observation[], + key: SortKey, + reverse = false, +): Observation[] { + const sorted = [...observations].sort((a, b) => + sortValue(a, key).localeCompare(sortValue(b, key)), + ); + return reverse ? sorted.reverse() : sorted; +} + +export function filterObservations( + observations: Observation[], + options: FilterOptions, +): Observation[] { + return observations.filter(buildPredicate(options)); +} diff --git a/src/lib/format.ts b/src/lib/format.ts new file mode 100644 index 0000000..08509ff --- /dev/null +++ b/src/lib/format.ts @@ -0,0 +1,158 @@ +import chalk from "chalk"; +import { CliError, EXIT } from "./errors.js"; + +export type OutputFormat = "table" | "json" | "ndjson" | "csv"; + +const FORMATS: OutputFormat[] = ["table", "json", "ndjson", "csv"]; + +/** + * Default to a table on a terminal and JSON when piped, so `| jq` keeps + * working unchanged while interactive use stays readable. + */ +export function resolveFormat(explicit?: string): OutputFormat { + if (explicit === undefined || explicit === "") { + return process.stdout.isTTY ? "table" : "json"; + } + const normalised = explicit.toLowerCase(); + if (!FORMATS.includes(normalised as OutputFormat)) { + throw new CliError( + `Unknown --format "${explicit}". Use one of: ${FORMATS.join(", ")}.`, + EXIT.USAGE, + ); + } + return normalised as OutputFormat; +} + +export interface Column { + key: string; + header: string; + align?: "left" | "right"; + maxWidth?: number; +} + +export type Row = Record; + +const truncate = (value: string, maxWidth?: number): string => { + if (!maxWidth || value.length <= maxWidth) return value; + if (maxWidth <= 1) return value.slice(0, maxWidth); + return `${value.slice(0, maxWidth - 1)}…`; +}; + +const cellText = (row: Row, column: Column): string => { + const raw = row[column.key]; + if (raw === undefined || raw === null) return ""; + return truncate(String(raw), column.maxWidth); +}; + +const pad = (value: string, width: number, align: "left" | "right"): string => + align === "right" ? value.padStart(width) : value.padEnd(width); + +/** Plain two-space-separated table with a dim rule under the header. */ +export function renderTable(columns: Column[], rows: Row[]): string { + if (rows.length === 0) return chalk.dim("(no matches)"); + + const widths = columns.map((column) => + Math.max( + column.header.length, + ...rows.map((row) => cellText(row, column).length), + ), + ); + + const header = columns + .map((column, index) => pad(column.header, widths[index], "left")) + .join(" "); + const rule = widths.map((width) => "─".repeat(width)).join(" "); + + const body = rows.map((row) => + columns + .map((column, index) => + pad(cellText(row, column), widths[index], column.align ?? "left"), + ) + .join(" "), + ); + + return [chalk.bold(header), chalk.dim(rule), ...body].join("\n"); +} + +const csvEscape = (value: string): string => + /[",\n]/.test(value) ? `"${value.replace(/"/g, '""')}"` : value; + +export function renderCsv(columns: Column[], rows: Row[]): string { + const lines = [columns.map((column) => csvEscape(column.header)).join(",")]; + for (const row of rows) { + lines.push( + columns.map((column) => csvEscape(cellText(row, column))).join(","), + ); + } + return lines.join("\n"); +} + +/** + * Render a list for the chosen format. + * + * `table` and `csv` project through `columns`; `json` and `ndjson` + * always emit the untouched records so nothing is lost to column + * selection when piping into jq. + */ +export function renderList( + format: OutputFormat, + columns: Column[], + rows: Row[], + records: T[], +): string { + switch (format) { + case "table": + return renderTable(columns, rows); + case "csv": + return renderCsv(columns, rows); + case "ndjson": + return records.map((record) => JSON.stringify(record)).join("\n"); + case "json": + return JSON.stringify(records, null, 2); + } +} + +/** + * `--count` output: a bare number with no decoration, because the whole + * point is `$(comapeo-cloud ... --count)` inside a script. + * Grouped counts are tab-separated so `| column -t` still works. + */ +export function renderCount( + total: number, + counts: Array<[string, number]> | undefined, + format: OutputFormat, +): string { + if (!counts) return String(total); + + if (format === "json") { + return JSON.stringify(Object.fromEntries(counts), null, 2); + } + if (format === "csv") { + return [ + "value,count", + ...counts.map(([key, n]) => `${csvEscape(key)},${n}`), + ].join("\n"); + } + return counts.map(([key, n]) => `${key}\t${n}`).join("\n"); +} + +export interface SummaryLine { + label: string; + value: string; +} + +export function renderSummary(lines: SummaryLine[]): string { + if (lines.length === 0) return ""; + const width = Math.max(...lines.map((line) => line.label.length)); + return lines + .map( + (line) => + `${chalk.dim(line.label.padEnd(width))} ${chalk.bold(line.value)}`, + ) + .join("\n"); +} + +export function write(text: string): void { + if (text === "") return; + process.stdout.write(`${text}\n`); +} diff --git a/src/lib/group.ts b/src/lib/group.ts new file mode 100644 index 0000000..72b43b6 --- /dev/null +++ b/src/lib/group.ts @@ -0,0 +1,160 @@ +import { CliError, EXIT } from "./errors.js"; +import { countMedia } from "./filters.js"; +import type { Observation } from "./types.js"; + +export type GroupKey = + | "category" + | "preset" + | "day" + | "device" + | "media" + | `tag:${string}`; + +const UNKNOWN = "(none)"; + +const short = (value: string | undefined, length = 8): string => + value ? value.slice(0, length) : UNKNOWN; + +/** + * The deployed servers we have verified do not expose `/presets`, so a + * preset can only be referenced by its docId prefix. Grouping therefore + * yields `preset:abb1b841`, not a human preset name. + */ +export function groupValue(observation: Observation, key: GroupKey): string { + if (key.startsWith("tag:")) { + const tag = key.slice("tag:".length); + const value = observation.tags?.[tag]; + if (value === undefined || value === null) return UNKNOWN; + if (Array.isArray(value)) + return value.length > 0 ? value.join(",") : UNKNOWN; + return String(value) || UNKNOWN; + } + + switch (key as "category" | "preset" | "day" | "device" | "media") { + case "category": + return String(observation.tags?.categoryId ?? "") || UNKNOWN; + case "preset": + return `preset:${short(observation.presetRef?.docId)}`; + case "day": + return (observation.createdAt ?? "").slice(0, 10) || UNKNOWN; + case "device": + return short(observation.createdBy); + case "media": { + const media = countMedia(observation); + if (media.total === 0) return "none"; + const parts: string[] = []; + if (media.photos > 0) parts.push("photo"); + if (media.audio > 0) parts.push("audio"); + if (media.video > 0) parts.push("video"); + return parts.join("+") || "other"; + } + } +} + +export function isValidGroupKey(value: string): value is GroupKey { + return ( + value === "category" || + value === "preset" || + value === "day" || + value === "device" || + value === "media" || + (value.startsWith("tag:") && value.length > "tag:".length) + ); +} + +export function assertGroupKey(value: string): GroupKey { + if (!isValidGroupKey(value)) { + throw new CliError( + `Unknown --group-by "${value}". Use category, preset, day, device, media or tag:.`, + EXIT.USAGE, + ); + } + return value; +} + +/** Counts per distinct group value, most frequent first, then alphabetical. */ +export function groupCounts( + observations: Observation[], + key: GroupKey, +): Array<[string, number]> { + const counts = new Map(); + for (const observation of observations) { + const value = groupValue(observation, key); + counts.set(value, (counts.get(value) ?? 0) + 1); + } + return [...counts.entries()].sort( + (a, b) => b[1] - a[1] || a[0].localeCompare(b[0]), + ); +} + +export interface Summary { + total: number; + active: number; + deleted: number; + withoutLocation: number; + withoutAttachments: number; + photos: number; + audio: number; + devices: number; + categories: number; + firstCreatedAt: string | null; + lastCreatedAt: string | null; +} + +/** + * Summarise the observations that survived filtering, so the numbers + * always agree with what `--count` reports for the same flags. + */ +export function summarize(observations: Observation[]): Summary { + let active = 0; + let deleted = 0; + let withoutLocation = 0; + let withoutAttachments = 0; + let photos = 0; + let audio = 0; + const devices = new Set(); + const categories = new Set(); + let first: string | null = null; + let last: string | null = null; + + for (const observation of observations) { + if (observation.deleted) deleted += 1; + else active += 1; + + if ( + typeof observation.lat !== "number" || + typeof observation.lon !== "number" + ) { + withoutLocation += 1; + } + + const media = countMedia(observation); + photos += media.photos; + audio += media.audio; + if (media.total === 0) withoutAttachments += 1; + + if (observation.createdBy) devices.add(observation.createdBy); + const category = observation.tags?.categoryId; + if (category) categories.add(String(category)); + + const created = observation.createdAt; + if (created) { + if (first === null || created < first) first = created; + if (last === null || created > last) last = created; + } + } + + return { + total: observations.length, + active, + deleted, + withoutLocation, + withoutAttachments, + photos, + audio, + devices: devices.size, + categories: categories.size, + firstCreatedAt: first, + lastCreatedAt: last, + }; +} diff --git a/src/lib/observations.ts b/src/lib/observations.ts new file mode 100644 index 0000000..a8bab80 --- /dev/null +++ b/src/lib/observations.ts @@ -0,0 +1,47 @@ +import type { AxiosInstance } from "axios"; +import { CliError, EXIT } from "./errors.js"; +import type { SortKey } from "./filters.js"; +import type { ApiList, Observation } from "./types.js"; + +const SORT_KEYS: SortKey[] = [ + "createdAt", + "updatedAt", + "category", + "notes", + "media", + "id", +]; + +export function assertSortKey(value: string): SortKey { + if (!SORT_KEYS.includes(value as SortKey)) { + throw new CliError( + `Unknown --sort "${value}". Use one of: ${SORT_KEYS.join(", ")}.`, + EXIT.USAGE, + ); + } + return value as SortKey; +} + +/** + * Fetch every observation for a project. + * + * The endpoint accepts no query parameters and returns tombstones too + * (`includeDeleted: true` server-side), so pagination, filtering and + * counting all happen in this process — see lib/filters.ts. + */ +export async function fetchObservations( + client: AxiosInstance, + projectId: string, +): Promise { + const response = await client.get>( + `/projects/${projectId}/observations`, + ); + const data = response.data?.data; + if (!Array.isArray(data)) { + throw new CliError( + "Unexpected response from the server: expected an object with a `data` array.", + EXIT.FAILURE, + ); + } + return data; +} diff --git a/src/lib/types.ts b/src/lib/types.ts new file mode 100644 index 0000000..4f6de77 --- /dev/null +++ b/src/lib/types.ts @@ -0,0 +1,75 @@ +/** + * Wire types for the CoMapeo Cloud API. + * + * Mirrors `src/datatypes/observation.js` in digidem/comapeo-cloud. The + * deployed server also exposes `/track`, `/preset` and `/field` in newer + * builds, but this CLI only relies on the two endpoints that exist on + * every deployment we have verified: `/projects`, `/projects/:id/observation` + * (plus the deprecated `/observations` alias) and + * `/projects/:id/remoteDetectionAlerts`. + */ + +export type TagValue = + | boolean + | number + | string + | null + | Array; + +export interface Attachment { + url?: string; +} + +export interface PresetRef { + docId: string; + versionId: string; + url: string; +} + +export interface Observation { + docId: string; + versionId: string; + originalVersionId?: string; + schemaName: string; + createdAt: string; + updatedAt: string; + deleted: boolean; + lat?: number; + lon?: number; + attachments: Attachment[]; + tags: Record; + presetRef?: PresetRef; + createdBy?: string; + updatedBy?: string; + metadata?: Record; + links?: string[]; +} + +export interface Project { + projectId: string; + name: string; +} + +export interface RemoteAlert { + docId: string; + createdAt: string; + updatedAt: string; + deleted: boolean; + detectionDateStart: string; + detectionDateEnd: string; + sourceId: string; + metadata?: Record; + geometry?: { type: string; coordinates: number[] }; +} + +/** Every list endpoint wraps its payload in `{ data: [...] }`. */ +export interface ApiList { + data: T[]; +} + +export interface ApiItem { + data: T; +} + +/** Attachment media kind, derived from `/attachments///`. */ +export type MediaKind = "photo" | "audio" | "video" | "unknown"; diff --git a/src/lib/version.ts b/src/lib/version.ts new file mode 100644 index 0000000..3b8f2c7 --- /dev/null +++ b/src/lib/version.ts @@ -0,0 +1,19 @@ +import { readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +/** + * Read the version from package.json instead of hardcoding it. + * `src/lib/version.ts` and `dist/lib/version.js` are both two levels + * below the package root, so the same relative path works for the + * compiled binary and for ts-jest. + */ +export function getVersion(): string { + try { + const here = dirname(fileURLToPath(import.meta.url)); + const raw = readFileSync(join(here, "..", "..", "package.json"), "utf8"); + return (JSON.parse(raw) as { version?: string }).version ?? "0.0.0"; + } catch { + return "0.0.0"; + } +}