[FIX] Check QC overrides in the mooring description; CLI and docs sweep

CHANGELOG.md

@@ -8,7 +8,7 @@ All notable changes to oceanarray are documented here.
 
 ### Added
 
-- `oceanarray.export(mooring, *, mooring_dir=None, out_dir=None, array=None, grid=True, overwrite=False)` and `oceanarray export MOORING [--array YAML] [-o DIR] [--no-grid] [--overwrite]` — write a mooring's stage-3 files and grid as OceanSITES 1.4 files (default `<mooring folder>/oceansites/`); instruments that cannot be exported are listed and left out. The data-format page (`oceanarray_format`) now describes these files, with the earlier departures from OceanSITES (`N_LEVELS`, `contributor_*`, compact times) dropped.
+- `oceanarray.export(mooring, *, mooring_dir=None, out_dir=None, array=None, grid=True, overwrite=False, dry_run=False)` and `oceanarray export MOORING [--array YAML] [-o DIR] [--no-grid] [--force] [-n]` — write a mooring's stage-3 files and grid as OceanSITES 1.4 files (default `<mooring folder>/oceansites/`); instruments that cannot be exported are listed and left out. The data-format page (`oceanarray_format`) now describes these files, with the earlier departures from OceanSITES (`N_LEVELS`, `contributor_*`, compact times) dropped.
 - `oceanarray.pipeline.oceansites.to_oceansites(ds, mooring, *, array=None, stack=None, source_file=None)` — one instrument's stage-3 dataset as an OceanSITES 1.4 deployment data file: variables on `(TIME, DEPTH)` with `DEPTH` the nominal instrument depth, `<PARAM>_QC` (reference table 2, the same flags), a data mode per variable (`DM_indicator`; `D` only with the calibration-dip correction recorded) and the §2.2 globals. `oceansites_filename` gives `OS_<platform>_<YYYYMM>_<mode>_<type>-<serial>-<depth>m.nc`. An ADCP is one file with its bins on `DEPTH` (nominal transducer depth ± range, by orientation; refused without one), `PRES` the pressure at each bin and `TRANSDUCER_PRES(TIME)` the measured transducer pressure and a failed or suspect `seabed_qc` merged into the velocity flags. A grid becomes a gridded file (`GRD`, §4.2): `OS_<platform>-<YYYYMM>_<start>-<end>_GRD_<interval>-DP<step>.nc`, `DEPTH` the pressure levels converted with `gsw.z_from_p`, `PRES` the levels, and `history` naming the stage files (pass `stack=`); only a provisional (`P`) grid is exported. The mooring description gains `platform_code` and `deployment_code` (default: the month of `deployment_time`); `site_code` comes from the array description and is left out when absent. The OceanSITES reader also reads `CSPD`, `CDIR`, `DOX2` and `TURB`.
 - `oceanarray.pipeline.read_mooring.read_mooring(mooring, *, stage=None, serials=None, allow_missing=False, mooring_dir=None, raw_dir=None)` — reads every instrument of one mooring into a dict of datasets keyed by serial. A mooring's identity is its `name:` field (every file is `<name>_<serial>_stage<N>.nc`); `mooring` may be a dict, a `*.mooring.yaml` path, or a directory holding one. `stage=None` finds and reads each raw file (from `raw_dir`) as stage 1 does — a missing `filename` found by the recovery-file naming convention, the Nortek header (also in a legacy `nortek/` folder), the raw header's calibration metadata — and returns it unmarked; entries marked `skip` are left out; `stage=1|2|3` opens the stage files (from `mooring_dir`) and checks the marker; on every path the description's facts are stamped (per-instrument: serial, instrument, hab, clamp_id, sample_interval_seconds, file_type, source_file; per-mooring: mooring_name, latitude/longitude + source, deployment/recovery window + source, waterdepth, density_reference). A dataset whose `serial` contradicts the description, or whose `mooring_name` differs from `name:` by more than case, raises `MooringError`; a case-only difference is corrected with a `history` line. Position and window keep their placeholders when absent, loudly (`location_source`/`deployment_window_source` = `"none"`, a WARNING). A listed instrument with no file at the requested stage raises unless `allow_missing=True`.
 - `oceanarray.pipeline.write_mooring.write_mooring(datasets, mooring_dir, *, overwrite=False)` — writes a dict of per-instrument datasets to `<mooring_dir>/<instrument>/<name>_<serial>_stage<N>.nc`, taking the name, instrument, serial and stage from each dataset's attributes; refuses a dataset whose `serial` attribute does not match its key.
@@ -17,7 +17,7 @@ All notable changes to oceanarray are documented here.
 - `oceanarray.pipeline._stage` — the `processing_stage` marker: `stamp(ds, stage, *, source=)` writes the marker, a `history` line and `source_file`; `require(datasets, stage, next_stage)` raises `StageError` when a dataset is at the wrong stage, naming the stage to run first. A file written before the marker existed is tolerated for one release (a missing marker on read is a WARNING).
 - `oceanarray.mooring` — the single seam for reading a mooring description: `load` (path, name + `proc_dir`, or dict; validates and raises `MooringError` on an invalid config), `check` (never raises; adds an unknown-top-level-key warning), `template` and `save`, plus `oceanarray.MooringError`. Every stage, report and CLI path now receives the loaded dict instead of calling `yaml.safe_load` itself; a test (`test_config_sites`) fails if a new mooring-config `safe_load` appears outside the loader. `scripts/survey_mooring_keys.py` lists the top-level keys on a processed-data root that are not yet in `mooring.KNOWN_KEYS`.
 - `oceanarray.pipeline.stage1.stage1(datasets, mooring)`, `oceanarray.pipeline.stage2.stage2(datasets, mooring)`, `oceanarray.pipeline.stage3.stage3(datasets, mooring, *, caldip_dir=None)`, `oceanarray.pipeline.stack.stack(datasets, mooring, *, dt_seconds=60)` and `oceanarray.pipeline.grid.grid(ds_stack, *, pmin, pmax, dp)` — the processing steps as pure functions over datasets, with no file I/O. `MooringProcessor`, `Stage2Processor`, `Stage3Processor`, `MooringStacker` and `MooringGridder` now read, call these, and write; their outputs are unchanged in value. `oceanarray.pipeline.raw_files` holds stage 1's raw-file finding (`resolve_raw_file`, `guess_instrument_filename`) and reading (`read_raw_file`).
-- `oceanarray stamp FILE... --stage N` and `oceanarray.pipeline.stamp.stamp_file(path, stage)` — write the `processing_stage` marker into a stage file written before it existed (`history`: "declared via stamp"). Refuses a stage that contradicts the filename or an existing marker; leaves an already-marked file untouched.
+- `oceanarray stamp FILE... --declare N` and `oceanarray.pipeline.stamp.stamp_file(path, stage)` — write the `processing_stage` marker into a stage file written before it existed (`history`: "declared via stamp"). Refuses a stage that contradicts the filename or an existing marker; leaves an already-marked file untouched.
 - `oceanarray.process(mooring, *, stages=(1, 2, 3), serials=None, force=False, keep_going=True, mooring_dir=None, raw_dir=None, caldip_dir=None) -> list[Path]` — runs stages 1–3 for one mooring from its `*.mooring.yaml` path (or a directory holding one, or a loaded description with `mooring_dir=`), writing each stage's files and a log per stage, and returns every stage file of the stages run — written, or kept because it was already there (`force=True` rewrites; stage 1 also rewrites a file older than its raw file); each stage prints how many it wrote and kept. By default it carries on past a failed instrument (see `keep_going` below); `keep_going=False` raises `MooringError` naming the instruments that failed, after the stage has written the rest, and runs nothing after it. Stacking and gridding are not part of it. The previous form, `process(name, stage, proc_dir=…)` (all five stages, returns a bool), still works for one release (see *Deprecated*).
 - `oceanarray.run(mooring, *, stages=None, from_stage=None, serials=None, force=False, keep_going=True, mooring_dir=None, raw_dir=None, caldip_dir=None, dt_seconds=None, pmin=None, pmax=None, dp=None, stack_stage=3, report=False) -> list[Path]` — stages 1–3 (through `process`), then the stack (`<name>_stack.nc`) and the grid (`<name>_grid.nc`), rebuilt on every call; returns the stage files, then the stack and grid. `stages` lists the stages to run (default all); because the stack is rebuilt from the stage-3 files, they must run from the first one given through stage 3, so the stack never takes files older than the ones just written — `stages=(2, 3)` (or `from_stage=2`) re-runs stages 2 and 3 from the stage-1 files without `raw_dir`, while `stages=2` or `(1, 3)` is refused before anything runs. To run one stage on its own, use `process`. `stack_stage=2` (CLI `--stack-stage 2`) is a quick look at sea: it runs through stage 2 only and stacks and grids the stage-2 files (no QC flags, no interpolated pressure, no magnetic declination; every level's `source_stage` is 2). By default it carries on past failures (see `keep_going` below). With `keep_going=False` any failure raises, an instrument with no stage-3 file stops the stack, and a stack that left an instrument out is written but not gridded. The stack and grid settings come from the call, else from new optional `stack:` (`dt_seconds`) and `grid:` (`pmin`, `pmax`, `dp`) blocks in the mooring description, else 60 s and 200–1000 dbar every 20 dbar; the merged settings are checked before anything runs, and `mooring.check` refuses a malformed block. When a strict run stops after it has rewritten stage files, or its new stack cannot be gridded, the stack and grid that no longer match are renamed `<name>_stack.stale.nc` / `<name>_grid.stale.nc` (a line says so; one stale copy per product, replaced by the next) — whatever the error that stopped it.
 - `keep_going` on `process` and `run`, **on by default** — as much as possible on a first pass, troubleshoot later: a failed instrument is logged and left out and the next stage runs on what has files; a stage-3 failure is logged, not raised; `run` stacks each instrument's stage-3 file, else its stage-2 file. What was taken from stage 2 is recorded in the stack's new per-level `source_stage` variable and in a `stage2_instruments` attribute (the grid inherits it); instruments with neither, or that failed in this run (their older files are not used), are named in `instruments_omitted`; after a stage-3 failure every instrument is stacked from stage 2; what failed is recorded in a `keep_going_failures` attribute on the stack and grid; an instrument that failed in one stage is left out of the later stages of the same run (its older files are neither rebuilt from nor used as another instrument's pressure source); a stack that cannot be gridded (e.g. no pressure left) is kept, with a warning and no grid.
@@ -40,6 +40,11 @@ All notable changes to oceanarray are documented here.
 
 ### Changed
 
+- The command-line help says, for every subcommand, what it reads and writes, with an `Examples:` block; flags follow the family conventions. Renamed (old spellings accepted until 0.7.0 with a note on stderr): `process --report` → `--summary`, `plot --downsample` → `--downsample-seconds`, `export --overwrite` → `--force`, `stamp --stage` → `--declare`. All deprecated spellings now share one mechanism and message ("… is deprecated and removed in 0.7.0; use …").
+- `oceanarray validate` takes a mooring the way the other commands do (its `*.mooring.yaml`, its folder, or a name with `--proc-dir`) and gains `--strict` (exit 1 on warnings too; the default still exits 0 when there are only warnings).
+- A usage error (a removed flag, a missing `--raw-dir` or `--proc-dir`, an option that does not combine) exits 2 with an `ERROR:` line on stderr; a processing failure still exits 1. A mooring name with `--proc-dir` whose folder does not exist is an error naming the folder.
+- `python -m oceanarray` runs the CLI; `-v` is accepted after the subcommand as well as before it; `export -n/--dry-run` lists the files it would write.
+- `process --stage` help marks `stack` and `grid` as deprecated (use `run`), and their tuning flags are hidden from `process --help`.
 - Private helpers follow one naming table (CONTRIBUTING, "Private helpers"): one verb per job, figure helpers `_draw_<what>` / `_make_<what>_b64` with no `_fig` suffix, 85 renamed; their docstrings are one line, with the reasons the code cannot show kept as comments (≈ −1,400 lines). The three private `_safe_serial` wrappers are gone (`paths.safe_serial`). Four dead helpers deleted, the duplicate `_times_to_float` and the three module `_log`s collapsed (into `oceanarray.logger.log_message`), and four one-line wrappers inlined. No public name or behaviour changes; report `source:` lines name the new helpers.
 - `oceanarray process --dry-run` lists the file every stage would write (new, keep or rewrite) and, for stage 3, which instruments get interpolated pressure; it works by path and needs no `--raw-dir` (before: stage 3 only, by name under `--proc-dir`). The README, the demo notebook and the method pages use the verbs.
 - The report page switches lose their short forms (`-i`, `-s`, `-g`, `-A`; accepted with a note until 0.7.0): the short flags are `-o` (always an output directory), `-n` (`--dry-run`) and `-v`.
@@ -76,6 +81,8 @@ Every old name below still works, warns (`DeprecationWarning`, naming its replac
 
 ### Breaking changes
 
+- A mooring description whose QC override blocks (`qc_ranges`, `qc_spike`, `tilt_qc`, `qc_flat_line`, at mooring or instrument level) hold an unknown field, a span that is not two increasing numbers, a threshold that is not a number, or a new variable without a `fail_span` is now refused by `load` and reported as an ERROR by `oceanarray validate`; before, the mistake was dropped and QC ran with the defaults. A variable outside the registry is a WARNING. `qc_spike`, `tilt_qc` and `qc_flat_line` are no longer reported as unknown top-level keys.
+- `current_direction` is written with `standard_name = "sea_water_velocity_to_direction"`; CF deprecated `direction_of_sea_water_velocity`. Files written before keep the old name.
 - Modules moved to the package layout (no aliases: nothing outside oceanarray imported them). Old → new:
 
   | old | new |

CONTRIBUTING.md

@@ -81,7 +81,7 @@ For all other bugs:
 
 ### Scope
 
-oceanarray **processes moored-array observations**: raw instrument files from a mooring (MicroCATs, RBRs, Nortek and RDI current meters and ADCPs, SBE 16/56, Seapoint, …) → CF-NetCDF at each stage — stage 1 (raw → NetCDF), stage 2 (deployment window, clock), stage 3 (QC, velocity rotation, calibration-dip offsets applied) — then a stack on a common time axis and a grid on pressure levels, each step documented in an HTML report. The OceanSITES export of the stacked and gridded products is the next in-scope step.
+oceanarray **processes moored-array observations**: raw instrument files from a mooring (MicroCATs, RBRs, Nortek and RDI current meters and ADCPs, SBE 16/56, Seapoint, …) → CF-NetCDF at each stage — stage 1 (raw → NetCDF), stage 2 (deployment window, clock), stage 3 (QC, velocity rotation; calibration-dip offsets once the apply step lands — the interface is in place) — then a stack on a common time axis and a grid on pressure levels, each step documented in an HTML report, and an OceanSITES 1.4 export of the stage-3 files and the grid. Exporting the stack is the next in-scope step.
 
 **Fixed — not open to variation:** the output contract. Variable names, CF standard names and the canonical unit spellings come from one registry (`config/parameters.py`) and are the same in every file the package writes; QC flags follow the QARTOD/OceanSITES vocabulary; every processing parameter is recorded in the output's attributes so a file can be reprocessed from itself. Contributions that change these change every user's files, and are discussed first, as an issue.
 
@@ -91,7 +91,7 @@ oceanarray **processes moored-array observations**: raw instrument files from a
 - **Corrections to individual instruments.** How calibration-dip offsets and other per-instrument corrections (drift, pressure-sensor offset) are applied in stage 3 — constant, linear in time, pre/post weighted — is a method choice; the *derivation* of caldip offsets stays in `caldip`, oceanarray applies its output.
 - **QC tests.** New QARTOD-style tests or thresholds, as long as the flags land in the existing `_qc` companions with their thresholds recorded.
 - **Diagnostic figures and report panels.** New figures are welcome, and the report is meant to let a user choose their own selection of panels per page (a config-driven profile); contributions that make that selection easier are in scope.
-- **Readers.** Instrument readers live in `seasenselib`; oceanarray's stage 1 dispatches on the file type (`config/parameters.INSTRUMENT_FILE_TYPES`) and keeps only a small set of its own readers (`readers/`: OceanSITES, RODB and the old Nortek CSV). A reader for an instrument `seasenselib` does not cover is welcome — but write it in `seasenselib` first and add the file type here; a reader that only oceanarray would ever use can live in `readers/`. (Where exactly that line is drawn is not written down yet — ask.)
+- **Readers.** Instrument readers live in `seasenselib`; oceanarray's stage 1 dispatches on the file type (`config/parameters.INSTRUMENT_FILE_TYPES`) and keeps only a small set of its own readers and helpers (`readers/`: its own netCDF, OceanSITES, RODB, and the Nortek and Sea-Bird header parsers). A reader for an instrument `seasenselib` does not cover is welcome — but write it in `seasenselib` first and add the file type here; a reader that only oceanarray would ever use can live in `readers/`. (Where exactly that line is drawn is not written down yet — ask.)
 
 **Out of scope, because a sibling owns it:** shipboard CTD processing (`ctdcast`); deriving calibration-dip offsets (`caldip`); glider data (`glidertest`, `seagliderOG1`); transport and AMOC estimates from the gridded products (`AMOCatlas`); general-purpose plotting. Also out: features that need cruise-specific raw data in the repository, heavy new dependencies, or that break the CF-NetCDF contract. If you are unsure whether an idea fits, open an issue describing it rather than deciding alone.
 
@@ -150,7 +150,7 @@ Co-authored-by: Name <email@example.com>
 - **Type annotations**: all public function arguments and return values.
 - **Lint**: `ruff check .` must be clean. No bare `except:`. No suppressed warnings without a comment explaining why.
 - **Units**: every registered variable carries the registry's canonical spelling (`config/parameters.VARIABLES[var]["units"]`), and that spelling is identical in every file the package writes — except N², kept as `s-2` or `rad2 s-2` as it arrives (`config/unit_tables.UNIT_SPELLINGS_KEPT`). The pipeline rewrites known spellings through the shared units tables (`config/unit_tables.TABLES`) and refuses unknown ones. A new spelling is added **with evidence** — a reader that emits it — never from memory: to `config/unit_tables.UNIT_ALIAS_EXTRAS` if only oceanarray's readers emit it, or upstream in ctdcast's vendored `units_tables.py` if it belongs to the family. A mass↔volume conversion is never a scalar alias. Use `gsw` for all seawater property calculations.
-- **Plot style**: discrete colorbars (`BoundaryNorm`, ≤ 20 levels). Report figures are styled at one choke point — `reports/_encode.render_b64` applies the vendored `report.mplstyle` and `reports/_slots.render` draws each figure at its slot width — so a report plotter does not call `plt.style.use` itself.
+- **Plot style**: discrete colorbars (`BoundaryNorm`, ≤ 20 levels). Report figures are styled at one choke point — `reports/_vendored/figure_png.render_b64` applies the vendored `report.mplstyle` and `reports/_slots.render` draws each figure at its slot width — so a report plotter does not call `plt.style.use` itself.
 - **Private helpers**: a leading underscore keeps a helper out of the public API; it may be shared inside its own subpackage (`reports/`, `cli/`, `pipeline/`). Its docstring is **one line** plus a typed signature; a reason the code cannot show (a units or flag-table detail, a precedence order, an edge case) goes in a `#` comment at the line it explains. The name says what comes back, with one verb per job:
 
   | verb | returns |