PR 7: adopt CosmoSIS native SACC likelihood (sacc_like), validated against the converter path by cailmdaley · Pull Request #255 · CosmoStat/sp_validation

cailmdaley · 2026-07-10T15:09:48Z

Closes #254. Row 7 of PRD #241 — the final PR of the SACC/Smokescreen series: the inference pipeline consumes the assembled {version}.sacc natively through CosmoSIS's likelihood/sacc/sacc_like.py, with the PR-3 converter path (SACC → 2pt-FITS → 2pt_like) retained as the cross-validating twin. Each path validates the other, per PRD §2.

The "prototype-grade" verdict, grounded

The PRD flagged sacc_like's ξ± path as prototype-grade. A source read of CSL @ 4fd2f1c found the concrete mechanisms:

No angular-unit conversion. sacc_likelihoods/twopoint.py builds the theory spline on block[section, "theta"] (radians, CosmoSIS convention) and evaluates it at raw SACC theta tags (arcmin, SACC/firecrown convention). 2pt_like explicitly converts everything to radians (2pt_like.py L179–183); sacc_like never does — the spline is evaluated ~3437× outside its grid, SpectrumInterp returns 0 there, and χ² silently collapses to dᵀC⁻¹d. Its only production use (HSC Y3) was the ℓ-space Cℓ path, which is unit-free — hence never caught.
Theory↔data ordering assumed, never enforced. The data vector is get_mean() (insertion order); theory is a type-major loop. A comment claims to_canonical_order was called on load — it never is. Single-pair files align by construction; tomographic pair-major files would silently misalign.
(Latent, not touched here: sacc_like.py L106 references undefined t1, t2 in the keep_tracers removal path.)

Adoption = subclass shim, not fork

src/sp_validation/sacc_like_unions.py subclasses upstream SaccClLikelihood (factory-loaded from csl_dir), containing exactly two fixes: theory extraction runs against a radian-θ copy of the loaded SACC (self.sacc_data stays arcmin, so scale cuts keep arcmin ini ergonomics and upstream's save_theory/save_realization never write radian tags), and a loud ordering guard that reconstructs the upstream theory-loop order and requires it to match insertion order. Everything else — scale-cut grammar, Sellentin/Hartlap, windows for Cℓ — rides upstream unmodified. n(z) loads natively via CSL's number_density/load_nz_sacc. like_name = 2pt_like keeps every block key identical across engines, so chain post-processing is engine-agnostic.

Validation (all observed, in-process, no sampler/CAMB needed)

Both likelihood modules run standalone against identical synthetic theory DataBlocks:

Machine-zero equality shim-on-SACC vs 2pt_like-on-converter-FITS: synthetic (realistic covariance) χ² = 1.18829133107 both, Δχ² = 0, max|Δtheory| = 0; real data (SP_v1.4.6_leak_corr, ξ-only, N=40) χ² = 420310.753739 both, Δχ² = 0. (Re-verified 2026-08-31 in the current env — py3.12 / scipy 1.17, CSL upstream @530e2b1b — all 8 tests pass, identical χ² to all printed digits.)
Scale cuts equivalent: same arcmin ranges → same post-cut N, identical χ².
Tripwire: raw upstream sacc_like on the same arcmin SACC is ~631× off — this test failing one day means upstream fixed units and the shim should be retired.
Ordering guard raises loudly on a pair-major tomographic file; θ-conversion scoped to real-category types only (COSEBIs n, spectrum ell untouched); save_theory output verified to carry arcmin tags and theory values, with re-execution χ²-stable.

Workflow revival

inference_prep honors PR 4's dormant-note contract: input = the assembled {version}.sacc (+ the two ini templates, DAG-tracked), outputs = converter 2pt-FITS + generated 2pt_like ini + generated sacc_like ini (workflow/scripts/generate_inference_config.py, dual-mode like assemble_sacc.py). cosmosis_fitting.py is retired from the real-data path; the glass-mock rules keep it, and the PSF/xi_sys pipeline variant stays on 2pt_like (both flagged in comments — follow-up work, not silent scope creep). snakemake -n inference_fiducial resolves the full chain: parts → assemble_sacc → inference_prep → inference_fiducial.

Environment

New inert [cosmosis] extra (cosmosis>=3.25; pip-installs cleanly on py3.12). Tests gate on importorskip("cosmosis") + a CSL_DIR env var pointing at a CSL checkout — they skip in CI (image carries neither) and run on candide (verified against upstream CSL @530e2b1b). CSL dependency: the pipeline needs a CSL that is current on likelihood/sacc (real-space support is recent) and carries the UNIONS fork's modules (sample_S8, z-dependent IA) — i.e. the fork updated toward upstream, tracked in #316. csl_dir in config stays as-is until that lands.

Adversarial review

Fresh-context multi-lens review (5 lenses, 3 refuters per finding) before this draft opened; two MEDIUM findings confirmed and fixed: (1) save_theory/save_realization wrote radian θ tags into saved SACCs (in-place mutation leak — now a swap-in copy, regression-tested); (2) the ini templates were rule params rather than inputs, so template edits didn't propagate to generated configs (now DAG edges, asserted by the dry-run guard).

Stack note

Base = feat/sacc-6-blinding (PR #253) — the stack is linear, #251 → #253 → #255; PRs 1–3 (#243/#245/#249) are merged into develop. The diff against the base is this PR's own surface: sacc_like_unions.py, test_sacc_like.py, test_generate_inference_config.py, cosmosis_pipeline_A_ia_sacc.ini, generate_inference_config.py, plus inference.smk, config.yaml, pyproject.toml, test_bmodes_workflow_dry_run.py.

Recommended follow-ups (team call): file the unit gap + the L106 keep_tracers bug upstream at joezuntz/cosmosis-standard-library; migrate the PSF/xi_sys variant and glass-mock prep; tomographic support in both the converter and the shim's guard-relaxation.

— Claude (Fable) on behalf of Cail.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt

PRD #241 row 1. sacc>=2.4 joins core dependencies (sp_validation.sacc_io will be core library code). firecrown v1.15.1 + smokescreen 1.5.6 + an exact pyccl pin form the new [blinding] extra: the blind must be exactly recomputable from the seed at unblinding time, so the theory stack is pinned as a set. firecrown is not on PyPI and hard-depends on numcosmo-py (conda-forge only) plus the cosmosis/cobaya sampler connectors we never import — the new uv-overrides.txt drops those three from resolution; the Dockerfile and README carry the --overrides invocation. Plain installs without the blinding extra are unaffected. requires-python moves 3.11 → 3.12: Smokescreen and firecrown both set a 3.12 floor, and the container base (shapepipe:develop) already runs python:3.12-slim-bookworm — this aligns pyproject with the actual runtime. ruff target-version follows. Validated: uv resolution of '.[test,glass,blinding]' with overrides (235 packages, py3.12) and the full firecrown-core + smokescreen import chain in a fresh venv on candide. The CI image build on this branch is the container-side proof. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bGhVMAuhTy1gF6DdUc1dQ

Follow-up hardening after empirical validation of the blinding stack on candide. Two facts surfaced that the resolve-only check couldn't see: - firecrown 1.15.1 subclasses npt.NDArray (DataVector); numpy 2.5 made that a non-subclassable typing alias, so firecrown breaks at import on the numpy>=2.0 resolution (2.5.1). The [blinding] extra now carries numpy>=2.2,<2.5 (2.4.3 verified against the full compiled stack + the fast suite), and the Dockerfile requests the bound explicitly to dodge uv #8410 non-movement of an already-installed numpy. - pip-installed firecrown hits NumCosmo (conda-forge-only) at import time through two paths unrelated to cosmic shear: eager re-export of LSST predefined n(z) bins in generators/__init__ (defeating upstream's own lazy __getattr__), and the cluster likelihoods -> lsstdesc-crow -> Ncm.IntegralND C-subclass at module load. scripts/patch_firecrown.py makes the bin re-export lazy, the cluster imports optional, and ships a loud numcosmo_py shim (raises on any real use). Exact-string surgery against pinned v1.15.1, idempotent, fails loudly on a version bump, ends with an import check. Verified: fresh application on a pristine install, idempotent re-run, and the full toy Smokescreen blind end-to-end after patching. CI now also smoke-tests the blinding imports in the built image - the fast suite never imports firecrown, so a broken blinding stack would otherwise ship green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019bGhVMAuhTy1gF6DdUc1dQ

Add sp_validation.sacc_io: the writer/reader layer for the two-file SACC layout that becomes the package's standard data-product format. {version}.sacc analysis vector — NZ tracers, coarse xi+/-, pseudo-Cl (EE/BB/EB) with a shared BandpowerWindow, COSEBIs, pure E/B, rho/tau PSF diagnostics; one FullCovariance assembled block-diagonally (zero cross-blocks). {version}_xi_fine COSEBIs/pure-EB integration input — same NZ tracers, fine-grid xi+/-, DiagonalCovariance from TreeCorr varxip/varxim. Covariance order is point-insertion order (SACC preserves it bitwise through FITS). Writers insert in the canonical order — xi+ then xi-, Cl (ee, bb, eb), COSEBIs (all En then all Bn), pure E/B in _EB_KEYS order (xip_E, xim_E, xip_B, xim_B, xip_amb, xim_amb, matching b_modes.calculate_eb_statistics), rho, then tau — but readers never assume global order: every getter resolves indices through s.indices(dtype, tracers, **tags). assemble_covariance validates that blocks are contiguous, ascending and tile the data vector exactly, failing loud otherwise. Custom data types (pure E/B, rho, tau) all parse under sacc.parse_data_type_name. Tag filters are plain kwargs; the tags={...} form silently selects nothing and is never used. Test suite (test_sacc_io.py, all synthetic and fast): per-writer round-trips (arrays/tags/windows/NZ bitwise), covariance block alignment and zero cross-blocks, assemble_covariance failure modes, DiagonalCovariance round-trip, extract() sub-covariance alignment, a tomographic multi-pair case, reader/writer mirroring on a mixed file, and the end-to-end two-file layout. 20 passed. Co-Authored-By: Claude Opus <noreply@anthropic.com>

Fresh-eyes review caught a correctness bug: readers re-sorted selections by theta/ell/n, but covariance blocks and bandpower windows stay in insertion order. On a non-ascending grid the reader output silently desynchronised from its covariance, and get_pseudo_cl returned sorted cl arrays against unsorted window columns — internally inconsistent within one return tuple. Fix by construction, not by sort: - Writers validate their grids. add_xi/add_pure_eb/add_rho/add_tau require strictly ascending theta; add_pseudo_cl requires strictly ascending ell_eff (add_cosebis is inherently safe — it enumerates the mode index). Out-of-order grids raise a loud ValueError naming the argument. - Readers drop the sort entirely and return in s.indices (insertion) order, so every getter is covariance- and window-aligned for ANY file, and ascending for canonical files. The _sorted_* helpers are replaced by plain insertion-order accessors (_mean/_tag). Also: - _pair normalises (i, j) -> sorted, so get_xi(s, (1, 0)) addresses the same symmetric shear-shear pair as (0, 1) instead of a silent empty read. - Module docstring documents the tomographic ξ covariance ordering: insertion is pair-major ([pair0 xip; pair0 xim; pair1 xip; …]), supplied to assemble_covariance as one contiguous block matching add_xi call order; type-major converters (DES 2pt-FITS) permute explicitly via s.indices. - extract() docstring states tracers takes SACC names, not integer bins. New tests (26 total, was 20): writers reject non-ascending theta/ell; (1, 0) == (0, 1) round-trip; a 3-pair tomographic ξ covariance assembled as one contiguous pair-major block with per-pair sub-blocks recovered via extract(); get_pseudo_cl window column j <-> ell_eff[j] via window_ind tags. Co-Authored-By: Claude Opus <noreply@anthropic.com>

Add sacc_writers.py: pure per-statistic *_to_sacc functions (xi coarse/fine, pseudo-Cl, COSEBIs, pure-E/B, rho/tau) that turn computed arrays into single-statistic SACC "parts", plus assemble_analysis_sacc, which rebuilds the single {version}.sacc analysis file with a FullCovariance assembled block- diagonally in canonical order (per the SACC layout contract — not concatenate_data_sets, whose BlockDiagonalCovariance the contract rules out). Add bandpower_window_from_workspace to pseudo_cl.py: the NEW plumbing that threads NaMaster's get_bandpower_windows() into sacc_io.add_pseudo_cl (the EE/BB/EB diagonal window is verified identical). Tests: 12 fast tests, each writer round-tripped and checked against the contract; a real small-nside NaMaster window round-trip; analysis assembly proven to cover every point with aligned blocks and zero cross-blocks, incl. the reload-from-disk DAG path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…sis_fitting Add sp_validation.twopoint_convert.sacc_to_twopoint_fits, converting an analysis SACC into the CosmoSIS 2pt-FITS that 2pt_like (and Sacha's rho/tau 2pt_like_xi_sys fork) reads. Output reproduces today's cosmo_inference cosmosis_fitting.py assembly HDU-for-HDU and byte-for-byte: NZDATA, XI_PLUS/MINUS, CELL_EE, the blocked COVMAT (STRT_i offsets) and separate COVMAT_CELL, and the TAU_{0,2}_PLUS + verbatim RHO_STATS tables. The data vector and covariance are permuted from SACC's pair-major order to the 2pt-FITS type-major layout via s.indices. The tau covariance is laid in as one contiguous [tau_0+; tau_2+] block, preserving the tau_0<->tau_2 cross-correlation that covdat_to_fits carries. Rho/tau HDUs need the rho/tau sidecar FITS: the RHO_STATS varrho_* variance columns are not stored in the analysis SACC, so the converter copies them verbatim (never fabricates them); xi, Cl, n(z) and the covariance are fully reconstructed from SACC alone. Tests: - test_twopoint_convert.py: byte-equal vs current cosmosis_fitting.py builders on deterministic synthetic inputs for all three product shapes (plain xi; xi+Cl; xi+rho/tau), plus the tau_0<->tau_2 cross-correlation and perturbation teeth. - test_twopoint_convert_realdata.py (skipif on candide paths): build a SACC from a real product's own contents, convert, byte-compare vs the current writer. Observed byte-equal for SP_v1.4.6_leak_corr and a glass_mock sibling; the stale on-disk files differ only by an extra CELL_BB HDU (older script version) and ~1e-17 float noise on bin edges, documented and guarded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add sp_validation.one_covariance_io, the two small pieces of glue between the SACC layout and OneCovariance (which knows nothing of SACC and only exchanges files): - write_nz / nz_table / nz_config_stanza: the source_i NZ tracers of an analysis SACC -> OneCovariance's combined whitespace n(z) file (column 0 = shared z grid, one n(z) column per tomographic bin, no bin edges) plus the [redshift] config stanza pointing at it. Config keys verified against upstream OneCovariance config.ini (zlens_directory/zlens_file/value_loc_in_lensbin); the UNIONS template (cosmo_val/pseudo_cl.py._modify_onecov_config) uses the z_directory alias, exposed via dir_key. Fails fast on missing bins or disagreeing z grids. - covariance_blocks: OneCovariance's flat covariance_list_*.dat output -> dense square block(s) paired with SACC selectors for sacc_io.assemble_covariance, reusing statistics.cov_from_one_covariance for the per-block reshape (col 10 gaussian / col 9 gauss+non-gaussian). Single-block and multi-block (tomography-ready) forms. Tests (test_one_covariance_io.py, 9 passing): reshape to a hand-built matrix + gaussian/gauss+ng column selection + perturbation teeth; blocks feed assemble_covariance and round-trip s.covariance.dense; n(z) write/read round-trip, UNIONS dir_key, header/no-header, and fail-fast on mismatched grids / missing bin / bad value_loc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The theoretical tau covariance is not (6·nbin)² over [τ+; τ−]. Read from the write-side (shear_psf_leakage.rho_tau_cov.CovTauTh.build_cov): it is a (3·nbin)² k-major matrix over {τ0, τ2, τ5} with plus/minus folded into one component per index — exactly the flavor today's CosmoSIS chain consumes via covdat_to_fits. rho_tau_to_sacc now scatters that plus-only block into the τ-plus rows/columns of the 6·nbin τ block (per-k [+;−] insertion order), leaves τ-minus a vartau diagonal, and keeps plus↔minus cross zero. The kwarg is renamed tau_cov -> tau_cov_th to name the flavor. tau_cov_th=None keeps a compact DiagonalCovariance placeholder. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…CC part calculate_pseudo_cl_map/_catalog already call pseudo_cl_to_sacc_part (from the prior commit); this defines it — building the EE/BB/EB part via pseudo_cl_to_sacc(nz, meta, ell_eff, cl_all, wsp) and saving it as the native pseudo_cl_{ver}.sacc product. Drops the legacy save_pseudo_cl FITS writer (no external callers) and the unused _NMT_* constants. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…tics mixins Thin *_to_sacc_part methods that turn each already-computed diagnostic into its single-statistic SACC part via the born-as-SACC writers: - cosebis.py: cosebis_to_sacc_part picks the fiducial scale cut's {En,Bn,cov} (find_conservative_scale_cut_key, else the widest cut — mirroring plot_cosebis) and writes it; the multi-cut .npz sidecar (the PTE scan) is untouched. - pure_eb.py: pure_eb_to_sacc_part writes the six PURE_KEYS blocks at gg.meanr with the results['cov'] block. - psf_systematics.py: calculate_rho_tau_stats now writes a rho_tau_{base}.sacc part per version from the handler tables, passing cov_tau_{base}_th.npy as tau_cov_th when present (the tau-plus CovTauTh inference block), else a loud diagonal-placeholder fallback. nz/metadata come from the shared core helpers sacc_nz / sacc_metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…mble) - run_2pcf.py: after calculate_2pcf, write {ver}_xi_coarse.sacc via xi_to_sacc(grid='coarse', theta_nom=rnom, npairs, weight) — no covariance (added at assembly). Drops the save_fits threading (mixin no longer writes DES-style xi FITS). - run_2pcf_highres.py: on rank 0, write the terminal {version}_xi_fine.sacc via xi_to_sacc(grid='fine', variances=[varxip; varxim]) as a DiagonalCovariance. Deletes write_xi_fits (DES-FITS); keeps the .txt. sacc_io import works on the bare-host MPI path (no healpy), n(z) read from shear.redshift_path. - generate_pseudo_cl.py: native product is now pseudo_cl_{ver}.sacc (the mixin writes it); report reads it back via sacc_io.get_pseudo_cl. - assemble_sacc.py (new, dual-mode): loads the per-statistic parts in canonical order and calls assemble_analysis_sacc. Injects the xi-coarse block from the CosmoCov .txt (already [xi+;xi-]-ordered) and the pseudo-Cl block from the NaMaster iNKA cov FITS as block-diagonal [EE_EE;BB_BB;EB_EB] (cross-spectrum blocks dropped, TODO flagged). --allow-placeholder attaches a documented diagonal for cov-less parts so DAG dry-runs / tests produce a valid FullCovariance; production requires the real cov inputs (fail-fast otherwise). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ather Adversarial-review hardening (one HIGH, one MEDIUM, one coverage gap): - HIGH: n_bins>1 silently truncated — n_bins drove only the NZDATA column count while data/covariance were read from bin (0, 0), so a 2-bin SACC emitted a plausible-looking FITS carrying 1/3 of the data. The converter now fails fast unless n_bins == 1 and the ξ tracer pairs are exactly {(source_0, source_0)}; tomographic emission lands with the tomographic round. - MEDIUM: a ξ-less SACC wrote an empty XI_PLUS and a (0, 0) COVMAT silently; now a loud ValueError. - Coverage: every prior covariance test exercised the identity permutation (single-pair SACC order is already type-major). A (row, col)-encoded covariance test now pins the exact np.ix_ gather of COVMAT (block-diag ξ + joint non-adjacent [τ0+; τ2+]) and COVMAT_CELL — any transposition, offset, or swapped block fails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Ko89uZF84ez6HEDcTWbJe

…t_config The migrated pseudo-Cl SACC part calls get_redshift(), which reads cc[version]['shear']['redshift_path']; the synthetic fixture wrote the dndz file but never pointed that key at it (KeyError at core.py:422). Verified: the KeyError layer is resolved; the end-to-end test now fails one layer deeper (test asserts the legacy ELL/EE/BB/EB FITS columns while the migrated writer emits the SACC part - the open wiring seam). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Ko89uZF84ez6HEDcTWbJe

…-flight Working-tree state of the wiring subagent when it died on the session limit (its final turn also went unreviewed by the safety classifier). Wires cosmo_val.smk/twopoint.smk rules and cv_* scripts to the SACC parts + assemble_sacc rule. NOT dry-run-tested (no snakemake in the shared venv), NOT reviewed; test_calculate_pseudo_cl_catalog_end_to_end still fails at the writer/consumer seam (test reads legacy ELL columns). Next session: verify this diff against the DAG design in the PR-4 plan, reconcile the pseudo-Cl end-to-end test with the born-as-SACC output, add writer-path tests, run the suite, then de-WIP. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Ko89uZF84ez6HEDcTWbJe

… test calculate_pseudo_cl_catalog is born-as-SACC (pseudo_cl_to_sacc_part writes EE/BB/EB + a shared bandpower window), so the end-to-end test can no longer read legacy ELL/EE/EB/BB FITS columns. Round-trip through sacc_io.get_pseudo_cl instead and assert the bandpower window rides the part per the layout contract. The pinned golden spectra are unchanged (identical computation, SACC serialization); they held bitwise through the round-trip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

… parts The born-as-SACC parts existed but nothing assembled them. Wire the DAG: per-statistic parts (xi_coarse, pseudo_cl, cosebis, pure_eb, rho_tau) → assemble_sacc rule → terminal {version}.sacc. - rule xi: give the coarse ξ± .sacc output the same reporting-binning wildcards as the .txt (Snakemake requires one wildcard set per rule; the bare {version}_xi_coarse.sacc name left the binning wildcards unbound and broke DAG resolution). run_2pcf.py writes the part to the declared output path. - rule rho_tau_stats: declare the rho_tau .sacc part (already written by calculate_rho_tau_stats) as a real output; run_rho_tau.py verifies it. - new rule assemble_sacc + assemble_sacc_all: load the five parts in canonical order via assemble_sacc.py, injecting a documented diagonal placeholder for the cov-less ξ/pseudo-Cℓ blocks (real CosmoCov/NaMaster covariance plugs into the same --xi-cov/--pseudo-cl-cov seam later — PR-3 converter territory). - cosmo_val_all: request the terminal {version}.sacc per version. - cv_pseudo_cl.py docstring: born-as-SACC, no longer FITS. Dry-runs clean for assemble_sacc_all, cosmo_val_all, bmode_summary.json, and the fine-grid xi_fine target. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

… seam The pure assembler is covered in test_sacc_writers; this pins the DAG-facing script that loads per-statistic .sacc part *files* and rebuilds {version}.sacc: canonical block order across all five statistics, the diagonal placeholder for the cov-less ξ± part, real CosmoCov ξ covariance .txt injection, the fail-loud path when a required covariance is absent, and the pseudo_cl config toggle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

The MPI xi_highres rule shells out to run_2pcf_highres.py by absolute path through the deprecated pure_eb/ compat symlink (it can't use Snakemake's script: directive — mpiexec wraps the python call). Anchor the path on common.py's own location (WORKFLOW_SCRIPTS), which resolves to the generic workflow/scripts of whatever checkout parses the DAG, regardless of which paper composes it — workflow.basedir is unreliable here (under module composition it reflects the composing paper, verified: it resolved to papers/cosmo_val/scripts). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

… real cov The analysis {version}.sacc must be byte-comparable against today's cosmosis_fitting.py assembly (the SACC layout contract's stated requirement for PR-3's converter). cosmosis consumes the tagged, blinded pseudo-Cl product (blind=A, powspace, nbins=32 per config harmonic.fiducial), not the untagged cv_pseudo_cl diagnostic — so assemble_sacc now inputs the tagged part and injects its real NaMaster covariance from the matching pseudo_cl_cov FITS (COVAR_EE_EE/BB_BB/EB_EB → block-diagonal). The untagged cv_pseudo_cl part stays the cv_summarize_bmodes B-mode diagnostic, unchanged. The ξ± coarse block keeps a documented diagonal placeholder: its real CosmoCov covariance is blind/gaussian/mask-keyed in the inference tree, and wiring it couples cosmo_val to the whole inference covariance DAG — that sourcing is PR-3 converter territory, ready at the --xi-cov seam. test_assemble_sacc: add the pseudo-Cl COVAR-FITS injection path (the live default) alongside the existing ξ .txt and placeholder paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

…OFILE test_bmodes_workflow_dry_runs invoked a bare python3.12 (resolves off PATH to intel-python without snakemake) and inherited the login shell's slurm SNAKEMAKE_PROFILE (forces an executor plugin the test env need not have). Use sys.executable (the interpreter pytest/snakemake live in) and drop the profile for the dry-run. The guard now passes and exercises the migrated rule graph (rule xi's coarse-sacc output, the WORKFLOW_SCRIPTS path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

…red to PR 7 The SACC migration removed the xi_plus/xi_minus FITS and switched pseudo_cl to .sacc, so inference_prep's DAG no longer resolves. It is not reachable from the cosmo_val suite (cosmo_val_all never requests it), so the cosmo_val DAG stays clean. Per the PR-4 scope this subsystem is left dormant — a comment block at the rule head names the stale inputs and states PR 7 (native-SACC inference consumption) rewires it to consume the assembled {version}.sacc directly, retiring cosmosis_fitting.py's per-product FITS assembly. Comment-only; no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

…collision Both pseudo-Cl rules call calculate_pseudo_cl, which hardcoded the untagged native pseudo_cl_{ver}.sacc; generate_pseudo_cl.py then renamed it to the tagged name. In one cosmo_val_all DAG both fire: if the untagged cv_pseudo_cl (blind= None) ran first, the tagged rule's (blind=A) skip-if-exists silently adopted the blind=None file and the rename deleted cv_pseudo_cl's declared output — rebuild loops, and wrong-blind n(z) stamped into the terminal {version}.sacc's pseudo-Cl part. Thread out_path through calculate_pseudo_cl so each part is born directly at its final declared name (tagged for the producer, untagged for the diagnostic). No shared native basename, no rename; skip-if-exists keys on the declared path, so the two rules' paths are provably disjoint and the blind that computes each file matches its name. Multi-version + out_path now raises. generate_pseudo_cl.py and its CLI updated to pass the declared/native out_path directly. Regression tests: out_path is honoured (native name untouched) and the multi-version guard fires. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

…covariance {version}.sacc is the terminal science file; shipping it with a var=1.0 diagonal as the LEADING (ξ±) covariance block — ~20 orders off the real variance — is a silent catastrophic χ²/PTE for any consumer. Drop the unconditional placeholder_var=1.0. Default: no real ξ-cov wired → assemble_sacc.py's existing ValueError fires. The placeholder is now gated behind an explicit opt-in (cosmo_val.allow_placeholder_cov: true), for dry-run / test configs only. The pseudo-Cℓ block stays real (pseudo_cl_cov input); the integration tests pass placeholder_var explicitly and are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

test_bmodes_workflow_dry_runs uses papers/bmodes, whose config has no cosmo_val block, so cosmo_val.smk (the born-as-SACC + assemble rules) is never included — the guard couldn't catch a break in them. Add test_cosmo_val_workflow_assemble_ dry_runs targeting assemble_sacc_all in papers/cosmo_val (the only paper that includes cosmo_val.smk): asserts the DAG resolves and each assemble_sacc job pulls the tagged, blinded pseudo-Cl part + its NaMaster cov (not the untagged diagnostic) plus all five per-statistic parts. Shared _dry_run helper factored out of the bmodes guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

…f call run_xi_sweep.py called run_2pcf(..., save_fits=True) — a kwarg the SACC migration removed, so every invocation TypeErrors. The sweep consumes only the .txt dump; drop the kwarg. run_2pcf is born-as-SACC, so give its coarse part a grid-qualified sacc_out (the default {ver}_xi_coarse.sacc carries no binning, so the reporting + integration grids would collide per version). Stale "+ ξ+/ξ- FITS" docstring dropped. test_cli_seams: bind the sweep's exact run_2pcf call against the live signature (and assert save_fits no longer binds) so this CLI seam can't silently rot — the compute is cluster-only and never exercised by the fast suite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

_from_snakemake built part_paths with a hasattr filter, so a typo'd input keyword (cosebi for cosebis) silently omitted that statistic from the terminal {version}.sacc — the exact silent-truncation class this series has been bitten by. assemble_sacc now takes an `expected` list (the statistics the caller wired, from its config toggles) and raises if any is missing from part_paths or names a non-CANONICAL statistic. The rule derives expected from cv_assemble_inputs so it tracks the include_pseudo_cl toggle. CLI path is already typo-safe (argparse rejects unknown flags), so it passes expected=None. Tests: typo'd input key raises; typo in the expected list itself raises. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPfbe2XVTGzbnPn4g7BdN6

Adopt CosmoSIS's native SACC likelihood for the ξ± inference path through a thin sp_validation-owned subclass that fixes two upstream defects: 1. arcmin→rad theta-tag conversion — upstream sacc_like builds its theory spline on block[section,"theta"] (radians) but evaluates it at raw SACC theta tags (arcmin), silently returning theory≈0 and collapsing χ². 2pt_like converts to radians for exactly this reason; sacc_like never does. 2. an ordering guard — the data vector (get_mean, insertion order) and the theory loop (get_data_types × tracer_combinations × points) agree only for type-major files; a pair-major tomographic file would silently misalign. The guard reconstructs the theory-loop order and requires it == arange. Factory pattern (setup imports the upstream class from csl_dir and subclasses at call time) so importing the module never needs cosmosis; not imported by __init__. build_data calls super() first (scale cuts run in arcmin, matching 2pt_like's angle_range grammar) then converts + guards. Tests (cosmosis + CSL_DIR gated, skip cleanly otherwise) prove in-process equality against 2pt_like on the PR-3 converter FITS: machine-zero equality on synthetic (χ²=1.18829133107, Δχ²=0) and real data (χ²=420310.753739, Δχ²=0, N=40), the unit-gap tripwire (raw sacc_like 631× off), scale-cut equivalence, identical perturbation response, the ordering guard, and real-type-scoped theta conversion. pyproject: cosmosis>=3.25 extra (inert in CI, which installs [test,glass,blinding]). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt

Add the CosmoSIS pipeline ini template for the native SACC path (cosmosis_pipeline_A_ia_sacc.ini): copies the A_ia template, swapping load_nz_fits→load_nz_sacc (native n(z) from the SACC's source_i NZ tracers) and 2pt_like→sacc_like (the shim, via SP_VALIDATION_MODULES, csl_dir=COSMOSIS_DIR, like_name=2pt_like for block-key parity). Same numeric scale cuts as A_ia, in the sacc_like angle_range grammar (full data-type names + source_0 pairs). generate_inference_config.py fills a template's [DEFAULT] section with concrete paths (SCRATCH, FITS_FILE|SACC_FILE, COSMOSIS_DIR, SP_VALIDATION_MODULES resolved from sp_validation.__file__). Dual-mode (snakemake object OR argparse CLI) like assemble_sacc.py; plain text processing (the pipeline.sh sed idiom) that REPLACES an existing DEFAULT key in place and prepends only new ones — no configparser round-trip (which would strip comments + %(...)s interpolation) and no duplicate DEFAULT key (which CosmoSIS's strict parser rejects). Tests (no cosmosis needed): both templates fill + interpolate cleanly, the referenced module file exists, no %(...)s placeholder survives, missing-DEFAULT raises. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt

Revive the real-data inference_prep from its dormant, pre-SACC state. It now consumes the assembled analysis {version}.sacc (cosmo_val.smk's assemble_sacc, bound lazily through cv_analysis_sacc) and emits the A_ia (IA-only, ξ±) file-prep products: (a) the converter 2pt-FITS (sacc_to_twopoint_fits, pure ξ — no rho/tau sidecars, A_ia scope) + a generated 2pt_like ini (the validating/legacy path), and (b) a generated sacc_like ini pointing at the SACC (the native path, validated bit-for-bit against (a)). The cosmosis_fitting.py real-data assembly is retired from this rule; the glass-mock rules keep it (their SACC migration is out of scope, noted inline). inference_fiducial extends to all three prep outputs. CSL_DIR is read lazily (inference.smk is parsed by every paper workflow, but only cosmo_val carries inference.csl_dir) — a missing key still fails loudly, just at DAG time rather than at parse time for an unrelated (bmodes) workflow. config.yaml: add inference.csl_dir. The workflow dry-run guard gains a test covering the revived inference_prep DAG (SACC in, converter FITS + both engine inis out, no cosmosis_fitting.py real-data assembly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt

The merged layer order ran the [blinding] editable install before COPY ., so uv-overrides.txt (and the source tree) did not exist in the build context. The blinding install is itself 'uv pip install -e .[blinding]', so the old final --no-deps editable layer is redundant and is dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01766vawzi2XqrgoyHmeHEY9

The feat/sacc-3 merge brought the guard-era sacc_io whose save() now requires a keyword-only `type={'data','mock'}` provenance tag (PRD #241 §4) and whose load() refuses unblinded `type='data'`. Application code and tests from the pr4/pr7 lineage still called the old `save(s, path)` and referenced a template file the #236 cleanup had relocated, leaving 21 CI failures outside the merge-conflict set. This adapts the callers to the reviewed contract (sacc_io is unchanged). save() type= threading: - CosmologyValidation gains a `sacc_type` ctor kwarg (default "data"), stamped by every part-writer: cosebis/pure_eb/psf_systematics/pseudo_cl now `save(..., type=self.sacc_type)`. - assemble_sacc inherits provenance from its parts: `type=metadata["type"]`. - run_2pcf uses `cv.sacc_type`; run_2pcf_highres stamps "data" (real-catalogue fine covariance). - Synthetic-data tests adopt the reviewed idiom `type="mock"` (loads freely): test_sacc_writers _roundtrip + reload helper, test_assemble_sacc part writer, and the test_pseudo_cl fixture (`sacc_type="mock"`). Relocated FITS engine template: - The #236 folder cleanup moved cosmosis_pipeline_A_ia.ini into cosmosis_config/templates/ (still used by the legacy cosmosis_fitting.py path). The new pr7 Snakemake regime — inference.smk's INFERENCE_TEMPLATE_DIR, generate_inference_config, and its tests — resolves templates from cosmosis_config/ directly, where pr7 placed A_ia_sacc.ini but never copied its FITS sibling. Restore A_ia.ini alongside the sacc template. Fast suite: 254 passed, 1 skipped. Two remaining failures (test_calculate_pure_eb pinned-value drift; test_configured_paths_exist_on_candide) are pre-existing and environmental — they fail identically with these changes stashed, and neither is in the API-drift set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01766vawzi2XqrgoyHmeHEY9

Bring PR #255 current with its heavily-reworked base. The base restructure wins where it touched shared ground; PR 7's own additions ride on top. Base state adopted: - xi grids renamed coarse->reporting, fine->integration; the integration part stays a per-version intermediate and is NOT gathered into the terminal {version}.sacc (#247 ruling). - Analysis covariance is a BlockDiagonalCovariance via sacc_io.assemble_covariance (not FullCovariance / concatenate_data_sets). - add_cosebis(s, bins, En, scale_cut, Bn=None); cv_cosebis/cv_pure_eb re-derive E-modes from the integration/reporting parts via the b_modes seams (en_override / eb_override). - Fail-closed allow_unblinded gate in assemble_sacc.py; sacc_io.save(type="data"). PR 7 additions preserved: - inference.smk inference_prep revived LIVE: consumes the assembled {version}.sacc and emits the converter 2pt-FITS + both engine inis (2pt_like, sacc_like). - sacc_io converters (sacc_to_twopoint_fits, write_nz, covariance_blocks) and the Converters docstring paragraph kept alongside the base's Optionality paragraph. - sacc_like_unions shim and test_sacc_like; test_cosmo_val_inference_prep_dry_runs kept on top of the base's _xi_reporting_ assemble assertion. Reconciled test_sacc_like to the base's add_cosebis keyword-Bn signature. Tests: 258 passed, 1 skipped (not slow); only the known off-fiducial test_configured_paths_exist_on_candide env failure remains. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD

# Conflicts: # src/sp_validation/cosmo_val/core.py # src/sp_validation/sacc_io.py

# Conflicts: # workflow/rules/inference.smk

…s; PR-7 live rule uses it)

The Smokescreen-fork replan made the blinding deps core (fork + pyccl + cryptography + sacc, pinned via uv.lock); firecrown is gone from the project, so everything this branch still carried for it is dead weight: - delete scripts/patch_firecrown.py and uv-overrides.txt - revert the Dockerfile to the base (uv sync from lock + editable install) - drop the CI blinding-stack import smoke test (the fast suite in the image already covers the blinding path) - drop the [blinding] extra and the firecrown-flavoured python-floor comment from pyproject; drop the override/patch recipe from the README Also trims PR archaeology from the new comments and docstrings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014hSQTC6WwTH1p9w4FuKJGz

…stack)

inference.smk resolved wholly in favour of this branch's rewrite; #251 no longer touches that file.

The dormant inference subsystem was being half-edited here (inference_prep deleted, comments trimmed, pseudo_cl_tag adopted) while #255 rewrites it wholesale. Restored to its merge-base-with-develop state; both paper workflows still dry-run.

…251) * feat(sacc_io): SACC read/write for the standard data-product layout Add sp_validation.sacc_io: the writer/reader layer for the two-file SACC layout that becomes the package's standard data-product format. {version}.sacc analysis vector — NZ tracers, coarse xi+/-, pseudo-Cl (EE/BB/EB) with a shared BandpowerWindow, COSEBIs, pure E/B, rho/tau PSF diagnostics; one FullCovariance assembled block-diagonally (zero cross-blocks). {version}_xi_fine COSEBIs/pure-EB integration input — same NZ tracers, fine-grid xi+/-, DiagonalCovariance from TreeCorr varxip/varxim. Covariance order is point-insertion order (SACC preserves it bitwise through FITS). Writers insert in the canonical order — xi+ then xi-, Cl (ee, bb, eb), COSEBIs (all En then all Bn), pure E/B in _EB_KEYS order (xip_E, xim_E, xip_B, xim_B, xip_amb, xim_amb, matching b_modes.calculate_eb_statistics), rho, then tau — but readers never assume global order: every getter resolves indices through s.indices(dtype, tracers, **tags). assemble_covariance validates that blocks are contiguous, ascending and tile the data vector exactly, failing loud otherwise. Custom data types (pure E/B, rho, tau) all parse under sacc.parse_data_type_name. Tag filters are plain kwargs; the tags={...} form silently selects nothing and is never used. Test suite (test_sacc_io.py, all synthetic and fast): per-writer round-trips (arrays/tags/windows/NZ bitwise), covariance block alignment and zero cross-blocks, assemble_covariance failure modes, DiagonalCovariance round-trip, extract() sub-covariance alignment, a tomographic multi-pair case, reader/writer mirroring on a mixed file, and the end-to-end two-file layout. 20 passed. Co-Authored-By: Claude Opus <noreply@anthropic.com> * fix(sacc_io): enforce ascending grids; readers in insertion order Fresh-eyes review caught a correctness bug: readers re-sorted selections by theta/ell/n, but covariance blocks and bandpower windows stay in insertion order. On a non-ascending grid the reader output silently desynchronised from its covariance, and get_pseudo_cl returned sorted cl arrays against unsorted window columns — internally inconsistent within one return tuple. Fix by construction, not by sort: - Writers validate their grids. add_xi/add_pure_eb/add_rho/add_tau require strictly ascending theta; add_pseudo_cl requires strictly ascending ell_eff (add_cosebis is inherently safe — it enumerates the mode index). Out-of-order grids raise a loud ValueError naming the argument. - Readers drop the sort entirely and return in s.indices (insertion) order, so every getter is covariance- and window-aligned for ANY file, and ascending for canonical files. The _sorted_* helpers are replaced by plain insertion-order accessors (_mean/_tag). Also: - _pair normalises (i, j) -> sorted, so get_xi(s, (1, 0)) addresses the same symmetric shear-shear pair as (0, 1) instead of a silent empty read. - Module docstring documents the tomographic ξ covariance ordering: insertion is pair-major ([pair0 xip; pair0 xim; pair1 xip; …]), supplied to assemble_covariance as one contiguous block matching add_xi call order; type-major converters (DES 2pt-FITS) permute explicitly via s.indices. - extract() docstring states tracers takes SACC names, not integer bins. New tests (26 total, was 20): writers reject non-ascending theta/ell; (1, 0) == (0, 1) round-trip; a 3-pair tomographic ξ covariance assembled as one contiguous pair-major block with per-pair sub-blocks recovered via extract(); get_pseudo_cl window column j <-> ell_eff[j] via window_ind tags. Co-Authored-By: Claude Opus <noreply@anthropic.com> * One SACC file per catalogue version: fine xi rides as grid='fine' points The two-file split (analysis + {version}_xi_fine.sacc) was premised on a 10000-bin fine grid; the production operating point (Paper II B-modes) is 1000 bins, where a dense per-pair fine covariance block is ~32 MB and the CosmoCov integration-binning covariance — which feeds pure-E/B and COSEBIs error propagation — has a natural home as a BlockDiagonal block alongside the analysis blocks. Layout test replaced with the one-file end-to-end case (dense fine block, extract() sub-covariance alignment, zero cross-blocks) plus a varxi-diagonal fallback test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019R5eiy11Lihkgn4MKufXSp * Word fine-block covariance source tool-agnostically (OneCovariance go-forward) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019R5eiy11Lihkgn4MKufXSp * feat(sacc_io): type=data|mock stamp with fail-closed load; merge + update_statistic PRD #241 §4 (Mocks vs data): save() requires type='data'|'mock' and stamps it into metadata; load() raises on type='data' files lacking the concealed=True blinding stamp, so skipping the blind can never silently expose real data. Mocks load freely. allow_unblinded=True is the loud escape hatch reserved for the blinding/unblinding tooling. merge() wraps sacc.concatenate_data_sets thinly: per-statistic files combine in order, shared tracers stored once, covariance block-diagonal (library-enforced all-or-none), metadata union with loud conflicts. update_statistic() is the value-only merge-back for the extract -> conceal -> merge blinding flow: matches each sub point by (data_type, tracers, tags) and overwrites the value, leaving order and covariance untouched. Tests: all four data/mock x concealed/unconcealed quadrants, the escape hatch, stamp validation, merge (points, covariance, metadata conflict, mixed-covariance failure) and update_statistic (values-only, unique-match). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QWF72ofwJh6ekgnCt9Xx6C * Address review: rename grid values to reporting/integration; clarify comments - grid tag values: coarse -> "reporting", fine -> "integration" (descriptive, not relative); tag kept because both grids share data type and tracer pair, so the tag is the sole disambiguator. Swept module docstrings and tests. - Module docstring now spells out why insertion order is load-bearing: covariance row/column i refers to the i-th inserted data point. - new_sacc: comment that psf_stars rides in the same tracer list as the NZ tracers only because a Sacc has one flat tracer namespace (bookkeeping, not physics). - source_name/_pair helpers kept (4 and 8 call sites) with one-line justifications of the conventions they centralize. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QWF72ofwJh6ekgnCt9Xx6C * Simplify sacc_io writers/readers and test builders - Factor the theta-tagged insertion loop into _add_theta_series (add_rho, add_tau, add_pure_eb); add_pure_eb zips PURE_TYPES.values() against its signature order, and PURE_KEYS is derived from PURE_TYPES instead of restating it. - Factor the (theta, plus, minus) read pattern into _get_pm (get_xi, get_rho, get_tau). - add_xi hoists the optional-tag None-filtering out of the point loop; extract and merge lose their throwaway mutable dicts; get_cosebis builds its scale-cut tags in one expression. - Tests: shared _add_xi default-ξ builder and _xi_block/_cl_block/ _cosebi_block canonical index-block helpers replace ~60 lines of copy-pasted setup; test_readers_on_mixed_file builds on _multi_statistic_sacc. Behaviour unchanged; 41/41 tests green in the container. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QWF72ofwJh6ekgnCt9Xx6C * fix(sacc_io): fail loud on unmatched selections; merge/update guards Adversarial review (pass 2) findings: - Sacc.indices returns an EMPTY array (warning only) on an unmatched selection; every reader, extract(), and covariance-block selector now funnels through a shared _indices guard that raises instead — a typo'd tag or non-bitwise-identical float scale cut can no longer propagate empty arrays downstream (assemble_covariance previously died with an opaque IndexError on the same path). - get_cosebis(scale_cut=None) on a file carrying several scale cuts silently concatenated them; now raises and asks for an explicit cut. - update_statistic let two sub points claim the same target point (last-write-wins); now raises. - merge: the library keeps the FIRST input's tracer on a name clash with no equality check; shared tracers are now verified identical (z, nz) across inputs before concatenation. 7 regression tests; 48 total. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzUt7VbtXwr2SCHUdiQTyt * Rebuild PR4 on feat/sacc-2-sacc-io: drop vendored sacc_io, keep migration The branch previously carried a stale vendored copy of sacc_io.py / test_sacc_io.py from before PR2's review rounds. Rebuilt directly on feat/sacc-2-sacc-io (8bd3817) so the canonical module is inherited, bringing over the cosmo_val + Snakemake born-as-SACC migration work from the old tip unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzUt7VbtXwr2SCHUdiQTyt * Migrate to canonical sacc_io API: typed save, fail-closed load, grid rename Sweep the migration code onto PR2's canonical vocabulary and contracts: - grid='coarse'/'fine' -> 'reporting'/'integration' everywhere (writer calls, readers, tests), including the internal DAG intermediates: {version}_xi_coarse* -> _xi_reporting*, _xi_fine -> _xi_integration, the xi_coarse/xi_fine Snakemake output keys, CANONICAL part names, cv_xi_reporting_sacc, write_xi_integration_sacc. - save(s, path) -> save(s, path, type=...): 'data' at every production writer (the cosmo_val pipeline measures the real UNIONS catalogues; GLASS mocks do not flow through these writers), 'mock' for synthetic test fixtures. assemble_sacc propagates its parts' type stamp rather than hardcoding, so mock parts assemble into a mock analysis file. - load(path) -> fail-closed load: pipeline-internal readbacks of freshly written pre-blind data parts pass allow_unblinded=True (blinding is a downstream Smokescreen step); mock fixtures load freely. Readers raising on unmatched selections needed no call-site changes: every get_* reads a statistic guaranteed present in the file just written or assembled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzUt7VbtXwr2SCHUdiQTyt * sacc_io: guard merge() against inconsistent theta grids Consistency follows tagging semantics: the grid tag declares which binning a set of points lives on, so all same-length theta arrays under one tag value must be bitwise identical (sacc never validates angles, and grids diverging at floating-point level choke CosmoSIS downstream). Different lengths within a tag group pass (scale-cut subsets); grids under different tag values are unconstrained (reporting vs integration differ by design). The rho/tau/pure-EB writers now tag their points grid="reporting" by default (overridable via grid=), so they join xi's consistency group and no untagged group appears in our own files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MN9VazXKHUHQg16kiG7Ufk * sacc_io: extend merge guard to ell grids and bandpower windows; tag pseudo-Cl grid="reporting" The theta consistency guard generalizes to both angular domains: theta and ell points are grouped separately by grid tag value, and within a tag value all same-length grids must be bitwise identical. Two ell series sharing a grid must also carry equal bandpower windows (window ells and weight matrix); series without windows skip that check. add_pseudo_cl now stamps grid="reporting" on every point by default (overridable), joining the merge guard's consistency groups; since sacc's add_ell_cl accepts no extra tags, its per-point insertion (ell + shared window + window_ind) is inlined. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MN9VazXKHUHQg16kiG7Ufk * Store covariance block-diagonally in assemble_covariance assemble_covariance now passes sacc.add_covariance a list of per-block matrices instead of a dense zero-filled N×N array. sacc.BaseCovariance.make turns a list into a BlockDiagonalCovariance (one FITS table per block, Σ block² on disk), so cross-blocks are zero and implicit rather than materialized. Validation (contiguous/ascending indices, no gap/overlap, square blocks matching their index span) is unchanged. Every existing consumer already read through the polymorphic .dense property, so only the type assertions needed updating (FullCovariance -> BlockDiagonalCovariance). Added a test that merging two files that already carry a BlockDiagonalCovariance (assembled via assemble_covariance) stays block-diagonal through merge and a save/load round-trip. Updated the module docstring's storage-cost discussion accordingly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Unify pure-EB integration-grid default to 1000 bins pure_eb.py's calculate_pure_eb/plot_pure_eb defaulted nbins_int=100; cosebis.py's calculate_cosebis already defaulted to 1000, and every production config (papers/bmodes, papers/cosmo_val fiducial/pure_eb blocks) already overrides to 1000. This aligns the function-signature default with what every caller actually uses; config plumbing is untouched, so any explicit override still wins. The papers/cosmo_val/config/config.yaml cosebis.nbins_int (currently 2000, production numerics) is deliberately left unchanged — see report. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * sacc_io: make optional statistic components genuinely optional Writers no longer force components an analysis may not have computed: add_pseudo_cl's BB/EB, add_cosebis's Bn, and add_pure_eb's B/ambiguous blocks now default to None and are simply not written when omitted (add_pure_eb still requires each +/- pair together). Structural arguments (grids, tracer/bin identifiers, the value for a component you ARE adding) remain required with no default. Composite readers (get_pseudo_cl, get_cosebis, get_pure_eb) return None / omit the key for an absent optional component instead of raising, via a new _mean_optional helper; a selection naming a missing component explicitly (s.indices, _mean, extract) still fails loud, per the existing empty-selection guard. merge() and assemble_covariance work unchanged on files with only a subset of components. Documents the optionality contract in the module docstring, and adds partial-file round-trip, merge, and explicit-selection-fails-loud tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * cosmo_val config: align COSEBIs integration grid with the 1000-bin default cosebis.nbins_int was 2000; every other integration-grid entry in this config (pure_eb, the fiducial block) is already 1000, and cosebis.py's own function defaults are 1000. Unify. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sacc): one terminal file + fail-closed assembly Fold the integration-grid ξ± into the single terminal {version}.sacc, and make part loading fail closed on unblinded real data. Integration ξ± (grid='integration') is no longer its own terminal product. The xi_highres part is now gathered by rule assemble_sacc into {version}.sacc as tagged points, next to the reporting ξ± block. It is fiducial-only (the 10k-bin MPI run emits only the fiducial part), so it joins the fiducial version's terminal file alone. assemble_sacc.py adds xi_integration to CANONICAL; its own DiagonalCovariance passes straight through. assemble_sacc.py no longer loads every part with allow_unblinded=True. The run type (data|mock, from config, default data) gates it: mock runs load freely, data runs fail closed unless a part carries the concealed=True stamp. This is the seam for PR #253's blind-at-birth — a concealed data part then assembles with allow_unblinded=False untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * docs(workflow): mark glass-mock A/B/C 'blind' as distinct from Smokescreen The pseudo_cl / pseudo_cl_cov rules and the shared BLINDS list use a 'blind' wildcard that is the glass-mock multi-catalogue A/B/C variant, not Smokescreen blinding. Add prominent comments at the BLINDS definition, the wildcard constraint, and both rules so the two axes are not confused. A full rename is avoided: 'blind' is baked into on-disk filenames we do not own (external nz_{version}_{A|B|C}.txt) and into the covariance / inference path builders. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * fix(sacc): reconcile writers with the merged sacc_io API Resolve two silent semantic conflicts from the base merge (no textual conflict, but the tests broke). - add_cosebis reordered its params to (s, bins, En, scale_cut, Bn=None). cosebis_to_sacc still passed the old positional order; call it by keyword. - assemble_covariance now builds a BlockDiagonalCovariance (one FITS table per block, validated ordering), not a dense FullCovariance. Update the assembler docstrings, the smk comments, and the assembled-file test assertions to the block-diagonal type. The assembled .dense is unchanged (cross-blocks are zero either way); single-part writer covariances stay FullCovariance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * chore(deps): restore develop's uv.lock SSOT; drop stale firecrown-era override machinery The branch had accidentally superseded #266's reproducible-deps model (uv.lock as source of truth, `uv sync --frozen` in the Dockerfile) with an older firecrown/Smokescreen-blinding scheme: uv.lock deleted, dependency resolution done via `uv pip install --overrides uv-overrides.txt`, and a patch script to make pip-installed firecrown importable without NumCosmo. Firecrown has been dropped from the project, so all of that goes. Restores develop's pyproject.toml, uv.lock, Dockerfile, and the deploy-image blinding-stack smoke test wholesale; removes uv-overrides.txt and scripts/patch_firecrown.py. Keeps one genuine PR4-driven change: tightens the sacc constraint to >=2.4,<3, since sacc_io.py (this branch) uses concatenate_data_sets/BlockDiagonalCovariance, both from sacc's 2.x rewrite (develop's lock already resolves sacc to 2.4; this just makes the pyproject floor honest). Drops the unused numexpr addition — no code in the tree imports it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * chore(ci): restore develop's lint.yml (PR #264 server-side ruff autofix) lost in stale merge * revert(sacc): drop xi_integration from terminal file per #247 ruling Keep the integration-grid ξ± as its own per-part intermediate ({version}_xi_integration.sacc) rather than folding it into the terminal {version}.sacc. Per the #247 ruling (comment 5033716753), the terminal file carries the analysis vector only; COSEBIs/pure-E/B consume the integration part directly, and Snakemake provenance covers its traceability. This keeps the terminal file at tens-of-MB scale. Removes xi_integration from CANONICAL, the cv_xi_integration_sacc helper, and the fiducial-gated assemble input. Keeps the fail-closed allow_unblinded gate, glass A/B/C comments, and dependency restorations from the prior rework. The integration part stays blinded at birth on data runs (per #253). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * chore: drop files resurrected from a stale base (deleted on develop) The "Rebuild PR4" base re-added the cosmo_inference/ analysis tree that develop removed in #236 (Clean up cosmo_inference folder); the develop merge kept them because a delete-vs-readd does not conflict. Remove the 80 resurrected files (notebooks, cosmosis_config .ini set, cosmo_inference scripts, get_chi2 notebooks, pipeline shells) to match develop. None is PR4 scope — PR4 is the SACC migration (workflow/ rules + scripts, sacc-related src + tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * chore(2pcf): default integration grid to 1000 bins The B-modes paper found no substantial 1k-vs-10k difference on the integration grid, and develop already unified COSEBIs/pure-EB to nbins_int=1000. Drop the run_2pcf_highres.py 10000-bin default to 1000 and reword the docstring/comments (the Asgari 10k figure becomes context, not the operative number). The grid is config-driven (nbins_int); the MPI path stays available but single-process is the default at 1000 bins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * feat(cosmo_val): cv_cosebis/cv_pure_eb consume the ξ± SACC parts; per-version xi_highres COSEBIs and pure-E/B now derive their E-mode outputs from the born-as-SACC ξ± parts on disk instead of only the raw catalogue recompute. This makes born blinding automatic downstream: whatever the part carries (blinded on data runs in PR #253) flows into the E-modes with no blind-aware consumer code. b_modes: add the values-only seams cosebis_from_xi and pure_eb_from_xi (copied from the sacc-6b spur, de-blinded) — same cosmo_numba kernels as the raw path. cosebis_to_sacc_part / pure_eb_to_sacc_part: add en_override / eb_override so the E-mode En (COSEBIs) and the six pure-mode arrays (pure-E/B) written to the SACC part come from the consumed part; Bn and the covariance stay blind-invariant from the raw estimator run. No concealment machinery (that is PR #253's). cv_cosebis.py / cv_pure_eb.py: keep the raw plot_* run (Bn + jackknife covariance need the catalogue and are blind-invariant), then load the integration part (COSEBIs) / reporting + integration parts (pure-E/B), re-derive the E-modes through the seams, and override both the SACC part and the diagnostic npz. pure-E/B reads the reporting-grid bin edges from the raw reporting gg (SACC stores centers only). Unconditional and version-agnostic. xi_highres: per version (was fiducial-hardcoded); in-container single-process at the config-driven 1000-bin grid (the 10k-bin bare-host MPI path is unnecessary). Grid comes from a dedicated cosmo_val.integration block ([0.08, 300] @ 1000) so the one part serves both consumers (pure-E/B full range, COSEBIs scale-cuts to 0.9); decoupled from covariance.smk's FIDUCIAL grid. Shared twopoint.smk falls back to the fiducial integration grid for configs without a cosmo_val section (e.g. papers/bmodes). The raw .txt byproduct is left undeclared to avoid an AmbiguousRuleException with rule xi; nothing in the DAG consumes it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD * Restore cosmo_inference/ to develop state (stale-base residue reverted newer develop edits) * One binning-agnostic xi rule (grid + covariance are config, not a second rule) The reporting and integration ξ± measurements were two rules and two drivers for one computation. Collapse them: rule `xi` measures any binning, files are named by that binning (.txt + .sacc), and the grid label + covariance treatment are resolved from the wildcards via XI_GRIDS (reporting → no covariance, the ξ block arrives at assembly from CosmoCov; integration → DiagonalCovariance from TreeCorr varxip/varxim). Workflows with no cosmo_val block fall back to their fiducial grids; an unnamed binning measures as plain reporting. run_2pcf.py gains --grid/--covariance and drops run_2pcf_highres.py entirely (its hand-rolled catalogue loader, cat_config resolver and bare-host MPI path were a second implementation of CosmologyValidation.calculate_2pcf). The SACC part now stamps the npatch actually measured. calculate_2pcf writes patch results + covariance only for npatch > 1, so the fine grid no longer serialises a dense (2*nbins)^2 block that nothing reads. Renames {version}_xi_reporting_<binning>.sacc and {version}_xi_integration.sacc to {version}_xi_<binning>.sacc; consumers go through cv_xi_sacc(version, grid). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014hSQTC6WwTH1p9w4FuKJGz * Restore develop files wrongly deleted by stale-base cleanup (realspace paper, image-sims, mask configs, candide profile) * Restore develop content wrongly reverted by stale base (.gitignore uv.lock, CONTRIBUTING lint gate, calibration/catalog modules) * simplify: trim comments to Google style; dedupe helpers; drop dead knobs Comment/docstring pass over the PR-4 diff: cut PR/issue archaeology, restated defaults and "identical/bit-matching" claims (not true under blinding), keeping the load-bearing contracts (canonical part order, placeholder-cov consequence, scale_cut semantics, TreeCorr bin edges, CovTauTh k-major plus-folded layout). Code: - common.py gains base_version() and pseudo_cl_tag(); generate_cosmocov_ini imports both instead of copying build_redshift_path, fixing the drift where the cov_th lookup stripped only _leak_corr while the n(z) path also stripped _ecut{N}. cosmo_val.smk / inference.smk share the one pseudo-Cl tag. - run_cosmocov_chain.sh anchors the checkout on its own location; container, bind list and cosmocov binary come from the environment. - assemble_sacc: drop the unused --pseudo-cl-cov-hdu knob (the three named HDUs are always written; missing ones now raise). - assemble_analysis_sacc seeds tracers + metadata from parts[0], retiring _n_source_bins and the nz/metadata arguments. - Covariance mode derives from the grid: XI_GRIDS loses its "covariance" field, xi_grid_of returns just the label and compares binnings numerically, and run_2pcf loses its --covariance flag. - inference.smk: drop the dead inference_prep / inference_fiducial rules. Also restores bandpower_window_from_workspace (deleted in error by f14774b) and develop's NON_PATH_KEYS overlay entries in test_config_paths_exist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014hSQTC6WwTH1p9w4FuKJGz * cv_cosebis: derive Bn from the integration part; unpatched integration path in claims Both COSEBIs values now come from the (blindable) integration ξ± SACC part through the cosebis_from_xi seam; only the jackknife covariance still comes from the raw patched measurement, where the patches live. papers/bmodes' integration ξ± path said npatch=FIDUCIAL['npatch']; the integration grid (XI_GRIDS, twopoint.smk) is unpatched, so pin it to 1. * test: make the COSEBIs override stub work under the blinded-part writer too * Leave inference.smk untouched: it is #255's to rewrite The dormant inference subsystem was being half-edited here (inference_prep deleted, comments trimmed, pseudo_cl_tag adopted) while #255 rewrites it wholesale. Restored to its merge-base-with-develop state; both paper workflows still dry-run. * Assemble against the real CosmoCov ξ± covariance; drop the placeholder The ξ± block now comes from the CosmoCov-processed covariance on the reporting binning, wired as a DAG input of assemble_sacc (cv_xi_cov), so requesting the terminal file builds the covariance chain. With a real block always present, the allow_placeholder_cov switch and the diagonal stand-in are gone: a missing covariance is a missing Snakemake input, and a cov-less part with nothing to inject still raises. The tests use a CosmoCov-format .txt fixture written with the same writer the seam reads. * covariance_process: run the launched checkout's cosmocov_process.py The rule shelled out to an absolute path through the deprecated pure_eb symlink, i.e. a different checkout on an unrelated branch — now on the critical path, since assemble_sacc depends on this rule. REPO_ROOT (common.py) anchors on the module's realpath, the same way the rest of the workflow resolves the running checkout under `module` composition. * Leave cosmo_inference/README.md untouched (inference is out of scope here) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NmGA86b7YyQn54JFj78ssM * Restore image_sims include + README section (out of scope here) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NmGA86b7YyQn54JFj78ssM * README: state the 3.12 requirement, not its rationale Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NmGA86b7YyQn54JFj78ssM * Comment sweep: each concept once, at the thing itself Kills repeats (the same grid/naming/canonical-order explanation restated at producer, rule and script) and comments that narrated what downstream code does — consumer names, rule wiring, pipeline shape — which rot as soon as the consumer moves. What stays is each thing's own contract plus the invariants a reader would violate: the inclusive scale-cut selection, the strict sub-range the pure-E/B integration grid must satisfy, why the integration grid carries no covariance, why an already-written part may be read back unblinded, and the covariance ordering assembly depends on. Also drops the stale rename note in generate_pseudo_cl (parts are born at their final path) and the duplicated REPO_ROOT anchoring rationale in common.py. * Each grid carries its own covariance; COSEBIs derives from ξ± arrays A grid is now a binning plus how its covariance is estimated, so one TreeCorr run per grid produces a part complete enough to work from: the dense jackknife block where there are patches, the varxip/varxim diagonal, or nothing. The COSEBIs scan gains an arrays-and-covariance seam (cosebis_scan_from_xi): the COSEBIs covariance was already the ξ± covariance through the same linear kernel as the modes, so a caller holding a data vector needs no estimator re-run — only the realisation count behind the covariance, for Hartlap. calculate_cosebis delegates to it, and the scale-cut heatmap takes bin edges rather than a correlation object; log_bin_edges reconstructs those edges from a binning, which is what a part-based consumer has. cv_cosebis becomes such a consumer: one part in, and the SACC part, the .npz scan and the figures all out of those values. The en/bn_override machinery it needed while values and covariance came from different runs goes with it. * cv_cosebis: bind the COSEBIs grid's part, and declare the figures The rule's inputs are now one ξ± part instead of a TreeCorr .txt plus the integration part, and the three companion figures are declared outputs rather than byproducts landing beside them. The COSEBIs grid (0.9–300 at 1000 bins, patched) is a row in the grid table, so the same xi rule measures it. * Pure E/B derives from its parts; per-patch vectors are never written The pure-E/B jackknife needed the patched correlation objects, which is the one thing a part cannot carry — and per Cail's ruling jackknife and blinding are a deprecated pair for derived statistics. So the covariance now comes from the Monte Carlo path calculate_pure_eb already had, extracted as pure_eb_covariance_mc: draws from the CosmoCov gaussian covariance on the integration grid, around a theory mean, through the same kernel as the modes. It depends on the covariance model and the grids, never on the measured vector, so it is blind-invariant by construction. cv_pure_eb consumes two parts plus that one covariance file and touches no catalogue. The results dict is now self-describing — the grids travel with the modes (theta, edges, the raw ξ± and their variances, n_eff) instead of a TreeCorr object riding along — so the statistics, plots and npz all work from values. That deletes the FakeGG stub the paper PTE script needed, and turns calculate_eb_statistics into a function of its results alone. calculate_2pcf no longer writes patch results into the .txt dump: a per-patch ξ± realisation is an unblinded data vector, nothing reads one back, and the covariance a consumer needs is the matrix the part carries. * cv_summarize_bmodes reads the products instead of recomputing them It re-ran plot_pure_eb / plot_cosebis / plot_pseudo_cl in-process to repopulate the in-memory result dicts, which meant the terminal diagnostic reached the catalogue — and rewrote the two consumers' figures from unblinded values on the way past. It now reads what those rules wrote: the pure-E/B PTE matrices and the COSEBIs B-mode PTE from their .npz products, and the pseudo-Cℓ BB spectrum from its part against the NaMaster covariance, both declared inputs. The summary can no longer disagree with the products it summarises. The table itself moves to print_bmode_summary, so the in-memory path (summarize_bmodes, for notebooks) and the file path print the same thing. A test pins the .npz key contract the two rules meet on. * Review round: one pseudo-Cl producer, analytic covariance wins, grid tags canonical Six findings from the review. One pseudo-Cl producer. The untagged diagnostic part and its rule are gone; the summary, the terminal file and now the figures all read the analysis part with its matching NaMaster covariance, so the three cannot disagree. The figures become a plot-only ingest rule like the other B-mode ones, over a plot_pseudo_cl_spectrum that draws one spectrum for every version — the three near-identical EE/EB/BB blocks were one function all along. The analytic covariance wins, loudly. A part whose analysis covariance is external (ξ± reporting, pseudo-Cl) now always takes the supplied block, which replaces the estimate it was born with rather than losing a race with it, and says so on stdout; a missing injection raises instead of silently keeping the part's own. The reporting part keeps carrying its jackknife — it is a real diagnostic, just not the analysis covariance. Grid tags are canonical. xi_binning stamped raw YAML (maxsep=300) while the measurement wrote float-normalised names (maxsep=300.0), so producer and consumer would have asked for different paths on the first real run. The table coerces once, where it is built. It moved to common.py to be testable at all, and the tests pin the round trip a filename makes. Also: cosebis_from_xi deleted (no callers, and its centre-based cut diverged from the edge-based one in use); the paper PTE script's unread cov_integration and npatch dropped, with the rule docstring corrected to say plainly that nothing there varies by blind; and the header DAG comment rewritten to the SACC-part graph it actually describes. * chore: delete dead cosmocov drivers and repoint dangling references `workflow/scripts/run_cosmocov_chain.sh` invoked `cosmo_inference/scripts/cosmocov_process.py`, which was deleted on develop (#236) and relocated as the snakemake-only `workflow/scripts/cosmocov_process.py` (driven by `workflow/rules/covariance.smk`). The chain script's only caller was `papers/bmodes/scripts/run_cov_sweep.sh`, whose own only caller was a human, so both are dead end-to-end and are removed. Tree sweep for the same class of rot, repointing what survives: - README: drop the firecrown/Smokescreen override paragraph. Both files it names (`uv-overrides.txt`, `scripts/patch_firecrown.py`) were removed in f5eb417 when firecrown was dropped, and there is no `blinding` extra left in pyproject. - CLAUDE.md / CONTRIBUTING.md: the single-test example named `tests/test_cosmology.py`, which does not exist; use `tests/test_cosmo_val.py`. - CLAUDE.md: the cosmo_inference section documented a `./pipeline.sh` driver with flags. There is no such script; the pipeline is Snakemake-orchestrated, so the section now carries the invocation from `cosmo_inference/README.md`. - `papers/bmodes/config/ecut_spec.md`: `workflow/config/config.yaml` and `workflow/rules/claims.smk` moved under `papers/bmodes/`; the catalog config is `cosmo_val/cat_config.yaml`, not `code/sp_validation/cosmo_val/…`. - `papers/bmodes/scripts/update_survey_stats.py`: usage docstring still gave its pre-move `workflow/scripts/` path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * papers Snakefiles: read cat_config through code/, not the deprecated pure_eb symlink Same file as common.py's CAT_CONFIG; the symlink is slated for removal. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * papers/bmodes: read the pseudo-Cl spectrum from its SACC part The generic pseudo_cl rule writes a SACC part, so the paper's C_ell consumers read that instead of the FITS PSEUDO_CELL table: one shared reader (pseudo_cl_io.load_pseudo_cl_data over sacc_io.get_pseudo_cl) for the six scripts, and the claims.smk path helper points at the .sacc part. The bandpower covariance still comes from pseudo_cl_cov's FITS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeMhD6yCyrgBz5oz87bCtR * Drop generate_cosmocov_ini.py and the README install section generate_cosmocov_ini.py was deleted on develop (#236) and came back with this branch's stale-base rebuild; nothing calls it. The README "Local Installation" section repeats docs/source/installation.rst and is not part of the SACC migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeMhD6yCyrgBz5oz87bCtR --------- Co-authored-by: Claude Opus <noreply@anthropic.com>

cailmdaley and others added 30 commits July 10, 2026 01:49

Merge branch 'feat/sacc-3-2pt-converter' into feat/sacc-7-sacc-like

c37bdcb

cailmdaley and others added 4 commits July 20, 2026 22:30

Merge branch 'feat/sacc-6-blinding' into feat/sacc-7-sacc-like

45c44b4

# Conflicts: # src/sp_validation/cosmo_val/core.py # src/sp_validation/sacc_io.py

cailmdaley mentioned this pull request Aug 30, 2026

Scrub legacy blinding vocabulary (pre-Smokescreen blind A/B/C) #312

Open

cailmdaley and others added 3 commits August 30, 2026 04:01

Merge branch 'feat/sacc-6-blinding' into feat/sacc-7-sacc-like

feff1e1

# Conflicts: # workflow/rules/inference.smk

inference.smk: restore CHAINS_DIR (dropped upstream with dormant rule…

be5b9fd

…s; PR-7 live rule uses it)

cailmdaley mentioned this pull request Aug 30, 2026

cosmo_val + Snakemake migration to the SACC writers (PRD #241 row 4) #251

Merged

cailmdaley added 3 commits August 31, 2026 02:09

Merge feat/sacc-4-cosmo-val-sacc into feat/sacc-7-sacc-like

352aa99

Merge feat/sacc-4-cosmo-val-sacc into feat/sacc-7-sacc-like

8f83a98

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like (linearize the …

d64dc23

…stack)

cailmdaley changed the base branch from feat/sacc-4-cosmo-val-sacc to feat/sacc-6-blinding August 31, 2026 00:29

This was referenced Aug 31, 2026

CSL fork needs updating toward upstream: scipy ≥1.15 crash + likelihood/sacc predates real-space support #316

Closed

Bring UNIONS CSL modules onto current upstream UNIONS-WL/cosmosis-standard-library#1

Closed

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

298bca8

inference.smk resolved wholly in favour of this branch's rewrite; #251 no longer touches that file.

cailmdaley added 7 commits August 31, 2026 02:59

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

f30d1c8

Merge branch 'feat/sacc-6-blinding' into feat/sacc-7-sacc-like

8408c74

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

beb23e9

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

4870bfb

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

018c42d

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

ac913e8

Merge feat/sacc-6-blinding into feat/sacc-7-sacc-like

75a6216

cailmdaley marked this pull request as ready for review August 31, 2026 16:13

cailmdaley marked this pull request as draft September 28, 2026 03:07

LisaGoh deleted the branch feat/sacc-6-blinding October 20, 2026 11:39

LisaGoh closed this Oct 20, 2026

cailmdaley mentioned this pull request Oct 20, 2026

PR 7: adopt CosmoSIS native SACC likelihood (sacc_like), validated against the converter path #417

Open

Navigation Menu

Repository navigation

PR 7: adopt CosmoSIS native SACC likelihood (sacc_like), validated against the converter path - #255

PR 7: adopt CosmoSIS native SACC likelihood (sacc_like), validated against the converter path#255
cailmdaley wants to merge 57 commits into
feat/sacc-6-blindingfrom
feat/sacc-7-sacc-like

cailmdaley commented Jul 10, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cailmdaley commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The "prototype-grade" verdict, grounded

Adoption = subclass shim, not fork

Validation (all observed, in-process, no sampler/CAMB needed)

Workflow revival

Environment

Adversarial review

Stack note

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cailmdaley commented Jul 10, 2026 •

edited

Loading