Skip to content

Feature/305 adapt tomographic 2pcf script to sacc io format - #306

Open
LisaGoh wants to merge 5 commits into
developfrom
feature/305-adapt-tomographic-2pcf-script-to-sacc_io-format
Open

LisaGoh wants to merge 5 commits into
developfrom
feature/305-adapt-tomographic-2pcf-script-to-sacc_io-format

Conversation

@LisaGoh

@LisaGoh LisaGoh commented Aug 4, 2026 •

Copy link
Copy Markdown
Member

Closes #305. Part of #376.

Writes tomographic ξ± as one SACC part, and puts the OneCovariance ξ± covariance (#256) into SACC in the right order.

develop already wrote the non-tomographic ξ± as a sealed part (xi_to_sacc, #251/#253). This extends that same writer to tomographic bin pairs rather than adding a second one, as agreed in #375/#376. The pure_eb, test and plotting fixes from the first version of this PR are dropped (superseded on develop), and so is the earlier save_2pcf_sacc, which targeted the old sacc_io API.

OneCovariance order ≠ SACC order
OneCovariance writes the real-space matrix type-major: all ξ+ (bin pairs i ≤ j, i outer, then θ), then all ξ−. A ξ± part is pair-major: each pair's ξ+ then ξ−. The two agree only for a single bin, so a tomographic .dat attached as-is (what assemble_sacc did) would silently misalign the covariance. The matrix is now reordered to the part's points before it's attached.

What changes

  • cosmo_val/sacc_writers.py
    • xi_pairs_to_sacc: one ξ± part over any set of bin pairs, pair-major, with an optional dense covariance or diagonal. xi_to_sacc is now its single-pair form.
    • onecov_xi_covariance(part, cov): OneCovariance's matrix permuted to the part's insertion order; refuses a matrix whose size doesn't match.
  • sacc_io.read_onecov_matrix: reads the .dat (np.savetxt with a # header) and checks it's square and symmetric.
  • cosmo_val/real_space.py, calculate_2pcf_version(compute_tomography=True):
    • every bin pair goes into one part, with a source_i tracer per bin;
    • pairs go in as (0,0), (0,1), …, (n-1,n-1), which is OneCovariance's order;
    • jackknife covariance via treecorr.estimate_multi_cov, so the correlations between pairs are kept; shot noise gives the diagonal;
    • the part is sealed under the catalogue's blind and kept in cv.xi_tomo_parts[ver, grid];
    • the returned values are xi_correlation views of the sealed part, not raw TreeCorr objects, with the same attributes.
  • cosmo_val/core.py
    • sacc_nz(version, tomography=True): bin i takes column i + 1 of redshift_path.
    • Blinded catalogues now conceal tomographic ξ± (and ⟨M_ap²⟩ derived from it) instead of refusing it; blinding.shear already works per tracer pair. Tomographic pseudo-Cℓ are still refused.
  • Workflow
    • run_2pcf.py --tomo; the xi rule output is …_npatch={npatch}{tomo_suffix}.sacc (also removes a duplicated npatch param).
    • New rule xi_onecov → …{tomo_suffix}_onecov.sacc: the ξ± part with the OneCovariance matrix of its binning (NG on the reporting grid, G on the integration grid) as its covariance, reordered, keeping the part's blind. Script: workflow/scripts/xi_onecov_sacc.py (also runs from the command line).
    • assemble_sacc.py reorders --xi-cov the same way.

Requirements

  • Tomographic ξ± needs an n(z) column per bin. A single-column file for a multi-bin catalogue is refused.
  • SACC bin i is the i-th smallest catalogue bin ID, and must be column i + 1 of the n(z) file. OneCovariance reads the same file, so its bins line up as long as columns are in ascending bin-ID order.

Tested

  • New tests:
    • the multi-pair layout;
    • the type-major → pair-major reorder, against an expected order built independently;
    • the identity case for one bin, and refusal of a mismatched matrix;
    • read_onecov_matrix;
    • a tomographic part on a synthetic catalogue: tracers and n(z) per bin, pair order, jackknife correlations between pairs;
    • refusal of a single-column n(z);
    • sacc_nz(tomography=True);
    • concealment on a blinded catalogue (needs smokescreen).
  • Updated fixtures: two existing tomographic tests now carry a per-bin n(z).
  • Suites: test_cosmo_val, test_sacc_writers, test_sacc_io, test_assemble_sacc and the regression tests pass, apart from three failures that are already on the base commit (pure-E/B synthetic run, pure-E/B mapping, τ centring).
  • Dry runs resolve for …_tomo.sacc, …_tomo_onecov.sacc and …_onecov.sacc; the cosmo_val paper DAG is unchanged.
  • End-to-end: xi_onecov_sacc.py on a synthetic two-bin part with a type-major .dat recovers the expected pair-major covariance (to text precision) and keeps the blind stamp.

Not yet checked

  • The reorder against a real tomographic OneCovariance run: it follows OneCovariance's source code, but there's no tomographic catalogue yet to produce one.

Out of scope

🤖 Generated with Claude Code

LisaGoh and others added 2 commits August 4, 2026 16:24
Add RealSpaceMixin.save_2pcf_sacc, which serialises the TreeCorr output of
calculate_2pcf_version into the sacc_io layout: the source_{i} n(z) tracers
and one xi+/xi- block per tomographic bin pair.

Serialising lives on the class because it needs the version's n(z) and its
tomographic bin map; run_2pcf.py only decides where the file goes, from the
sacc_path param under Snakemake and from --sacc on the CLI.

Insertion order is load-bearing: add_xi writes one pair as [xi+; xi-], so the
data vector is pair-major and the covariance is tied to it by position alone.
The bin pairs are sorted once and that list drives every subsequent loop.

Catalog bin ids are mapped to 0-based SACC indices from the ids actually
present, rather than by subtracting 1, so a catalog that starts elsewhere or
skips a bin still lands on the right source_{i} tracer. A mismatch between the
n(z) column count and the measured bin count raises: the columns are never
summed or truncated to make up the difference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@LisaGoh
LisaGoh marked this pull request as draft August 5, 2026 08:33
@LisaGoh LisaGoh linked an issue Aug 5, 2026 that may be closed by this pull request
@LisaGoh
LisaGoh marked this pull request as ready for review August 6, 2026 09:37
@LisaGoh
LisaGoh marked this pull request as draft August 6, 2026 09:38
@LisaGoh
LisaGoh changed the base branch from feature/sp_validation-extend-to-tomography to develop August 6, 2026 10:25
@LisaGoh
LisaGoh changed the base branch from develop to feature/sp_validation-extend-to-tomography August 6, 2026 10:25
LisaGoh and others added 2 commits August 6, 2026 13:38
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cailmdaley

Copy link
Copy Markdown
Collaborator

@LisaGoh i'm going to start working on getting #253 into shape now, in the meantime this can at least be synced up with all of the changes from develop!

…e/305-adapt-tomographic-2pcf-script-to-sacc_io-format
@LisaGoh
LisaGoh marked this pull request as ready for review October 9, 2026 10:13
@LisaGoh
LisaGoh changed the base branch from feature/sp_validation-extend-to-tomography to develop October 20, 2026 09:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adapt tomographic 2PCF script to sacc_io format

2 participants