🤖 Filed by an agent at @adelorme's request, as post-merge follow-up for the EEGLAB test-port stack (#329–#372).
@suraj-ranganath reported running the external parity harness against each PR and the cumulative tip: EEGPrep and live MATLAB/EEGLAB through eeglabcompat, 13 subjects × 7 pipeline stages; import and re-referencing exactly identical; ASR/ICA differing by at most 0.000014 µV; two MATLAB R2026a ICLabel outliers reproduced identically on unmodified develop; the older exact golden result generated under MATLAB R2025b.
This is the strongest evidence attached to the stack and it should be recorded somewhere durable rather than living in a chat thread. Three things:
1. Provenance. Which commit was the harness run against, and can the summary output be attached to this issue or the epic?
2. ASR mode. Was the run made with BurstRejection='off' and WindowCriterion='off', and what fraction of samples did ASR rewrite?
This is a documented property of this harness, not a doubt about the result. With BurstRejection='on', clean_artifacts builds its retained-sample mask as |original - repaired| < 1e-8, so every sample ASR rewrote is discarded and never reaches the output — the comparison is then structurally unable to see ASR reconstruction error. Measured previously in this repo: EEGPrep vs MATLAB ASR burst reconstruction differed by ~234 µV while the same comparison with rejection on was bit-identical; separately, perturbing the ASR coefficient table by ~1e-13 relative still gave bit-identical step-3 output. So ≤ 0.000014 µV is consistent both with a correct ASR path and with a run blind to reconstruction. If rejection mode was on, the figure should be recorded as a rejection-decision check so nobody later cites it as covering reconstruction.
3. Test validation. Were any of the 99 new test files validated against MATLAB at any point, even out of tree? None reference an oracle in-tree. If some were, it narrows the scope of the binary-reader validation issue.
For reference, the harness drives these entry points: pop_loadset, pop_saveset, eeg_checkset, eeg_interp, iclabel, pop_load_frombids, clean_artifacts, pop_subcomp, pop_select, pop_reref, eeg_picard, eeg_idx, clean_rawdata, pop_rmbase, pop_epoch, eeg_icflag, pop_importbids, pop_icflag — 8 of which the stack modifies.
🤖 Filed by an agent at @adelorme's request, as post-merge follow-up for the EEGLAB test-port stack (#329–#372).
@suraj-ranganath reported running the external parity harness against each PR and the cumulative tip: EEGPrep and live MATLAB/EEGLAB through
eeglabcompat, 13 subjects × 7 pipeline stages; import and re-referencing exactly identical; ASR/ICA differing by at most 0.000014 µV; two MATLAB R2026a ICLabel outliers reproduced identically on unmodifieddevelop; the older exact golden result generated under MATLAB R2025b.This is the strongest evidence attached to the stack and it should be recorded somewhere durable rather than living in a chat thread. Three things:
1. Provenance. Which commit was the harness run against, and can the summary output be attached to this issue or the epic?
2. ASR mode. Was the run made with
BurstRejection='off'andWindowCriterion='off', and what fraction of samples did ASR rewrite?This is a documented property of this harness, not a doubt about the result. With
BurstRejection='on',clean_artifactsbuilds its retained-sample mask as|original - repaired| < 1e-8, so every sample ASR rewrote is discarded and never reaches the output — the comparison is then structurally unable to see ASR reconstruction error. Measured previously in this repo: EEGPrep vs MATLAB ASR burst reconstruction differed by ~234 µV while the same comparison with rejection on was bit-identical; separately, perturbing the ASR coefficient table by ~1e-13 relative still gave bit-identical step-3 output. So ≤ 0.000014 µV is consistent both with a correct ASR path and with a run blind to reconstruction. If rejection mode was on, the figure should be recorded as a rejection-decision check so nobody later cites it as covering reconstruction.3. Test validation. Were any of the 99 new test files validated against MATLAB at any point, even out of tree? None reference an oracle in-tree. If some were, it narrows the scope of the binary-reader validation issue.
For reference, the harness drives these entry points:
pop_loadset,pop_saveset,eeg_checkset,eeg_interp,iclabel,pop_load_frombids,clean_artifacts,pop_subcomp,pop_select,pop_reref,eeg_picard,eeg_idx,clean_rawdata,pop_rmbase,pop_epoch,eeg_icflag,pop_importbids,pop_icflag— 8 of which the stack modifies.