You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sorry, incorrectly explained what long names I meant. I meant internal Igor names inside pxp experiment - long folder (and likely variable/string) names allowed from Igor 7 or 8, I think. (32+ characters long) I have long folder names in my test pxp file and those fail to extract correctly while the ones <= 31 characters extract correctly.
Thanks for the clarification. Could you please create a zip file with the failing file and attach it to this issue?
We can use it to reproduce the issue and then create a test case for it as well.
I cannot allocate time right now, but maybe somebody else would be willing to create a PR.
I have modified the original Igor experiment from 150MB into this but hopefully it will show the correct issue. I also asked Claude which has been helping me with the coding to generate documentation on the observed issues which would be helpful and it is clear what the issue. Hopefully this will help.
Title: packed.load() silently drops folders/waves with names longer than 31 characters (Igor Pro 8 long-name format)
Summary
When loading a packed experiment (.pxp) saved by Igor Pro 8 (or later), igor2.packed.load() silently drops every data folder and wave whose name exceeds 31 characters. The records exist in the file but are encoded with two new packed-record type IDs that igor2 does not recognise (types 26 and 33), so they get treated as UnknownRecord and the data they reference never appears in the returned filesystem dict.
There is no warning, no exception, no log message — the affected folders are simply absent from the output. For experiments with many long sample names this can mean half or more of the data is silently invisible.
Version
igor2: 0.5.13 (current PyPI)
Python: 3.13 (also reproduced on 3.12)
OS: macOS 14 and Windows 11 Reproduction
fromigor2importpackedrecords, fs=packed.load("test.pxp")
# Walk top-level foldersprint(sorted(fs["root"].keys()))
# Returns only short-named folders; folders like# "AMC_25_crystalline_04_07_26_10min_0036" (38 chars) are missing.
A minimal experiment that reproduces the bug is attached as test.pxp. It contains:
short-named folders (e.g. AMC_04_07_26_3min_0033, 22 chars) → appear in fs["root"]
long-named folders (e.g. AMC_25_crystalline_04_07_26_10min_0036, 38 chars) → silently absent
Counting in the raw byte stream confirms the long-named folders are present in the file — their names appear inside VariablesRecord and WaveRecord payloads — but no FolderStartRecord (type 9) ever carries them.
Root cause (as far as I've traced it)
Igor Pro 8 introduced new packed-record types to support folder and wave names longer than 31 characters. Walking the byte stream directly, two new IDs appear that aren't in _RECORD_TYPE:
type 26 — typically 52 bytes, payload begins with bytes that look like \x10\x00\x10\x00\x00\x00...
type 33 — typically 4 bytes, payload is a 32-bit little-endian integer that looks like an ID/hash (e.g. 0x0019AB15)
The pattern I see consistently is:
type=33 (4 bytes, ID) <-- precedes a long-named record
type= 1 (Variables, n=...) <-- holds something the type-33 ID references?
type= 3 (Wave, ...) <-- the actual wave (often v7 format)
There is no documented spec I can find — Wavemetrics PTN003 covers wave-file versions 1/2/3/5/7 but not these packed-record extensions. My best guess is that type 33 carries an ID into a string pool somewhere, and the resolved long name is stored separately rather than inline with the FolderStart payload — but that's just a guess.
Wave version 7 (long-name waves), by contrast, is documented in PTN003 and turns out to be binary-compatible with v5 in its data section. I worked around the "invalid binary wave version: 7" error in my own code by patching the version field 7 → 5 before calling _loadibw, and that recovers the wave's numeric data correctly. But folder records appear to need a different fix because the name itself isn't in the FolderStart payload.
What I'd consider helpful
In rough order of helpfulness:
Warn rather than silently skip. Even before full support, a warning like "skipped Igor 8+ long-name record (type 26/33); folder/wave with long name will be missing from the filesystem" would prevent users from silently losing data. Document the limitation in the README so downstream packages can handle it explicitly. Add v7-wave support (binary-compatible with v5; just need to accept the version number and read the trailing long-name section). I'd be happy to submit a PR for this if helpful. Decode types 26/33. Hardest, no spec — I haven't tried.
I'm not blocked: the .h5xp (Wavemetrics HDF5 packed-experiment) format has no name-length limit and works perfectly via h5py, so my project recommends users re-save as .h5xp when this comes up. Filing this so others hitting the same silent-drop see a record of it, and in case any of the above is easy to land upstream.
Happy to attach the diagnostic scripts I wrote for tracking this down (a byte-level record walker and a folder-tree printer) if useful.
Thanks for maintaining igor2 — it's been very useful for everything else!
Activity
paulmueller commentedon May 26, 2026
There shouldn't be any limitation in path length. If you are on Windows, you might be hitting this: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry
What error message do you get?
jilavsky commentedon May 28, 2026
Sorry, incorrectly explained what long names I meant. I meant internal Igor names inside pxp experiment - long folder (and likely variable/string) names allowed from Igor 7 or 8, I think. (32+ characters long) I have long folder names in my test pxp file and those fail to extract correctly while the ones <= 31 characters extract correctly.
paulmueller commentedon May 29, 2026
Thanks for the clarification. Could you please create a zip file with the failing file and attach it to this issue?
We can use it to reproduce the issue and then create a test case for it as well.
I cannot allocate time right now, but maybe somebody else would be willing to create a PR.
jilavsky commentedon Jun 1, 2026
Test.pxp.zip
I have modified the original Igor experiment from 150MB into this but hopefully it will show the correct issue. I also asked Claude which has been helping me with the coding to generate documentation on the observed issues which would be helpful and it is clear what the issue. Hopefully this will help.
Title: packed.load() silently drops folders/waves with names longer than 31 characters (Igor Pro 8 long-name format)
Summary
When loading a packed experiment (.pxp) saved by Igor Pro 8 (or later), igor2.packed.load() silently drops every data folder and wave whose name exceeds 31 characters. The records exist in the file but are encoded with two new packed-record type IDs that igor2 does not recognise (types 26 and 33), so they get treated as UnknownRecord and the data they reference never appears in the returned filesystem dict.
There is no warning, no exception, no log message — the affected folders are simply absent from the output. For experiments with many long sample names this can mean half or more of the data is silently invisible.
Version
igor2: 0.5.13 (current PyPI)
Python: 3.13 (also reproduced on 3.12)
OS: macOS 14 and Windows 11
Reproduction
A minimal experiment that reproduces the bug is attached as test.pxp. It contains:
short-named folders (e.g. AMC_04_07_26_3min_0033, 22 chars) → appear in fs["root"]
long-named folders (e.g. AMC_25_crystalline_04_07_26_10min_0036, 38 chars) → silently absent
Counting in the raw byte stream confirms the long-named folders are present in the file — their names appear inside VariablesRecord and WaveRecord payloads — but no FolderStartRecord (type 9) ever carries them.
Root cause (as far as I've traced it)
Igor Pro 8 introduced new packed-record types to support folder and wave names longer than 31 characters. Walking the byte stream directly, two new IDs appear that aren't in _RECORD_TYPE:
The pattern I see consistently is:
There is no documented spec I can find — Wavemetrics PTN003 covers wave-file versions 1/2/3/5/7 but not these packed-record extensions. My best guess is that type 33 carries an ID into a string pool somewhere, and the resolved long name is stored separately rather than inline with the FolderStart payload — but that's just a guess.
Wave version 7 (long-name waves), by contrast, is documented in PTN003 and turns out to be binary-compatible with v5 in its data section. I worked around the "invalid binary wave version: 7" error in my own code by patching the version field 7 → 5 before calling _loadibw, and that recovers the wave's numeric data correctly. But folder records appear to need a different fix because the name itself isn't in the FolderStart payload.
What I'd consider helpful
In rough order of helpfulness:
Warn rather than silently skip. Even before full support, a warning like "skipped Igor 8+ long-name record (type 26/33); folder/wave with long name will be missing from the filesystem" would prevent users from silently losing data.
Document the limitation in the README so downstream packages can handle it explicitly.
Add v7-wave support (binary-compatible with v5; just need to accept the version number and read the trailing long-name section). I'd be happy to submit a PR for this if helpful.
Decode types 26/33. Hardest, no spec — I haven't tried.
I'm not blocked: the .h5xp (Wavemetrics HDF5 packed-experiment) format has no name-length limit and works perfectly via h5py, so my project recommends users re-save as .h5xp when this comes up. Filing this so others hitting the same silent-drop see a record of it, and in case any of the above is easy to land upstream.
Happy to attach the diagnostic scripts I wrote for tracking this down (a byte-level record walker and a folder-tree printer) if useful.
Thanks for maintaining igor2 — it's been very useful for everything else!
tests: load trimmed Igor Pro 8 experiment from #28
Merge pull request #31 from paulmueller-agent/test-igor8-long-folder-pxp
paulmueller commentedon Nov 3, 2026
Fixed in #30 with a test in #31