Skip to content

Rename chunk_dim to stream_dim, following ezmsg core - #39

Merged
cboulay merged 2 commits into
devfrom
cboulay/stream-dim
Sep 9, 2026
Merged

cboulay merged 2 commits into
devfrom
cboulay/stream-dim

Conversation

@cboulay

@cboulay cboulay commented Sep 9, 2026

Copy link
Copy Markdown
Member

Tracks the rename of AxisArray.chunk_dim to stream_dim in ezmsg core (ezmsg-org/ezmsg#265).

Why the name changed

chunk was already spoken for in three directions:

  • xarray, dask, zarr, HDF5 all use chunk for multi-dimensional memory/IO layout. Since AxisArray deliberately echoes xarray.DataArray — which has a .chunk() method — borrowing the word invites exactly the wrong reading.
  • ezmsg itself already uses it a second way: array_chunker / chunk_len mean "how much data per message".
  • Neither is what the field says. It names the dimension the stream extends along — the one whose extent is arbitrary because successive messages append there.

stream_dim names that role directly and reuses ezmsg's own vocabulary (InputStream / OutputStream). append_dim (zarr) and concat_dim (xarray) were the closest semantic matches but were rejected: in ezmsg-sigproc those words already name the target argument of an append or concat operation. netCDF's unlimited_dims describes a static property of the dimension rather than what is happening to it.

What's in here

Pure rename — no behaviour change.

This package uses chunk in the payload sense throughout ("chunk boundaries", "next chunk", carry-over across chunks in the kernel units); all of that is left alone.

Uses of chunk that mean how much data a message carries are deliberately left alone; only the dimension sense is renamed, including the prose that describes it.

The ezmsg floor moves >=3.10.0b2 → >=3.10.0b3. 3.10.0b1 and b2 are on PyPI carrying chunk_dim, so the floor has to move for the rename to be meaningful; 3.10.0b3 is the first release with stream_dim.

Testing

170 passed, run against the renamed ezmsg core and ezmsg-baseproc installed editable.

Merge order: ezmsg 3.10.0b3 must be on PyPI before this merges, or CI here resolves an ezmsg that has no stream_dim.

Tracks the upstream rename of `AxisArray.chunk_dim`. `chunk` was
overloaded -- xarray's `DataArray.chunk`, dask, zarr and HDF5 all use it
for memory/IO layout, and this package uses it a third way, for how much
data a message carries ("chunk boundaries", "next chunk"). Those uses are
left alone; only the dimension sense is renamed.

Bumps the ezmsg floor to 3.10.0b3, the first release carrying the new
name. Pure rename otherwise. 170 passed.
A wedged job used to run to GitHub's 6-hour default. One did: an
ezmsg-simbiophys run hung immediately after `collected 161 items`,
produced not one test dot, and sat on a runner for 3h09m until it was
cancelled by hand -- long enough to starve the macOS queue for every
other repo in the org.

`timeout-minutes: 30` bounds that; the suite runs in well under a minute
on every platform, so the margin is generous. The concurrency group makes
a new push to a PR cancel the run already in flight for that ref, so a
stuck job cannot hold a slot while its own replacement queues behind it.
@cboulay
cboulay merged commit ec107cb into dev Sep 9, 2026
4 checks passed
@cboulay
cboulay deleted the cboulay/stream-dim branch September 9, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant