Adopt the scverse cookiecutter template - #1197
Open
Zethson wants to merge 29 commits into
Open
Conversation
Link the repository to cookiecutter-scverse via .cruft.json so future template changes can be pulled in with `cruft update`, and bring the project in line with the template it now tracks. Tooling: - pyproject.toml is restructured onto the template layout: classifiers, a `doc` (was `docs`) and `typecheck` dependency group, hatch envs for tests/docs/checks, and the ruff/mypy/pytest/coverage/cruft tables. hatch-vcs versioning is kept, so .bumpversion.cfg (which pointed at a static version that does not exist) is dropped. - The CI test matrix now lives in [tool.hatch.envs.hatch-test] instead of the workflow file, covering the lowest and highest supported Python versions, the pinned dependency floor, and pre-release dependencies, each across Linux, macOS and Windows. - .mypy.ini moves into [tool.mypy]. The previous pre-commit hook ran mypy in an isolated environment with --ignore-missing-imports, which made everything from pandas, xarray, anndata, dask and geopandas `Any`; running it against a real environment surfaces 499 errors in 32 modules. Those modules are listed explicitly as grandfathered so that new and refactored code is type-checked from now on. - prettier is replaced by biome, and pyproject-fmt and zizmor join the hook set. asv.conf.json gets a biome override because asv's parser accepts comments but not trailing commas. CI: - test.yaml, build.yaml and release.yaml come from the template. Tests are driven through hatch, actions are pinned to commit SHAs, coverage upload uses codecov OIDC, and PyPI upload uses trusted publishing. - build_image.yml is hardened the same way so zizmor passes. - Issue templates become GitHub issue forms, keeping the guidance on reproducing bugs with the `blobs` datasets. Docs: - conf.py adopts the template's structure, switching mathjax for katex and adding opengraph and scverse-misc, while keeping the linkcode, intersphinx and tutorial-submodule configuration. - The contributing guide is replaced by the template's hatch/prek based one, with the spatialdata release process, cross-repo integration testing and profiling sections carried over. - Docs are built with `hatch run docs:build`, so docs/Makefile is gone and warnings are now errors. The package also ships py.typed, and the empty CHANGELOG.md is removed since release notes are curated on GitHub Releases. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
The template's ruff configuration ignores E501, D400, D401 and enables RUF100, so the noqa directives those rules required are now unused and removed. `format.docstring-code-format` normalises quoting and spacing in doctest examples, and biome reformats the JSON test fixtures (their parsed content is unchanged). Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
…W build `docs/extensions/typed_returns.py` overrides napoleon's Returns parser, so a numpydoc type line followed by an indented description is not understood and docutils reports "Unexpected indentation" for every class inheriting BaseTransformation.inverse and BaseTransformation.to_affine_matrix. Write both the way the rest of the package does, with the description starting directly under the underline. The contributing guide cites Virshup_2023 as the example citation, so add that entry to references.bib. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1197 +/- ##
==========================================
- Coverage 91.89% 91.31% -0.58%
==========================================
Files 53 53
Lines 7942 8392 +450
==========================================
+ Hits 7298 7663 +365
- Misses 644 729 +85
🚀 New features to boost your workflow:
|
Zethson
marked this pull request as draft
August 29, 2026 12:28
Removes the [[tool.mypy.overrides]] block that set ignore_errors on 32 modules. That was suppression, not type checking. Fixed so far, by root cause: - SpatialData.__init__ reused one loop variable across five loops over differently typed mappings, so mypy joined them to the first one. - SpatialData._write_element dispatches on an element_type string while holding a SpatialElement union; each branch now checks the element type it is about to write and raises TypeError otherwise. - get_model() accepted AnnData at runtime but was annotated as taking only a SpatialElement, and validated through a helper that passed the whole union to schema.validate(). Each branch now validates after narrowing. - _parse_version() indexed a zarr attribute payload typed as JSON. - _read_points() and _read_table() passed the Array | Group returned by zarr.open() to code that requires a Group. 500 errors are down to 470 in 30 files. CI is red until the rest are fixed; the pull request stays a draft in the meantime. Also unwraps two sentences that were soft-wrapped across lines. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Follows the approach of scverse/anndata#2577: dependencies that are installed but ship no py.typed marker are read from source through follow_untyped_imports rather than collapsing to Any, and ignore_missing_imports is scoped to the optional dependencies that are absent when the check runs. Reading real types out of ome_zarr, pyarrow, dask_image and friends surfaces more errors, not fewer, which is the point. Narrowing is written out at each site rather than routed through a generic helper, because a helper taking `object` and returning a concrete type is a cast wearing a hat. Fixed here: - Indexing a zarr group and opening a zarr store both yield Array | Group, while every reader and writer here needs a Group. - store.root exists only on LocalStore, so reading or writing the parquet file of a points or shapes element now says so instead of failing with AttributeError on any other store. - coordinateTransformations and the spatialdata attrs payload are typed as arbitrary JSON and were indexed without being checked. - _resolve_zarr_store() returns a Store on every path but was annotated StoreLike, which made every .close() and .root on the result an error. - read_zarr() rebound its attrs dict to None, and DataArray.dims is a tuple of Hashable rather than of str. 460 errors left, down from 500. Tests stay green. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Series.values is ndarray | ExtensionArray, so the bin coordinates and the relabeled instance ids now go through to_numpy(), which carries a real numpy dtype into np.zeros() and into the fancy indexing. The csc-versus-dense branch was decided by a boolean flag, so nothing downstream knew which of the two AnnData.X actually was; the branches now narrow the matrix itself. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
RasterSchema.parse rebound its `data` parameter through three different shapes (input array, spatial image, multiscale tree) and its `dims` parameter between `Sequence[str] | None` and the tuple of Hashable that xarray hands back, so no line after the first assignment had a type that matched the value. Each stage now has its own name. _validate_labels_dtype declared neither self nor cls yet was called as cls._validate_labels_dtype(data); it only worked because the single argument lined up with the missing first parameter. It is a staticmethod. The parse() overrides on Labels2DModel and Labels3DModel named their class parameter self. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
get_axes_names() registers an implementation for pandas DataFrame, which the fallback signature did not accept, and its DataArray and DataTree implementations returned xarray's tuple of Hashable behind a type: ignore[no-any-return] that did not even match the error. Indexing a DataTree returns DataTree | DataArray, so `scale0` is narrowed before its data variables are iterated. force_2d() built Point, Polygon and MultiPolygon values into one variable first bound to a Point. ShapesModel used shapely's private _ndim; get_coordinate_dimension() is the public equivalent. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
The guard read `dtype is str or table.obs[instance_key].dtype is str`. A dtype object is never the `str` type, so both operands were always False and the TypeError could not be raised. It now compares string dtypes through pandas. Also replaces the reflection in _gen_elements() with iteration over the typed element containers, narrows the region and instance key columns to pandas Series, and rejects a table that annotates another table instead of failing later on a missing .index. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
filter_by_coordinate_system() and transform_to_coordinate_system() both collected elements into a dict[str, dict[str, SpatialElement]] keyed by element type and splatted it into the SpatialData constructor, which takes four separately typed mappings. Both now build those four mappings directly. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
init_from_elements(), subset() and __setitem__ dispatched on the schema returned by get_model() and then wrote the still-unnarrowed element into a container typed for one specific element type. Each branch now asserts the element type that get_model() just established. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Narrows the places where an element looked up by name can be a table but the code that follows only handles spatial elements, threads the raster format out of the parsed format mapping before handing it to the raster writers, pins the container zarr format to the two values zarr accepts, and stops joining xarray's Hashable dims as if they were strings. src/spatialdata/_core/spatialdata.py is now clean. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
_match_rows declared its mask as a pandas Series while every caller passes the boolean array from np.isin, and its match_rows parameter as str where pandas.merge needs one of its join literals. _inplace_fix_subset_categorical_obs was called with the result of a filter that can come back empty; the hasattr(subset_adata, 'obs') guard was standing in for a None check, which the signature now states. table.obs is narrowed to a pandas DataFrame before groupby and reset_index, and annsel's .an accessor is reached through AnnselAccessor, since it is registered on AnnData at import time and is invisible to a type checker. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
The right, inner and left joins return None when nothing matches, but their signatures, the join dispatcher and join_spatialelement_table all declared a plain AnnData. The callers that cannot cope with an empty join now say so instead of failing later on None. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
get_values() indexed the element for dataframe origins and the table for obs, var and obsm origins without narrowing either, and its obsm branch can return a sparse array where the signature promises a data frame or a dense one. src/spatialdata/_core/query/relational_query.py is now clean. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
get_values() indexed the element for dataframe origins and the table for obs, var and obsm origins without narrowing either, and its obsm branch can return a sparse array where the signature promises a data frame or a dense one. src/spatialdata/_core/query/relational_query.py is now clean. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Without scale factors parse() returns a DataArray and with them a multiscale DataTree, but the single signature returned the union, so every caller that knows it passed no scale factors still had to deal with a DataTree it can never get. Labels2DModel and Labels3DModel only overrode parse() to reject c_coords, through a (*args, **kwargs) signature that erased the overloads. The base tells labels from images by whether its dims contain a channel axis, so both overrides are gone. Also imports datashader's reductions from the module that defines them, since datashader re-exports them through a star import. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
rasterize_images_labels() was annotated to take any SpatialElement while it only handles rasters, the multiscale branch pulled data variables out of a DataTree without narrowing the node it indexed, and the SpatialData branch collected results of rasterize() as if a single element could come back as a SpatialData object. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
The bounding box and polygon queries can return slice selections when asked for the request only, and one result per box when given several, so their signatures now say so instead of promising a single element. ImageTilesDataset used .A, the alias of .toarray() that scipy removed, and reached for its table on the path where it was built without one. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
_dissolve_on_overlaps() is annotated to return a GeoDataFrame while it returns a (label, geometry) pair, to_circles() used .A1 (the alias of a flattened dense view that scipy removed), and both entry points pulled the single scale out of a DataTree without narrowing the node. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
xrspatial's stats() returns a pandas data frame for numpy-backed values and a dask one for dask-backed values, and .compute() was called on both. Also narrows the elements handed to the shapes and raster aggregation paths, which the dispatch above has already established the model of. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
xrspatial's stats() returns a pandas data frame for numpy-backed values and a dask one for dask-backed values, and .compute() was called on both. pandas-stubs models the missing attribute as a Series, which is how the type check surfaced it. Also narrows the elements handed to the shapes and raster aggregation paths, which the dispatch above has already established the model of. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
_resolve_zarr_store() tested `isinstance(path.store, zarr.storage.ConsolidatedMetadataStore)`, which zarr 3 does not define, so a group backed by anything other than a LocalStore or an FsspecStore raised AttributeError rather than the ValueError below it. It then tested `isinstance(path, zarr.storage.StoreLike)`, a PEP 695 type alias that isinstance() rejects with TypeError, so an already-resolved store never reached its branch either. Also narrows the ome and omero attribute payloads, which zarr types as arbitrary JSON, before they are indexed and written back. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Narrows the DataTree traversal that these modules all share, replaces the ome-zarr Format base annotation on the points and tables writers with the spatialdata format types whose API they actually use, and imports dask's assert_eq from dask.dataframe.utils rather than from its test module. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Narrows the schema returned by get_model() to a raster schema where the element is known to be a raster, walks DataTree nodes without assuming what indexing one returns, imports dask's delayed from the module that defines it, and narrows table.obs to a pandas frame before renaming columns or writing to them. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
… array zarr 3 dropped the object_codec argument of create_array, so writing a shapes element with a string or object index in the v01 format raised TypeError. A variable-length string array stores the same values and reads back the same way. Also replaces the ome-zarr Format base annotations on the shapes writers with the shapes format types whose API they use. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Narrows the element format out of the parsed format mapping at the points, shapes and table write sites, the AnnData.X matrix that the bins rasterization indexes, and the elements looked up by name in the blobs datasets. mypy now reports no errors over src, with no cast(), no # type: ignore added and no module excluded from the check. Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Comments that restate the line below them are gone: the sphinx config header, the editable-install note, the section labels in docs/conf.py, and the CI job descriptions that repeat the job names. What is left explains something the code does not say, one sentence per line. Restores line_length, which was lost when .editorconfig was replaced with the template's copy, and drops the trailing period from the copyright string, which the theme appends itself and which therefore rendered as "scverse..". Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net>
Zethson
commented
Aug 29, 2026
| license = { file = "LICENSE" } | ||
| maintainers = [ | ||
| {name = "scverse", email = "giov.pll@gmail.com"}, | ||
| { name = "scverse", email = "giov.pll@gmail.com" }, |
Member
Author
There was a problem hiding this comment.
This should be Luca + team
| { | ||
| "template": "https://github.com/scverse/cookiecutter-scverse", | ||
| "commit": "eb4523fde2e18bcfc121ff2cd722c9037ff8b910", | ||
| "checkout": null, |
Member
Author
There was a problem hiding this comment.
Should not be null but rather main or 0.8.0
| "package_name": "spatialdata", | ||
| "project_description": "Spatial data format.", | ||
| "author_full_name": "scverse", | ||
| "author_email": "giov.pll@gmail.com", |
| "scipy!=1.17", | ||
| "scverse-misc[datasets]>=0.1", | ||
| # for debug logging (referenced from the issue template) | ||
| "session-info2", |
Member
Author
There was a problem hiding this comment.
Maybe this shouldn't be a runtime dependency
| "scverse-misc[datasets]>=0.1", | ||
| # for debug logging (referenced from the issue template) | ||
| "session-info2", | ||
| "setuptools", |
Member
Author
There was a problem hiding this comment.
Still required? In 2026?
| mamba install -c conda-forge spatialdata napari-spatialdata spatialdata-io spatialdata-plot | ||
| ``` | ||
|
|
||
| ## Limitations |
Member
Author
There was a problem hiding this comment.
We can keep this if you like. I always point Windows users to WSL and don't bother with Windows but that's totally up to you
Zethson
marked this pull request as ready for review
August 29, 2026 14:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Links the repository to
cookiecutter-scversevia.cruft.json, and brings packaging, tooling, CI, issue templates and docs configuration in line with the template.Adopting the template's mypy hook surfaced 500 errors, fixed here without adding a
cast(), a# type: ignoreor an excluded module, among them these bugs:validate_table_in_spatialdataguarded ondtype is str, which is never true of a dtype object, so the instance-key dtype mismatch has never been reported._resolve_zarr_storebranched onzarr.storage.ConsolidatedMetadataStore, which zarr 3 removed, so a group backed by anything other than a Local or Fsspec store raisedAttributeErrorinstead of theValueErroron the next line._resolve_zarr_storethen branched onisinstance(path, zarr.storage.StoreLike), a PEP 695 alias thatisinstancerejects withTypeError, so an already-resolved store never reached its branch.object_codectocreate_array, which zarr 3 removed, raisingTypeError._aggregate_image_by_labelscalled.compute()on the result ofxrspatial.stats, which is a pandas frame for numpy-backed values and only a dask one for dask-backed values.ImageTilesDatasetused.Aandto_circlesused.A1, the scipy sparse aliases removed in 1.14.Merging needs PyPI trusted publishing and codecov OIDC enabled on the repo, since
release.yamlandtest.yamlno longer usePYPI_API_TOKENandCODECOV_TOKEN.