Skip to content

Include support for jpk-qi-data files - #206

Open
tobyallwood wants to merge 227 commits into
mainfrom
tobyallwood/adjusted_volumes
Open

tobyallwood wants to merge 227 commits into
mainfrom
tobyallwood/adjusted_volumes

Conversation

@tobyallwood

@tobyallwood tobyallwood commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Adds support for .jpk-qi-data files, allowing the loading of the image for the selected channel. It also allows for the raw force-distance curve data as well as metadata for each curve to be loaded efficiently, using a lazy data structure approach that loads each section of the dataset into memory only when it is required.
The updates also allow for the conversion of .jpk-qi-data files into a HDF5 file format (.h5-jpk) for much faster mass analysis. Hence, the h5-jpk loading code has also been updated to support the loading of these converted files.

The update also allows .bin with different binary formats to be loaded.

…files and removing the ability to run curve analysis directly in the reader
…g to memory then saving in one go when duplicating data to h5
…sampled curves then are streamed directly into h5

@ahobbs7 ahobbs7 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, small thing about flip_image?

Comment thread AFMReader/h5_jpk.py Outdated
dict
A dictionary containing the QI curve data for each channel and segment for a pixel.
"""
if flip_image is None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to still allow flip image from the dataclass to overridden by the argument of the function, I believe it is used in napari-TopoStats

Comment thread AFMReader/h5_saver.py
predicted_points_per_channel_segment[segment_name][channel] = (
int(np.max(points_for_channel_segment[segment_name][channel]) * 1.1) * num_of_curves
)
max_points + (max_points + 9) // 10

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: not sure what the +9 is for? Is it to ensure that there is a minimum 'buffer' above the curves if the curves have very few data points?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahobbs7 yeah it's still +10% but using integer values can avoid floating point errors. // rounds down so we add 9 to push it to the next integer (a.k.a rounding up so there's always headroom).

Comment thread AFMReader/jpk_qi.py


def _get_channel_scaling(loaded_jpk_properties: dict, channel_index: str) -> tuple[float, float, str]:
def _get_channel_scaling(loaded_jpk_properties: dict, channel_index: int) -> tuple[float, float, str]:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

@tobyallwood
tobyallwood marked this pull request as ready for review November 10, 2026 11:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants