Add load_templates_clusters parameter (kilosort/phy output)

[pas-calc]

Added a new parameter 'load_templates_clusters' to control loading of templates or clusters from Kilosort output. Updated logic to handle loading based on the parameter value.

Have "auto" as default to ensure backward compatibility, as this was the previous implementation.

related:
#4373

We could also store a kwarg dict / attribute so that we later know from where we got the clusters, especially if "auto" was chosen.

[chrishalcrow]

Hi @pas-calc, thanks for this! It sounds like it's correct for the default to be "templates" since this always represents the "raw" outputs. This function is very new and hasn't been in a release version yet, so we don't need to be so careful about backward compatibility. Hence I would vote to make "templates" the default and remove the "auto" option. What do you think?

Maybe we should also make it clear in the docstring that this function is only tested for data that has not been modified by Phy.

EDIT: also, do you know if kilosort 2.5 also outputs both of these files?

[pas-calc]

Basically I wrapped the original code inside the if load_templates_clusters=="auto": so I'd interpret that "auto" is then the method in the implementation before that commit.

Kilosort2.5 outputs "spike_templates.npy"

[chrishalcrow]

Hi @pas-calc ,

Sorry for the delay with all the kilosort loading stuff.

I've been looking at this alongside other PRs. Kilosort output has spike_clusters.npy and spike_templates.npy. If you then do manual curation templates is left alone, and clusters is modified. Phy also generates a bunch of metrics e.g. cluster_waveform_baseline_flatness.tsv. These metrics correspond to the curated units - e.g. the ones in clusters. When we use load_phy it also loads these metrics and this is why we use clusters by default. If you were to use the templates, the metrics and units wouldn't match.

So if you want to load the templates, we shouldn't load any of the metrics. I've made the adjustments to the PR.

After understanding this and implementing the don't-get-metrics stuff, I'm not quite sure what the use case for this is. What would you use this feature for?