Note
This page was generated from a Jupyter notebook. Not all interactive visualization will work on this web page. Consider downloading the notebooks for full Python-backed interactivity.
[ ]:
# holoviews initialization
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.Dimension: Use method 'get_param_values' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
3. Cross-session registration¶
3.1. calculate centroids¶
We start by calculating a centroid of spatial footprint for each cell. The centroid location is the only source of information used to register cells across sessions.
[10]:
%%time
cents = calculate_centroids(A_shifted, window)
computing centroids
[######## ] | 20% Completed | 15.3s
/usr/share/miniconda/envs/minian/lib/python3.8/site-packages/scipy/ndimage/measurements.py:1406: RuntimeWarning: invalid value encountered in double_scalars
results = [sum(input * grids[dir].astype(float), labels, index) / normalizer
[########################################] | 100% Completed | 17.1s
[########################################] | 100% Completed | 17.2s
CPU times: user 14.8 s, sys: 11 s, total: 25.8 s
Wall time: 17.2 s
3.2. calculate centroid distance¶
We then calculate pairwise distance between cells in all pairs of sessions. Note that at this stage, since we are computing something along the session dimension, it is no longer considered as a metadata dimension, so we remove it.
[11]:
%%time
id_dims.remove("session")
dist = calculate_centroid_distance(cents, index_dim=id_dims)
creating parallel schedule
computing distances
CPU times: user 2.2 s, sys: 8.45 ms, total: 2.21 s
Wall time: 2.29 s
3.3. threshold centroid distances¶
We threshold the centroid distances and keep only cell pairs with distance less than param_dist.
[12]:
dist_ft = dist[dist['variable', 'distance'] < param_dist].copy()
dist_ft = group_by_session(dist_ft)
3.4. generate mappings¶
Finally we generate mapping of cells across session in three steps:
We filter the pairwise distances into pairwise mappings by applying a mutual nearest-neighbour criteria, using
calculate_mapping.We extend/merge pairwise mappings into multi-session mappings and drop any conficting mappings, using
resolve_mapping.We fill in “mappings” that represent cells only appeared in single sessions, using
fill_mapping.
Please see the API reference for more detail on the output dataframe format.
[13]:
%%time
mappings = calculate_mapping(dist_ft)
mappings_meta = resolve_mapping(mappings)
mappings_meta_fill = fill_mapping(mappings_meta, cents)
mappings_meta_fill.head()
CPU times: user 1.5 s, sys: 275 µs, total: 1.5 s
Wall time: 1.5 s
[13]:
| group | session | variable | ||
|---|---|---|---|---|
| group | session1 | session2 | distance | |
| 0 | (session1, session2) | 100.0 | 108.0 | NaN |
| 1 | (session1, session2) | 72.0 | 89.0 | NaN |
| 2 | (session1, session2) | 57.0 | 64.0 | NaN |
| 3 | (session1, session2) | 142.0 | 144.0 | NaN |
| 4 | (session1, session2) | 162.0 | 177.0 | NaN |
3.5. visualize mappings¶
We visualize the matching of cells by color-mapping cells 3 arbitrary sessions into RGB channels and plot the overlay image. Please see API reference for more details on the tools available in this visualization.
[14]:
hv.output(size=int(output_size * 0.7))
alnviewer = AlignViewer(minian_ds, cents, mappings_meta_fill, shiftds)
alnviewer.show()
WARNING:param.BokehRenderer: Use method 'get_param_values' via param namespace
WARNING:param.BokehRenderer: Use method 'get_param_values' via param namespace
[######################## ] | 60% Completed | 9.9s
/home/runner/work/minian/minian/minian/visualization.py:1495: RuntimeWarning: All-NaN slice encountered
amax = np.nanmax(a)
/home/runner/work/minian/minian/minian/visualization.py:1496: RuntimeWarning: All-NaN slice encountered
amin = np.nanmin(a)
[########################################] | 100% Completed | 12.3s
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.RGBPlot: Use method 'params' via param namespace
WARNING:param.RGBPlot: Use method 'params' via param namespace
WARNING:param.RGBPlot: Use method 'params' via param namespace
WARNING:param.RGBPlot: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
WARNING:param.ParameterizedMetaclass: Use method 'params' via param namespace
[14]:
3.6. save results¶
If everything looks good, we commit by saving the mappings into pickle file. Optionally we also save centroids cents and shiftds in case they come in handy in down-stream analysis.
[15]:
mappings_meta_fill.to_pickle(os.path.join(dpath, "mappings.pkl"))
cents.to_pickle(os.path.join(dpath, "cents.pkl"))
shiftds.to_netcdf(os.path.join(dpath, "shiftds.nc"))