Note

This page was generated from a Jupyter notebook. Not all interactive visualization will work on this web page. Consider downloading the notebooks for full Python-backed interactivity.

# holoviews initialization

4. Motion Correction

4.1. estimate motion

Recall the parameters for estimate_motion:

param_estimate_motion

{'dim': 'frame'}

By default the motion estimation process is simple: for each pair of frames it calculates a phase correlation between the two frames using fft. Then the peak of the phase correlation will correspond to the translational shift between the two frames. The argument dim specifies along which dimension to run the motion estimation, and should always be set to "frame" here. Usually this step is parameter-free, but see API reference for estimate_motion for more advanced tweaking of the parameters. By default, the results from estimate_motion are saved in a two dimensional DataArray called motion, with two labels on the shift_dim dimension, representing the shifts along "height" and "width" directions.

%%time
motion = estimate_motion(varr_ref.sel(subset_mc), **param_estimate_motion)

CPU times: user 185 ms, sys: 12.1 ms, total: 197 ms
Wall time: 194 ms

4.2. save motion

Here is the first time we save a variable to the final output folder using param_save_minian:

param_save_minian

{'dpath': './demo_movies/minian',
 'meta_dict': {'session': -1, 'animal': -2},
 'overwrite': True}

As mentioned before param_save_minian decides how your data will be saved and what metadata will be stored. Additionally we use the chk variable earlier to make sure all our data have same chunk size along same dimension.

%%time
motion = save_minian(
    motion.rename("motion").chunk({"frame": chk["frame"]}), **param_save_minian
)

CPU times: user 3.7 s, sys: 415 ms, total: 4.11 s
Wall time: 1min 59s

4.3. visualization of motion

Here we visualize motion as a fluctuating curve across frames.

hv.output(size=output_size)
visualize_motion(motion)

4.4. apply transform

After determining each frame’s motion, we use the function apply_transform to correct for the motion. Notably, we have to decide what to do with pixels that are shifted from outside of the FOV. The default is to fill them with 0.

Y = apply_transform(varr_ref, motion, fill=0)

4.5. save result

Here we save two versions of the motion-corrected movie Y. Their contents are identical. The only difference is how they are chunked. Also note that we convert the data to float type for better downstream processing.

%%time
Y_fm_chk = save_minian(Y.astype(float).rename("Y_fm_chk"), intpath, overwrite=True)
Y_hw_chk = save_minian(
    Y_fm_chk.rename("Y_hw_chk"),
    intpath,
    overwrite=True,
    chunks={"frame": -1, "height": chk["height"], "width": chk["width"]},
)

CPU times: user 1.23 s, sys: 331 ms, total: 1.57 s
Wall time: 34.5 s

4.6. visualization of motion-correction

Here we visualize the result of motion correction Y_fm_chk side by side with input varr_ref.

hv.output(size=int(output_size * 0.7))
if interactive:
    vaviewer = VArrayViewer(
        [varr_ref.rename("before_mc"), Y_fm_chk.rename("after_mc")],
        framerate=5,
        summary=None,
        layout=True,
    )
    display(vaviewer.show())

A potentially better visualization is to look at max projcetion across all frames before and after the motion correction. You should see that the cells after motion correction have much more defined borders.

im_opts = dict(
    frame_width=500,
    aspect=varr_ref.sizes["width"] / varr_ref.sizes["height"],
    cmap="Viridis",
    colorbar=True,
)
(
    regrid(
        hv.Image(
            varr_ref.max("frame").compute().astype(np.float32),
            ["width", "height"],
            label="before_mc",
        ).opts(**im_opts)
    )
    + regrid(
        hv.Image(
            Y_hw_chk.max("frame").compute().astype(np.float32),
            ["width", "height"],
            label="after_mc",
        ).opts(**im_opts)
    )
)

4.7. generate video for motion-correction

Finally, we can generate a mp4 video for the original movie and motion-corrected movie side-by-side so that we can play it back at original speed.

%%time
vid_arr = xr.concat([varr_ref, Y_fm_chk], "width").chunk({"width": -1})
write_video(vid_arr, "minian_mc.mp4", dpath)

CPU times: user 2.27 s, sys: 2.66 s, total: 4.93 s
Wall time: 47.3 s

'/home/runner/work/minian/minian/demo_movies/minian_mc.mp4'

3. Pre-processing 5. Initialization