Skip to content

Example using 3D multichannel image.#4946

Merged
grlee77 merged 26 commits intoscikit-image:mainfrom
mkcor:interact-3D-image
Apr 21, 2021
Merged

Example using 3D multichannel image.#4946
grlee77 merged 26 commits intoscikit-image:mainfrom
mkcor:interact-3D-image

Conversation

@mkcor
Copy link
Copy Markdown
Member

@mkcor mkcor commented Aug 28, 2020

Description

I just started a gallery example where I'd like to show possible ways of interacting with 3D multichannel images. I'm using the kidney tissue fluorescence data by @GenevieveBuckley of course. 💞

There's nothing interesting yet. The script does not even run as is, since #4939 isn't merged in yet.

I thought I would dive straight into the 3D challenges, then I realized that rendering the colours wasn't trivial: The image is closer to BGR than it is to RGB, but it's not exactly BGR either... 🙃

Checklist

For reviewers

  • Check that the PR title is short, concise, and will make sense 1 year
    later.
  • Check that new functions are imported in corresponding __init__.py.
  • Check that new features, API changes, and deprecations are mentioned in
    doc/release/release_dev.rst.

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

Thanks @mkcor!

@pep8speaks
Copy link
Copy Markdown

pep8speaks commented Sep 30, 2020

Hello @mkcor! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-04-21 11:24:37 UTC

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Sep 30, 2020

Now you understand why it's called confocal fluorescence microscopy!! w00t @GenevieveBuckley ❤️
newplot
Achieved with:

import plotly.express as px

px.imshow(data[n_plane // 2], zmin=v_min, zmax=v_max)

@emmanuelle @nicolaskruchten thanks for making imshow so powerful and flexible in Plotly Express!! I was looking at PR plotly/plotly.py#2691 and I simply used https://plotly.com/python/imshow/#defining-the-data-range-covered-by-the-color-range-with-zmin-and-zmax (great documentation, btw).

I cannot achieve it with:

from skimage import io

io.imshow(data[n_plane // 2], plugin='matplotlib', vmin=v_min, vmax=v_max)

fluo_fail
which roughly amounts to:

import matplotlib.pyplot as plt

plt.imshow(data[n_plane // 2], vmin=v_min, vmax=v_max)

It's as if I didn't pass any min/max values...

@emmanuelle
Copy link
Copy Markdown
Member

@mkcor yeah matplotlib's imshow uses vmin/vmax only for scalar (single-channel) data, it is ignored for multichannel data. We might checks with specialists (@GenevieveBuckley) if it's ok to change the contrast for each channel but since channels are not blended too much intermediate hues are not important. If it's confirmed that it's ok to change the contrast for fluorescence data then I'm really happy that we included this possibility in plotly's imshow. Maybe it's worth opening an issue in matplotlib's tracker to discuss this point?

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Oct 1, 2020

We might check with specialists (@GenevieveBuckley) if it's ok to change the contrast for each channel but since channels are not blended too much intermediate hues are not important. If it's confirmed that it's ok to change the contrast for fluorescence data then I'm really happy that we included this possibility in plotly's imshow.

@emmanuelle yes, I would also prefer double-checking with a specialist of the field if and how we should change the contrast in a fluorescence microscopy image. Notably, should we do it on a per channel basis? I understand that my doing

px.imshow(slice, zmin=v_min, zmax=v_max)

is equivalent to

px.imshow(
    slice,
    zmin=[v_min, v_min, v_min],
    zmax=[v_max, v_max, v_max]
)

and, while the maximum value happens to be the same across channels (in this case, 4095), the minimum value is not (it goes like [10, 68, 35]). Going for

px.imshow(
    slice,
    zmin=[data[:, :, :, 0].min(), data[:, :, :, 1].min(), data[:, :, :, 2].min()],
    zmax=[v_max, v_max, v_max]
)

doesn't feel great, but maybe it's done...? I would say that

px.imshow(
    slice,
    zmin=[0, 0, 0],
    zmax=[v_max, v_max, v_max]
)

looks most natural but, again, I'm not familiar enough with confocal fluorescence microscopy.

Let's poke some biology experts who might know better!
@GMFranceschini @slowkow @fran6w @ThomasWalter @kmader @IvoLeist

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Oct 1, 2020

@mkcor yeah matplotlib's imshow uses vmin/vmax only for scalar (single-channel) data, it is ignored for multichannel data.

Oh, right, the docs read "When using scalar data ..." indeed, but I didn't get that the parameter would just be ignored for multichannel data.

Maybe it's worth opening an issue in matplotlib's tracker to discuss this point?

Ok, taking care of it now.

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

Changing the contrast independently for each channel for visualization is fine.

@kmader
Copy link
Copy Markdown
Contributor

kmader commented Oct 2, 2020

Great demo! I would also say it's fine to normalize on a per-channel basis and for most cases, it will prevent really poorly scaled images (why is my figure all black/white? questions) and having things just work out of the box is nice.

I might have a note of it since many scikit-image tutorials end up as figures in papers (and comparing two groups with normalized channels might lead to wrong conclusions).

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Oct 2, 2020

@GenevieveBuckley @kmader thank you for your replies, they are really helpful!

Great demo! I would also say it's fine to normalize on a per-channel basis and for most cases, it will prevent really poorly scaled images (why is my figure all black/white? questions) and having things just work out of the box is nice.

@kmader ok, in this case I will favour the following option:

px.imshow(
    slice,
    zmin=[data[:, :, :, 0].min(), data[:, :, :, 1].min(), data[:, :, :, 2].min()],
    zmax=[data[:, :, :, 0].max(), data[:, :, :, 1].max(), data[:, :, :, 2].max()]
)

I might have a note of it since many scikit-image tutorials end up as figures in papers (and comparing two groups with normalized channels might lead to wrong conclusions).

Oh, I would be very interested in this note! Thank you so much, again.

@fran6w
Copy link
Copy Markdown

fran6w commented Oct 2, 2020

@mkcor Thank you for the poke! Python, bioinformatics... but I am not a biology expert ;-)

@GMFranceschini
Copy link
Copy Markdown

@mkcor glad you thought about me! Unfortunately, I have never worked with images so I can't help with this issue!

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Oct 2, 2020

@fran6w @GMFranceschini I know your expertise is in bioinformatics rather than cell imaging... I reached out just in case and we got answers anyway, so it's all good! Thanks anyway.

@mkcor mkcor marked this pull request as ready for review December 31, 2020 16:31
@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Mar 30, 2021

Re-posting:

@scikit-image/core please review. I'm happy to add an entry to the release notes, but I can't figure out which file under doc/release/ I'm supposed to edit... 🤔

@grlee77 grlee77 self-requested a review March 30, 2021 16:55
Copy link
Copy Markdown
Member

@jni jni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkcor apologies for the delay in reviewing! This is a great little doc! 😊 I've made two very minor suggestions, but this is ok to merge with or without them. Thank you!

Since this is just a doc change I'll merge in ~24h unless anyone has objections.

# We turn to `plotly`'s implementation of the `imshow` function, for it
# supports `value ranges
# <https://plotly.com/python/imshow/#defining-the-data-range-covered-by-the-color-range-with-zmin-and-zmax>`_
# beyond ``(0.0, 1.0)`` for floats and ``(0, 255)`` for integers.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this callout to the docs. =)

Co-authored-by: Juan Nunez-Iglesias <[email protected]>
Copy link
Copy Markdown
Contributor

@grlee77 grlee77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mkcor, this looks great.

My only question is about how we can make this and other Plotly-based examples more useful for users that try to run them from the command line.

One possibility it calling plotly.io.show on each figure, for example.:

fig = px.imshow(data[n_plane // 2], zmax=vmax)
plotly.io.imshow(fig)

If I do that everywhere in the example, I end up with one browser tab per plotly figure. I didn't check if that will interfere with the gallery example, though? Let me know what you think.

Adding a plt.show() at the end of the script can also keep the Matplotlib figure from disappearing immediately once the script completes. We have that in many of the existing Matplotlib-based examples.

@grlee77
Copy link
Copy Markdown
Contributor

grlee77 commented Mar 31, 2021

I'm happy to add an entry to the release notes, but I can't figure out which file under doc/release/ I'm supposed to edit

I think just doc/release/release_dev.rst.

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Apr 19, 2021

I think just doc/release/release_dev.rst.

@grlee77 oh, thanks! 84d65a8 I replaced the entry mentioning a previous biology example (#4648) with this current one, because #4648 made it into the 0.18 release so... I guess it should be pruned from release_dev.rst, once it lands in a given release? Not sure because a bunch of other Documentation entries can be found in both release_dev.rst and release_0.18.rst... 😕

@jni
Copy link
Copy Markdown
Member

jni commented Apr 20, 2021

@mkcor I think what's happened is that release_dev wasn't pruned after release. That would be my fault. Perhaps look at the log to see what happened in that file, if anything, since 0.18, and delete everything that was there at the 0.18 branch point. (663e9e4, I think.)

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Apr 21, 2021

Hi @grlee77,

My only question is about how we can make this and other Plotly-based examples more useful for users that try to run them from the command line.

Thank you for keeping this in mind! I admit I forgot about this.

One possibility is calling plotly.io.show on each figure, for example.:

fig = px.imshow(data[n_plane // 2], zmax=vmax)
plotly.io.imshow(fig)

or simply (sparing import plotly):

fig = px.imshow(data[n_plane // 2], zmax=vmax)
fig.show()

If I do that everywhere in the example, I end up with one browser tab per plotly figure.

Right. I think it's quite nice, but...

I didn't check if that will interfere with the gallery example, though? Let me know what you think.

It does! 😭 I tried and ended up with the same display issue as in: https://scikit-image.org/docs/dev/auto_examples/data/plot_3d.html It used to work, so I'm not sure when the issue arose (maybe a particular mix of Plotly and Sphinx versions...?). I'm noticing a display issue with the thumbnail as well.

Adding a plt.show() at the end of the script can also keep the Matplotlib figure from disappearing immediately once the script completes. We have that in many of the existing Matplotlib-based examples.

Good point. In this gallery example, only one figure (i.e., the first one) is a Matplotlib figure... But it's better than nothing: done 1a0a7af.

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Apr 21, 2021

@mkcor I think what's happened is that release_dev wasn't pruned after release. That would be my fault. Perhaps look at the log to see what happened in that file, if anything, since 0.18, and delete everything that was there at the 0.18 branch point. (663e9e4, I think.)

@jni ok, thanks. The overall cleanup could/should be a different PR, so I have pruned only the Documentation section (37b2b07, a0b0de7) by comparing release_dev.rst and release_0.18.rst side by side. I'll take care of item "Documentation has been added to the contributing notes about how to submit a gallery example" separately (either deleting it or finding corresponding PRs).

PR should be good to go now! :shipit:

@grlee77 grlee77 changed the title Start example using 3D multichannel image. Example using 3D multichannel image. Apr 21, 2021
@grlee77 grlee77 merged commit 5b47e84 into scikit-image:main Apr 21, 2021
@grlee77
Copy link
Copy Markdown
Contributor

grlee77 commented Apr 21, 2021

I opened #5341 about pruning the release_dev.rst file so we don't forget

@grlee77
Copy link
Copy Markdown
Contributor

grlee77 commented Apr 21, 2021

I tried and ended up with the same display issue as in: https://scikit-image.org/docs/dev/auto_examples/data/plot_3d.html

Is there an issue for that? Do you want to open a PR removing the call to show for now so things show up in the gallery as expected?

@mkcor mkcor deleted the interact-3D-image branch April 21, 2021 19:02
@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Apr 21, 2021

I tried and ended up with the same display issue as in: https://scikit-image.org/docs/dev/auto_examples/data/plot_3d.html

Is there an issue for that? Do you want to open a PR removing the call to show for now so things show up in the gallery as expected?

Yes, I'll take care of it tomorrow (there's currently no issue for that).

tupui pushed a commit to tupui/scikit-image that referenced this pull request Apr 27, 2021
* Start example using 3D multichannel image

* Add plotly as docs dependency

* set plotly renderer in conf.py

* Update data loading

* Visualize image slice with plotly's imshow

* Drop deprecated function

* Use second plot as thumbnail

* Improve code style and wording

* Add three interactive visualizations

* Tell a better story

* Include biological expertise by @GenevieveBuckley

* Apply suggestions from code review

Co-authored-by: Genevieve Buckley <[email protected]>

* Stick with usual convention for 3D spatial coords

* Revert "Stick with usual convention for 3D spatial coords"

This reverts commit 9656e41.

* Name dimensions [plane, row, column, channel]

* Apply suggestions from code review

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

* Add entry to release notes

* Keep the Matplotlib figure when running script from CLI

* Prune documentation items found in 0.18 release notes

* Prune documentation itemequivalent to one found in 0.18 release notes

Co-authored-by: Emmanuelle Gouillart <[email protected]>
Co-authored-by: Genevieve Buckley <[email protected]>
Co-authored-by: Juan Nunez-Iglesias <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.