Port 3D image processing tutorial into gallery. by mkcor · Pull Request #4850 · scikit-image/scikit-image

mkcor · 2020-07-16T15:35:24Z

Description

In this PR, we are porting an existing tutorial, i.e., https://github.com/scikit-image/skimage-tutorials/blob/master/lectures/three_dimensional_image_processing.ipynb, into the gallery of examples. This addition falls under #4601 and makes use of the 3D dataset available with Pooch (namely, cells.tif).

I have only included the first two sections of the original tutorial:

Load and display 3D images
Adjust exposure (in experimental images)

I think that the segmentation of cells and nuclei could be its own tutorial, and so could feature extraction. Each example should be self-contained, but could nevertheless refer to another example (e.g., this introduction to 3D image processing where we only explore slices and adjust exposure).

If core devs prefer keeping everything in one place, I'm happy to keep going and append sections to the same file.

Checklist

Docstrings for all functions
Gallery example in ./doc/examples (new features only)
Benchmark in ./benchmarks, if your changes aren't covered by an
existing benchmark
Unit tests
Clean style in the spirit of PEP8

For reviewers

Check that the PR title is short, concise, and will make sense 1 year
later.
Check that new functions are imported in corresponding __init__.py.
Check that new features, API changes, and deprecations are mentioned in
doc/release/release_dev.rst.

pep8speaks · 2020-07-16T15:35:28Z

Hello @mkcor! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file doc/examples/applications/plot_3d_image_processing.py:

Line 220:21: E703 statement ends with a semicolon
Line 281:31: E703 statement ends with a semicolon

Comment last updated at 2020-07-28 16:31:44 UTC

mkcor · 2020-07-17T08:18:34Z

@emmanuelle @alexdesiqueira @jni Following up on our sprint conversations, please have a look when you get a chance. Thanks!

doc/examples/applications/plot_3d_image_processing.py

emmanuelle · 2020-07-19T12:45:40Z

doc/examples/applications/plot_3d_image_processing.py

+
+This tutorial is an introduction to three-dimensional image processing. Images
+are represented as `numpy` arrays. A single-channel, or grayscale, image is a
+2D matrix of pixel intensities of shape `(row, column)`. We can construct a 3D


since you are porting markdown to rst, there are several places like here where you need the text to be enclosed in double backticks instead of single ones.

Oh, I didn't notice because it rendered nicely (sort of):

now I realize this is italic, not code-style!

indeed :-).

but are you ok changing the single backticks to double ones?

I'm definitely ok, in the sense that I don't have the strongest opinion. But I have hesitations in terms of consistency and pedagogy: Words in code style should refer to words... in code. For example, "the shape (plane, row, column)" (lines 9--10) should match the code we have line 98:

- (n_plane, n_row, n_col) = data.shape + (plane, row, column) = data.shape

There is some confusion because we are using "(plane, row, column)" to mean:

the number of planes, rows, columns (shape);

the dimensions (axes) as in line 34;

the indices (coordinates) along the plane, row, column dimensions as in the summary table, line 16.
In this context, I think that using italic style for one of the meanings not found in code would make sense.

Ok, I went for the following: e8c23f9. I hope it's not too heavy in terms of wording...
In the table, coordinates use the square-bracket notation, as they should in general and recalling/announcing code line 103--105.
Alternatively, I considered keeping the italic style to describe dimensions:

which could also make sense.

doc/examples/applications/plot_3d_image_processing.py

emmanuelle · 2020-07-19T13:01:33Z

doc/examples/applications/plot_3d_image_processing.py

+
+_, (a, b, c) = plt.subplots(ncols=3, figsize=(15, 5))
+
+show_plane(a, data[32], title="Plane = 32")


maybe write

(l_plane, l_row, l_col) = data.shape

and use l_plane // 2 etc. below?

Good point, hard-coding isn't justified here. Done 9bb3123. I have used notation (n_plane, n_row, n_col) though, it comes more natural to me.

doc/examples/applications/plot_3d_image_processing.py

emmanuelle · 2020-07-19T13:23:04Z

Thanks @mkcor ! I left a couple of comments. I think it's very useful to port the tutorials to the main doc, one reason being that they'll go through the traditional review process! For now I'm fine with having several tutorials instead of one very big one, as long as there is proper cross-referencing between them. See for example https://github.com/scikit-image/scikit-image/blob/master/doc/examples/applications/plot_human_mitosis.py#L50 for the syntax of such links.

At some point maybe I'll contribute a plotly paragraph to show how to visualize these data in plotly (without the need for a Jupyter kernel :-)) but I don't have the time at the moment.

Co-authored-by: Emmanuelle Gouillart <[email protected]>

mkcor · 2020-07-27T19:44:48Z

I think it's very useful to port the tutorials to the main doc, one reason being that they'll go through the traditional review process!

Absolutely!

For now I'm fine with having several tutorials instead of one very big one, as long as there is proper cross-referencing between them.

Ok.

See for example https://github.com/scikit-image/scikit-image/blob/master/doc/examples/applications/plot_human_mitosis.py#L50 for the syntax of such links.

Looks like it's not the link you meant to share 🤔

At some point maybe I'll contribute a plotly paragraph to show how to visualize these data in plotly (without the need for a Jupyter kernel :-)) but I don't have the time at the moment.

Oh, good idea!!

mkcor · 2020-07-27T19:47:04Z

doc/examples/applications/plot_3d_image_processing.py

+
+
+(n_plane, n_row, n_col) = data.shape
+_, (a, b, c) = plt.subplots(ncols=3, figsize=(15, 5))


I'm confused because the current guidelines say "Examples in the gallery should have a maximum figure width of 8 inches" but we keep using widths of 15, 16 inches throughout examples (not only this one)... 🤔 @emmanuelle

@stefanv is probably best placed to answer this question. I think it's not a huge deal because the gallery rescales the images. But it might affect font sizes, for example. Anyway, the answer is I don't know. =P

I have no idea of where this 8 inches come from :-).

The rule of thumb should be to always take a look at the generated html...

The rule of thumb should be to always take a look at the generated html...

Sure. So maybe this line

scikit-image/CONTRIBUTING.txt

Line 418 in ebb53a6

Note that gallery examples should have a maximum figure width of 8 inches.

could be removed or edited.

jni

Great! I've made only some very minor comments. Thank you @mkcor!

doc/examples/applications/plot_3d_image_processing.py

jni · 2020-07-28T01:57:01Z

doc/examples/applications/plot_3d_image_processing.py

+
+
+(n_plane, n_row, n_col) = data.shape
+_, (a, b, c) = plt.subplots(ncols=3, figsize=(15, 5))


@stefanv is probably best placed to answer this question. I think it's not a huge deal because the gallery rescales the images. But it might affect font sizes, for example. Anyway, the answer is I don't know. =P

doc/examples/applications/plot_3d_image_processing.py

jni · 2020-07-28T02:09:40Z

doc/examples/applications/plot_3d_image_processing.py

+# highest extremes. Clipping the darkest and brightest 0.5% of pixels will
+# increase the overall contrast of the image.
+
+vmin, vmax = np.percentile(data, q=(0.5, 99.5))


Personally, I prefer using quantiles, though it's a personal preference, consider this comment optional. =)

Suggested change

vmin, vmax = np.percentile(data, q=(0.5, 99.5))

vmin, vmax = np.quantile(data, q=(0.005, 0.995))

I went for a popularity contest to decide and percentile won. ;-)
I searched for "cell biology" "percentile" vs "cell biology" "quantile".

I also did not know that numpy had a quantile function :-). (but I'm happy that I learned this, since I also prefer a bit quantile to percentile, but I'm fine with letting the pop contest decide.)

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

emmanuelle · 2020-08-03T14:53:29Z

Merging, thanks a lot @mkcor!

This PR is an addition to the 3D image processing introduction tutorial, which landed with #4850. It relates to #4850 (comment) and #4762. I've made a slice explorer with Plotly so it renders even in the static HTML page. In future iterations, I'll add a 3D visualization showing the position of the selected slice within the dataset. I've chosen the density_heatmap function so I could readily use its animation_frame argument for the slider interaction (selection). This function is made for aggregation, which I'm bypassing by setting the number of bins to the data size... Ideally, I wanted to use imshow to view the image slices (planes) but this wouldn't offer such a convenient abstraction in terms of slider interaction. One issue when trying with this is that occasionally the Plotly plot would behave strangely when dragging the animation slider (the slider itself sometimes moved to a lower position on the screen mid-drag and the image would quit updating). Hitting the "home" button at top to reset the axes should restore it to a working state again. It might be somehow related to dragging too quickly as I never saw the issue if I just clicked around on different frames, only when dragging the slider. Co-authored-by: Emmanuelle Gouillart <[email protected]> Co-authored-by: Juan Nunez-Iglesias <[email protected]> Co-authored-by: Gregory R. Lee <[email protected]>

mkcor added 5 commits July 12, 2020 17:16

Port tutorial on 3D image processing

6af1175

Port tutorial up to first visualization

a07b874

Complete introduction text

5cb12dc

Port tutorial up to slice explorer

b988144

Port section about exposure

b5cabc3

mkcor added 3 commits July 16, 2020 17:38

Fix PEP 8 issues

bce9bbd

Add ipywidgets in docs requirements

d1f9574

Fix PEP 8 issue: 2 blank lines after function definition

8052ab6

emmanuelle reviewed Jul 19, 2020

View reviewed changes

doc/examples/applications/plot_3d_image_processing.py Outdated Show resolved Hide resolved

emmanuelle reviewed Jul 19, 2020

View reviewed changes

doc/examples/applications/plot_3d_image_processing.py Show resolved Hide resolved

emmanuelle reviewed Jul 19, 2020

View reviewed changes

doc/examples/applications/plot_3d_image_processing.py Show resolved Hide resolved

emmanuelle reviewed Jul 19, 2020

View reviewed changes

doc/examples/applications/plot_3d_image_processing.py Show resolved Hide resolved

emmanuelle reviewed Jul 19, 2020

View reviewed changes

doc/examples/applications/plot_3d_image_processing.py Show resolved Hide resolved

emmanuelle reviewed Jul 19, 2020

View reviewed changes

doc/examples/applications/plot_3d_image_processing.py Outdated Show resolved Hide resolved

emmanuelle mentioned this pull request Jul 19, 2020

2020's calendar of community management #4486

Closed

mkcor and others added 5 commits July 20, 2020 19:48

Specify suitable range along planes for slice exploration

225cb87

Co-authored-by: Emmanuelle Gouillart <[email protected]>

Fix indentation

e6b8a4e

Update doc/examples/applications/plot_3d_image_processing.py

8499f5a

Co-authored-by: Emmanuelle Gouillart <[email protected]>

Use exposure figure as thumbnail image

b5e6e4d

Apply suggestions from code review

9bb3123

alexdesiqueira added the 📄 type: Documentation Updates, fixes and additions to documentation label Jul 25, 2020

mkcor added 3 commits July 27, 2020 18:01

Clarify that widget does not show in static HTML

0e58bb1

Adapt data display to HTML static page

c1efdd4

Clarify notation of shape and coordinates

e8c23f9

mkcor commented Jul 27, 2020

View reviewed changes

jni approved these changes Jul 28, 2020

View reviewed changes

mkcor and others added 7 commits July 28, 2020 15:44

Update doc/examples/applications/plot_3d_image_processing.py

53c9ece

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

Update doc/examples/applications/plot_3d_image_processing.py

3997ec0

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

Update doc/examples/applications/plot_3d_image_processing.py

3096b99

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

Update doc/examples/applications/plot_3d_image_processing.py

d06a515

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

Update doc/examples/applications/plot_3d_image_processing.py

bfd752c

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

Update doc/examples/applications/plot_3d_image_processing.py

c295fc9

Co-authored-by: Juan Nunez-Iglesias <[email protected]>

Break line at 80 characters

5d801f5

emmanuelle merged commit 798b495 into scikit-image:master Aug 3, 2020

mkcor deleted the port-3D-tutorial branch August 31, 2020 07:14

mkcor mentioned this pull request Aug 31, 2020

Add (interactive) slice explorer in plotly. #4953

Merged

mkcor mentioned this pull request Feb 15, 2021

Proofread and crosslink examples with immunohistochemistry image. #5231

Merged

mkcor mentioned this pull request Jul 17, 2021

Replacing filters .format() calls with f-strings #5478

Merged


		_, (a, b, c) = plt.subplots(ncols=3, figsize=(15, 5))

		show_plane(a, data[32], title="Plane = 32")



		(n_plane, n_row, n_col) = data.shape
		_, (a, b, c) = plt.subplots(ncols=3, figsize=(15, 5))

	vmin, vmax = np.percentile(data, q=(0.5, 99.5))
	vmin, vmax = np.quantile(data, q=(0.005, 0.995))

Uh oh!

Conversation

mkcor commented Jul 16, 2020

Description

Checklist

For reviewers

Uh oh!

pep8speaks commented Jul 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2020-07-28 16:31:44 UTC

Uh oh!

mkcor commented Jul 17, 2020

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

emmanuelle commented Jul 19, 2020

Uh oh!

mkcor commented Jul 27, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jni left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emmanuelle commented Aug 3, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pep8speaks commented Jul 16, 2020 •

edited

Loading