Skip to content

Add fluorescence microscopy images to data registry.#4939

Merged
emmanuelle merged 9 commits intoscikit-image:masterfrom
mkcor:add-data-by-genevieve
Sep 2, 2020
Merged

Add fluorescence microscopy images to data registry.#4939
emmanuelle merged 9 commits intoscikit-image:masterfrom
mkcor:add-data-by-genevieve

Conversation

@mkcor
Copy link
Copy Markdown
Member

@mkcor mkcor commented Aug 24, 2020

Description

This PR is a simple follow-up on this one https://gitlab.com/scikit-image/data/-/merge_requests/6 by @GenevieveBuckley. We will then be able to load these images, that she obtained in confocal fluorescence microscopy, within scikit-image (via pooch).

We will thus have a 2D multichannel image (lily-of-the-valley-fluorescence.tif) and a 3D multichannel image (kidney-tissue-fluorescence.tif) to play with! I'm excited for the upcoming gallery examples, notably more biomedical ones.

The only hesitation I had when submitting this was whether I should keep the descriptive (short enough) filenames or shorten them into lily.tif and kidney.tif, respectively.

Let me refer to #3384 and #4601, and thank @GenevieveBuckley again!

Checklist

For reviewers

  • Check that the PR title is short, concise, and will make sense 1 year
    later.
  • Check that new functions are imported in corresponding __init__.py.
  • Check that new features, API changes, and deprecations are mentioned in
    doc/release/release_dev.rst.

@pep8speaks
Copy link
Copy Markdown

pep8speaks commented Aug 24, 2020

Hello @mkcor! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 133:80: E501 line too long (90 > 79 characters)
Line 134:80: E501 line too long (88 > 79 characters)
Line 141:80: E501 line too long (106 > 79 characters)
Line 142:80: E501 line too long (109 > 79 characters)

Comment last updated at 2020-09-02 19:26:18 UTC

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 24, 2020

@emmanuelle fyi, and tagging @jni for review

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

I vote for "lily" and "kidney" as names, it's a good idea to shorten them.

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 25, 2020

I vote for "lily" and "kidney" as names, it's a good idea to shorten them.

Sold! Done f70dbf9. Yes, this way they're also in line with the other datasets in scikit-image, which are named with one word: cells, eagle, mitosis, ... And if they ever become famous, the world will talk about "the kidney image" just like "the iris dataset," you know. 😆

Thanks for weighing in, @GenevieveBuckley!

Copy link
Copy Markdown
Member

@sciunto sciunto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for folliwing up!

@sciunto sciunto self-requested a review August 27, 2020 05:45
@sciunto
Copy link
Copy Markdown
Member

sciunto commented Aug 27, 2020

A function must be added and we need to had an example in examples/data. For the gallery, a new page must be created for 3D dataset and we need to think how we can provide a preview -- we do not have the ability to interact with images yet.

@sciunto sciunto added this to the 0.18 milestone Aug 27, 2020
@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 27, 2020

we need to think how we can provide a preview -- we do not have the ability to interact with images yet.

@sciunto tomorrow I want to try something out with plotly :-)

@sciunto
Copy link
Copy Markdown
Member

sciunto commented Aug 27, 2020

we need to think how we can provide a preview -- we do not have the ability to interact with images yet.

@sciunto tomorrow I want to try something out with plotly :-)

Cool! :) (In case you haven't seen it: #4762)

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

we need to think how we can provide a preview -- we do not have the ability to interact with images yet.

@sciunto tomorrow I want to try something out with plotly :-)

You could also load the image into napari, switch it to 3D view and then take a screenshot (I had to split the color channels into separate layers to give them different colormaps). The kidney tissue is a pretty thin slice, so it doesn't look quite as impressive in 3D as other datasets might. Feel free to use these in the docs if they work for you.

kidney-screenshot-napari-viewer

kidney-screenshot2-napari-viewer

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

I don't know if you already have images in the docs for the brain image too, but it also looks pretty cool visualized in napari

Axial plane
brain-axial

3D view
brain-3D

@sciunto
Copy link
Copy Markdown
Member

sciunto commented Aug 28, 2020

It's a good idea, but we want to do not commit images to the main repository anymore.
Either the preview you suggest is added to the data repository, or we work on a visualization method for our documentation, or we make a simple 2D view and we improve it when we are ready to make nice views in our doc.

my taste for small incremental steps goes for the last solution.

my 2 cents.

@jni
Copy link
Copy Markdown
Member

jni commented Aug 28, 2020

I agree with @sciunto, we want to create our views programmatically from the data within our docs. Axis-aligned max projection is easy enough to do for now, and we can add something like the plotly Volume render eventually: https://plotly.com/python/3d-volume-plots/ (Though I don't really love those visuals...)

@jni
Copy link
Copy Markdown
Member

jni commented Aug 28, 2020

@GenevieveBuckley and if you meant run napari within the docs, let's first figure that out in the napari docs! 😂 Also we don't yet have a camera model, though Nick has that as his next priority.

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 28, 2020

Cool! :) (In case you haven't seen it: #4762)

Indeed, thanks for the pointer!

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 28, 2020

we need to {have, add} an example in examples/data

@sciunto not sure I understand correctly: Why should the example(s) necessarily fall under https://scikit-image.org/docs/stable/auto_examples/#data? I was taking a stab at an 'application' example ("Longer examples and demonstrations"): #4946

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 28, 2020

A function must be added

@sciunto do you mean data.kidney() and data.lily()? I didn't realize that! I guess we would also want data.mitosis() and data.cells(), which is not available at the moment:

def test_cells_3d():
"""Needs internet connection."""
path = fetch('data/cells.tif')
image = io.imread(path)
assert image.shape == (60, 256, 256)

@sciunto
Copy link
Copy Markdown
Member

sciunto commented Aug 28, 2020

A function must be added

@sciunto do you mean data.kidney() and data.lily()? I didn't realize that! I guess we would also want data.mitosis() and data.cells(), which is not available at the moment:

def test_cells_3d():
"""Needs internet connection."""
path = fetch('data/cells.tif')
image = io.imread(path)
assert image.shape == (60, 256, 256)

Absolutely. The aim of this page is to display the content of our dataset

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

@GenevieveBuckley and if you meant run napari within the docs, let's first figure that out in the napari docs! 😂 Also we don't yet have a camera model, though Nick has that as his next priority.

Nope, I didn't mean that! We're in agreement here.

I did not understand earlier that we wanted the figures/images for the docs to be auto-generated.

@hmaarrfk
Copy link
Copy Markdown
Member

A function must be added and we need to had an example in examples/data. For the gallery, a new page must be created for 3D dataset and we need to think how we can provide a preview -- we do not have the ability to interact with images yet.

I agree with this statement.

The file you are looking for is here:
https://github.com/scikit-image/scikit-image/blob/master/skimage/data/__init__.py

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 31, 2020

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 31, 2020


Returns
-------
lily : (922, 922, 4) uint16 ndarray
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this shape acceptable for our datasets?

I Feel like the output shape should be (4, 922, 922), in C-contiguous order

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, yeah that wasn't clear to me. I was assuming x, y, z

def kidney():
"""Microscopy image of kidney tissue.
See `kidney-tissue-fluorescence.tif` entry at
https://gitlab.com/scikit-image/data/-/blob/master/README.md#data
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there should be more fleshed out information here.

If you want to cross reference, then sphinx will have to fill things in for you.

I think it is important to be rather verbose in
https://scikit-image.org/docs/stable/api/skimage.data.html#skimage.data.brick

rather than sending the user around a multi click hunt.

@mkcor mkcor mentioned this pull request Aug 31, 2020
3 tasks
@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Aug 31, 2020

@hmaarrfk a3c73c2 I went the verbose/copying-pasting route you suggested.

@emmanuelle
Copy link
Copy Markdown
Member

@mkcor
Copy link
Copy Markdown
Member Author

mkcor commented Sep 2, 2020

Thanks @mkcor and reviewers! Could you just add lily to https://github.com/scikit-image/scikit-image/blob/master/doc/examples/data/plot_scientific.py#L16 before we merge this PR?

Sure! Done 973508e. Good to know of this list.

@emmanuelle
Copy link
Copy Markdown
Member

Merging, thanks !

We have some pretty bad CI failures :-(. But they are unrelated to this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants