Skip to content

Update profiling code with NVTX handlers#310

Merged
wyli merged 6 commits intoProject-MONAI:masterfrom
bhashemian:profile-handlers
Aug 20, 2021
Merged

Update profiling code with NVTX handlers#310
wyli merged 6 commits intoProject-MONAI:masterfrom
bhashemian:profile-handlers

Conversation

@bhashemian
Copy link
Copy Markdown
Member

@bhashemian bhashemian commented Aug 13, 2021 •
edited
Loading

Description

This PR update the NVTX profiling to include NVTX handlers, and also update the RangePush/RangePop to the simpler interface of Range.

This requires MONAI/PR2765 to be merged first.

Status

Ready

Checks

@bhashemian bhashemian mentioned this pull request Aug 13, 2021
Merged
6 tasks
@review-notebook-app
Copy link
Copy Markdown

review-notebook-app bot commented Aug 19, 2021

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@bhashemian
minor changes
23d3673 
Signed-off-by: Behrooz <[email protected]>
@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented Aug 19, 2021 •
edited
Loading

@Nic-Ma @wyli, this PR is ready to be reviewed/merged. Thanks

@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented Aug 19, 2021

As I mentioned in #308, this notebook now tries to download two large histopathology images. Although the runner should work but it might take a while depending on the server speed (about 1 hour). Do you want me to exclude it from runner.sh again or it's fine since it is a cron job?

@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Aug 19, 2021

Hi @wyli ,

I feel 1 hour is acceptable for cron job, but it should be stable enough.
What do you think?

Thanks.

@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented Aug 19, 2021 •
edited
Loading

I saw that the cron job has 150min timeout for all the notebook. I'll move things to our google drive (instead of outside ftp) to speed up the download, and will change this notebook accordingly.

@bhashemian
Copy link
Copy Markdown
Member Author

bhashemian commented Aug 20, 2021

@Nic-Ma @wyli,
I updated the notebook and now it's only doing profiling for training (not validation) and only with the smallest Camelyon tumor image. I moved the image to our google drive and now it's very fast to download and depending on internet speed it should take more than few minutes.

I don't have write access to this repo so can any of you merge it. Thanks

wyli
wyli approved these changes Aug 20, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for enhancing the tests!

@wyli wyli merged commit 25700d0 into Project-MONAI:master Aug 20, 2021
@bhashemian bhashemian deleted the profile-handlers branch August 20, 2021 00:33
@wyli wyli mentioned this pull request Aug 24, 2021
Closed
boneseva pushed a commit to boneseva/MONAI-tutorials that referenced this pull request Apr 21, 2024
@bhashemian
Update profiling code with NVTX handlers (Project-MONAI#310)
4280d4b 
* Update profiling code with NVTX handlers

Signed-off-by: Behrooz <[email protected]>

* Change a range name

Signed-off-by: Behrooz <[email protected]>

* Enable test for camelyon and add it to doesnt_contain_max_epochs

Signed-off-by: Behrooz <[email protected]>

* Add data prepration and dependencies

Signed-off-by: Behrooz <[email protected]>

* minor changes

Signed-off-by: Behrooz <[email protected]>

* Update with a smaller gdrive-hosted  WSI

Signed-off-by: Behrooz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wyli wyli wyli approved these changes

@Nic-Ma Nic-Ma Nic-Ma approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bhashemian @Nic-Ma @wyli