Skip to content

Epoch/Iteration interval#1870

Merged
wyli merged 2 commits intoProject-MONAI:masterfrom
danieltudosiu:1549_tensorboard
Mar 29, 2021
Merged

Epoch/Iteration interval#1870
wyli merged 2 commits intoProject-MONAI:masterfrom
danieltudosiu:1549_tensorboard

Conversation

@danieltudosiu
Copy link
Copy Markdown
Contributor

Description

Allowing the choice of epoch and iteration interval.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@danieltudosiu
Copy link
Copy Markdown
Contributor Author

@Nic-Ma @wyli

For the new code, how do we test that the event is fired x times in a given time frame?

Also, should we further normalize the steps of the loggers from epoch to iteration so we have the iteration and epoch-based loggers refer to the same time axis?

@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Mar 28, 2021

Hi @danieltudosiu ,

Thanks for your contribution.
May I know what's the feature request? When do you want to set intervals for TensorBoard logging? Maybe your training is too fast that every iteration doesn't have enough time to write TensorBoard?

Thanks.

@danieltudosiu
Copy link
Copy Markdown
Contributor Author

danieltudosiu commented Mar 28, 2021

Hi @Nic-Ma,

Not all networks need logging every epoch or every iteration (for example I log every 25 epochs in my research), so I am suggesting adding an interval argument so users can control the interval at which the logging happens.

This falls in line with the usual Ignite handlers and it is more of a quality of life improvement.

The pull request is in regards to Issue #1549

Cheers!

@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Mar 28, 2021

Hi @danieltudosiu ,

OK, that makes sense.
The CI failure of your PR is because you didn't sign off it, please refer to:
https://github.com/Project-MONAI/MONAI/blob/master/CONTRIBUTING.md#signing-your-work

Others look good to me.

Thanks.

@danieltudosiu
Copy link
Copy Markdown
Contributor Author

@Nic-Ma, perfect! Will do it today. Thanks!

danieltudosiu and others added 2 commits March 29, 2021 10:16
Allowing the choice of epoch and iteration interval.

Signed-off-by: Petru-Daniel Tudosiu <[email protected]>
@wyli wyli enabled auto-merge (squash) March 29, 2021 09:22
@wyli wyli merged commit 9f6ceeb into Project-MONAI:master Mar 29, 2021
@danieltudosiu danieltudosiu deleted the 1549_tensorboard branch April 4, 2021 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants