Skip to content

Made the APIs of the Tensorboard Logger and WandB Logger same#2826

Merged
vfdev-5 merged 12 commits intopytorch:masterfrom
guptaaryan16:master
Jan 18, 2023
Merged

Made the APIs of the Tensorboard Logger and WandB Logger same#2826
vfdev-5 merged 12 commits intopytorch:masterfrom
guptaaryan16:master

Conversation

@guptaaryan16
Copy link
Copy Markdown
Contributor

@guptaaryan16 guptaaryan16 commented Jan 16, 2023

Fixes #2791

Description: Added a getattr method to the Tensorboard file

Check list:

  • New tests are added (if a new feature is added)
  • New doc strings: description and/or example code are in RST format
  • Documentation is updated (if required)

@github-actions github-actions bot added the module: contrib Contrib module label Jan 16, 2023
Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guptaaryan16 thanks for a PR, I left few comments how to fix it

@guptaaryan16
Copy link
Copy Markdown
Contributor Author

Hey @vfdev-5, I made the changes, and I apologize for messing things but I have now tested them and they seem to be correct


wrapper(mock_engine, mock_logger, Events.ITERATION_STARTED)
mock_logger.add_scalar.assert_called_once_with("lr/group_0", 0.01, 123)
mock_logger.writer.add_scalar.assert_called_once_with("lr/group_0", 0.01, 123)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the test we still however want to test __getattr__, so let's keep mock_logger.add_scalar instead of mock_logger.writer.add_scalar and others where we call mock_logger.writer.some_func -> mock_logger.some_func

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok let me make the change

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey I did that and it failed

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, mock does not have __getattr__ replicated from the spec. Let's find another way to test that user can access methods like writer.add_scalar directly using logger.add_scalar. Do you have any ideas or suggestions ?

Copy link
Copy Markdown
Contributor Author

@guptaaryan16 guptaaryan16 Jan 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can have a multiple inheritance in the tensorboard logger such that it supports all methods without the writer instance variable. I am testing a few things in this area.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is the dict of tensorboard logger when I added SummaryWriter as second inheritance in TensorboardLogger class .
{'_mock_return_value': sentinel.DEFAULT, '_mock_parent': None, '_mock_name': None, '_mock_new_name': '', '_mock_new_parent': None, '_mock_sealed': False, '_spec_class': <class 'ignite.contrib.handlers.tensorboard_logger.TensorboardLogger'>, '_spec_set': None, '_spec_signature': <Signature (*args: Any, **kwargs: Any)>, '_mock_methods': ['_SummaryWriter__append_to_scalar_dict', 'abstractmethods', 'class', 'delattr', 'dict', 'dir', 'doc', 'enter', 'eq', 'exit', 'format', 'ge', 'getattr', 'getattribute', 'gt', 'hash', 'init', 'init_subclass', 'le', 'lt', 'module', 'ne', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref', '_abc_impl', '_check_caffe2_blob', '_create_opt_params_handler', '_create_output_handler', '_encode', '_get_comet_logger', '_get_file_writer', 'add_audio', 'add_custom_scalars', 'add_custom_scalars_marginchart', 'add_custom_scalars_multilinechart', 'add_embedding', 'add_figure', 'add_graph', 'add_graph_deprecated', 'add_histogram', 'add_histogram_raw', 'add_hparams', 'add_image', 'add_image_with_boxes', 'add_images', 'add_mesh', 'add_onnx_graph', 'add_openvino_graph', 'add_pr_curve', 'add_pr_curve_raw', 'add_scalar', 'add_scalars', 'add_text', 'add_video', 'attach', 'attach_opt_params_handler', 'attach_output_handler', 'close', 'export_scalars_to_json', 'flush'], '_spec_asyncs': [], '_mock_children': {'writer': }, '_mock_wraps': None, '_mock_delegate': None, '_mock_called': False, '_mock_call_args': None, '_mock_call_count': 0, '_mock_call_args_list': [], '_mock_mock_calls': [], 'method_calls': [], '_mock_unsafe': False, '_mock_side_effect': None, 'writer': }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess through this we can remove the need of writer.add_scalar

Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 Jan 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we could inherit also from SummaryWriter but this would imply tensorboard package as static ignite dependency vs right now it is a runtime dependency and TensorboardLogger class is exposed in ignite even if tensorboard package is not installed...

I think we can proceed with __getattr__ solution and just add a specific test such that we can ensure that methods logger.add_scalar is propagated to logger.writer.add_scalar.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you are right, get_attr is returning the self.writer methods but Magic block doesn't seem to recognise it so when I printed the dict of a tensorboardlogger I got the dict of the SummaryWriter

wrapper(mock_engine, mock_logger, Events.EPOCH_STARTED)
mock_logger.writer.add_scalar.assert_called_once_with("weights_norm/fc2/weight", 12.0, 5)
mock_logger.writer.reset_mock()
mock_logger.reset_mock()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's revert this line as well

@guptaaryan16
Copy link
Copy Markdown
Contributor Author

I think this will work @vfdev-5

@@ -1,5 +1,6 @@
import math
import os
import unittest
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using pytest so let's stick to their formalism without creating TestCase classes and just directly defining test functions.

Comment on lines +38 to +42
def test_add_scalar(self, mock_writer):
# Create a TensorboardLogger instance
logger = TensorboardLogger()
# Call the add_scalar method
logger.add_scalar("loss", 0.5)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check that logger.writer.add_scalar was called with ("loss", 0.5)

Copy link
Copy Markdown
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @guptaaryan16 , lgtm !

@vfdev-5 vfdev-5 merged commit e530c39 into pytorch:master Jan 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module: contrib Contrib module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WandBLogger and TensorboardLogger have different APIs for logging audio

2 participants