Skip to content

Disable the logging of imported and third-party libraries#725

Merged
mahendrark merged 3 commits intodevfrom
fix/decrease_log_messages
Dec 18, 2020
Merged

Disable the logging of imported and third-party libraries#725
mahendrark merged 3 commits intodevfrom
fix/decrease_log_messages

Conversation

@mahendrark
Copy link
Copy Markdown
Contributor

@mahendrark mahendrark commented Dec 16, 2020 •
edited
Loading

Fix #717

Changes proposed in this pull request:

  • Remove logging of log messages from third party libraries in the log file

The following steps were realized, as well (if applies):

  • Use in-line comments to explain your code
  • Write docstrings to your code (example docstring)
  • For new functionalities: Explain in readthedocs
  • Write test(s) for your new patch of code (pytests, assertion debug messages)
  • Update the CHANGELOG.md
  • Apply black (black . --exclude docs/)
  • Check if benchmark tests pass locally (EXECUTE_TESTS_ON=master pytest)

Please mark above checkboxes as following:

  • Open
  • Done

❌ Check not applicable to this PR

For more information on how to contribute check the CONTRIBUTING.md.

@mahendrark mahendrark marked this pull request as draft December 16, 2020 14:42
@mahendrark mahendrark added the enhancement New feature or request label Dec 16, 2020
@mahendrark mahendrark changed the title Disable the logging of websockets and asyncio modules to reduce logs Disable the logging of imported and third-party libraries Dec 17, 2020
@mahendrark
Copy link
Copy Markdown
Contributor Author

mahendrark commented Dec 17, 2020

@smartie2076

I managed to suppress all, but the messages from dash (about 11 lines). I could not find a solution for those yet, and my issue is still open on their repo without a comment. I will try their community forum next and see if someone responds.

I will push the changes. Can you please test it in your PC and let me know if everything is good? Thanks!

@mahendrark mahendrark force-pushed the fix/decrease_log_messages branch from 4573727 to 245b049 Compare December 17, 2020 09:03
smartie2076
smartie2076 approved these changes Dec 17, 2020
View reviewed changes
Copy link
Copy Markdown
Collaborator

@smartie2076 smartie2076 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better! Thank you!

I have only those error message types left:

2020-12-17 15:07:20,026 - DEBUG - files - write 3939 bytes to 'D:\\PycharmProjects\\mvs_eland\\MVS_outputs\\energy_system_graph'
2020-12-17 15:07:20,026 - DEBUG - backend - run ['dot', '-Tpng', '-O', 'energy_system_graph']
...
2020-12-17 15:11:40,123 - DEBUG - connectionpool - Starting new HTTP connection (1): a.tile.komoot.de:80
...
2020-12-17 15:11:40,424 - DEBUG - PngImagePlugin - STREAM b'IHDR' 16 13
...
2020-12-17 15:12:27,290 - INFO - dash - Dash is running on http://127.0.0.1:8050/
2020-12-17 15:12:30,322 - DEBUG - client - client > GET /devtools/browser/81cf389f-f453-4bc8-a8a4-b7151c0f013a HTTP/1.1
...

@mahendrark mahendrark marked this pull request as ready for review December 17, 2020 14:20
@mahendrark
Copy link
Copy Markdown
Contributor Author

mahendrark commented Dec 17, 2020

Much better! Thank you!

I have only those error message types left:

2020-12-17 15:07:20,026 - DEBUG - files - write 3939 bytes to 'D:\\PycharmProjects\\mvs_eland\\MVS_outputs\\energy_system_graph'
2020-12-17 15:07:20,026 - DEBUG - backend - run ['dot', '-Tpng', '-O', 'energy_system_graph']
...
2020-12-17 15:11:40,123 - DEBUG - connectionpool - Starting new HTTP connection (1): a.tile.komoot.de:80
...
2020-12-17 15:11:40,424 - DEBUG - PngImagePlugin - STREAM b'IHDR' 16 13
...
2020-12-17 15:12:27,290 - INFO - dash - Dash is running on http://127.0.0.1:8050/
2020-12-17 15:12:30,322 - DEBUG - client - client > GET /devtools/browser/81cf389f-f453-4bc8-a8a4-b7151c0f013a HTTP/1.1
...

Aha! I have a different set of messages, 11 lines only from dash.

@smartie2076
Copy link
Copy Markdown
Collaborator

smartie2076 commented Dec 17, 2020

Hi @mahendrark! Just for completeness sake - this is the complete log file:
mvs_logfile.txt

@smartie2076
Copy link
Copy Markdown
Collaborator

smartie2076 commented Dec 17, 2020 •
edited
Loading

Do you plan to check my log messages out? If not, which would be fine, change your first post to adresses #717 and mention in it that there are logging messages left to be removed, then merge.

@mahendrark
Copy link
Copy Markdown
Contributor Author

mahendrark commented Dec 17, 2020

Do you plan to check my log messages out? If not, which would be fine, change your first post to adresses #717 and mention in it that there are logging messages left to be removed, then merge.

Of course! Thats why I asked you to try running from this branch and let me know :P

I am still waiting for the answer from dash guys as to how to suppress the messages from that library. If they help out, I can make them disappear too and then I would like to look at the ones in your logs.

@mahendrark
Copy link
Copy Markdown
Contributor Author

mahendrark commented Dec 17, 2020

Hi @mahendrark! Just for completeness sake - this is the complete log file:
mvs_logfile.txt

Holy hell! I thought connectionpool and PngImagePlugin disappeared ... Do have the latest version of this branch @smartie2076 ?

I added code to suppress the messages from those libraries latter. Can you please check and confirm? Danke!

@smartie2076
Copy link
Copy Markdown
Collaborator

smartie2076 commented Dec 17, 2020

Hi @mahendrark! Just for completeness sake - this is the complete log file:
mvs_logfile.txt

Holy hell! I thought connectionpool and PngImagePlugin disappeared ... Do have the latest version of this branch @smartie2076 ?

I added code to suppress the messages from those libraries latter. Can you please check and confirm? Danke!

False alarm! So weird, I did pull it from github before executing, but maybe I should have also updated my project first? Strange.

So, everything is all right, there are 10 max (maybe less) plotly dash messages left :D

@mahendrark
Copy link
Copy Markdown
Contributor Author

mahendrark commented Dec 18, 2020

As the dash community did not respond to my questions on both their git repo issues section or their community forum, and considering that the current log file is quite manageable in terms of the number of lines, I am merging and closing this PR for now. In the future, if I come across a fix for the dash logging, I will implement it.

@mahendrark mahendrark merged commit 0a13c1d into dev Dec 18, 2020
@mahendrark mahendrark deleted the fix/decrease_log_messages branch December 18, 2020 08:37
Bachibouzouk
Bachibouzouk reviewed Jan 4, 2021
View reviewed changes
src/multi_vector_simulator/A0_initialization.py
"websockets.protocol",
"websockets.client",
"urllib3.connectionpool",
"PIL.PngImagePlugin",
Copy link
Copy Markdown
Collaborator

@Bachibouzouk Bachibouzouk Jan 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"werkzeug" for dash (it is the one of flask, on which dash piggybacks) and "pyppeteer" for the browser emulator. It is wierd to me that the commands on line

multi-vector-simulator/src/multi_vector_simulator/F2_autoreport.py

Line 37 in a2d27f9

flask_log = logging.getLogger("werkzeug")
does not work to suppress dash messages?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Bachibouzouk Bachibouzouk Bachibouzouk left review comments

@smartie2076 smartie2076 smartie2076 approved these changes

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Supress logging messages from packages

3 participants

@mahendrark @smartie2076 @Bachibouzouk