Skip to content

[AIRFLOW-3980] Unify logger#4804

Merged
Fokko merged 1 commit intoapache:masterfrom
mik-laj:logger
Mar 10, 2019
Merged

[AIRFLOW-3980] Unify logger#4804
Fokko merged 1 commit intoapache:masterfrom
mik-laj:logger

Conversation

@mik-laj
Copy link
Copy Markdown
Member

@mik-laj mik-laj commented Mar 1, 2019

Make sure you have checked all steps below.

Jira

Description

The logger was used incorrectly in several places. We should not format the text before passing it to the logger, because it can be ignored in the logger.

CC: @BasPH

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
    • All the public functions and the classes in the PR contain docstrings that explain what it does

Code Quality

  • Passes flake8

@XD-DENG
Copy link
Copy Markdown
Member

XD-DENG commented Mar 1, 2019

Description

The logger was used incorrectly in several places. We should not format the text before passing it to the logger, because it can be ignored in the logger.

Hi @mik-laj , does logger ignore format method? In which cases?

@BasPH
Copy link
Copy Markdown
Contributor

BasPH commented Mar 1, 2019

The logger does know str.format() but it's best practice in Python to use this form with logging: logger.info("Hello %s", "world") since that creates a string only when it actually gets logged. str.format() and f-strings create strings before they get passed to the logger, even if the string is never logged. So it's considered inefficient and not advised.

@mik-laj Nice work! Also got rid of some of the **locals()👍.

@mik-laj
Copy link
Copy Markdown
Member Author

mik-laj commented Mar 3, 2019

If parameters and text are pass separately, they can also be independently analyzed if more advanced loggers are used.

@mik-laj mik-laj force-pushed the logger branch 2 times, most recently from c68b88b to 02e0986 Compare March 3, 2019 03:22
@mik-laj
Copy link
Copy Markdown
Member Author

mik-laj commented Mar 3, 2019

I can not find any more problems. Can I ask you to review and accept changes?

@mik-laj mik-laj force-pushed the logger branch 2 times, most recently from a4d2af7 to bba53a9 Compare March 4, 2019 10:40
Comment thread airflow/contrib/sensors/sagemaker_training_sensor.py Outdated
Comment thread tests/utils/log/test_es_task_handler.py Outdated
Copy link
Copy Markdown
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two comments, apart from that: LGTM.

Thanks for picking this up

@mik-laj
Copy link
Copy Markdown
Member Author

mik-laj commented Mar 4, 2019

I updated a PR with all suggestion.

@Fokko
Copy link
Copy Markdown
Contributor

Fokko commented Mar 6, 2019

Thanks @mik-laj

Unfortunately the CI is sad now:

======================================================================
33) FAIL: test_exception_propagation (tests.executors.test_celery_executor.CeleryExecutorTest)
----------------------------------------------------------------------
   Traceback (most recent call last):
    tests/executors/test_celery_executor.py line 162 in test_exception_propagation
      self.assertIn('AttributeError', args[1])
   AssertionError: 'AttributeError' not found in AttributeError("'dict' object has no attribute 'state'",)

@mik-laj mik-laj force-pushed the logger branch 2 times, most recently from 35a1ef3 to c415804 Compare March 7, 2019 23:52
@codecov-io
Copy link
Copy Markdown

codecov-io commented Mar 8, 2019

Codecov Report

Merging #4804 into master will increase coverage by <.01%.
The diff coverage is 71.53%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4804      +/-   ##
==========================================
+ Coverage   75.28%   75.29%   +<.01%     
==========================================
  Files         450      450              
  Lines       29026    29019       -7     
==========================================
- Hits        21852    21849       -3     
+ Misses       7174     7170       -4
Impacted Files Coverage Δ
airflow/contrib/auth/backends/ldap_auth.py 0% <ø> (ø) ⬆️
airflow/sensors/external_task_sensor.py 94.64% <ø> (ø) ⬆️
airflow/contrib/operators/s3_list_operator.py 100% <ø> (ø) ⬆️
airflow/hooks/webhdfs_hook.py 36.36% <ø> (ø) ⬆️
airflow/sensors/hive_partition_sensor.py 0% <ø> (ø) ⬆️
...ntrib/sensors/aws_glue_catalog_partition_sensor.py 100% <ø> (ø) ⬆️
airflow/utils/log/gcs_task_handler.py 0% <ø> (ø) ⬆️
airflow/bin/cli.py 67.22% <0%> (ø) ⬆️
airflow/operators/check_operator.py 60.16% <0%> (+1.89%) ⬆️
airflow/hooks/http_hook.py 95.45% <0%> (ø) ⬆️
... and 45 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8dacae...c415804. Read the comment docs.

@mik-laj
Copy link
Copy Markdown
Member Author

mik-laj commented Mar 8, 2019

@Fokko Now Travis is green.

@Fokko
Copy link
Copy Markdown
Contributor

Fokko commented Mar 10, 2019

Thanks @mik-laj

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants