WASB remote logging too verbose in task logger #16224
Description
Apache Airflow version: 2.1.0
Environment:
apache-airflow-providers-microsoft-azure == 2.0.0
What happened:
When wasb remote logger wants to create container (always), the log is part of the task logger.
[2021-06-02 13:55:51,619] {wasb.py:385} INFO - Attempting to create container: xxxxx
[2021-06-02 13:55:51,479] {_universal.py:419} INFO - Request URL: 'XXXXXXX'
[2021-06-02 13:55:51,483] {_universal.py:420} INFO - Request method: 'HEAD'
[2021-06-02 13:55:51,490] {_universal.py:421} INFO - Request headers:
[2021-06-02 13:55:51,495] {_universal.py:424} INFO - 'x-ms-version': 'REDACTED'
[2021-06-02 13:55:51,499] {_universal.py:424} INFO - 'Accept': 'application/xml'
[2021-06-02 13:55:51,507] {_universal.py:424} INFO - 'User-Agent': 'azsdk-python-storage-blob/12.8.1 Python/3.7.10 (Linux-5.4.0-1046-azure-x86_64-with-debian-10.9)'
[2021-06-02 13:55:51,511] {_universal.py:424} INFO - 'x-ms-date': 'REDACTED'
[2021-06-02 13:55:51,517] {_universal.py:424} INFO - 'x-ms-client-request-id': ''
[2021-06-02 13:55:51,523] {_universal.py:424} INFO - 'Authorization': 'REDACTED'
[2021-06-02 13:55:51,529] {_universal.py:437} INFO - No body was attached to the request
[2021-06-02 13:55:51,541] {_universal.py:452} INFO - Response status: 404
[2021-06-02 13:55:51,550] {_universal.py:453} INFO - Response headers:
[2021-06-02 13:55:51,556] {_universal.py:456} INFO - 'Transfer-Encoding': 'chunked'
[2021-06-02 13:55:51,561] {_universal.py:456} INFO - 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
[2021-06-02 13:55:51,567] {_universal.py:456} INFO - 'x-ms-request-id': 'xxxx'
[2021-06-02 13:55:51,573] {_universal.py:456} INFO - 'x-ms-client-request-id': 'xxxx'
[2021-06-02 13:55:51,578] {_universal.py:456} INFO - 'x-ms-version': 'REDACTED'
[2021-06-02 13:55:51,607] {_universal.py:456} INFO - 'x-ms-error-code': 'REDACTED'
[2021-06-02 13:55:51,613] {_universal.py:456} INFO - 'Date': 'Wed, 02 Jun 2021 13:55:50 GMT'
What you expected to happen:
HTTP verbose logging not showing and "Attempting to create container" maybe it's not useful in the task log