Skip to content

Log the HTTP response in HttpToS3Operator when log_response is set - #72951

Merged
vincbeck merged 1 commit into
apache:mainfrom
bingqin2:http-to-s3-log-response
Sep 16, 2026
Merged

vincbeck merged 1 commit into
apache:mainfrom
bingqin2:http-to-s3-log-response

Conversation

@bingqin2

Copy link
Copy Markdown
Contributor

HttpToS3Operator documents log_response ("Log the response (default: False)") and stores it, but execute() never reads it, so the response body is never logged no matter what the user sets. This logs response.text after the request when the flag is set, the same way HttpOperator does.

Changes

  • providers/amazon/src/airflow/providers/amazon/aws/transfers/http_to_s3.py: log the response text when log_response is true
  • providers/amazon/tests/unit/amazon/aws/transfers/test_http_to_s3.py: test_execute_logs_response_only_when_requested, parametrized over both values

Testing

  • providers/amazon: tests/unit/amazon/aws/transfers/test_http_to_s3.py
  • mypy on the changed module, prek hooks on the changed files

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Fable 5.1) following the guidelines. I reviewed and understand all changes; the tests were run locally as listed above.


🤖 Generated with Claude Code

@bingqin2

Copy link
Copy Markdown
Contributor Author

@vincbeck @o-nikolas this has been sitting with green CI for five days. It is a two-line fix for a documented option that is never read, found by the audit in #72982. Could one of you take a look when you have a moment? Thanks.

@vincbeck
vincbeck merged commit 4783927 into apache:main Sep 16, 2026
83 checks passed
@bingqin2
bingqin2 deleted the http-to-s3-log-response branch September 30, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants