Skip to content

Comments

Add S3CreateObjectOperator#22758

Merged
eladkal merged 5 commits intoapache:mainfrom
vincbeck:s3_create_object
Apr 20, 2022
Merged

Add S3CreateObjectOperator#22758
eladkal merged 5 commits intoapache:mainfrom
vincbeck:s3_create_object

Conversation

@vincbeck
Copy link
Contributor

@vincbeck vincbeck commented Apr 5, 2022

Create a new operator S3CreateObjectOperator to create a new file in S3

@eladkal eladkal self-requested a review April 18, 2022 17:46
Copy link
Contributor

@eladkal eladkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vincbeck would you like to extend the example dag as @o-nikolas mentioned?

@github-actions github-actions bot added the okay to merge It's ok to merge this PR as it does not require more tests label Apr 19, 2022
@github-actions
Copy link

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@vincbeck
Copy link
Contributor Author

I updated the example which was already there to have some data which look like more "real world" example. I dont know if it's enough? Usually we do not mix up services in our sample dags, that's why I did not include any Xcom example. If you think it is really needed to have an example which uses another service and Xcom to push the results into S3, I'll do that

@eladkal
Copy link
Contributor

eladkal commented Apr 20, 2022

Usually we do not mix up services in our sample dags

It doesn't have to be. Example dags may serve more than one purpose - in this case my goal was to show that the data is actually metadata but I'm OK with leaving it as is.

@eladkal eladkal changed the title Add S3CreateObjectOperator Add S3CreateObjectOperator Apr 20, 2022
@eladkal eladkal merged commit 9e1ac6e into apache:main Apr 20, 2022
@ephraimbuddy ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Apr 26, 2022
@vincbeck vincbeck deleted the s3_create_object branch May 2, 2022 15:54
@simonprydden simonprydden mentioned this pull request Oct 23, 2023
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) kind:documentation okay to merge It's ok to merge this PR as it does not require more tests provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants