Skip to content

[ISSUE #4082] Support S3 file source connector #4132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
xwm1992 merged 5 commits into from
Aug 15, 2023
Merged

[ISSUE #4082] Support S3 file source connector #4132

xwm1992 merged 5 commits into from
Aug 15, 2023

Conversation

@TheR1sing3un
Copy link
Member

@TheR1sing3un TheR1sing3un commented Jun 17, 2023

Fixes #4082 .

Motivation

Explain the content here.
Explain why you want to make the changes and what problem you're trying to solve.

Modifications

Describe the modifications you've done.

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

@TheR1sing3un
Copy link
Member Author

TheR1sing3un commented Jun 17, 2023

@xwm1992 PTAL~

@Alonexc
Copy link
Contributor

Alonexc commented Jun 20, 2023

please fix the build error.

qqeasonchen
qqeasonchen previously approved these changes Jun 20, 2023
View reviewed changes
@TheR1sing3un
Copy link
Member Author

TheR1sing3un commented Jun 25, 2023

please fix the build error.

Could u rerun these ci？Some build error bugfix pr just have been merged to master branch just now

@xwm1992
Copy link
Contributor

xwm1992 commented Jun 25, 2023

please fix the build error.

Could u rerun these ci？Some build error bugfix pr just have been merged to master branch just now

you can just merge the master branch code to your branch, this will re trigger the ci run.

@TheR1sing3un TheR1sing3un force-pushed the support_s3_connector branch from 59db8a6 to 388b6d5 Compare June 25, 2023 11:28
@TheR1sing3un
Copy link
Member Author

TheR1sing3un commented Jun 25, 2023

please fix the build error.

Could u rerun these ci？Some build error bugfix pr just have been merged to master branch just now

you can just merge the master branch code to your branch, this will re trigger the ci run.

I have pushed the branch after rebasing, but it seems like that these ci need a maintainer to trigger~

@codecov
Copy link

codecov bot commented Jun 29, 2023 •
edited
Loading

Codecov Report

Merging #4132 (6ad1fd8) into master (857ad6b) will decrease coverage by 0.06%.
Report is 2 commits behind head on master.
The diff coverage is 0.00%.

❗ Current head 6ad1fd8 differs from pull request most recent head 8fb302e. Consider uploading reports for the commit 8fb302e to get more accurate results

@@             Coverage Diff              @@
##             master    #4132      +/-   ##
============================================
- Coverage     17.81%   17.75%   -0.06%     
  Complexity     1502     1502              
============================================
  Files           594      599       +5     
  Lines         25350    25426      +76     
  Branches       2388     2391       +3     
============================================
  Hits           4515     4515              
- Misses        20400    20476      +76     
  Partials        435      435
Files Changed Coverage Δ
.../eventmesh/connector/s3/config/S3ServerConfig.java 0.00% <0.00%> (ø)
...eventmesh/connector/s3/server/S3ConnectServer.java 0.00% <0.00%> (ø)
...esh/connector/s3/source/config/S3SourceConfig.java 0.00% <0.00%> (ø)
...nector/s3/source/config/SourceConnectorConfig.java 0.00% <0.00%> (ø)
...nnector/s3/source/connector/S3SourceConnector.java 0.00% <0.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@TheR1sing3un
Copy link
Member Author

TheR1sing3un commented Jul 3, 2023

@Alonexc @qqeasonchen @xwm1992 Hi~ PTAL again!

mxsm
mxsm requested changes Jul 28, 2023
View reviewed changes
Copy link
Member

@mxsm mxsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TheR1sing3un The code is good, but the code structure of the project needs to be renamed and changed with reference to the existing structure

@TheR1sing3un
Copy link
Member Author

TheR1sing3un commented Jul 28, 2023

@TheR1sing3un The code is good, but the code structure of the project needs to be renamed and changed with reference to the existing structure

Got it! I will update it later.

@TheR1sing3un TheR1sing3un force-pushed the support_s3_connector branch from 493c469 to a196bc6 Compare August 8, 2023 14:48
@TheR1sing3un
feat(connector): add S3 file source connector with specified format
0dc5c05 
1. add S3 file source connector with specified format

Closes apache#4082
@TheR1sing3un
test(connector): add S3SourceConnectorTest to verify
80d260c 
1. add S3SourceConnectorTest to verify

Closes apache#4082
@TheR1sing3un
feat(connector): add S3SourceWorker
7b1668b 
1. add S3SourceWorker

Closes apache#4082
@TheR1sing3un
style(connector): add header license
532c8ec 
1. add header license

Closes apache#4082
@TheR1sing3un
refactor(connector): refactor S3 source connector
8fb302e 
1. refactor S3 source connector

Closes apache#4082
mxsm
mxsm approved these changes Aug 14, 2023
View reviewed changes
Copy link
Member

@mxsm mxsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xwm1992 xwm1992 merged commit c29ada6 into apache:master Aug 15, 2023
@pandaapo pandaapo added this to the 1.10 milestone Dec 5, 2023
@xwm1992 xwm1992 changed the title [ISSUE #4082][Task1] Support S3 file source connector [ISSUE #4082] Support S3 file source connector Dec 12, 2023
xuhongjia pushed a commit to Deckers-Ohana/eventmesh that referenced this pull request Mar 13, 2025
@TheR1sing3un
[ISSUE apache#4082][Task1] Support S3 file source connector (apache#4132
be669bd 
)

* feat(connector): add S3 file source connector with specified format

1. add S3 file source connector with specified format

Closes apache#4082

* test(connector): add S3SourceConnectorTest to verify

1. add S3SourceConnectorTest to verify

Closes apache#4082

* feat(connector): add S3SourceWorker

1. add S3SourceWorker

Closes apache#4082

* style(connector): add header license

1. add header license

Closes apache#4082

* refactor(connector): refactor S3 source connector

1. refactor S3 source connector

Closes apache#4082
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xwm1992 xwm1992 xwm1992 approved these changes

@mxsm mxsm mxsm approved these changes

@Alonexc Alonexc Alonexc approved these changes

@qqeasonchen qqeasonchen qqeasonchen left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.10.0

Development

Successfully merging this pull request may close these issues.

[Feature] Add S3 source/sink connector

6 participants

@TheR1sing3un @Alonexc @xwm1992 @mxsm @qqeasonchen @pandaapo