Skip to content

[Feature][Resource Center] Add support for Huawei Cloud OBS #14643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
rickchengx merged 17 commits into from
Aug 14, 2023
Merged

[Feature][Resource Center] Add support for Huawei Cloud OBS #14643

rickchengx merged 17 commits into from
Aug 14, 2023

Conversation

@ediconss
Copy link
Contributor

@ediconss ediconss commented Jul 27, 2023 •
edited
Loading

Purpose of the pull request

Add support for Huawei Cloud OBS

Verify this pull request

UT and manually tested

  1. configure in the common.properties
 resource.huawei.cloud.access.key.id=<your-access-key-id>
 resource.huawei.cloud.access.key.secret=<your-access-key-secret>
 resource.huawei.cloud.obs.bucket.name=dolphinscheduler
 resource.huawei.cloud.obs.endpoint=obs.cn-southwest-2.huaweicloud.com
  1. Upload files and directory in the resource center
    obs

SbloodyS
SbloodyS reviewed Jul 27, 2023
View reviewed changes
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add it to the docs too. @ediconss

@SbloodyS SbloodyS added first time contributor First-time contributor miss:docs missing documents in PR 3.2.0 for 3.2.0 version labels Jul 27, 2023
@SbloodyS SbloodyS added this to the 3.2.0 milestone Jul 27, 2023
@SbloodyS SbloodyS added the feature new feature label Jul 27, 2023
@ediconss
Copy link
Contributor Author

ediconss commented Jul 27, 2023 •
edited by SbloodyS
Loading

Please add it to the docs too. @ediconss

this doc file?
docs/docs/zh/guide/resource/configuration.md

@SbloodyS
Copy link
Member

SbloodyS commented Jul 27, 2023

Please add it to the docs too. @ediconss

this doc file? docs/docs/zh/guide/resource/configuration.md

Both docs/docs/zh/guide/resource/configuration.md and docs/docs/en/guide/resource/configuration.md.

Radeity
Radeity reviewed Jul 27, 2023
View reviewed changes
@rickchengx rickchengx self-requested a review July 27, 2023 10:17
rickchengx
rickchengx reviewed Jul 27, 2023
View reviewed changes
@SbloodyS SbloodyS removed the miss:docs missing documents in PR label Jul 28, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jul 28, 2023 •
edited
Loading

Codecov Report

Merging #14643 (2a3b92d) into dev (dde45db) will decrease coverage by 0.07%.
The diff coverage is 30.73%.

❗ Current head 2a3b92d differs from pull request most recent head 9c182ee. Consider uploading reports for the commit 9c182ee to get more accurate results

@@             Coverage Diff              @@
##                dev   #14643      +/-   ##
============================================
- Coverage     38.70%   38.64%   -0.07%     
- Complexity     4627     4651      +24     
============================================
  Files          1285     1287       +2     
  Lines         43973    44216     +243     
  Branches       4859     4874      +15     
============================================
+ Hits          17020    17087      +67     
- Misses        25075    25238     +163     
- Partials       1878     1891      +13
Files Changed Coverage Δ
...e/dolphinscheduler/common/constants/Constants.java 75.00% <ø> (ø)
.../plugin/storage/obs/ObsStorageOperatorFactory.java 0.00% <0.00%> (ø)
...olphinscheduler/plugin/task/api/TaskConstants.java 0.00% <ø> (ø)
...heduler/plugin/storage/obs/ObsStorageOperator.java 31.22% <31.22%> (ø)
...e/dolphinscheduler/common/enums/ResUploadType.java 100.00% <100.00%> (ø)

... and 4 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@rickchengx
Copy link
Contributor

rickchengx commented Aug 1, 2023

Hi, @ediconss could you please give some screenshots of the files uploaded through the resource center in OBS

@ediconss
Copy link
Contributor Author

ediconss commented Aug 2, 2023 •
edited
Loading

Hi, @ediconss could you please give some screenshots of the files uploaded through the resource center in OBS

sure

@ediconss ediconss closed this Aug 2, 2023
@ediconss ediconss reopened this Aug 2, 2023
rickchengx
rickchengx approved these changes Aug 8, 2023
View reviewed changes
Copy link
Contributor

@rickchengx rickchengx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work @ediconss
LGTM if CI pass

@rickchengx rickchengx added 4.0.0 and removed 3.2.0 for 3.2.0 version labels Aug 10, 2023
@sonarqubecloud
Copy link

sonarqubecloud bot commented Aug 14, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 25 Code Smells

33.8% 33.8% Coverage
17.5% 17.5% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

@rickchengx rickchengx merged commit 25480ae into apache:dev Aug 14, 2023
zhongjiajie pushed a commit that referenced this pull request Aug 30, 2023
@ediconss @SbloodyS
@rickchengx @zhongjiajie
[Feature][Resource Center] Add support for Huawei Cloud OBS (#14643)
3e0569b 
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center

* add license and add doc

* add 3-party dependency license

* Update LICENSE

* fix

* Update pom.xml

* fix

* fix

---------

Co-authored-by: sunkang <[email protected]>
Co-authored-by: xiangzihao <[email protected]>
Co-authored-by: Rick Cheng <[email protected]>
(cherry picked from commit 25480ae)
@ediconss ediconss deleted the resource-center-obs branch December 21, 2023 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SbloodyS SbloodyS SbloodyS left review comments

@Radeity Radeity Radeity left review comments

@rickchengx rickchengx rickchengx approved these changes

@caishunfeng caishunfeng Awaiting requested review from caishunfeng caishunfeng is a code owner

@zhuangchong zhuangchong Awaiting requested review from zhuangchong

@zhongjiajie zhongjiajie Awaiting requested review from zhongjiajie

@EricGao888 EricGao888 Awaiting requested review from EricGao888 EricGao888 is a code owner

Assignees

@ediconss ediconss

Labels

4.0.0 backend document feature new feature first time contributor First-time contributor ready-to-merge

Projects

None yet

Milestone

3.2.0

Development

Successfully merging this pull request may close these issues.

5 participants

@ediconss @SbloodyS @codecov-commenter @rickchengx @Radeity