[Feature-16396][storage-plugin] Add Tencent Cloud COS Storage Plugin #16565
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mighten
requested review from
EricGao888,
Gallardot,
SbloodyS,
caishunfeng and
ruanwenjun
as code owners
Mighten
changed the title
Mighten
changed the title
Mighten
changed the title
SbloodyS
requested changes
Sep 2, 2024
Member
SbloodyS
left a comment
SbloodyS
left a comment
There was a problem hiding this comment.
Since we don't have any tencent cos support for integration testing. I will -1 on this until we can find a way to test against this in out UT, IT and E2E.
ruanwenjun
reviewed
Sep 2, 2024
dolphinscheduler-storage-plugin/dolphinscheduler-storage-cos/src/test/resources/logback.xml
Show resolved
Hide resolved
...eduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/TaskConstants.java
Outdated
Show resolved
Hide resolved
ruanwenjun
reviewed
Sep 2, 2024
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Fixed
Show fixed
Hide fixed
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Fixed
Show fixed
Hide fixed
Member
we should remove bcprov-jdk15on-1.69
|
fuchanghai
reviewed
Sep 21, 2024
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
fuchanghai
reviewed
Sep 21, 2024
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperatorFactory.java
Outdated
Show resolved
Hide resolved
fuchanghai
reviewed
Sep 21, 2024
...ge-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageConstants.java
Outdated
Show resolved
Hide resolved
fuchanghai
previously approved these changes
Sep 22, 2024
Member
If |
SbloodyS
reviewed
Sep 23, 2024
...-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/CosRemoteLogHandler.java
Outdated
Show resolved
Hide resolved
...-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/CosRemoteLogHandler.java
Outdated
Show resolved
Hide resolved
...-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/CosRemoteLogHandler.java
Outdated
Show resolved
Hide resolved
...-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/CosRemoteLogHandler.java
Outdated
Show resolved
Hide resolved
...-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/CosRemoteLogHandler.java
Outdated
Show resolved
Hide resolved
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
...age-cos/src/main/java/org/apache/dolphinscheduler/plugin/storage/cos/CosStorageOperator.java
Outdated
Show resolved
Hide resolved
SbloodyS
reviewed
Sep 23, 2024
SbloodyS
changed the title
fuchanghai
previously approved these changes
Sep 25, 2024
SbloodyS
reviewed
Sep 25, 2024
dolphinscheduler-storage-plugin/dolphinscheduler-storage-cos/pom.xml
Outdated
Show resolved
Hide resolved
dolphinscheduler-storage-plugin/dolphinscheduler-storage-cos/pom.xml
Outdated
Show resolved
Hide resolved
fuchanghai
approved these changes
Sep 27, 2024
* Fixed SonarCloud Quality Readability B (required >= A) * Added missing license files * YAML-ized Tencent COS configuration from common.properties * Added remote logger support with Tencent COS
* Removed redundant dependency: `bcprov-jdk15on-1.69.jar` * Relocated Tencent COS Constants, and reduced magic values in COS with constants * Isolated cloud storage configuration in `common.properties` -> `resource-center.yaml` * Fixed CodeQL: Uncontrolled data used in path expression
* Updated log statements in `CosRemoteLogHandler` * Updated `CosStorageOperator` * Added *maven-shade-plugin* in storage-plugin-cos/pom.xml
* Refactored `pom.xml`
SbloodyS
approved these changes
Sep 27, 2024
Member
SbloodyS
left a comment
SbloodyS
left a comment
There was a problem hiding this comment.
+1, Thanks for your contribution.
|
|
Awesome work, congrats on your first merged pull request! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose of the pull request
This pull request adds a Storage Plugin to the DS Resource Center to support Tencent Cloud COS and aims to close #16396 .
Brief change log
dolphinscheduler-storage-cosindolphinscheduler-storage-pluginCosRemoteLogHandlerindolphinscheduler-commonVerify this pull request
Pull Request Notice
Pull Request Notice
If your pull request contain incompatible change, you should also add it to
docs/docs/en/guide/upgrede/incompatible.md