[DSIP-40][APIService] Add LogClient to fetch log #17165
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request refactors log fetching across Worker, Master, and API services by consolidating duplicated logic into a shared LogServiceImpl and introducing a new LogClientDelegate.
- Worker and Master log services now extend LogServiceImpl to centralize log operations.
- Updated response classes in the extract module now include status code and message fields.
- LoggerServiceImpl and the new LogClientDelegate enable more robust local-remote log fetching.
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/rpc/WorkerLogServiceImpl.java | Removed duplicated log fetching and extended LogServiceImpl. |
| dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/rpc/MasterLogServiceImpl.java | Updated to extend LogServiceImpl, consolidating common logic. |
| dolphinscheduler-extract/.../TaskInstanceLogPageQueryResponse.java & TaskInstanceLogFileDownloadResponse.java | Added “code” and “message” fields to support error reporting. |
| dolphinscheduler-extract/.../LogServiceImpl.java | New shared log service implementation with exception handling. |
| dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/LoggerServiceImpl.java | Integrated LogClientDelegate for better log fetching abstraction. |
| dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/executor/logging/LogClientDelegate.java | New delegate encapsulating local and remote log fetching logic. |
Comments suppressed due to low confidence (1)
dolphinscheduler-extract/dolphinscheduler-extract-common/src/main/java/org/apache/dolphinscheduler/extract/common/service/impl/LogServiceImpl.java:46
- [nitpick] Consider logging the caught exception in the catch block to provide better observability when local log retrieval fails.
byte[] bytes = LogUtils.getFileContentBytesFromLocal(taskInstanceLogFileDownloadRequest.getTaskInstanceLogAbsolutePath());
...org/apache/dolphinscheduler/extract/common/transportor/TaskInstanceLogPageQueryResponse.java
Show resolved
Hide resolved
.../apache/dolphinscheduler/extract/common/transportor/TaskInstanceLogFileDownloadResponse.java
Show resolved
Hide resolved
ruanwenjun
reviewed
May 9, 2025
...er-api/src/main/java/org/apache/dolphinscheduler/api/executor/logging/LogClientDelegate.java
Outdated
Show resolved
Hide resolved
...er-api/src/main/java/org/apache/dolphinscheduler/api/executor/logging/LogClientDelegate.java
Outdated
Show resolved
Hide resolved
...er-api/src/main/java/org/apache/dolphinscheduler/api/executor/logging/LogClientDelegate.java
Outdated
Show resolved
Hide resolved
ruanwenjun
reviewed
May 12, 2025
...duler-api/src/main/java/org/apache/dolphinscheduler/api/executor/logging/LocalLogClient.java
Outdated
Show resolved
Hide resolved
...er-api/src/main/java/org/apache/dolphinscheduler/api/executor/logging/LogClientDelegate.java
Outdated
Show resolved
Hide resolved
...on/src/main/java/org/apache/dolphinscheduler/extract/common/service/impl/LogServiceImpl.java
Outdated
Show resolved
Hide resolved
...on/src/main/java/org/apache/dolphinscheduler/extract/common/service/impl/LogServiceImpl.java
Outdated
Show resolved
Hide resolved
det101
force-pushed
the
DSIP-40
branch
2 times, most recently
from
b146d9c to
34d4316
Compare
Contributor
Author
|
@ruanwenjun @SbloodyS @caishunfeng |
det101
force-pushed
the
DSIP-40
branch
2 times, most recently
from
7e7bf79 to
074f863
Compare
ruanwenjun
reviewed
May 23, 2025
Member
ruanwenjun
left a comment
ruanwenjun
left a comment
There was a problem hiding this comment.
LGTM, we need to support cut-off the log when fetch whole log. I create a new issue #17204
added 5 commits
June 9, 2025 09:35
…tch local logs first; if that fails, pull from the remote. 2.The logic in MasterLogServiceImpl and WorkerLogServiceImpl is the same. Abstract the shared logic into LoggerServiceImpl to maintain a single codebase.
added 2 commits
June 9, 2025 09:35
|
Member
|
@SbloodyS PTAL |
davidzollo
pushed a commit
to davidzollo/dolphinscheduler
that referenced
this pull request
Oct 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1.Pull local and remote task logs via the LogClientDelegate proxy. Fetch local logs first; if that fails, pull from the remote.
2.The logic in MasterLogServiceImpl and WorkerLogServiceImpl is the same. Abstract the shared logic into LoggerServiceImpl to maintain a single codebase.
close #15966
Purpose of the pull request
Brief change log
Verify this pull request
This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(or)