[Feature][Zeta][Core] Support api get logs #7818

zhangshenghang · 2024-10-11T13:12:42Z

Purpose of this pull request

reslove [Feature][Zeta][Core] Add support for accessing job execution output log files through an API. #7724

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update plugin-mapping.properties and add new connector information in it
2. Update the pom file of seatunnel-dist
3. Add ci label in label-scope-conf
4. Add e2e testcase in seatunnel-e2e
5. Update connector plugin_config
Update the release-note.

liugddx · 2024-10-11T13:43:39Z

Please add test cases in RestApiIT

hailin0 · 2024-10-11T14:25:07Z

suggestion

Browse this node log files through browser

list log files name

GET /logs
----------------

seatunnel-engine-server.log
job-xxx1.log
job-xxx2.log
...

click filename to output log content text

GET /logs/job-xxx1.log
----------------

[INFO] ....
[INFO] ....
[INFO] ....
.....

Browse job all logs through browser

request master get job log of all host node

GET /logs/xxx1
----------------

node1-job-xxx1.log
node2-job-xxx1.log
node3-job-xxx1.log
....

request master get job log of node1

GET /logs/xxx1/node1-job-xxx1.log
----------------

[INFO] ....
[INFO] ....
[INFO] ....
.....

Example: Browse files through browser

hailin0 · 2024-10-11T14:45:29Z

link #7712, #7787

liugddx · 2024-10-15T05:16:13Z

According to #7647, we need to implement the V2 version of the api

zhangshenghang · 2024-10-15T11:57:06Z

effect

liugddx · 2024-10-15T12:24:09Z

https://github.com/apache/seatunnel/blob/dev/docs/en/seatunnel-engine/rest-api-v2.md Please maintain documentation here.

zhangshenghang · 2024-10-15T12:28:07Z

https://github.com/apache/seatunnel/blob/dev/docs/en/seatunnel-engine/rest-api-v2.md Please maintain documentation here.

ok, thank you for your reminder

hailin0 · 2024-10-15T13:08:58Z

good pr

liugddx · 2024-10-17T07:33:34Z

Please resolve code conflicts

...ver/src/main/java/org/apache/seatunnel/engine/server/rest/servlet/CurrentNodeLogServlet.java

davidzollo · 2024-10-18T02:02:13Z

good job

Hisoka-X · 2024-10-18T06:31:31Z

Please update https://github.com/apache/seatunnel/blob/934434cc75afc747339182df6468f89a5d0fb1b2/docs/en/seatunnel-engine/logging.md to tell user we can get logs by rest api.

docs/en/seatunnel-engine/logging.md

...nector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/joblog/JobLogIT.java

...erver/src/main/java/org/apache/seatunnel/engine/server/rest/RestHttpGetCommandProcessor.java

...-server/src/main/java/org/apache/seatunnel/engine/server/rest/servlet/AllLogNameServlet.java

docs/zh/seatunnel-engine/rest-api-v2.md

Hisoka-X · 2024-10-18T12:05:05Z

docs/en/seatunnel-engine/rest-api-v2.md

+
+Returns a list of logs from the requested node.
+
+#### Examples


Can I use http://localhost:5801/log/12345 to get job 12345's logs in current node?

Can I use http://localhost:5801/log/12345 to get job 12345's logs in current node?

Currently not supported, /log can only be matched accurately

liugddx

LGTM

Hisoka-X

LGTM. Thanks @zhangshenghang

zhangshenghang added 3 commits September 1, 2024 10:31

Merge remote-tracking branch 'upstream/dev' into dev

cf958f9

Merge remote-tracking branch 'upstream/dev' into HEAD

d9b836d

init

e82cbb9

github-actions bot added document Zeta Zeta Rest API e2e labels Oct 11, 2024

Merge remote-tracking branch 'upstream/dev' into feature-add-api-log

8ab3ff9

update doc

537b12f

dailai mentioned this pull request Oct 12, 2024

[Feature][Engine UI] partial pages of the engine #7602

Merged

hailin0 mentioned this pull request Oct 13, 2024

SeaTunnel community meeting Topic collect #7828

Closed

3 tasks

zhangshenghang added 3 commits October 14, 2024 22:42

modify implement plan

dc4f10c

restore doc

43d6299

add-v2

6599a0f

github-actions bot added the api label Oct 15, 2024

zhangshenghang added 4 commits October 15, 2024 21:16

update doc and add test case

a607ed0

test log4j

29d9ce8

test log

5824a95

test log

d226836

zhangshenghang added 2 commits October 17, 2024 21:18

merge

d002526

improve

5050209

zhangshenghang added 4 commits October 17, 2024 21:18

merge

a636114

improve

f71e5b7

update doc

213c188

add doc

c43a71d

Hisoka-X reviewed Oct 18, 2024

View reviewed changes

...ver/src/main/java/org/apache/seatunnel/engine/server/rest/servlet/CurrentNodeLogServlet.java Show resolved Hide resolved

Hisoka-X reviewed Oct 18, 2024

View reviewed changes

docs/en/seatunnel-engine/logging.md Outdated Show resolved Hide resolved

Hisoka-X reviewed Oct 18, 2024

View reviewed changes

improve

d7c9652

liugddx approved these changes Oct 20, 2024

View reviewed changes

liugddx requested review from Hisoka-X and hailin0 October 20, 2024 11:57

github-actions bot added approved reviewed labels Oct 20, 2024

Hisoka-X approved these changes Oct 21, 2024

View reviewed changes

Hisoka-X merged commit f640eda into apache:dev Oct 21, 2024

hailin0 mentioned this pull request Oct 28, 2024

SeaTunnel community meeting Topic collect #7929

Closed

3 tasks

Hisoka-X mentioned this pull request Oct 29, 2024

[Feature][Zeta][Core] Add support for accessing job execution output log files through an API. #7724

Closed

3 tasks

hailin0 mentioned this pull request Jan 14, 2025

SeaTunnel community meeting Topic collect #8517

Closed

3 tasks

hailin0 mentioned this pull request Mar 26, 2025

[Feature][Zeta] Support view master/worker node log files #9050

Closed

3 tasks

hailin0 mentioned this pull request Apr 16, 2025

[Feature][Zeta] Support basic authentication for web ui #9171

Merged

4 tasks


		Returns a list of logs from the requested node.

		#### Examples

[Feature][Zeta][Core] Support api get logs #7818

[Feature][Zeta][Core] Support api get logs #7818

Uh oh!

Conversation

zhangshenghang commented Oct 11, 2024

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

Uh oh!

liugddx commented Oct 11, 2024

Uh oh!

hailin0 commented Oct 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Browse this node log files through browser

Browse job all logs through browser

Example: Browse files through browser

Uh oh!

hailin0 commented Oct 11, 2024

Uh oh!

liugddx commented Oct 15, 2024

Uh oh!

zhangshenghang commented Oct 15, 2024

Uh oh!

liugddx commented Oct 15, 2024

Uh oh!

zhangshenghang commented Oct 15, 2024

Uh oh!

hailin0 commented Oct 15, 2024

Uh oh!

liugddx commented Oct 17, 2024

Uh oh!

Uh oh!

davidzollo commented Oct 18, 2024

Uh oh!

Hisoka-X commented Oct 18, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Hisoka-X Oct 18, 2024

Choose a reason for hiding this comment

Uh oh!

zhangshenghang Oct 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liugddx left a comment

Choose a reason for hiding this comment

Uh oh!

Hisoka-X left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hailin0 commented Oct 11, 2024 •

edited

Loading

zhangshenghang Oct 20, 2024 •

edited

Loading