Skip to content

ci: tcp logger with real logger server#9392

Merged
juststillthinking merged 4 commits into
apache:masterfrom
Sn0rt:sn0rt/new-ci-4-tcp-logger
May 6, 2023
Merged

ci: tcp logger with real logger server#9392
juststillthinking merged 4 commits into
apache:masterfrom
Sn0rt:sn0rt/new-ci-4-tcp-logger

Conversation

@Sn0rt

@Sn0rt Sn0rt commented Apr 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

#9391

the vector is a popular logger service implemented by Rust

to add the vector as the logger server for receiving the log from the user client.

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

bzp2010
bzp2010 reviewed Apr 28, 2023
View reviewed changes

@bzp2010 bzp2010 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the content, I see no problem. But I have a question: is there any reason why you chose vector?

@Sn0rt

Sn0rt commented Apr 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

For the content, I see no problem. But I have a question: is there any reason why you chose vector?

Simple, easy to configure, well documented.

There is another reason why I tried to use Fluentd and encountered a strange case. I spent half a day trying to fix it to no avail.

@bzp2010

bzp2010 commented Apr 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Simple, easy to configure, well documented.

There is another reason why I tried to use Fluentd and encountered a strange case. I spent half a day trying to fix it to no avail.

OK, @Sn0rt

For the first point, that's very good. And for the second point, have you tried fluent bit (it is the next generation of fluentd)?

@Sn0rt

Sn0rt commented Apr 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Simple, easy to configure, well documented.
There is another reason why I tried to use Fluentd and encountered a strange case. I spent half a day trying to fix it to no avail.

OK, @Sn0rt

For the first point, that's very good. And for the second point, have you tried fluent bit (it is the next generation of fluentd)?

I took a brief look at the document construction of fluent-bit, which is not as good as that of vector. My requirement is to cooperate with multiple reporting log formations. fluent-bit does not have an example available out of the box, but vector provides it.

In this PR, I expect the vector to become the basic component for the CI log plugins test, and to cooperate with other log plugins later.

@bzp2010

bzp2010 commented Apr 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

@Sn0rt Understand and recognize your position.

But I'm not sure if it's the right strategy to keep using vector in the future because essentially the open source project is a vendor-neutral project, the plugin is called tcp-logger and users may use any logging pipeline project that supports TCP, I'm not sure....

Maybe consider adding testing via other log pipeline projects in a future PR. Approved this PR now.

bzp2010
bzp2010 previously approved these changes Apr 28, 2023
View reviewed changes
shreemaan-abhishek
shreemaan-abhishek reviewed Apr 28, 2023
View reviewed changes

@shreemaan-abhishek shreemaan-abhishek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Neilblaze
Neilblaze previously approved these changes Apr 28, 2023
View reviewed changes
soulbird
soulbird reviewed Apr 30, 2023
View reviewed changes
Comment thread ci/pod/vector/vector_logs_ca.key
@@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----

@soulbird soulbird Apr 30, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd better move certs to t/certs

@Sn0rt Sn0rt May 4, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@Sn0rt
Sn0rt dismissed stale reviews from Neilblaze and bzp2010 via 25b16e2 May 4, 2023 01:54
juststillthinking
juststillthinking reviewed May 4, 2023
View reviewed changes
Comment thread t/plugin/tcp-logger.t Outdated
@soulbird

soulbird commented May 5, 2023

Copy link
Copy Markdown
Contributor

We need to adjust the existing test cases. Currently, the log is forwarded to port 5044, which is a simulated log service.

@Sn0rt

Sn0rt commented May 5, 2023

Copy link
Copy Markdown
Contributor Author

We need to adjust the existing test cases. Currently, the log is forwarded to port 5044, which is a simulated log service.

copy that. I rewrite some of the tcp-logger test cases.

@juststillthinking
juststillthinking merged commit 02ee09e into apache:master May 6, 2023
hongbinhsu added a commit to fitphp/apix that referenced this pull request May 13, 2023
@hongbinhsu
Merge branch 'upstream/master' into github/master
416f768 
* upstream/master: (42 commits)
  fix: add `rust-check` target to the makefile for deps pre-checking (apache#9453)
  test: remove duplicate `proto` unit test file and tweak test case title (apache#9445)
  feat: proxy-mirror add grpc and grpcs support (apache#9388)
  test: replace all httpbin upstream (apache#9452)
  fix: 404 error in ci test (apache#9447)
  fix: body-transformer plugin return raw body anytime (apache#9446)
  fix(ci): fips: ensure apisix compiles with openssl3 (apache#9427)
  chore(deps): bump actions/stale from 7 to 8 (apache#9159)
  chore(deps): bump actions/setup-node from 3.5.1 to 3.6.0 (apache#8625)
  chore(test): split the access log of apisix and test server (apache#8819)
  docs: fix grammar (apache#9395)
  fix(wolf-rbac): other plugin in consumer not effective when consumer used wolf-rbac plugin (apache#9287) (apache#9298)
  docs: use shell instead of Python to configure protos resources (apache#9414)
  docs: add guide to authenticate websocket connections (apache#9369)
  docs: updated Stream Proxy doc (apache#9367)
  feat: proxy_rewrite support miltiple regex pattern matching (apache#9194)
  feat: route-level MTLS (apache#9322)
  ci: tcp logger with real logger server (apache#9392)
  fix: always parse domain when host is domain name (apache#9332)
  docs: CHANGELOG.md Chinese docs lint (apache#9411)
  ...
AlinsRan pushed a commit to AlinsRan/apisix that referenced this pull request Jun 26, 2023
@Sn0rt @AlinsRan
ci: tcp logger with real logger server (apache#9392)
33de9a8
@Sn0rt
Sn0rt deleted the sn0rt/new-ci-4-tcp-logger branch September 20, 2023 02:39
@Sn0rt
Sn0rt restored the sn0rt/new-ci-4-tcp-logger branch September 20, 2023 02:39
@Sn0rt
Sn0rt deleted the sn0rt/new-ci-4-tcp-logger branch September 20, 2023 02:39
@Sn0rt
Sn0rt restored the sn0rt/new-ci-4-tcp-logger branch September 20, 2023 02:39
@Sn0rt
Sn0rt deleted the sn0rt/new-ci-4-tcp-logger branch September 20, 2023 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@soulbird soulbird soulbird approved these changes
@shreemaan-abhishek shreemaan-abhishek Awaiting requested review from shreemaan-abhishek
@bzp2010 bzp2010 Awaiting requested review from bzp2010
@Neilblaze Neilblaze Awaiting requested review from Neilblaze
+1 more reviewer
@juststillthinking juststillthinking juststillthinking approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Sn0rt @bzp2010 @soulbird @juststillthinking @Neilblaze @shreemaan-abhishek