[h2olog] h2olog v2 part 1 (core) by gfx · Pull Request #3081 · h2o/h2o

gfx · 2022-08-04T06:47:08Z

This is a core feature of h2olog v2, which uses an internal logger (ptlslog, h2o/picotls#406) and a UNIX domain socket server.

Motivation: #3059

Done

Introduces new configuration h2olog.path to enable the new logging system
h2olog(1) has -u <unix-socket-path> option for the new logging system
- it does not support the other flags (should be supported in the future)
Adding a test file 90h2olog-qlog.t to make sure h2olog v1 and v2 generate the same QLog outputs (raw outputs are slightly different because it's too difficult to make sure the perfect compatibility)

TODOs for the further development

Implement the other h2olog flags, including -S and -A
Cover USDTs defined in h2o (it only covers picotls and quicly so far)
Remove BCC dependency from h2olog (and of course, the h2o project)

kazuho

Thank you for working on this. Looking really good.

gfx · 2023-12-13T01:38:11Z

NOTE: #3149 should be merged to this branch first.

Conflict resolution on searchindex.js is required after that merge.

…(headers_util is a shared file and debatable)

…r stealing the socket (would not happen in practice)

…eed to add that as we start using HTTP

…ior (revert db13c29)

[h2olog2] change the h2olog-h2o protocol to be cleartext HTTP/1

kazuho

Thank you for the PR. Looks good modulo nitpicks.

kazuho · 2024-01-17T04:17:12Z

+  group: h2o # optional
+  permission: 666 # optional
+  sndbuf: 65536 # optional
+EOT


I think the directive is now host-level and the Unix socket is specified differently?

Done in 62827f3.

Co-authored-by: Kazuho Oku <[email protected]>

gfx added 6 commits August 3, 2022 04:15

extract https://github.com/h2o/picotls @ 5490f21 () at deps/picotls

ae21e64

extract https://github.com/h2o/picotls @ b394a8c () at deps/picotls

7c192d2

extract https://github.com/h2o/quicly @ 85015182 () at deps/quicly

c564c68

extract https://github.com/h2o/quicly @ 84f5e05 () at deps/quicly

8770be3

extract https://github.com/h2o/picotls @ 215a0702 () at deps/picotls

9fb8e5f

MVP

99f96ff

kazuho reviewed Aug 8, 2022

View reviewed changes

Comment thread deps/picotls/lib/picotls.c Outdated

Comment thread deps/picotls/t/cli.c Outdated

Comment thread deps/picotls/t/picotls.c Outdated

Comment thread deps/picotls/include/picotls.h Outdated

Comment thread lib/common/probe_log.c Outdated

gfx added 13 commits August 8, 2022 04:11

add missing changes

8abbea9

extract https://github.com/h2o/quicly @ a6c68cafa1a () at deps/quicly

50d5b76

extract https://github.com/h2o/picotls @ ec3e4db () at deps/picotls

1acd671

extract https://github.com/h2o/picotls @ 2fa9b69 () at deps/picotls

f05f7cb

tweaks

c51bd73

rename to "h2olog-socket"

7a3c532

tweaks

54552f8

extract https://github.com/h2o/picotls @ 64e24ea () at deps/picotls

6f8294e

add "h2olog.lost" metrics

7b15508

create h2olog socket with h2o's standard routine

83e6770

add experimental h2olog v2

bc95bd2

fix lisence notations

1549c09

rewrite experimental-h2olog-v2 in C

3625953

gfx mentioned this pull request Aug 17, 2022

add ptlslog as an alternative to USDT probes h2o/picotls#406

Merged

gfx added 7 commits August 17, 2022 08:48

extract https://github.com/h2o/picotls @ 6d51eb7501b () at deps/picotls

9e3728a

extract https://github.com/h2o/quicly @ 3191349deedec () at deps/quicly

fa0c94e

checkout deps/quicly/CMakeLists.txt as of 6b62d4a

cb268a4

reorder files

3f3415c

define H2O_LOG and H2O_LOG_CONN

969f5a1

add t/90 to make sure qlog-adapter work for h2olog v1 & v2

3a30483

merge experimental-h2olog-v2(1) into h2olog(1)

9ccc6be

gfx marked this pull request as ready for review August 26, 2022 04:52

gfx changed the title ~~[WIP] h2olog v2~~ MVP of h2olog v2 Aug 26, 2022

kazuho and others added 5 commits April 14, 2023 13:56

Merge branch 'master' into gfx/h2olog2

d298825

Merge branch 'master' into gfx/h2olog2

7dcdee4

Merge remote-tracking branch 'origin/master' into gfx/h2olog2

dd44ce9

add doc for h2olog directive

203010d

Merge branch 'gfx/h2olog2' into gfx/h2olog2-as-h2o-handler

8dd5f3d

gfx requested a review from kazuho December 11, 2023 01:57

kazuho added 15 commits January 16, 2024 16:22

if we are to reorder the files, that should be in alphabetical order …

274d51b

…(headers_util is a shared file and debatable)

[xcode] add files

e78ca9a

path is portion of an endpoint identified as an URI

2223712

no need to subtype

8e37e49

use one way of obtaining fd, call h2o_fatal when error happens afte…

a65881f

…r stealing the socket (would not happen in practice)

avoid constructing an object

a2c7233

consolidate HTTP parsing logic

2e8744c

haven't logged connection establishment / teardown, maybe we do not n…

db13c29

…eed to add that as we start using HTTP

oopses

cfd04b0

missing break

10bb7d8

refactor for readability

5463a05

until we remove USDT-based variant, it is easier to retain same behav…

c41b9fd

…ior (revert db13c29)

fix race condition

4d82793

fix HTTP parsing bug and refactor

4bcfdac

Merge pull request #3149 from h2o/gfx/h2olog2-as-h2o-handler

2beb53e

[h2olog2] change the h2olog-h2o protocol to be cleartext HTTP/1

kazuho reviewed Jan 17, 2024

View reviewed changes

gfx and others added 5 commits January 17, 2024 14:17

fix comments in src/main.c

ba79f9a

Co-authored-by: Kazuho Oku <[email protected]>

Merge branch 'master' into gfx/h2olog2

7bdf4c3

reflect current state of the directive

62827f3

regen doc

ac10f30

[h2olog] only allow access on UNIX sockets

fd68694

kazuho merged commit e7502f9 into master Feb 7, 2024

kazuho mentioned this pull request Feb 7, 2024

Hide h2olog endpoint unless accessed via UNIX socket, update docs #3350

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[h2olog] h2olog v2 part 1 (core)#3081

[h2olog] h2olog v2 part 1 (core)#3081
kazuho merged 122 commits into
masterfrom
gfx/h2olog2

gfx commented Aug 4, 2022 •

edited

Loading

Uh oh!

kazuho left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gfx commented Dec 13, 2023 •

edited

Loading

Uh oh!

kazuho left a comment

Uh oh!

Uh oh!

kazuho Jan 17, 2024

Uh oh!

kazuho Feb 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

gfx commented Aug 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Done

TODOs for the further development

Uh oh!

kazuho left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gfx commented Dec 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kazuho left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kazuho Jan 17, 2024

Choose a reason for hiding this comment

Uh oh!

kazuho Feb 6, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gfx commented Aug 4, 2022 •

edited

Loading

gfx commented Dec 13, 2023 •

edited

Loading