Skip to content

trace: add /logs.json endpoint to admin server#1642

Merged
olix0r merged 32 commits intomainfrom
eliza/log-streaming-demo
May 20, 2022
Merged

trace: add /logs.json endpoint to admin server#1642
olix0r merged 32 commits intomainfrom
eliza/log-streaming-demo

Conversation

@hawkw
Copy link
Contributor

@hawkw hawkw commented May 2, 2022 •
edited
Loading

This branch adds a new /logs.json endpoint to the proxy's admin
server. This endpoint accepts HTTP GET requests with a log filter as a
query string, or QUERY requests with a filter string as a body, and
responds with a stream of JSON objects representing each tracing event
that matches that filter, as described in linkerd/linkerd2#7250.
Multiple concurrent /logs.json requests with different filters are
supported, and each request will only recieve the logged events that
match the filter it requested.

The /logs.json endpoint is feature flagged, and is only accessible
from localhost, similarly to the /proxy-log-level endpoint. If the
feature flag is not enabled, /logs.json will respond with a 404 error.
Currently, the log-streaming feature is not enabled by default, so the
/logs.json endpoint won't do anything unless the proxy is compiled
with the feature flag enabled.

In the future, we'll want to figure out a better auth story for the logs
endpoint (and the proxy's admin server more generally); when we do, we
can enable the feature flag in release builds. There's also room for
some additional work to make the implementation cleaner and improve
performance, but some of this will require upstream library changes as
well. I've tried to comment on most of the future work where
appropriate.

Closes linkerd/linkerd2#7250

hawkw added 22 commits April 27, 2022 15:35
@hawkw
wip
4092893 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
wip
654d658 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
sketch out more types
d5db9c9 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
further wip
e2a4578
@hawkw
yessss hahahaha it works!!!
d1b94ce 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
add a test for multiple concurrent streams
9f7ce46 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
clean up error boilerplate
9f4afab 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
comments and stuff
018e49d 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
pub public apis at the top
ce5361c 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
report if any events were dropped
30f5283 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
note for the future
aa6b900 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
send errors as JSON
498e92f 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
clippy nonsense
1bf6d50 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
validate log-streaming requests a bit more nicely
b849bd9 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
remove log streams when the request ends
5b685fe 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
fixup compilation with feature flags disabled
0acd1b8 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
test that requests actually don't error
5c61dc6 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
send a query string in tests
cdc94cc 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
add separate tests for HTTP QUERY requests
f0810b3 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
feature-flag log streaming
a852e29 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
rename endpoint to /logs.json
8fbbb5d 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw
nop logs from log forwarding tasks
abddbc8 
Signed-off-by: Eliza Weisman <[email protected]>
@hawkw hawkw requested a review from a team as a code owner May 2, 2022 21:08
@olix0r
Copy link
Member

olix0r commented May 17, 2022

Unfortunately there seems to be some conflict with the recently merged uptime PR. Looks like there are some merge conflicts.

@hawkw
Copy link
Contributor Author

hawkw commented May 17, 2022

Unfortunately there seems to be some conflict with the recently merged uptime PR. Looks like there are some merge conflicts.

fixed it, thanks! i also added some additional comments; hope that's helpful!

hawkw and others added 2 commits May 17, 2022 15:00
@hawkw
rustfmt
a46fccc
@olix0r
Sort cargos toml
94139ca 
Signed-off-by: Oliver Gould <[email protected]>
olix0r added a commit that referenced this pull request May 18, 2022
@olix0r
Simplify admin server feature-flagging in #1642
c3552c1 
* Rather than push feature flagging down into the `serve` logic, we can
  avoid even matching the logs.json endpoint when the feature isn't
  disabled. This avoids the need for conditional feature gatingc in the
  handler.
* The log module is split into two submodules: level and stream. The
  stream module is feature-gated. This allows to extract function-local
  defintions into the module, which simplifies scoping and makes things
  easier to read (imo).

Signed-off-by: Oliver Gould <[email protected]>
olix0r and others added 2 commits May 18, 2022 13:31
@olix0r @dependabot
Simplify admin server feature-flagging in #1642 (#1682)
9c764c1 
* opencensus: Include empty generated protobuf (#1676)

prost generates empty files for google protobuf types. This change adds
these generated files so that they are not regenerated on `make test`.

Signed-off-by: Oliver Gould <[email protected]>

* build(deps): bump rustls from 0.20.5 to 0.20.6 (#1679)

Bumps [rustls](https://github.com/rustls/rustls) from 0.20.5 to 0.20.6.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md)
- [Commits](rustls/rustls@v/0.20.5...v/0.20.6)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert "build(deps): bump socket2 from 0.4.4 to 0.4.5 (#1654)" (#1681)

socket2 v0.4.5 has been yanked (rust-lang/socket2#308).

This reverts commit 9004d9c.

Signed-off-by: Oliver Gould <[email protected]>

* build(deps): bump EmbarkStudios/cargo-deny-action from 1.2.17 to 1.3.0 (#1678)

Bumps [EmbarkStudios/cargo-deny-action](https://github.com/EmbarkStudios/cargo-deny-action) from 1.2.17 to 1.3.0.
- [Release notes](https://github.com/EmbarkStudios/cargo-deny-action/releases)
- [Commits](EmbarkStudios/cargo-deny-action@3481b77...b655a95)

---
updated-dependencies:
- dependency-name: EmbarkStudios/cargo-deny-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump clang-sys from 1.3.1 to 1.3.2 (#1680)

Bumps [clang-sys](https://github.com/KyleMayes/clang-sys) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/KyleMayes/clang-sys/releases)
- [Changelog](https://github.com/KyleMayes/clang-sys/blob/master/CHANGELOG.md)
- [Commits](https://github.com/KyleMayes/clang-sys/commits)

---
updated-dependencies:
- dependency-name: clang-sys
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Simplify admin server feature-flagging in #1642

* Rather than push feature flagging down into the `serve` logic, we can
  avoid even matching the logs.json endpoint when the feature isn't
  disabled. This avoids the need for conditional feature gatingc in the
  handler.
* The log module is split into two submodules: level and stream. The
  stream module is feature-gated. This allows to extract function-local
  defintions into the module, which simplifies scoping and makes things
  easier to read (imo).

Signed-off-by: Oliver Gould <[email protected]>

* bump thingbuf to 1.0.3

Signed-off-by: Oliver Gould <[email protected]>

* to_stream => into_stream

Signed-off-by: Oliver Gould <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@olix0r
Merge branch 'main' into eliza/log-streaming-demo
4a080a6
olix0r added a commit that referenced this pull request May 20, 2022
@olix0r
tracing: Simplify trace initialization (for #1642)
67b9120 
While reviewing #1642, I found myself wanting clearer documentation
about what is being initilalized. This change attempts to reorganize the
builder for clarity.

Signed-off-by: Oliver Gould <[email protected]>
@olix0r olix0r mentioned this pull request May 20, 2022
Merged
olix0r added 2 commits May 20, 2022 12:03
@olix0r
tracing: Simplify trace initialization (for #1642) (#1688)
2350e83 
While reviewing #1642, I found myself wanting clearer documentation
about what is being initilalized. This change attempts to reorganize the
builder for clarity.

Signed-off-by: Oliver Gould <[email protected]>
@olix0r
touch up type alias helpers
6c09ece 
Signed-off-by: Oliver Gould <[email protected]>
olix0r
olix0r approved these changes May 20, 2022
View reviewed changes
Copy link
Member

@olix0r olix0r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! one last question

olix0r and others added 2 commits May 20, 2022 19:35
@olix0r
fixup
dc52e46 
Signed-off-by: Oliver Gould <[email protected]>
@hawkw
document log stream channel configuration
8fb8bc8 
Signed-off-by: Eliza Weisman <[email protected]>
@olix0r olix0r merged commit dd57a37 into main May 20, 2022
@olix0r olix0r deleted the eliza/log-streaming-demo branch May 20, 2022 20:09
olix0r added a commit to linkerd/linkerd2 that referenced this pull request May 24, 2022
@olix0r
proxy: v2.179.0
f9d3971 
This release fixes a problem with HTTP/1.1 `CONNECT` requests. When a
server responds to a `CONNECT` request with `content-length` or
`transfer-encoding` headers (in violation of RFC 7231), the proxy must
actively strip these headers to avoid making the Hyper server fail the
response.

This release also fixes an issue with the way proxies discover control
plane components via DNS. When `SRV` records cannot be resolved, the
proxy would no not necessarily fall back to resolving `A` records. This
has been fixed.

Finally, the inbound proxy can now discover policies dynamically. Ports
that are configured in the `LINKERD2_PROXY_INBOUND_PORTS` are discovered
as the proxy starts up; but now the proxy will discover policies for
ports that are not in this list. The pod's default policy is used
initially, but once a policy is received from the control plane it is
used.

---

* build(deps): bump syn from 1.0.93 to 1.0.94 (linkerd/linkerd2-proxy#1664)
* build(deps): bump tj-actions/changed-files from 19 to 20 (linkerd/linkerd2-proxy#1665)
* build(deps): bump rustls from 0.20.4 to 0.20.5 (linkerd/linkerd2-proxy#1666)
* build(deps): bump ryu from 1.0.9 to 1.0.10 (linkerd/linkerd2-proxy#1667)
* build(deps): bump tokio-util from 0.7.1 to 0.7.2 (linkerd/linkerd2-proxy#1668)
* build(deps): bump itoa from 1.0.1 to 1.0.2 (linkerd/linkerd2-proxy#1669)
* build(deps): bump tonic from 0.7.1 to 0.7.2 (linkerd/linkerd2-proxy#1652)
* dns: Fall back to A record when SRV resolution fails (linkerd/linkerd2-proxy#1670)
* Consolidate error chain inspection with `cause_ref` (linkerd/linkerd2-proxy#1671)
* ci: change how warnings are denied on CI (linkerd/linkerd2-proxy#1662)
* build(deps): bump proc-macro2 from 1.0.38 to 1.0.39 (linkerd/linkerd2-proxy#1673)
* build(deps): bump libc from 0.2.125 to 0.2.126 (linkerd/linkerd2-proxy#1674)
* build(deps): bump syn from 1.0.94 to 1.0.95 (linkerd/linkerd2-proxy#1675)
* ci: Use the cargo-action-fmt setup action (linkerd/linkerd2-proxy#1672)
* opencensus: Include empty generated protobuf (linkerd/linkerd2-proxy#1676)
* build(deps): bump rustls from 0.20.5 to 0.20.6 (linkerd/linkerd2-proxy#1679)
* Revert "build(deps): bump socket2 from 0.4.4 to 0.4.5 (linkerd/linkerd2-proxy#1654)" (#1681)
* build(deps): bump EmbarkStudios/cargo-deny-action from 1.2.17 to 1.3.0 (linkerd/linkerd2-proxy#1678)
* build(deps): bump clang-sys from 1.3.1 to 1.3.2 (linkerd/linkerd2-proxy#1680)
* cache: generalize `Cache` into a key-value cache (linkerd/linkerd2-proxy#1683)
* build(deps): bump once_cell from 1.10.0 to 1.11.0 (linkerd/linkerd2-proxy#1687)
* build(deps): bump EmbarkStudios/cargo-deny-action from 1.3.0 to 1.3.1 (linkerd/linkerd2-proxy#1686)
* trace: add `/logs.json` endpoint to admin server (linkerd/linkerd2-proxy#1642)
* Dynamically discover policies for undocumented ports (linkerd/linkerd2-proxy#1677)
* Don't allow a policy to be used if `check_port_allowed` fails (linkerd/linkerd2-proxy#1689)
* ci: Simplify release workflow (linkerd/linkerd2-proxy#1690)
* build(deps): bump petgraph from 0.6.0 to 0.6.1 (linkerd/linkerd2-proxy#1696)
* build(deps): bump actions/upload-artifact from 3.0.0 to 3.1.0 (linkerd/linkerd2-proxy#1692)
* build(deps): bump tj-actions/changed-files from 20 to 20.1 (linkerd/linkerd2-proxy#1693)
* build(deps): bump http-body from 0.4.4 to 0.4.5 (linkerd/linkerd2-proxy#1694)
* build(deps): bump regex from 1.5.5 to 1.5.6 (linkerd/linkerd2-proxy#1695)
* build(deps): bump regex-syntax from 0.6.25 to 0.6.26 (linkerd/linkerd2-proxy#1697)
* http: Strip illegal headers from CONNECT responses (linkerd/linkerd2-proxy#1699)
* dev: Replace `Makefile` with `justfile` (linkerd/linkerd2-proxy#1691)

Signed-off-by: Oliver Gould <[email protected]>
@olix0r olix0r mentioned this pull request May 24, 2022
Merged
olix0r added a commit to linkerd/linkerd2 that referenced this pull request May 24, 2022
@olix0r
proxy: v2.179.0 (#8543)
c7b9c65 
This release fixes a problem with HTTP/1.1 `CONNECT` requests. When a
server responds to a `CONNECT` request with `content-length` or
`transfer-encoding` headers (in violation of RFC 7231), the proxy must
actively strip these headers to avoid making the Hyper server fail the
response.

This release also fixes an issue with the way proxies discover control
plane components via DNS. When `SRV` records cannot be resolved, the
proxy would no not necessarily fall back to resolving `A` records. This
has been fixed.

Finally, the inbound proxy can now discover policies dynamically. Ports
that are configured in the `LINKERD2_PROXY_INBOUND_PORTS` are discovered
as the proxy starts up; but now the proxy will discover policies for
ports that are not in this list. The pod's default policy is used
initially, but once a policy is received from the control plane it is
used.

---

* build(deps): bump syn from 1.0.93 to 1.0.94 (linkerd/linkerd2-proxy#1664)
* build(deps): bump tj-actions/changed-files from 19 to 20 (linkerd/linkerd2-proxy#1665)
* build(deps): bump rustls from 0.20.4 to 0.20.5 (linkerd/linkerd2-proxy#1666)
* build(deps): bump ryu from 1.0.9 to 1.0.10 (linkerd/linkerd2-proxy#1667)
* build(deps): bump tokio-util from 0.7.1 to 0.7.2 (linkerd/linkerd2-proxy#1668)
* build(deps): bump itoa from 1.0.1 to 1.0.2 (linkerd/linkerd2-proxy#1669)
* build(deps): bump tonic from 0.7.1 to 0.7.2 (linkerd/linkerd2-proxy#1652)
* dns: Fall back to A record when SRV resolution fails (linkerd/linkerd2-proxy#1670)
* Consolidate error chain inspection with `cause_ref` (linkerd/linkerd2-proxy#1671)
* ci: change how warnings are denied on CI (linkerd/linkerd2-proxy#1662)
* build(deps): bump proc-macro2 from 1.0.38 to 1.0.39 (linkerd/linkerd2-proxy#1673)
* build(deps): bump libc from 0.2.125 to 0.2.126 (linkerd/linkerd2-proxy#1674)
* build(deps): bump syn from 1.0.94 to 1.0.95 (linkerd/linkerd2-proxy#1675)
* ci: Use the cargo-action-fmt setup action (linkerd/linkerd2-proxy#1672)
* opencensus: Include empty generated protobuf (linkerd/linkerd2-proxy#1676)
* build(deps): bump rustls from 0.20.5 to 0.20.6 (linkerd/linkerd2-proxy#1679)
* Revert "build(deps): bump socket2 from 0.4.4 to 0.4.5 (linkerd/linkerd2-proxy#1654)" (#1681)
* build(deps): bump EmbarkStudios/cargo-deny-action from 1.2.17 to 1.3.0 (linkerd/linkerd2-proxy#1678)
* build(deps): bump clang-sys from 1.3.1 to 1.3.2 (linkerd/linkerd2-proxy#1680)
* cache: generalize `Cache` into a key-value cache (linkerd/linkerd2-proxy#1683)
* build(deps): bump once_cell from 1.10.0 to 1.11.0 (linkerd/linkerd2-proxy#1687)
* build(deps): bump EmbarkStudios/cargo-deny-action from 1.3.0 to 1.3.1 (linkerd/linkerd2-proxy#1686)
* trace: add `/logs.json` endpoint to admin server (linkerd/linkerd2-proxy#1642)
* Dynamically discover policies for undocumented ports (linkerd/linkerd2-proxy#1677)
* Don't allow a policy to be used if `check_port_allowed` fails (linkerd/linkerd2-proxy#1689)
* ci: Simplify release workflow (linkerd/linkerd2-proxy#1690)
* build(deps): bump petgraph from 0.6.0 to 0.6.1 (linkerd/linkerd2-proxy#1696)
* build(deps): bump actions/upload-artifact from 3.0.0 to 3.1.0 (linkerd/linkerd2-proxy#1692)
* build(deps): bump tj-actions/changed-files from 20 to 20.1 (linkerd/linkerd2-proxy#1693)
* build(deps): bump http-body from 0.4.4 to 0.4.5 (linkerd/linkerd2-proxy#1694)
* build(deps): bump regex from 1.5.5 to 1.5.6 (linkerd/linkerd2-proxy#1695)
* build(deps): bump regex-syntax from 0.6.25 to 0.6.26 (linkerd/linkerd2-proxy#1697)
* http: Strip illegal headers from CONNECT responses (linkerd/linkerd2-proxy#1699)
* dev: Replace `Makefile` with `justfile` (linkerd/linkerd2-proxy#1691)

Signed-off-by: Oliver Gould <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@olix0r olix0r olix0r approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

proxy: Add a /logs.json admin endpoint

2 participants

@hawkw @olix0r