inbound: Record policy metrics for opaque-transport connections#1780
Merged
inbound: Record policy metrics for opaque-transport connections#1780
Conversation
When the inbound receives a "direct" inbound connection--a connection that targets the proxy's inbound port--it uses the "opaque transport" wrapper to discover the actual destination port for the connection. In this case, the direct stack does a one-off authorization check instead of using the proper TCP policy layer, which means that these connections don't get the proper authorization metrics. This change updates the direct TCP stack to use the TcpPolicy middleware to enforce authorization. It also removes the authorization checking utilities from the public `AllowPolicy` type so that all authorizations must go through the metrics-tracking middlewares. The `policy` tests are moved under the `policy::tcp` module so they have access to these functions. The TCP accept stack no longer performs eager authorization denials when a default-deny policy is used, as these failures bypassed the metrics code also. Signed-off-by: Oliver Gould <[email protected]>
hawkw
approved these changes
Jun 23, 2022
Contributor
hawkw
left a comment
hawkw
left a comment
There was a problem hiding this comment.
looks good to me! i commented on a couple very minor nits, but no blockers!
Comment on lines
+131
to
+140
| .check_new_service::<AuthorizedLocalTcp, _>() | ||
| .push_map_target(|(permit, tcp): (policy::ServerPermit, LocalTcp)| { | ||
| AuthorizedLocalTcp { | ||
| addr: tcp.server_addr, | ||
| client_id: tcp.client_id, | ||
| permit, | ||
| } | ||
| }) | ||
| .check_new_service::<(policy::ServerPermit, LocalTcp), _>() | ||
| .push(policy::NewTcpPolicy::layer(rt.metrics.tcp_authz.clone())) |
Contributor
There was a problem hiding this comment.
IMO it feels kind of like it would be nicer to me if the state transition from LocalTcp -> AuthorizedLocalTcp happened inside of the NewTcpPolicy layer...but, i guess trying to abstract over types that can be turned into an authorized target type by adding a ServerPermit with some kind of trait is probably not worth the complexity, so, this is fine IMO...
Comment on lines
+183
to
+184
| server: &ServerPolicy, | ||
| dst: OrigDstAddr, |
Contributor
There was a problem hiding this comment.
nit, not a big deal: it kinda feels like this should just take an &AllowPolicy instead of two pieces of it (the ServerPolicy and orig dst)... it seems like the only reason it doesn't is because we want to borrow the ServerPolicy for the tracing event, but...we could just move that into this function?
Contributor
There was a problem hiding this comment.
or maybe it's like that for the tests? in which case, no worries. not a big deal regardless.
Member
Author
There was a problem hiding this comment.
yeah, i want to test this function without all of the extra watch stuff.
olix0r
added a commit
to linkerd/linkerd2
that referenced
this pull request
Jun 30, 2022
This release updates the proxy's service discovery module to avoid redundant load balancer updates that could cause unnecessary connection churn. This release also includes improvements to the proxy's retry handling of gRPC requests. The proxy would not retry requests when a response's status code was emitted in a TRAILERS frame. This has been fixed. This release also includes a number of internal changes that set up for per-route authorization. There should be no user-facing impact at this point except for the introduction of additional metrics labels. --- * build(deps): bump mio from 0.8.3 to 0.8.4 (linkerd/linkerd2-proxy#1760) * build(deps): bump quote from 1.0.18 to 1.0.19 (linkerd/linkerd2-proxy#1761) * build(deps): bump tower-service from 0.3.1 to 0.3.2 (linkerd/linkerd2-proxy#1762) * build(deps): bump proc-macro2 from 1.0.39 to 1.0.40 (linkerd/linkerd2-proxy#1763) * build(deps): bump syn from 1.0.96 to 1.0.98 (linkerd/linkerd2-proxy#1764) * build(deps): bump prettyplease from 0.1.12 to 0.1.14 (linkerd/linkerd2-proxy#1766) * build(deps): bump anyhow from 1.0.57 to 1.0.58 (linkerd/linkerd2-proxy#1767) * dev: Update build settings (linkerd/linkerd2-proxy#1765) * Dedupe discovery updates (linkerd/linkerd2-proxy#1759) * build(deps): bump quote from 1.0.19 to 1.0.20 (linkerd/linkerd2-proxy#1768) * deny: Remove tokio-util from exceptions (linkerd/linkerd2-proxy#1769) * dev: Update memory contraints (linkerd/linkerd2-proxy#1770) * Reorganize `server-policy` to set up for routes (linkerd/linkerd2-proxy#1771) * inbound: Rename policy-enforcement layers (linkerd/linkerd2-proxy#1772) * ci: Split fuzzer logic into a script (linkerd/linkerd2-proxy#1773) * build(deps): bump prettyplease from 0.1.14 to 0.1.15 (linkerd/linkerd2-proxy#1775) * build(deps): bump indexmap from 1.9.0 to 1.9.1 (linkerd/linkerd2-proxy#1776) * integration: Cleanup test server (linkerd/linkerd2-proxy#1777) * http-retry: Move the ReplayBody type into a module (linkerd/linkerd2-proxy#1778) * inbound: Add route authorization labels (linkerd/linkerd2-proxy#1774) * Rename HTTPRoutePermit to HttpRoutePermit (linkerd/linkerd2-proxy#1779) * retry gRPC requests are immediately terminated by trailers (linkerd/linkerd2-proxy#1706) * inbound: Record policy metrics for opaque-transport connections (linkerd/linkerd2-proxy#1780) * build(deps): bump tj-actions/changed-files from 23 to 23.1 (linkerd/linkerd2-proxy#1782) * build(deps): bump derive_arbitrary from 1.1.2 to 1.1.3 (linkerd/linkerd2-proxy#1783) * build(deps): bump arbitrary from 1.1.2 to 1.1.3 (linkerd/linkerd2-proxy#1784) * inbound: Record TCP metrics for forwarded TLS connections (linkerd/linkerd2-proxy#1785) * inbound: Cleanup in preparation for route policies #1781 (linkerd/linkerd2-proxy#1786) * Add HTTP route matchers to support the Gateway API (linkerd/linkerd2-proxy#1787) * build(deps): bump unicode-normalization from 0.1.19 to 0.1.20 (linkerd/linkerd2-proxy#1789) * build(deps): bump linked-hash-map from 0.5.4 to 0.5.6 (linkerd/linkerd2-proxy#1790) * build(deps): bump smallvec from 1.8.0 to 1.8.1 (linkerd/linkerd2-proxy#1791) * build(deps): bump jemalloc-sys from 0.5.0+5.3.0 to 0.5.1+5.3.0-patched (linkerd/linkerd2-proxy#1792) * Introduce per-route authorization policies (linkerd/linkerd2-proxy#1781) * inbound: Add a header-modification route filter (linkerd/linkerd2-proxy#1793) * docs: update justfile man page link (linkerd/linkerd2-proxy#1794) Signed-off-by: Oliver Gould <[email protected]>
olix0r
added a commit
to linkerd/linkerd2
that referenced
this pull request
Jun 30, 2022
This release updates the proxy's service discovery module to avoid redundant load balancer updates that could cause unnecessary connection churn. This release also includes improvements to the proxy's retry handling of gRPC requests. The proxy would not retry requests when a response's status code was emitted in a TRAILERS frame. This has been fixed. This release also includes a number of internal changes that set up for per-route authorization. There should be no user-facing impact at this point except for the introduction of additional metrics labels. --- * build(deps): bump mio from 0.8.3 to 0.8.4 (linkerd/linkerd2-proxy#1760) * build(deps): bump quote from 1.0.18 to 1.0.19 (linkerd/linkerd2-proxy#1761) * build(deps): bump tower-service from 0.3.1 to 0.3.2 (linkerd/linkerd2-proxy#1762) * build(deps): bump proc-macro2 from 1.0.39 to 1.0.40 (linkerd/linkerd2-proxy#1763) * build(deps): bump syn from 1.0.96 to 1.0.98 (linkerd/linkerd2-proxy#1764) * build(deps): bump prettyplease from 0.1.12 to 0.1.14 (linkerd/linkerd2-proxy#1766) * build(deps): bump anyhow from 1.0.57 to 1.0.58 (linkerd/linkerd2-proxy#1767) * dev: Update build settings (linkerd/linkerd2-proxy#1765) * Dedupe discovery updates (linkerd/linkerd2-proxy#1759) * build(deps): bump quote from 1.0.19 to 1.0.20 (linkerd/linkerd2-proxy#1768) * deny: Remove tokio-util from exceptions (linkerd/linkerd2-proxy#1769) * dev: Update memory contraints (linkerd/linkerd2-proxy#1770) * Reorganize `server-policy` to set up for routes (linkerd/linkerd2-proxy#1771) * inbound: Rename policy-enforcement layers (linkerd/linkerd2-proxy#1772) * ci: Split fuzzer logic into a script (linkerd/linkerd2-proxy#1773) * build(deps): bump prettyplease from 0.1.14 to 0.1.15 (linkerd/linkerd2-proxy#1775) * build(deps): bump indexmap from 1.9.0 to 1.9.1 (linkerd/linkerd2-proxy#1776) * integration: Cleanup test server (linkerd/linkerd2-proxy#1777) * http-retry: Move the ReplayBody type into a module (linkerd/linkerd2-proxy#1778) * inbound: Add route authorization labels (linkerd/linkerd2-proxy#1774) * Rename HTTPRoutePermit to HttpRoutePermit (linkerd/linkerd2-proxy#1779) * retry gRPC requests are immediately terminated by trailers (linkerd/linkerd2-proxy#1706) * inbound: Record policy metrics for opaque-transport connections (linkerd/linkerd2-proxy#1780) * build(deps): bump tj-actions/changed-files from 23 to 23.1 (linkerd/linkerd2-proxy#1782) * build(deps): bump derive_arbitrary from 1.1.2 to 1.1.3 (linkerd/linkerd2-proxy#1783) * build(deps): bump arbitrary from 1.1.2 to 1.1.3 (linkerd/linkerd2-proxy#1784) * inbound: Record TCP metrics for forwarded TLS connections (linkerd/linkerd2-proxy#1785) * inbound: Cleanup in preparation for route policies #1781 (linkerd/linkerd2-proxy#1786) * Add HTTP route matchers to support the Gateway API (linkerd/linkerd2-proxy#1787) * build(deps): bump unicode-normalization from 0.1.19 to 0.1.20 (linkerd/linkerd2-proxy#1789) * build(deps): bump linked-hash-map from 0.5.4 to 0.5.6 (linkerd/linkerd2-proxy#1790) * build(deps): bump smallvec from 1.8.0 to 1.8.1 (linkerd/linkerd2-proxy#1791) * build(deps): bump jemalloc-sys from 0.5.0+5.3.0 to 0.5.1+5.3.0-patched (linkerd/linkerd2-proxy#1792) * Introduce per-route authorization policies (linkerd/linkerd2-proxy#1781) * inbound: Add a header-modification route filter (linkerd/linkerd2-proxy#1793) * docs: update justfile man page link (linkerd/linkerd2-proxy#1794) Signed-off-by: Oliver Gould <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the inbound receives a "direct" inbound connection--a connection
that targets the proxy's inbound port--it uses the "opaque transport"
wrapper to discover the actual destination port for the connection. In
this case, the direct stack does a one-off authorization check instead
of using the proper TCP policy layer, which means that these connections
don't get the proper authorization metrics.
This change updates the direct TCP stack to use the TcpPolicy middleware
to enforce authorization. It also removes the authorization checking
utilities from the public
AllowPolicytype so that all authorizationsmust go through the metrics-tracking middlewares. The
policytests aremoved under the
policy::tcpmodule so they have access to thesefunctions.
The TCP accept stack no longer performs eager authorization denials when
a default-deny policy is used, as these failures bypassed the metrics
code also.
Signed-off-by: Oliver Gould [email protected]