Skip to content

tracing: use Span::or_current when spawning tasks#1272

Merged
hawkw merged 2 commits intomainfrom
eliza/span-or-current
Sep 16, 2021
Merged

tracing: use Span::or_current when spawning tasks#1272
hawkw merged 2 commits intomainfrom
eliza/span-or-current

Conversation

@hawkw
Copy link
Contributor

@hawkw hawkw commented Sep 16, 2021

tracing v0.1.27 added a new Span::or_current method to aid with
efficiently propagating spans to spawned tasks (or threads). The method
is intended to be used when a spawned task should be instrumented with a
provided span if it is enabled, or the current span if the provided span
is not enabled. span.or_current() returns span if it is enabled,
or, if span is disabled, it clones the current span and returns it.

This offers better performance than writing code like this:

    .instrument(span)
    .in_current_span()

First, or_current will only look up the current span (requiring a
thread-local access) if span is not enabled, while
.instrument(span).in_current_span() will do this unconditionally.

Second, and more importantly, using .in_current_span() will wrap the
spawned future in two Instrumented futures. One will enter the
current span whenever the task is polled, and inside that, the second
instrument will enter the passed span. This means we are entering two
spans instead of one, in the case where both are enabled. That adds a
small amount of additional overhead in every poll.

Therefore, this branch changes code that instruments a future prior to
spawning it to use Span::or_current. A lot of this is test code, which
(it turns out) doesn't really handle span propagation at all. However,
this also touches the main accept loop, as well as where we spawn admin
background tasks.

`tracing` v0.1.27 added a new [`Span::or_current`] method to aid with
efficiently propagating spans to spawned tasks (or threads). The method
is intended to be used when a spawned task should be instrumented with a
provided span if it is enabled, or the current span if the provided span
is _not_ enabled. `span.or_current()` returns `span` if it is enabled,
or, if `span` is disabled, it clones the current span and returns it.

This offers better performance than writing code like this:
```rust
    .instrument(span)
    .in_current_span()
```

First, `or_current` will only look up the current span (requiring a
thread-local access) _if_ `span` is not enabled, while
`.instrument(span).in_current_span()` will do this unconditionally.

Second, and more importantly, using `.in_current_span()` will wrap the
spawned future in _two_ `Instrumented` futures. One will enter the
current span whenever the task is polled, and inside that, the second
`instrument` will enter the passed span. This means we are entering two
spans instead of one, in the case where both are enabled. That adds a
small amount of additional overhead in every poll.

Therefore, this branch changes code that instruments a future prior to
spawning it to use `Span::or_current`. A lot of this is test code, which
(it turns out) doesn't really handle span propagation at all. However,
this also touches the main accept loop, as well as where we spawn admin
background tasks.

[`Span::or_current`]: https://docs.rs/tracing/0.1.27/tracing/span/struct.Span.html#method.or_current
@hawkw hawkw requested review from a team, kleimkuhler and olix0r September 16, 2021 19:04
Copy link
Member

@olix0r olix0r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once it passes CI (there's an error in the test controller)

);
});
}
.instrument(span.exit().or_current()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this exits the above-entered span? and then we use that or the parent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup!

@hawkw hawkw merged commit 98fee53 into main Sep 16, 2021
@hawkw hawkw deleted the eliza/span-or-current branch September 16, 2021 21:34
olix0r added a commit to linkerd/linkerd2 that referenced this pull request Sep 23, 2021
This release improves the proxy's error handling, introducing a new
`l5d-proxy-connection` header to signal from an inbound proxy when its
peers outbound connections should be torn down.

Furthermore, error handling has been improved so that the
`l5d-proxy-error` header is only sent to trusted peers--the inbound
proxy only emits this header when its client is meshed; and the outbound
proxy can be configured to disable these headers via configuration.

---

* build(deps): bump hyper from 0.14.12 to 0.14.13 (linkerd/linkerd2-proxy#1273)
* build(deps): bump tracing-subscriber from 0.2.22 to 0.2.23 (linkerd/linkerd2-proxy#1274)
* tracing: use `Span::or_current` when spawning tasks (linkerd/linkerd2-proxy#1272)
* dns: Log TTL with resolution (linkerd/linkerd2-proxy#1275)
* error-respond: Support stack target configuration (linkerd/linkerd2-proxy#1276)
* build(deps): bump tracing-subscriber from 0.2.23 to 0.2.24 (linkerd/linkerd2-proxy#1277)
* build(deps): bump tracing from 0.1.27 to 0.1.28 (linkerd/linkerd2-proxy#1278)
* build(deps): bump tokio from 1.11.0 to 1.12.0 (linkerd/linkerd2-proxy#1279)
* build(deps): bump http from 0.2.4 to 0.2.5 (linkerd/linkerd2-proxy#1280)
* Support a `l5d-proxy-connection: close` header (linkerd/linkerd2-proxy#1281)
* Avoid emitting informational headers to untrusted clients (linkerd/linkerd2-proxy#1282)
* outbound: Only honor the `l5d-proxy-connection` header when meshed (linkerd/linkerd2-proxy#1283)
* outbound: Disable informational headers by config (linkerd/linkerd2-proxy#1284)
* outbound: Strip peer-sent `l5d-proxy-error` headers (linkerd/linkerd2-proxy#1285)
alpeb pushed a commit to linkerd/linkerd2 that referenced this pull request Sep 23, 2021
This release improves the proxy's error handling, introducing a new
`l5d-proxy-connection` header to signal from an inbound proxy when its
peers outbound connections should be torn down.

Furthermore, error handling has been improved so that the
`l5d-proxy-error` header is only sent to trusted peers--the inbound
proxy only emits this header when its client is meshed; and the outbound
proxy can be configured to disable these headers via configuration.

---

* build(deps): bump hyper from 0.14.12 to 0.14.13 (linkerd/linkerd2-proxy#1273)
* build(deps): bump tracing-subscriber from 0.2.22 to 0.2.23 (linkerd/linkerd2-proxy#1274)
* tracing: use `Span::or_current` when spawning tasks (linkerd/linkerd2-proxy#1272)
* dns: Log TTL with resolution (linkerd/linkerd2-proxy#1275)
* error-respond: Support stack target configuration (linkerd/linkerd2-proxy#1276)
* build(deps): bump tracing-subscriber from 0.2.23 to 0.2.24 (linkerd/linkerd2-proxy#1277)
* build(deps): bump tracing from 0.1.27 to 0.1.28 (linkerd/linkerd2-proxy#1278)
* build(deps): bump tokio from 1.11.0 to 1.12.0 (linkerd/linkerd2-proxy#1279)
* build(deps): bump http from 0.2.4 to 0.2.5 (linkerd/linkerd2-proxy#1280)
* Support a `l5d-proxy-connection: close` header (linkerd/linkerd2-proxy#1281)
* Avoid emitting informational headers to untrusted clients (linkerd/linkerd2-proxy#1282)
* outbound: Only honor the `l5d-proxy-connection` header when meshed (linkerd/linkerd2-proxy#1283)
* outbound: Disable informational headers by config (linkerd/linkerd2-proxy#1284)
* outbound: Strip peer-sent `l5d-proxy-error` headers (linkerd/linkerd2-proxy#1285)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants