trace: tweak tracing & test support for the multithreaded runtime#616
Merged
trace: tweak tracing & test support for the multithreaded runtime#616
Conversation
olix0r
approved these changes
Aug 5, 2020
olix0r
added a commit
to linkerd/linkerd2
that referenced
this pull request
Aug 5, 2020
This release enables a multi-threaded runtime. Previously, the proxy would only ever use a single thread for data plane processing; now, when the proxy is allocated more than 1 CPU share, the proxy allocates a thread per available CPU. This has shown substantial latency improvements in benchmarks, especially when the proxy is serving requests for many concurrent connections. --- * Add a `multicore` feature flag (linkerd/linkerd2-proxy#611) * Add `multicore` to default features (linkerd/linkerd2-proxy#612) * admin: add an endpoint to dump spawned Tokio tasks (linkerd/linkerd2-proxy#595) * trace: roll `tracing` and `tracing-subscriber` dependencies (linkerd/linkerd2-proxy#615) * stack: Add NewService::into_make_service (linkerd/linkerd2-proxy#618) * trace: tweak tracing & test support for the multithreaded runtime (linkerd/linkerd2-proxy#616) * Make FailFast cloneable (linkerd/linkerd2-proxy#617) * Move HTTP detection & server into linkerd2_proxy_http (linkerd/linkerd2-proxy#619) * Mark tap integration tests as flakey (linkerd/linkerd2-proxy#621) * Introduce a SkipDetect layer to preempt detection (linkerd/linkerd2-proxy#620)
adleong
pushed a commit
to linkerd/linkerd2
that referenced
this pull request
Aug 6, 2020
This release enables a multi-threaded runtime. Previously, the proxy would only ever use a single thread for data plane processing; now, when the proxy is allocated more than 1 CPU share, the proxy allocates a thread per available CPU. This has shown substantial latency improvements in benchmarks, especially when the proxy is serving requests for many concurrent connections. --- * Add a `multicore` feature flag (linkerd/linkerd2-proxy#611) * Add `multicore` to default features (linkerd/linkerd2-proxy#612) * admin: add an endpoint to dump spawned Tokio tasks (linkerd/linkerd2-proxy#595) * trace: roll `tracing` and `tracing-subscriber` dependencies (linkerd/linkerd2-proxy#615) * stack: Add NewService::into_make_service (linkerd/linkerd2-proxy#618) * trace: tweak tracing & test support for the multithreaded runtime (linkerd/linkerd2-proxy#616) * Make FailFast cloneable (linkerd/linkerd2-proxy#617) * Move HTTP detection & server into linkerd2_proxy_http (linkerd/linkerd2-proxy#619) * Mark tap integration tests as flakey (linkerd/linkerd2-proxy#621) * Introduce a SkipDetect layer to preempt detection (linkerd/linkerd2-proxy#620)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch makes a couple minor tweaks to improve tracing with the
multithreaded Tokio runtime.
It enables the option to print thread IDs when logging trace output,
which should make it somewhat clearer when events occur on different
threads.
Additionally, there's currently an issue where traces are not logged
from the proxy during integration tests. Only traces from the test
support code are logged. This is because we are using
tracing's scopedthread-local dispatcher in the test proxy, so that each test can create
its own subscriber. We set the test's subscriber as the default for the
test proxy thread, but not as the global default for all threads.
However, when the multithreaded runtime is enabled, the default runtime
returned by
tokio::runtime::Runtime::newis now a multithreadedruntime. Therefore, the test proxy runs on the runtime's worker threads,
rather than the main thread spawned by the tests, and the dispatcher is
not set for the worker threads.
This branch fixes that issue by explicitly using the basic (single
threaded) scheduler when creating the test proxy. This should
(hopefully) also improve test flakiness a bit by not spawning a whole
bunch of workers.