Skip to content

balance: Configure weights from keys, not services#281

Merged
olix0r merged 7 commits intotower-rs:masterfrom
olix0r:ver/lb-weight-key
May 9, 2019
Merged

balance: Configure weights from keys, not services#281
olix0r merged 7 commits intotower-rs:masterfrom
olix0r:ver/lb-weight-key

Conversation

@olix0r
Copy link
Collaborator

@olix0r olix0r commented May 9, 2019

The initial weighted balancing implementation required that the
underlying service implement HasWeight.

Practically, this doesn't work that well, since this may force
middlewares to implement this trait as well.

To fix this, we change the type bounds so that keys, not services,
must implement HasWeight.

This has a drawback, though, in that Weight, which contains a float,
cannot implement Hash or Eq, which is required by the balancer. This
tradeoff seems manageable, though (and is already addressed in linkerd,
for instance).

The initial weighted balancing implementation required that the
underlying service implement `HasWeight`.

Practically, this doesn't work that well, since this may force
middlewares to implement this trait as well.

To fix this, we change the type bounds so that _keys_, not services,
must implement `HasWeight`.

This has a drawback, though, in that Weight, which contains a float,
cannot implement `Hash` or `Eq`, which is required by the balancer. This
tradeoff seems manageable, though (and is already addressed in linkerd,
for instance).
@olix0r olix0r requested review from carllerche and seanmonstar May 9, 2019 18:15
@olix0r olix0r self-assigned this May 9, 2019
@seanmonstar
Copy link
Collaborator

This has a drawback, though, in that Weight, which contains a float,
cannot implement Hash or Eq, which is required by the balancer.

What if Weight didn't wrap a float, but a u64 instead?

@olix0r
Copy link
Collaborator Author

olix0r commented May 9, 2019

@seanmonstar Yeah, I agree that's a good change we should pursue. I think that's sufficiently separate, though, that we can merge this and address that in a followup.

@olix0r olix0r merged commit 9b27863 into tower-rs:master May 9, 2019
@olix0r olix0r deleted the ver/lb-weight-key branch May 9, 2019 19:17
hawkw pushed a commit to linkerd/linkerd2-proxy that referenced this pull request May 9, 2019
Tower must be updated in order to pickup tower-rs/tower#281
to address linkerd/linkerd2#2804.

This adopts released crates where possible.
hawkw added a commit to linkerd/linkerd2 that referenced this pull request May 10, 2019
commit 5f89351081eff47a4ab8cd88e2e1a69a04f86541
Author: Oliver Gould <[email protected]>
Date:   Thu May 9 16:39:24 2019 -0700

    Upgrade tower dependencies (#249)

    Tower must be updated in order to pickup tower-rs/tower#281
    to address #2804.

    This adopts released crates where possible.

commit 5d5eed6f8180b8db4090d995e71fdf7b0890c647
Author: Zahari Dichev <[email protected]>
Date:   Thu May 9 01:08:34 2019 +0300

    Assert that TLS connection is refused if identity is not certified yet (#243)

    This branch adds tls capability to the support cient used in tests. In addition to that it adds two tests verifying that a TLS connection is refused in case the identity is not certified yet. This attempts to fix ##2598 and provide facility to write tests for #2676.

    As these are still some of my first lines of Rust code, it is advised to approach everything with a healthy dose of doubt :)

    Signed-off-by: Zahari Dichev <[email protected]>

commit 1b9bb3745e44c959d1d41d14fed2b2822c82b5ba
Author: Oliver Gould <[email protected]>
Date:   Wed May 8 14:28:37 2019 -0700

    Introduce dispatch timeouts around buffers (#246)

    The proxy has several buffers, especially where it routes requests over
    shared stacks. If any of these routes is unavailable, then a request may
    remain buffered indefinitely. Previously, before service profiles were
    introduced, there was a default _response_ timeout that would cause
    these requests to fail; but since this response timeout is now optional
    (and is only applied once the request has been routed within a proxy),
    then we need a new mechanism to prevent requests from getting "stuck".

    This change does the following:
    - all proxied requests are annotated with a dispatch deadline;
    - each time a request is bufered, a timeout is registered.
    - if the timeout fires, the response exception fails, a 503 is returned,
      and the request is dropped.
    - if the request is processed into the inner stack, the timeout is
      ignored.

    The dispatch timeout limits the _time a request is buffered in a proxy_.
    This is distinct from the response timeout, as the server's response may
    naturally be delayed for any number of (non-proxy-related) reasons.

    The `insert_target` module has been generalized to `insert` to support
    setting the DispatchDeadline extension.

    The `buffer` module has been augmented with generic deadline-extraction
    logic.

    The `svc` module now exposes its own builder type that notably adds
    a `buffer_pending` helper. It's helpful to pull a builder type into the
    proxy to assist debugging type errors when modifying stacks.

    Fixes #2779 #2795

commit caf899557c3b041190f63544da865396231b3e30
Author: Oliver Gould <[email protected]>
Date:   Fri May 3 15:55:32 2019 -0700

    router: Fail requests when the route is not ready (#241)

    In #2779, we plan to expire requests while they are
    buffered. However, the router _implicitly_ buffers requests in the
    executor when the inner service is not ready.

    This change alters the route to wrap all inner layers in a `LoadShed`
    so it can expect all services to `poll_ready()` immediately.

commit 587bad101d9e5daeacb24b6733097c350a798356
Author: Eliza Weisman <[email protected]>
Date:   Fri May 3 14:18:08 2019 -0700

    Remove Destination service query concurrency limit (#244)

    Currently, the proxy enforces a limit on the number of concurrent
    queries (i.e., the number of gRPC streams) to the Destination service.
    This limit was added based on information about the behaviour of the
    Destination service that is now known to be incorrect.

    This branch removes the limit on concurrent queries from the proxy's
    `control::destination` module. Although it should now be possible to
    simplify this code as a result of this change, I've refrained from doing
    any major refactoring in this branch --- my intention is to do this
    after the DNS fallback behaviour has also been removed, as together with
    this change, that will result in a _significant_ simplification of the
    module. Additionally, I've removed the tests for the concurrency limit,
    as they are no longer relevant.

    The `LINKERD2_PROXY_DESTINATION_CLIENT_CONCURRENCY_LIMIT`
    environment variable was also removed; this is not a breaking change as
    neither the CLI nor the proxy injector will currently set this env var.

    Signed-off-by: Eliza Weisman <[email protected]>

commit cbdf45b44f7e4d852dc0497716062167ab9539fb
Author: Sean McArthur <[email protected]>
Date:   Thu May 2 11:47:48 2019 -0700

    Remove h2::Error requirement from metrics

    Signed-off-by: Sean McArthur <[email protected]>

commit 3276949d4608dc4344b7bed3de2fc4b3080c2c6e
Author: Sean McArthur <[email protected]>
Date:   Thu May 2 09:44:00 2019 -0700

    delete unused proxy::http::metrics::class module

    Signed-off-by: Sean McArthur <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
hawkw added a commit to linkerd/linkerd2 that referenced this pull request May 10, 2019
commit 5f89351081eff47a4ab8cd88e2e1a69a04f86541
Author: Oliver Gould <[email protected]>
Date:   Thu May 9 16:39:24 2019 -0700

    Upgrade tower dependencies (#249)

    Tower must be updated in order to pickup tower-rs/tower#281
    to address #2804.

    This adopts released crates where possible.

commit 5d5eed6f8180b8db4090d995e71fdf7b0890c647
Author: Zahari Dichev <[email protected]>
Date:   Thu May 9 01:08:34 2019 +0300

    Assert that TLS connection is refused if identity is not certified yet (#243)

    This branch adds tls capability to the support cient used in tests. In addition to that it adds two tests verifying that a TLS connection is refused in case the identity is not certified yet. This attempts to fix ##2598 and provide facility to write tests for #2676.

    As these are still some of my first lines of Rust code, it is advised to approach everything with a healthy dose of doubt :)

    Signed-off-by: Zahari Dichev <[email protected]>

commit 1b9bb3745e44c959d1d41d14fed2b2822c82b5ba
Author: Oliver Gould <[email protected]>
Date:   Wed May 8 14:28:37 2019 -0700

    Introduce dispatch timeouts around buffers (#246)

    The proxy has several buffers, especially where it routes requests over
    shared stacks. If any of these routes is unavailable, then a request may
    remain buffered indefinitely. Previously, before service profiles were
    introduced, there was a default _response_ timeout that would cause
    these requests to fail; but since this response timeout is now optional
    (and is only applied once the request has been routed within a proxy),
    then we need a new mechanism to prevent requests from getting "stuck".

    This change does the following:
    - all proxied requests are annotated with a dispatch deadline;
    - each time a request is bufered, a timeout is registered.
    - if the timeout fires, the response exception fails, a 503 is returned,
      and the request is dropped.
    - if the request is processed into the inner stack, the timeout is
      ignored.

    The dispatch timeout limits the _time a request is buffered in a proxy_.
    This is distinct from the response timeout, as the server's response may
    naturally be delayed for any number of (non-proxy-related) reasons.

    The `insert_target` module has been generalized to `insert` to support
    setting the DispatchDeadline extension.

    The `buffer` module has been augmented with generic deadline-extraction
    logic.

    The `svc` module now exposes its own builder type that notably adds
    a `buffer_pending` helper. It's helpful to pull a builder type into the
    proxy to assist debugging type errors when modifying stacks.

    Fixes #2779 #2795

commit caf899557c3b041190f63544da865396231b3e30
Author: Oliver Gould <[email protected]>
Date:   Fri May 3 15:55:32 2019 -0700

    router: Fail requests when the route is not ready (#241)

    In #2779, we plan to expire requests while they are
    buffered. However, the router _implicitly_ buffers requests in the
    executor when the inner service is not ready.

    This change alters the route to wrap all inner layers in a `LoadShed`
    so it can expect all services to `poll_ready()` immediately.

commit 587bad101d9e5daeacb24b6733097c350a798356
Author: Eliza Weisman <[email protected]>
Date:   Fri May 3 14:18:08 2019 -0700

    Remove Destination service query concurrency limit (#244)

    Currently, the proxy enforces a limit on the number of concurrent
    queries (i.e., the number of gRPC streams) to the Destination service.
    This limit was added based on information about the behaviour of the
    Destination service that is now known to be incorrect.

    This branch removes the limit on concurrent queries from the proxy's
    `control::destination` module. Although it should now be possible to
    simplify this code as a result of this change, I've refrained from doing
    any major refactoring in this branch --- my intention is to do this
    after the DNS fallback behaviour has also been removed, as together with
    this change, that will result in a _significant_ simplification of the
    module. Additionally, I've removed the tests for the concurrency limit,
    as they are no longer relevant.

    The `LINKERD2_PROXY_DESTINATION_CLIENT_CONCURRENCY_LIMIT`
    environment variable was also removed; this is not a breaking change as
    neither the CLI nor the proxy injector will currently set this env var.

    Signed-off-by: Eliza Weisman <[email protected]>

commit cbdf45b44f7e4d852dc0497716062167ab9539fb
Author: Sean McArthur <[email protected]>
Date:   Thu May 2 11:47:48 2019 -0700

    Remove h2::Error requirement from metrics

    Signed-off-by: Sean McArthur <[email protected]>

commit 3276949d4608dc4344b7bed3de2fc4b3080c2c6e
Author: Sean McArthur <[email protected]>
Date:   Thu May 2 09:44:00 2019 -0700

    delete unused proxy::http::metrics::class module

    Signed-off-by: Sean McArthur <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants