Skip to content

Time out protocol detect futures#464

Merged
olix0r merged 13 commits intomasterfrom
zd/accept-future
Apr 15, 2020
Merged

Time out protocol detect futures#464
olix0r merged 13 commits intomasterfrom
zd/accept-future

Conversation

@zaharidichev
Copy link
Member

@zaharidichev zaharidichev commented Mar 30, 2020

This PR changes the Accept trait to return a Future<Item = Future<()>>. The inner future is the one driving the connection while the outer future represents work such as protocol detect. This allows us to time out the outer future independently of the inner one. Additionally a timeout has been introduced to guard against serve first connections that do not run on a standard for the protocol port hanging and causing memory issues.

Fixes linkerd/linkerd2#4069

Signed-off-by: Zahari Dichev [email protected]

@zaharidichev zaharidichev force-pushed the zd/accept-future branch 3 times, most recently from a6f3a53 to 1816bf9 Compare March 31, 2020 12:36
@zaharidichev zaharidichev marked this pull request as ready for review March 31, 2020 19:46
@zaharidichev zaharidichev requested review from hawkw and olix0r March 31, 2020 19:46
@zaharidichev zaharidichev changed the title [WIP]Split connection and accept futures Time out protocol detect futures Mar 31, 2020
Copy link
Contributor

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think most of the indirection in this PR is necessary. In general, we should only need to return Box<dyn Future> when we might conditionally return different concrete types that need to be erased by boxing. Otherwise, we should continue returning the same concrete types we did before.

Signed-off-by: Zahari Dichev <[email protected]>
@zaharidichev zaharidichev requested a review from hawkw April 7, 2020 15:27
Copy link
Member

@olix0r olix0r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty close. I need to check it out and browse through it myself but I left some superficial comments (tioli)

Copy link
Contributor

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this change looks pretty good to me!

It would be good if there was an integration test asserting that a connection where no data is sent by the client times out at protocol detection correctly. I'm not sure how practical this is to test, but it would be nice to have.

In addition, I left some minor style comments. Hope they're helpful!

@zaharidichev
Copy link
Member Author

@hawkw I will add the test in a follow up PR if thats fine. At least it will be a good educational experience.

target.take().expect("polled after complete"),
io.prefix().as_ref(),
);
let inner_fut = accept.accept((target, BoxedIo::new(io)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting that there was a bug lurking in this change: we were no longer polling the accept to ready before dispatching. These changes were just superficial anyway, it seems, so I've backed this out to be as it was before.

Copy link
Contributor

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@olix0r's changes LGTM

@olix0r olix0r merged commit 8d8f409 into master Apr 15, 2020
@olix0r olix0r deleted the zd/accept-future branch April 15, 2020 21:04
olix0r added a commit to linkerd/linkerd2 that referenced this pull request Apr 15, 2020
This release includes a new protocol detection timeout, which prevents
clients from consuming resources indefinitely when they do not send any
data.

Additionally: the proxy's admin endpoint now supports a `/live` endpoint
for liveness checks, and a feature has been added to enrich tracing
metadata from a file of label/values.

---

* Add Labels from a path as oc-collector attributes (linkerd/linkerd2-proxy#463)
* Add liveness endpoint to admin server (linkerd/linkerd2-proxy#470)
* docker: Use buildkit for caching (linkerd/linkerd2-proxy#472)
* Makefile: Use STRIP variable with strip as default (linkerd/linkerd2-proxy#475)
* Add checksec to the release process (linkerd/linkerd2-proxy#476)
* Time out protocol detect futures (linkerd/linkerd2-proxy#464)
* Ensure that checksec is executable (linkerd/linkerd2-proxy#477)
* Fix the checksec URL (linkerd/linkerd2-proxy#478)
* Undo hardcoded release version (linkerd/linkerd2-proxy#479)
olix0r added a commit to linkerd/linkerd2 that referenced this pull request Apr 16, 2020
This release includes a new protocol detection timeout, which prevents
clients from consuming resources indefinitely when they do not send any
data.

Additionally: the proxy's admin endpoint now supports a `/live` endpoint
for liveness checks, and a feature has been added to enrich tracing
metadata from a file of label/values.

---

* Add Labels from a path as oc-collector attributes (linkerd/linkerd2-proxy#463)
* Add liveness endpoint to admin server (linkerd/linkerd2-proxy#470)
* docker: Use buildkit for caching (linkerd/linkerd2-proxy#472)
* Makefile: Use STRIP variable with strip as default (linkerd/linkerd2-proxy#475)
* Add checksec to the release process (linkerd/linkerd2-proxy#476)
* Time out protocol detect futures (linkerd/linkerd2-proxy#464)
* Ensure that checksec is executable (linkerd/linkerd2-proxy#477)
* Fix the checksec URL (linkerd/linkerd2-proxy#478)
* Undo hardcoded release version (linkerd/linkerd2-proxy#479)
hawkw added a commit that referenced this pull request May 5, 2020
This branch updates `master-tokio-0.2` from the latest `master`.

A lot of this diff is totally unrelated and is largely ignoreable (e.g.
changes to the docker build configuration, etc). However, there was also
a decently large chunk of code that had changed in both
`master-tokio-0.2` and on master. The majority of the merge conflicts
were related to the changes to `Accept` in #464, because `Accept` had
also been updated to use `std::future` on `master-tokio-0.2`. A review
should probably focus on the `Accept` codepaths.

Signed-off-by: Eliza Weisman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Excessive memory usage since upgrade to stable-2.7.0

3 participants