Skip to content

xds: fix for delta xDS reconnect bug in LDS/CDS#12174

Merged
rboyer merged 5 commits intomainfrom
fix-xds-delta-reconnect-wildcard
Jan 25, 2022
Merged

xds: fix for delta xDS reconnect bug in LDS/CDS#12174
rboyer merged 5 commits intomainfrom
fix-xds-delta-reconnect-wildcard

Conversation

@rboyer
Copy link
Copy Markdown
Member

@rboyer rboyer commented Jan 24, 2022

When a wildcard xDS type (LDS/CDS/SRDS) reconnects from a delta xDS stream, prior to envoy 1.19.0 it would populate the ResourceNamesSubscribe field with the full list of currently subscribed items, instead of simply omitting it to infer that it wanted everything (which is what wildcard mode means).

This upstream issue was filed in envoyproxy/envoy#16063 and fixed in envoyproxy/envoy#16153 which went out in Envoy 1.19.0 and is fixed in later versions (later refactored in envoyproxy/envoy#16855).

This PR conditionally forces LDS/CDS to be wildcard-only even when the connected Envoy requests a non-wildcard subscription, but only does so on versions prior to 1.19.0, as we should not need to do this on later versions.

This replaces PR #12160 by @fredwangwang that solved the failure case as described here: #11833 (comment)

fredwangwang and others added 2 commits January 24, 2022 10:10
when the envoy sidecar get disconnected from the xds stream, the reconnecting request will
contain `InitialResourceVersions` and `ResourceNamesSubscribe`. This is OK for endpoint and route type,
as these two are driven by (child types of) cluster and listener type respectively.

However, register `cluster` type as subscription instead of wildcard would cause envoy
not able to get any new cluster updates for the rest of this life. Same goes for `listener`.

This pr is to always set cluster and listener type to wildcard, to ensure the envoy sidecar will get
those updates after disconnecting from xds stream for whatever reason (network blip/consul restart/etc).
@rboyer rboyer requested a review from a team January 24, 2022 17:14
@github-actions github-actions bot added the theme/envoy/xds Related to Envoy support label Jan 24, 2022
@rboyer rboyer changed the title Fix xds delta reconnect wildcard xds: fix for delta xDS reconnect bug in LDS/CDS Jan 24, 2022
@vercel vercel bot temporarily deployed to Preview – consul-ui-staging January 24, 2022 17:19 Inactive
@vercel vercel bot temporarily deployed to Preview – consul January 24, 2022 17:19 Inactive
}
}

if handler, ok := handlers[req.TypeUrl]; ok {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to move this to after the version-sniff so we can use the detected version to change behavior.

@kisunji kisunji self-requested a review January 24, 2022 18:06
Copy link
Copy Markdown
Contributor

@kisunji kisunji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some Qs about clarity in comments

@kisunji kisunji requested a review from a team January 24, 2022 19:15
@vercel vercel bot temporarily deployed to Preview – consul January 24, 2022 20:56 Inactive
@vercel vercel bot temporarily deployed to Preview – consul-ui-staging January 24, 2022 20:56 Inactive
@vercel vercel bot temporarily deployed to Preview – consul January 24, 2022 20:57 Inactive
@vercel vercel bot temporarily deployed to Preview – consul-ui-staging January 24, 2022 20:57 Inactive
@rboyer rboyer requested review from eculver and kisunji January 24, 2022 20:57
@rboyer rboyer merged commit d2c0945 into main Jan 25, 2022
@rboyer rboyer deleted the fix-xds-delta-reconnect-wildcard branch January 25, 2022 17:24
@rboyer rboyer self-assigned this Jan 25, 2022
@hc-github-team-consul-core
Copy link
Copy Markdown
Collaborator

🍒 If backport labels were added before merging, cherry-picking will start automatically.

To retroactively trigger a backport after merging, add backport labels and re-run https://circleci.com/gh/hashicorp/consul/564788.

@hc-github-team-consul-core
Copy link
Copy Markdown
Collaborator

🍒✅ Cherry pick of commit d2c0945 onto release/1.11.x succeeded!

hc-github-team-consul-core pushed a commit that referenced this pull request Jan 25, 2022
When a wildcard xDS type (LDS/CDS/SRDS) reconnects from a delta xDS stream,
prior to envoy `1.19.0` it would populate the `ResourceNamesSubscribe` field
with the full list of currently subscribed items, instead of simply omitting it
to infer that it wanted everything (which is what wildcard mode means).

This upstream issue was filed in envoyproxy/envoy#16063 and fixed in
envoyproxy/envoy#16153 which went out in Envoy `1.19.0` and is fixed in later
versions (later refactored in envoyproxy/envoy#16855).

This PR conditionally forces LDS/CDS to be wildcard-only even when the
connected Envoy requests a non-wildcard subscription, but only does so on
versions prior to `1.19.0`, as we should not need to do this on later versions.

This fixes the failure case as described here: #11833 (comment)

Co-authored-by: Huan Wang <[email protected]>
@hc-github-team-consul-core
Copy link
Copy Markdown
Collaborator

🍒❌ Cherry pick of commit d2c0945 onto release/1.10.x failed! Build Log

rboyer added a commit that referenced this pull request Jan 25, 2022
When a wildcard xDS type (LDS/CDS/SRDS) reconnects from a delta xDS stream,
prior to envoy `1.19.0` it would populate the `ResourceNamesSubscribe` field
with the full list of currently subscribed items, instead of simply omitting it
to infer that it wanted everything (which is what wildcard mode means).

This upstream issue was filed in envoyproxy/envoy#16063 and fixed in
envoyproxy/envoy#16153 which went out in Envoy `1.19.0` and is fixed in later
versions (later refactored in envoyproxy/envoy#16855).

This PR conditionally forces LDS/CDS to be wildcard-only even when the
connected Envoy requests a non-wildcard subscription, but only does so on
versions prior to `1.19.0`, as we should not need to do this on later versions.

This fixes the failure case as described here: #11833 (comment)

Co-authored-by: Huan Wang <[email protected]>
rboyer added a commit that referenced this pull request Jan 25, 2022
When a wildcard xDS type (LDS/CDS/SRDS) reconnects from a delta xDS stream,
prior to envoy `1.19.0` it would populate the `ResourceNamesSubscribe` field
with the full list of currently subscribed items, instead of simply omitting it
to infer that it wanted everything (which is what wildcard mode means).

This upstream issue was filed in envoyproxy/envoy#16063 and fixed in
envoyproxy/envoy#16153 which went out in Envoy `1.19.0` and is fixed in later
versions (later refactored in envoyproxy/envoy#16855).

This PR conditionally forces LDS/CDS to be wildcard-only even when the
connected Envoy requests a non-wildcard subscription, but only does so on
versions prior to `1.19.0`, as we should not need to do this on later versions.

This fixes the failure case as described here: #11833 (comment)

Co-authored-by: Huan Wang <[email protected]>

Co-authored-by: Huan Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

theme/envoy/xds Related to Envoy support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants