Skip to content

feat: support plural targetRefs on policies#3581

Merged
guydc merged 16 commits intoenvoyproxy:mainfrom
liorokman:policies-multiple-targets
Jun 18, 2024
Merged

feat: support plural targetRefs on policies#3581
guydc merged 16 commits intoenvoyproxy:mainfrom
liorokman:policies-multiple-targets

Conversation

@liorokman
Copy link
Copy Markdown
Contributor

What this PR does / why we need it:
This PR adds support for plural targetRefs on the various policies.

Which issue(s) this PR fixes:
Fixes #2999

@liorokman liorokman requested a review from a team as a code owner June 10, 2024 09:14
@liorokman liorokman marked this pull request as draft June 10, 2024 09:14
@codecov
Copy link
Copy Markdown

codecov bot commented Jun 10, 2024

Codecov Report

Attention: Patch coverage is 97.36264% with 12 lines in your changes missing coverage. Please review.

Project coverage is 68.28%. Comparing base (28e1a48) to head (425f430).
Report is 1 commits behind head on main.

Files Patch % Lines
internal/gatewayapi/clienttrafficpolicy.go 92.59% 8 Missing and 2 partials ⚠️
internal/gatewayapi/backendtlspolicy.go 86.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3581      +/-   ##
==========================================
+ Coverage   68.17%   68.28%   +0.11%     
==========================================
  Files         168      169       +1     
  Lines       20496    20557      +61     
==========================================
+ Hits        13973    14038      +65     
+ Misses       5513     5510       -3     
+ Partials     1010     1009       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Xunzhuo Xunzhuo changed the title feat: Support plural targetRefs on policies feat: support plural targetRefs on policies Jun 10, 2024
@liorokman liorokman force-pushed the policies-multiple-targets branch from 1fa49b4 to ea5738c Compare June 11, 2024 05:50
@liorokman
Copy link
Copy Markdown
Contributor Author

/retest

@liorokman liorokman force-pushed the policies-multiple-targets branch 2 times, most recently from eb8cccc to 6285ffa Compare June 11, 2024 12:20
@liorokman
Copy link
Copy Markdown
Contributor Author

/retest

@liorokman liorokman marked this pull request as ready for review June 11, 2024 12:52
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldnt this need more fields so it can also incorporate Kind and Group ?

      group: gateway.networking.k8s.io
      kind: HTTPRoute
      selector/matchLabels:
        app: foo

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it's possible to add this, I don't think it's needed.

The selector is checked only against routes and gateways anyways right now, and if a developer/admin wants to limit the matched policies to a specific kind it's possible to add the kind as a label.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a project, we've tried to use strongly typed fields as much as possible, my suggestion would be to provide a kind and group field here along with the labelSelector field

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a kind and group field as part of the target selector defeats the purpose here. If I add a kind and group field, then how would that work?

Let's say I want to attach to all of my routes (HTTPRoute, GRPCRoute, TLSRoute, and TCPRoute) to add some default setting. What do I specify in the kind field?

And, considering we enforce that the group field is always a single value using CEL, what should I as a user do with the group field? Fill it, and make sure it always matches the exactly one valid value? Leave it empty and assume that the default single value is being filled in?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reg attaching to all xRoutes, 4 policies could be created per xRoute or 1 policy could be attached to the Gateway

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that since this API still needs some discussion, I will remove the targetSelector for now from the PR and keep only the plural targetRefs in this PR.

The target selectors can be done in a separate PR after the API has been designed and agreed upon.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure raised #3607 to track this work

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you rebase ? the changes here are unintentional

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we change this in a separate PR? It causes a lot of file changes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change didn't cause the file changes. I explicitly chmod-ed the yaml files which were marked as executable so that they wouldn't be.

This change only makes sure that new files being added are created without the executable bit set.

@liorokman liorokman force-pushed the policies-multiple-targets branch from 6033d8e to e8d3afb Compare June 14, 2024 09:48
@liorokman
Copy link
Copy Markdown
Contributor Author

Not sure why the docs-lint build step is not successful. It's got something to do with some other change?

[404] https://gateway.envoyproxy.io/latest/tasks/traffic/http-routing/
[404] https://gateway.envoyproxy.io/latest/tasks/security/secure-gateways/
[404] https://gateway.envoyproxy.io/latest/tasks/traffic/grpc-routing/
[404] https://gateway.envoyproxy.io/latest/tasks/traffic/http-traffic-splitting/
[404] https://gateway.envoyproxy.io/latest/tasks/traffic/global-rate-limit/

@liorokman liorokman force-pushed the policies-multiple-targets branch from e8d3afb to d51f777 Compare June 14, 2024 14:49
@liorokman
Copy link
Copy Markdown
Contributor Author

/retest

1 similar comment
@liorokman
Copy link
Copy Markdown
Contributor Author

/retest

liorokman added 13 commits June 14, 2024 20:58
Signed-off-by: Lior Okman <[email protected]>
Signed-off-by: Lior Okman <[email protected]>
Signed-off-by: Lior Okman <[email protected]>
BackendTrafficPolicy, EnvoyExtensionPolicy, and SecurityPolicy

Signed-off-by: Lior Okman <[email protected]>
Signed-off-by: Lior Okman <[email protected]>
agreement on how this API should look like.

Signed-off-by: Lior Okman <[email protected]>
// Map of Gateway to the routes attached to it
gatewayRouteMap := make(map[string]sets.Set[string])

handledPolicies := make(map[types.NamespacedName]*egv1a1.BackendTrafficPolicy)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious why we need this map ? if its being used to handle duplicates, any idea how we ended up with duplicates ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example:

There's a gateway with 3 different routes. You create a single policy resource that targets two of those three routes with two different targetRef elements.

This map is a way to prevent the policy resource from being added twice to the res array.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-reading, its because we have multiple targetRefs, can we execute res = append(res, policy) right after

for _, currPolicy := range backendTrafficPolicies {
    res = append(res, curPolicy)
    ....

and avoid extra memory

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not that simple. The code has two loops, the first handling the policies that target routes and the second handling the targets that target gateways. Once we have multiple targetRefs, it's possible for the same policy to target both a route and a different gateway.

The second loop needs to work on the same deep copy of the policy as the first loop, otherwise the status section of the policy is not correctly updated. If we do as you suggest in the first loop, then the second loop would need to go over all of the content of res and find the correct policy instance if its there. The easy way to do this is like the code does right now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah thanks for explaining
do we want to support a policy targeting a Gateway and HTTPRoute at the same time ? does our library gracefully handle this case when it comes to defaults and overrides ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's no different than two different policies targeting both the gateway and the route. The plural targetRefs is just a form of syntactic sugar.

ancestorRefs := []gwapiv1a2.ParentReference{
parent,
}
ancestorRefs := fixCurrentAncestorRefs(policy, parent)
Copy link
Copy Markdown
Contributor

@arkodg arkodg Jun 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: prefer

ancestorRefs := getAncestorRefs(policy)
ancestorRefs = append(ancestorRefs, parent)

simplifies reading the code w/o having to read the details in getAncestorRefs

Signed-off-by: Lior Okman <[email protected]>
@liorokman
Copy link
Copy Markdown
Contributor Author

/retest

Copy link
Copy Markdown
Contributor

@arkodg arkodg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks !

@arkodg arkodg requested review from a team June 17, 2024 23:46
@guydc guydc merged commit bdff5d5 into envoyproxy:main Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support attachment of policies to multiple Gateways

4 participants