Skip to content

Adding support for weighted k8s service#2293

Merged
srahul3 merged 13 commits intomainfrom
srahul3/support-for-weighted-k8s-service
Jun 19, 2023
Merged

Adding support for weighted k8s service#2293
srahul3 merged 13 commits intomainfrom
srahul3/support-for-weighted-k8s-service

Conversation

@srahul3
Copy link
Copy Markdown
Contributor

@srahul3 srahul3 commented Jun 7, 2023

Context

According to the documentation Consul supports setting weights for services . Unfortunately currently there is no way of doing that when using consul-k8s, as there is no annotation that can control that + there is no easy way of doing it via Consul API and it looks like the changes will be anyway overwritten.

Related link: https://hashicorp.atlassian.net/browse/NET-4266

Changes proposed in this PR:

  • While the k8s syncs to consul, the LB service annotation consul.hashicorp.com/service-weight: <number> is read and then the weight is divided among underlying ingress/ips. The calculated weight is persisted as passing weight in consulapi.CatalogRegistration

How I've tested this PR:
Added the Junit test for different scenarios

How I expect reviewers to test this PR:

Checklist:

  • Tests added
  • CHANGELOG entry added

    HashiCorp engineers only, community PRs should not add a changelog entry.
    Entries should use present tense (e.g. Add support for...)

@hashicorp-cla
Copy link
Copy Markdown

hashicorp-cla commented Jun 7, 2023

CLA assistant check
All committers have signed the CLA.

@srahul3 srahul3 self-assigned this Jun 7, 2023
Comment thread control-plane/catalog/to-consul/resource_test.go Outdated
Comment thread control-plane/catalog/to-consul/resource.go Outdated
Comment thread control-plane/catalog/to-consul/resource.go Outdated
@asheshvidyut
Copy link
Copy Markdown
Contributor

Documentation has passing and warning two values for service weights. Should we also use both?

@srahul3
Copy link
Copy Markdown
Contributor Author

srahul3 commented Jun 7, 2023

Documentation has passing and warning two values for service weights. Should we also use both?

Passing is the primary operation mode. According to last conversation with David, implementing passing is what is required for now.

Comment thread control-plane/catalog/to-consul/resource.go Outdated
Comment thread control-plane/catalog/to-consul/resource.go Outdated
Comment thread control-plane/catalog/to-consul/resource.go Outdated
Comment thread control-plane/catalog/to-consul/resource.go Outdated
return nil
}

var perAppWeight = weightI / appsCount
Copy link
Copy Markdown
Contributor

@asheshvidyut asheshvidyut Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can set weight to whatever value we are getting. We don't need to divide

Copy link
Copy Markdown
Contributor

@asheshvidyut asheshvidyut Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. One simple reason being - if a customer is setting weight - x for a service - and if they query, they will get another value from APIs which will be divided.
  2. Weight are proportional - if you have two services with weights 10 and 20 respectively, the service with weight 20 would receive twice as much traffic as the service with weight 10. If we divide, we will not get the correct traffic.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup! we should set the weight that is set on the annotation and we dont need the app count for the reasons @absolutelightning described!

}

// Sets the passing service weight
func setServiceWeight(weight string, appsCount int, r *consulapi.CatalogRegistration) error {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

id change this method to return the service weight or an error, providing the value of the annotation as an input. we should avoid sending in the consul registration into it. it will make this is a cleaner interface that is significantly easier to test.

@david-yu david-yu added the backport/1.2.x This release branch is no longer active. label Jun 7, 2023
@asheshvidyut
Copy link
Copy Markdown
Contributor

I think we need to add documentation of the new annotation in website/content/docs/k8s/annotations-and-labels.mdx


// Adding information about service weight.
// Overrides the existing weight if present
if weight, ok := svc.Annotations[annotationServiceWeight]; ok && weight != "" {
Copy link
Copy Markdown
Contributor

@asheshvidyut asheshvidyut Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these lines 567 - 577 and lines 518 - 528 are exactly the same. can we take it out in a function as well?

Copy link
Copy Markdown
Contributor Author

@srahul3 srahul3 Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code at both place is constructing the CatalogRegistration object, there is no logic which is duplicate. Read the comment #2293 (comment)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is ok to keep as is. it appears that the only behavior here is an assignment which IMO is not really duplication.

@srahul3 srahul3 requested a review from asheshvidyut June 8, 2023 15:42
@wilkermichael
Copy link
Copy Markdown
Contributor

A reminder that the 1.2.x branch is currently under a code freeze. If this isn't a bug fix, unless it's a critical feature for the 1.2.x-rc can you please refrain from merging this until after the 1.2.0-rc1 release (Tuesday, June 13th)

@david-yu david-yu added pr/no-backport signals that a PR will not contain a backport label and removed backport/1.2.x This release branch is no longer active. labels Jun 8, 2023
@srahul3 srahul3 merged commit 47d4063 into main Jun 19, 2023
@srahul3 srahul3 deleted the srahul3/support-for-weighted-k8s-service branch June 19, 2023 05:27
@srahul3
Copy link
Copy Markdown
Contributor Author

srahul3 commented Jun 19, 2023

@david-yu can you test it on latest. If thing are working as expected, I will backport the changes.

@david-yu
Copy link
Copy Markdown
Contributor

Adding backports to 1.0.x, 1.1.x and 1.2.x. Thanks @srahul3 could you follow the backport PRs to see they are back ported correctly and passing tests?

david-yu pushed a commit to hashicorp/consul that referenced this pull request Jul 6, 2023
@david-yu david-yu mentioned this pull request Jul 6, 2023