KEP: Enhance HPA Metrics Specificity #2055

DirectXMan12 · 2018-04-19T21:16:35Z

This KEP introduces additional tools into the autoscaling/v2 API for
working with metrics, including support for a metrics label selector,
and expanded support for the targetAverageValue field.

This PR also introduces a KEP folder for SIG Autoscaling.

DirectXMan12 · 2018-04-19T21:19:38Z

cc @kubernetes/sig-autoscaling-feature-requests @kubernetes/sig-instrumentation-feature-requests

MaciekPytel · 2018-04-19T21:40:41Z

If we're already changing the layout of the API maybe it would be worth it to move target to a separate unified struct (similar to what you do with moving fields describing metric to MetricSpec)? With that change your example would look like that:

- type: Object
  object:
    target:
      kind: StatefulSet 
      apiVersion: apps/v1
      name: foo
    targetValue: # not very happy with the name, I'd use target except the name would conflict
      type: Value
      value: 2
    metric:
      name: queue_length
      selector: "queue=some-jobs"

@thockin suggested that approach during his review of external metric API: kubernetes/kubernetes#60096 (review) (you need to unfold the comment). WDYT?

MaciekPytel · 2018-05-21T18:08:17Z

keps/sig-autoscaling/draft-20180419-enhance-hpa-metrics-specificity.md

+metrics adapters support the new changes -- this is up to those adapters
+maintainers.
+
+## Proposal


I think this misses a single part - what do we do if multiple time series are matched by selector? In external metrics API we sum all the values (we assume the user may not want to drill down on some dimension captured by metric, for example use all HTTP requests, regardless of response code). My assumption is we want to do the same? We should say it explicitly though.

If multiple series are matched, the adapter must deal with it, similarly to the current CM API (see my other comment).

MaciekPytel · 2018-05-21T18:11:12Z

keps/sig-autoscaling/draft-20180419-enhance-hpa-metrics-specificity.md

+API:
+
+```go
+type MetricValue struct {


In external metrics API we return a list of values, one for each time series. I think we need to do that in CM API as well or we need to say that it's adapters job to aggregate all time series into a single value.

Yeah, the intention was that it's the adapter's job to aggregate, if necessary. That's the approach we took before the specificity, and I missed that the external metrics API doesn't do that (I'd prefer if it did). I'll note that explicitly

DirectXMan12 · 2018-05-21T19:13:20Z

Factored in targetXXX unification, added note about what to do in case of multiple series.

@kubernetes/api-reviewers I'd also like to make the utilization serialize more sanely, but I want to make sure it's acceptible. Utilization would still be an int (e.g. 40) internally, but it would serialize as a string with a percent sign (e.g. 40%) to make it more obvious what's going on. Utilization continues to be one of the more confusing subjects for users, and I figure this will at least help a bit.

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

fejta-bot · 2018-08-28T17:15:54Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

@DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 67894, 64097). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. HPA metrics specificity improvements **What this PR does / why we need it**: Improves available specificity for HPA metrics by adding metric selector fields for metrics of Pods and Objects. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Implements this KEP: kubernetes/community#2055 **Special notes for your reviewer**: Need to add/update tests? **Release note**: ```release-note Introduces autoscaling/v2beta2 and custom_metrics/v1beta2, which implement metric selectors for Object and Pods metrics, as well as allowing AverageValue targets on Objects, similar to External metrics. ``` /assign @DirectXMan12 Kubernetes-commit: fdb5707194d56cbbd0da11c5be3a2a5653e714c9

fejta-bot · 2018-09-27T17:37:52Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

justaugustus · 2018-10-13T03:45:59Z

/kind kep
/remove-lifecycle rotten

justaugustus · 2018-11-20T04:42:07Z

REMINDER: KEPs are moving to k/enhancements on November 30. Please attempt to merge this KEP before then to signal consensus.
For more details on this change, review this thread.

Any questions regarding this move should be directed to that thread and not asked on GitHub.

This introduces a KEP folder for SIG autoscaling, similar to the ones for other SIGs.

DirectXMan12 · 2018-11-27T23:02:12Z

@jdumars @justaugustus this is ready to be merged (the implementation was done a bit ago), but it needs higher-level LGTM to be merged because it introduces the sig-autoscaling directory for the first time.

justaugustus · 2018-11-27T23:06:29Z

/test pull-community-verify

keps/sig-autoscaling/0032-enhance-hpa-metrics-specificity.md

This KEP introduces additional tools into the autoscaling/v2 API for working with metrics, including support for a metrics label selector, and expanded support for the `targetAverageValue` field.

justaugustus · 2018-11-27T23:36:08Z

KEP structure:
/lgtm

jdumars · 2018-11-28T01:16:53Z

/approve

k8s-ci-robot · 2018-11-28T01:16:58Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jdumars

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~keps/OWNERS~~ [jdumars]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…ect-metric-updates KEP: Enhance HPA Metrics Specificity

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 19, 2018

k8s-ci-robot requested review from bgrant0607 and calebamiles April 19, 2018 21:16

k8s-ci-robot added the sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. label Apr 19, 2018

k8s-ci-robot added sig/autoscaling Categorizes an issue or PR as relevant to SIG Autoscaling. kind/feature Categorizes issue or PR as related to a new feature. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. labels Apr 19, 2018

DirectXMan12 mentioned this pull request Apr 24, 2018

Arbitrary/Custom Metrics in the Horizontal Pod Autoscaler kubernetes/enhancements#117

Closed

7 tasks

DirectXMan12 force-pushed the proposal/hpa-object-metric-updates branch from 14cf566 to 7f01125 Compare April 24, 2018 17:14

DirectXMan12 mentioned this pull request Apr 24, 2018

Accept custom labels kubernetes-sigs/prometheus-adapter#60

Closed

DirectXMan12 force-pushed the proposal/hpa-object-metric-updates branch from 7f01125 to fe703c2 Compare May 8, 2018 14:51

damemi mentioned this pull request May 21, 2018

HPA metrics specificity improvements kubernetes/kubernetes#64097

Merged

MaciekPytel reviewed May 21, 2018

View reviewed changes

DirectXMan12 force-pushed the proposal/hpa-object-metric-updates branch from fe703c2 to 6889e19 Compare May 21, 2018 19:08

DirectXMan12 mentioned this pull request May 25, 2018

HPA Specificity Improvements kubernetes/website#8757

Merged

jdumars mentioned this pull request Jul 12, 2018

KEP #TBD: Enhance HPA Metrics Specificity kubernetes-retired/architecture-tracking#24

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 28, 2018

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Sep 27, 2018

k8s-ci-robot added kind/kep and removed lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. labels Oct 13, 2018

Introduce KEP folder for SIG Autoscaling

6b19697

This introduces a KEP folder for SIG autoscaling, similar to the ones for other SIGs.

DirectXMan12 force-pushed the proposal/hpa-object-metric-updates branch from 6889e19 to 008cfd4 Compare November 27, 2018 23:00

k8s-ci-robot added the sig/pm label Nov 27, 2018

justaugustus suggested changes Nov 27, 2018

View reviewed changes

keps/sig-autoscaling/0032-enhance-hpa-metrics-specificity.md Outdated Show resolved Hide resolved

KEP: Enhance HPA Metrics Specificity

9248f9e

This KEP introduces additional tools into the autoscaling/v2 API for working with metrics, including support for a metrics label selector, and expanded support for the `targetAverageValue` field.

DirectXMan12 force-pushed the proposal/hpa-object-metric-updates branch from 008cfd4 to 9248f9e Compare November 27, 2018 23:19

k8s-ci-robot assigned justaugustus Nov 27, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 27, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 28, 2018

k8s-ci-robot merged commit 8b556bf into kubernetes:master Nov 28, 2018

DirectXMan12 deleted the proposal/hpa-object-metric-updates branch November 28, 2018 18:11

justaugustus pushed a commit to justaugustus/community that referenced this pull request Dec 1, 2018

Merge pull request kubernetes#2055 from DirectXMan12/proposal/hpa-obj…

efae7bd

…ect-metric-updates KEP: Enhance HPA Metrics Specificity

johanneswuerbach mentioned this pull request Apr 22, 2020

custom metrics v1beta2 api with k8s-prometheus-adapter v0.7.0 prometheus-operator/kube-prometheus#508

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KEP: Enhance HPA Metrics Specificity #2055

KEP: Enhance HPA Metrics Specificity #2055

DirectXMan12 commented Apr 19, 2018

DirectXMan12 commented Apr 19, 2018

MaciekPytel commented Apr 19, 2018 •

edited

Loading

MaciekPytel May 21, 2018

DirectXMan12 May 21, 2018

MaciekPytel May 21, 2018

DirectXMan12 May 21, 2018

DirectXMan12 commented May 21, 2018

fejta-bot commented Aug 28, 2018

fejta-bot commented Sep 27, 2018

justaugustus commented Oct 13, 2018

justaugustus commented Nov 20, 2018

DirectXMan12 commented Nov 27, 2018

justaugustus commented Nov 27, 2018

justaugustus commented Nov 27, 2018

jdumars commented Nov 28, 2018

k8s-ci-robot commented Nov 28, 2018

KEP: Enhance HPA Metrics Specificity #2055

KEP: Enhance HPA Metrics Specificity #2055

Conversation

DirectXMan12 commented Apr 19, 2018

DirectXMan12 commented Apr 19, 2018

MaciekPytel commented Apr 19, 2018 • edited Loading

MaciekPytel May 21, 2018

Choose a reason for hiding this comment

DirectXMan12 May 21, 2018

Choose a reason for hiding this comment

MaciekPytel May 21, 2018

Choose a reason for hiding this comment

DirectXMan12 May 21, 2018

Choose a reason for hiding this comment

DirectXMan12 commented May 21, 2018

fejta-bot commented Aug 28, 2018

fejta-bot commented Sep 27, 2018

justaugustus commented Oct 13, 2018

justaugustus commented Nov 20, 2018

DirectXMan12 commented Nov 27, 2018

justaugustus commented Nov 27, 2018

justaugustus commented Nov 27, 2018

jdumars commented Nov 28, 2018

k8s-ci-robot commented Nov 28, 2018

MaciekPytel commented Apr 19, 2018 •

edited

Loading