Skip to content

[kubernetes] extended resources support#2242

Merged
saikonen merged 6 commits into
Netflix:masterfrom
valayDave:valay/extended-resources
Jun 17, 2026
Merged

[kubernetes] extended resources support#2242
saikonen merged 6 commits into
Netflix:masterfrom
valayDave:valay/extended-resources

Conversation

@valayDave

Copy link
Copy Markdown
Collaborator

@valayDave
valayDave requested review from saikonen and savingoyal and removed request for saikonen February 3, 2025 08:21
- support for kubernetes / argo
@valayDave
valayDave force-pushed the valay/extended-resources branch from f743d8c to 29fcbc1 Compare February 3, 2025 18:51
@valayDave valayDave changed the title [kubernetes] external resources support [kubernetes] extended resources support Jun 16, 2026
@greptile-apps

greptile-apps Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds support for Kubernetes extended resources to the @kubernetes decorator, allowing users to request arbitrary node-advertised resources (e.g. example.com/foo) alongside CPU/memory/GPU. The extended_resources dict is merged into both qos_requests and qos_limits — which is correct, since Kubernetes requires equal requests and limits for extended resources — and the change is consistently applied across the direct Kubernetes job path, the JobSet path, and the Argo Workflows path.

  • Adds extended_resources: Dict[str, str] parameter (default {}) to KubernetesDecorator, serialized as JSON for CLI forwarding and properly deserialized on the executor side.
  • The or {} null guard in kubernetes_job.py, kubernetes_jobsets.py, and argo_workflows.py safely handles None propagation from callers that omit the argument.
  • One latent foot-gun: if extended_resources contains a key that matches the GPU vendor resource string (e.g. nvidia.com/gpu), the GPU-specific limits block that merges last will silently create a requests/limits mismatch for that key, causing a Kubernetes API validation rejection. This is a user-error scenario but worth documenting at the merge site.

Confidence Score: 5/5

Safe to merge — the feature is correctly threaded through all execution paths (direct job, jobset, Argo) and the null-safety guard is in place everywhere it matters.

All changed files correctly propagate the new parameter end-to-end. The or {} guard prevents None-unpacking errors. The only observation is an undocumented interaction where specifying a GPU vendor key in extended_resources at the same time as the gpu parameter would produce a requests/limits mismatch and a Kubernetes API rejection — but this requires a deliberate (and unusual) user action and fails loudly rather than silently.

No files require special attention — the change is consistent across all three execution paths.

Important Files Changed

Filename Overview
metaflow/plugins/kubernetes/kubernetes_decorator.py Adds extended_resources parameter (default {}) to the decorator, serializes it as JSON for CLI forwarding, and documents it in the docstring.
metaflow/plugins/kubernetes/kubernetes_job.py Merges extended_resources into both qos_requests and qos_limits with a proper or {} null-safety guard before they are used in V1ResourceRequirements.
metaflow/plugins/kubernetes/kubernetes_jobsets.py Same extended_resources merge in JobSetSpec.dump() with proper null guard, mirroring the kubernetes_job.py change.
metaflow/plugins/argo/argo_workflows.py Applies extended_resources to both qos_requests and qos_limits before they are used in V1ResourceRequirements; passes the value through to the KubernetesArgoJobSet for the parallel-step path.
metaflow/plugins/kubernetes/kubernetes_cli.py Adds --extended-resources CLI option parsed as JSON and threads the value through to launch_job.
metaflow/plugins/kubernetes/kubernetes.py Adds extended_resources=None parameter to both execute (jobset path) and launch_job, forwarding the value to the underlying job/jobset builders.

Reviews (2): Last reviewed commit: "Apply suggestion from @valayDave" | Re-trigger Greptile

Comment thread metaflow/plugins/kubernetes/kubernetes_job.py Outdated
Comment thread metaflow/plugins/kubernetes/kubernetes_jobsets.py Outdated
Comment thread metaflow/plugins/kubernetes/kubernetes_decorator.py Outdated
@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 10.00000% with 9 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (master@9088ec4). Learn more about missing BASE report.

Files with missing lines Patch % Lines
metaflow/plugins/argo/argo_workflows.py 0.00% 3 Missing ⚠️
metaflow/plugins/kubernetes/kubernetes_job.py 0.00% 3 Missing ⚠️
metaflow/plugins/kubernetes/kubernetes_jobsets.py 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             master    #2242   +/-   ##
=========================================
  Coverage          ?   29.01%           
=========================================
  Files             ?      381           
  Lines             ?    52523           
  Branches          ?     9266           
=========================================
  Hits              ?    15242           
  Misses            ?    36246           
  Partials          ?     1035           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread metaflow/plugins/kubernetes/kubernetes_job.py Outdated
Comment thread metaflow/plugins/kubernetes/kubernetes_decorator.py
@valayDave
valayDave requested a review from saikonen June 17, 2026 19:34
Comment thread metaflow/plugins/argo/argo_workflows.py Outdated
@saikonen
saikonen merged commit 00b5cf1 into Netflix:master Jun 17, 2026
46 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants