KEP-4962: Standardizing the Representation of Cluster Network Topology by dmitsh · Pull Request #4965 · kubernetes/enhancements

dmitsh · 2024-11-15T17:29:49Z

One-line PR description:
Standardizing Cluster Network Topology Representation

Issue link: Standardizing the Representation of Cluster Network Topology #4962

Other comments:

k8s-ci-robot · 2024-11-15T17:29:57Z

Welcome @dmitsh!

It looks like this is your first PR to kubernetes/enhancements 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/enhancements has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2024-11-15T17:29:58Z

Hi @dmitsh. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

dmitsh · 2024-11-15T17:31:27Z

/cc @aojea

dmitsh · 2024-11-15T20:51:43Z

/cc @brianhammons @mwielgus @tardieu @mickeyboxell @arsenetar

k8s-ci-robot · 2024-11-15T20:51:46Z

@dmitsh: GitHub didn't allow me to request PR reviews from the following users: tardieu, arsenetar, brianhammons.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

/cc @brianhammons @mwielgus @tardieu @mickeyboxell @arsenetar

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

wojtek-t · 2024-11-18T14:00:43Z

+
+### Network QoS Annotation
+Format: `network.qos.kubernetes.io/switches: <QoS>`
+- `<QoS>`: A JSON object where each key is a switch name (matching the network topology label) with a value containing:


So this object contains N items (of below structure), where N is the number of predefined topology units (accelerator, block, datacenter, zone), right?

What I want to ensure is that those needs to be changed/updated when the cluster grows/shrinks (i.e. that the don't define distance between nodes themselves). But given that for a given node its contents don't depend on other nodes (rather on placements in the physical network), this seems to be fine.

@wojtek-t , that's correct, each node will contain QoS metrics between the node and every reachable switch.

We simplified this proposal - removed custom labels and removed annotations.

aojea · 2024-11-18T15:50:12Z

/assign @johnbelamaric

for sig-architecture

aojea · 2024-11-18T16:04:47Z

+- `<switch-name>`: Unique identifier for the switch
+
+### Network QoS Annotation
+Format: `network.qos.kubernetes.io/switches: <QoS>`


This is not really the switch, right, is the interface in the node that connects to the switch ... and we already have properties to define attributes on the network interfaces with DRA, see slice 14 https://docs.google.com/presentation/d/1Vdr7BhbYXeWjwmLjGmqnUkvJr_eOUdU0x-JxfXWxUT8/edit#slide=id.g2f750386db2_5_0 so I don't feel we need this additional annotation here

cc: @johnbelamaric @thockin

It is not a NIC on the node. These are QoS metrics from the node to every reachable switch. Also, the "switch" in this context could be a physical network device, or an aggregated entity defined by a CSP. For example, AWS returns 3 levels of switches per node, but the actual number of physical switches is unknown.

These are QoS metrics from the node to every reachable switch

how is the Node connected to the first switch :) ?

aojea · 2024-11-18T16:05:40Z

+network.qos.kubernetes.io/switches: {
+   "nvl10": {
+      "latency": "2us",
+      "bandwidth": "100Gbps"
+   },
+   "sw11": {
+      "latency": "50us",
+      "bandwidth": "40Gbps"
+   },
+   "sw21": {
+      "latency": "500us",
+      "bandwidth": "20Gbps"
+   },
+  "sw31": {
+      "latency": "1ms",
+      "bandwidth": "10Gbps"
+   }


These are network interfaces on the node, I think we should better model them with DRA https://github.com/kubernetes/enhancements/pull/4965/files#r1846865095 , that also allows us to provide dynamic capabilities to the interfaces

Again, as mentioned in the earlier comment, these QoS numbers represent node-to-reachable-switch metrics. They are not per NIC.

does it mean that some switches may not be connected directly?
how then the latency and bandwidth are obtained to guarantee those values?

aojea · 2024-11-18T16:14:52Z

+Format: `network.topology.kubernetes.io/<nw-switch-type>: <switch-name>`
+- `<nw-switch-type>`: Logical type of the network switch (can be one of the reserved names or a custom name)
+  - Reserved names: `accelerator`, `block`, `datacenter`, `zone`


this is the part we need to loop in SIG architecture, I briefly touched on this with @thockin and it seems it took some time to settle on the region/zone.

So my understanding is that we need to model a hierarchy, this KEP suggest to use nested structures: zone > datacenter > block > accelerator , but we should at least describe in alternative why weights are not better than this, it seems to me that weights are more easy to standardize different topologies, as you only focus on the distance provided between layers, and can support different architectures with multiple layers

This KEP is proposing to use reserved network types for typical network architectures, while allowing to extend network topology using custom network types.
We are providing means for weighted approach by specifying distance and/or bandwidth, latency or other metrics.
These are actual measurable physical characteristics of the network, and will be more accurate than specifying static weights.
Once again, we are providing QoS between a node and a switch, so the distance is the number of hops between the node and the switch. Same goes for bandwidth/latency.

aojea · 2024-11-18T16:17:20Z

+This proposal is designed with extensibility in mind, enabling the use of custom network types. This ensures that the standard can adapt to future advancements in cluster networking without requiring significant overhauls.
+
+For custom network types, Network QoS Annotations are required, with distance being the minimum mandatory metric. Specifying latency and bandwidth is optional, but including them can offer a more detailed view of link performance, enabling more efficient scheduling decisions.


IIUIC the use of custom network types means that environment that use them may not be compatible with other environments, that practically removes all the benefits of standarization, another thing where I think weighted/distances model can help better

this was already addressed.

aojea · 2024-11-18T16:20:05Z

+
+The same network topology depicted in Example 2 can be represented using custom network types.
+
+Let's use `tor` for top-of-rack switches, `area` for the second level of switches, and `center` for the third level.


After seen this example I'm definitevely not in favor of custom types as it is impossible for a generic tool to infer the distance between these custom types ... it will also create fragmentation and incompatibility as multiple tools can define the same name with different meaning ... the example also talks about levels, that reinforces my idea of weights, so something like network.topology.kubernetes.io/tier: 1

In this example network.topology.kubernetes.io/tor: sw13 will be

Node: network.topology.kubernetes.io/tier: 1

ResourceSlice:

apiVersion: resource.k8s.io/v1alpha3 kind: ResourceSlice … spec: devices: - basic: attributes: tier: int: 1 type: string: nvlink ip: string: 10.1.1.1/24 latency: string: "50us" bandwith: string: "100gbps"

When using custom network types, it is mandatory to define distance in the QoS annotation.
We explicitly expressed that in the KEP.

keithmattix · 2024-11-18T20:57:16Z

+  - Gang-scheduling auto-scaler
+  - DRA scheduler plugin
+
+### Goals


Are there existing topology formats or consumers that we want Kubernetes to integrate with? If so, are these integrations goals or non-goals?

To the best of my knowledge, only EKS exposes their custom node labels for network layers.
The goal of this KEP is to create a standard way of describing switch network topology.
Ultimately, we want to use this standard in development of Kubernetes-native network-aware scheduler plugin for multi-node workloads. We put this task as a "no goal", as it would be an independent effort.

Ultimately, we want to use this standard in development of Kubernetes-native network-aware scheduler plugin for multi-node workloads.

I'm (still) not seeing why that development work requires a KEP. An out-of-tree, out-of-project custom scheduler could use labels with keys outside of Kubernetes namespaces.

Kubernetes-native network-aware scheduler plugin for multi-node workloads

Perhaps a reference implementation to show how the end to end works or even use it as part of the integration test.

jackfrancis · 2024-12-11T20:13:29Z

+
+## Summary
+
+This document proposes a standard for declaring network topology in Kubernetes clusters,


Is "proposes a standard set of terminologies for declaring network topology"... more clear?

thanks, updated!

jackfrancis · 2024-12-11T20:18:40Z

+scheduling pods in close network proximity is becoming essential.
+Examples of such workloads include AI/ML training jobs or sets of interdependent, data-intensive services.
+
+However, Kubernetes currently lacks a standard method to describe cluster network topology, which is a key area for improvement.


I'm not sure this KEP is proposing a standard method, but rather it is proposing a set of common terms. A method would be "label your nodes using label XYZ" or "implement a custom operator". We don't seem to be advocating for one method or the other, but rather we want to define a standard terminology so that any chosen method can render the underlying network topology using a language that we all understand. We anticipate that different environments may have good reason to prefer one method over an other (device-plugin + node labels vs DRA vs custom scheduler vs custom operator).

Do other folks agree? I think "Kubernetes currently lacks a standard set of terms to describe network topology" would be a clearer expression of whatever we're doing, if so.

changed the wording, thank you!

Signed-off-by: Dmitry Shmulevich <[email protected]>

k8s-ci-robot · 2024-12-11T22:42:24Z

@dmitsh: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-enhancements-test	`d5407f7`	link	true	`/test pull-enhancements-test`
pull-enhancements-verify	`d5407f7`	link	true	`/test pull-enhancements-verify`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

danwinship · 2024-12-12T17:57:10Z

+
+### Goals
+
+- Introduce a standard way of representing network topology in Kubernetes clusters


I know this section was bigger before, but now it's too small. There has to be some goal of using that topology representation for something. Otherwise why would we want it?

danwinship · 2024-12-12T17:59:30Z

+We propose to use the following four network hierarchy layer types:
+1. `accelerator`: Network interconnect for direct accelerator communication (e.g., Multi-node NVLink interconnect between NVIDIA GPUs)
+2. `block`: Rack-level switches connecting hosts in one or more racks as a block.
+3. `spine`: Spine-level switches connecting multiple blocks inside a datacenter.
+4. `datacenter`: Zonal switches connecting multiple datacenters inside an availability zone.


This seems like a weird breakdown to me. I feel like every time people have talked about adding more topology labels beyond "region" and "zone", someone brings up "rack", but you just skip right over that layer...

In VM-based environments, you would want a layer for "hypervisor", but again, that doesn't correspond to anything here.

And you talk about people wanting to use these generically, but what if a given network is missing one of these layers? Eg, if my datacenter has no "block"s, but someone deploys a job that tries to use affinity by block, then what happens?

For that matter, if my application wants 50ms of latency between pods, then what level should I use for the pod affinity? Presumably that would be spine-level latency in some datacenters, block-level latency in others, and unachievable in yet others. With the existing "zone" and "region" levels, the assumption is that there's an order of magnitude of difference between the levels, so you can think about them in an agnostic way, but that doesn't really seem to work for these lower topology levels.

So, it seems to me like either:

The labels have cluster-specific-meanings (in which case they don't need to be standardized), or

The scheduling should be based not on topology per se, but on the actual latency/distance metrics that are computed from the topology (in which case the latency/distance information needs to be provided in a standardized way but the topology itself does not).

danwinship · 2024-12-12T18:00:26Z

+3. `spine`: Spine-level switches connecting multiple blocks inside a datacenter.
+4. `datacenter`: Zonal switches connecting multiple datacenters inside an availability zone.
+
+These types will accommodate the majority of common network hierarchies across different CSP and on-prem environments.


Anyone writing this KEP two years ago would not have included "accelerator" as a level... so I worry about how future-proof this approach is.

As technology advances, things change ...

danwinship · 2024-12-12T18:01:39Z

+
+As a data scientist running a data-intensive large-scale AI training job, I want to optimize the runtime
+by binding pods to nodes that are in close network proximity.
+This ensures better performance for my distributed workloads.


as others have noted, DRA is already addressing this use case

I'm. not sure DRA can be used to represent topology of the entire cluster

aojea · 2024-12-19T10:26:50Z

Discussed during SIG Architecture Meeting on 12 Dec 2024 https://docs.google.com/document/d/1BlmHq5uPyBUDlppYqAAzslVbAO8hilgjqZUTaNXUhKM/edit?tab=t.0

The overall agreement seems we're going to hold off on implementing this, mainly because the ecosystem is still a bit wild west and it's not clear what the "standard" will look like in the future, new patterns or technologies may succeed or become outdated quickly, and that will cause problems in the future because enforcing a specific way to represent network topology could limit that flexibility and might even break compatibility with other projects.

/hold

sftim · 2024-12-19T11:22:26Z

@dmitsh if you want help building an out-of-project but vendor-neutral way to capture this - and maybe to provide hints to the scheduler - then I think we can find some help for you.

The underlying ambition is important, even if right now this solution doesn't feel right.

kannon92 · 2025-03-09T16:17:25Z

@dmitsh if you want help building an out-of-project but vendor-neutral way to capture this - and maybe to provide hints to the scheduler - then I think we can find some help for you.

The underlying ambition is important, even if right now this solution doesn't feel right.

I believe that https://github.com/NVIDIA/topograph was a big motivating factor for this KEP and from what I could tell this does work across multiple vendors. Its not device agnostic but not sure if that is what we mean by "vendor-neutral".

sftim · 2025-03-09T16:25:41Z

I believe that https://github.com/NVIDIA/topograph was a big motivating factor for this KEP and from what I could tell this does work across multiple vendors. Its not device agnostic but not sure if that is what we mean by "vendor-neutral".

The logo for topograph: doesn't feel as vendor neutral as it could be. However the solution works with your Kubernetes or SLURM cluster no matter who sold you the equipment or rented you the node instances.

PS. We shouldn't recommend https://github.com/NVIDIA/topograph (my opinion) because it breaks some conventions and recommendations of the Kubernetes project. I'm fairly sure NVIDIA didn't mean harm, but using labels such as network.topology.kubernetes.io/spine implies Kubernetes endorsement that doesn't exist. I think they should have used something like networktopology.k8s.nvidia.com/spine or topograph.some-domain-they-buy.example/spine.

dmitsh · 2025-03-09T18:39:34Z

@dmitsh if you want help building an out-of-project but vendor-neutral way to capture this - and maybe to provide hints to the scheduler - then I think we can find some help for you.
The underlying ambition is important, even if right now this solution doesn't feel right.

I believe that https://github.com/NVIDIA/topograph was a big motivating factor for this KEP and from what I could tell this does work across multiple vendors. Its not device agnostic but not sure if that is what we mean by "vendor-neutral".

Major CSPs have already started applying network-topology related labels to the nodes:

AWS
topology.k8s.aws/network-node-layer-1=nn-abcd001
topology.k8s.aws/network-node-layer-2=nn-abcd002
topology.k8s.aws/network-node-layer-3=nn-abcd003

OCI
oci.oraclecloud.com/rdma.local_block_id=abcd001
oci.oraclecloud.com/rdma.network_block_id=abcd002
oci.oraclecloud.com/rdma.hpc_island_id=abcd003

GCP
cloud.google.com/gce-topology-subblock=abcd001
cloud.google.com/gce-topology-block=abcd002

This KEP proposes to standardize these labels. These labels can already be used today in pod affinity to improve topology-aware scheduling. As their utility becomes more apparent, other applications are likely to adopt them as well.

dmitsh · 2025-03-09T18:55:38Z

I believe that https://github.com/NVIDIA/topograph was a big motivating factor for this KEP and from what I could tell this does work across multiple vendors. Its not device agnostic but not sure if that is what we mean by "vendor-neutral".

The logo for topograph doesn't feel as vendor neutral as it could be. However the solution works with your Kubernetes or SLURM cluster no matter who sold you the equipment or rented you the node instances.

PS. We shouldn't recommend https://github.com/NVIDIA/topograph (my opinion) because it breaks some conventions and recommendations of the Kubernetes project. I'm fairly sure NVIDIA didn't mean harm, but using labels such as network.topology.kubernetes.io/spine implies Kubernetes endorsement that doesn't exist. I think they should have used something like networktopology.k8s.nvidia.com/spine or topograph.some-domain-they-buy.example/spine.

This KEP is focused on standardizing labels for topology-aware scheduling and does not involve Topograph. Your recommendations regarding the use of Topograph are outside the scope of this proposal.

I suggest focusing on the spirit of the proposal rather than the letter. If you have concerns about the label names, we can certainly explore alternative options.

If you suggest that the Topograph icon resembles a specific vendor, please provide the vendor's name to clarify the issue and avoid baseless accusations.

sftim · 2025-03-09T19:47:47Z

@dmitsh let's move the Topograph branding discussion to Slack; this discussion is about whether we can merge a KEP into our list of enhancements that have a written proposal.

sftim · 2025-03-09T19:51:01Z

My key points are:

I don't recommend making one particular way of node labelling official, even if it feels right in 2025. Mainly because I think it will date quickly and that making it official will unhelpfully calcify what should be a flexible mechanism.
I don't recommend endorsing the Topograph project as a vendor neutral way to achieve the same outcome.

Overall, I hope that we find a different approach (either as part of Kubernetes or as a separate project) that gets the vendor neutral outcome we want, without [my opinion] putting long term maintainability at risk.

k8s-triage-robot · 2025-06-07T20:51:06Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2025-07-07T21:15:44Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot · 2025-08-06T21:16:53Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2025-08-06T21:16:59Z

@k8s-triage-robot: Closed this PR.

Details

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

elmiko · 2025-09-24T16:57:17Z

this topic has come up during the most recent sig cloud provider meeting (24 september 2025). this seems like a topic that we should continue to work on.

we would like to promote the cloud controller manager maintainers to add this information where appropriate. it seems like there is a large challenge is gaining consensus about a standard way of handling these labels. it may be that for now, the best advice is that CCM teams should implement these as they see fit.

aojea · 2025-09-25T12:41:51Z

@elmiko I think a good first step should be to create a kind of "registry" where cloud providers can share this information with the community, the annecdotal evidence seems to indicate that as today only two cloud providers publish this information kubernetes-sigs/ai-conformance#5 (comment)

johnbelamaric · 2025-09-25T22:02:19Z

Rather than standardizing a set of labels, I am proposing a way to use logical multi-node DRA devices to model topology. This doc may be of interest: https://docs.google.com/document/d/1Fg9ughIRMtt1HmDqiGWV-w9OKdrcKf_PsH4TjuP8Y40/edit?tab=t.0#heading=h.fd6kgam8o1zo

elmiko · 2025-09-29T20:11:04Z

very cool! thank you both @aojea and @johnbelamaric , i've got a little more reading to do but this is interesting to me. i'll take some time to review the thread and the doc and see what questions i've got then ;)

thanks again 🙏

resker · 2026-05-11T17:04:48Z

@johnbelamaric - the Google Doc you reference seems to be access controlled... any chance you could open it up (if still applicable)? Or is this just a matter of joining the kubernetes-dev group?

johnbelamaric · 2026-05-11T18:47:53Z

@johnbelamaric - the Google Doc you reference seems to be access controlled... any chance you could open it up (if still applicable)? Or is this just a matter of joining the kubernetes-dev group?

I shared it with you. It is open to [email protected] as well as some sig/wg mailing lists.

The proposal there adds a layer of indirection over top of labels. It sill can't quite capture sub-node topology like that required in #6006 - but that layer of indirection provides a way to introduce that kind of thing.

That doc has a bunch of different ideas/options. Recently I am trying to refine them a bit and even prototype some. Any feedback is useful.

KEP-4962: Standardizing Cluster Network Topology Representation

c9f2407

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory labels Nov 15, 2024

k8s-ci-robot requested review from MikeZappa87 and thockin November 15, 2024 17:29

k8s-ci-robot added the sig/network Categorizes an issue or PR as relevant to SIG Network. label Nov 15, 2024

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 15, 2024

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 15, 2024

dmitsh mentioned this pull request Nov 15, 2024

Standardizing the Representation of Cluster Network Topology #4962

Closed

4 tasks

k8s-ci-robot requested a review from aojea November 15, 2024 17:31

update example for reserved network types

913c832

dmitsh force-pushed the ds-topology branch from ba21b04 to 913c832 Compare November 15, 2024 20:43

k8s-ci-robot requested a review from mickeyboxell November 15, 2024 20:51

k8s-ci-robot requested a review from mwielgus November 15, 2024 20:51

added 'Alternatives' section

fa36461

wojtek-t reviewed Nov 18, 2024

View reviewed changes

k8s-ci-robot assigned johnbelamaric Nov 18, 2024

thockin self-assigned this Nov 18, 2024

aojea reviewed Nov 18, 2024

View reviewed changes

keithmattix reviewed Nov 18, 2024

View reviewed changes

jackfrancis reviewed Dec 11, 2024

View reviewed changes

addressed comments

d5407f7

Signed-off-by: Dmitry Shmulevich <[email protected]>

danwinship reviewed Dec 12, 2024

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 19, 2024

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 7, 2025

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jul 7, 2025

k8s-ci-robot closed this Aug 6, 2025

kannon92 mentioned this pull request Sep 17, 2025

[Accelerators] Hardware Topology Awareness kubernetes-sigs/ai-conformance#5

Open

2 tasks

resker mentioned this pull request Apr 19, 2026

docs: post-#254 follow-ups — KEP-4962 note + Doc-Impact table rows NVIDIA/topograph#278

Merged

4 tasks

		This proposal is designed with extensibility in mind, enabling the use of custom network types. This ensures that the standard can adapt to future advancements in cluster networking without requiring significant overhauls.

		For custom network types, Network QoS Annotations are required, with distance being the minimum mandatory metric. Specifying latency and bandwidth is optional, but including them can offer a more detailed view of link performance, enabling more efficient scheduling decisions.


		The same network topology depicted in Example 2 can be represented using custom network types.

		Let's use `tor` for top-of-rack switches, `area` for the second level of switches, and `center` for the third level.


		## Summary

		This document proposes a standard for declaring network topology in Kubernetes clusters,


		### Goals

		- Introduce a standard way of representing network topology in Kubernetes clusters

Uh oh!

Conversation

dmitsh commented Nov 15, 2024

Uh oh!

k8s-ci-robot commented Nov 15, 2024

Uh oh!

k8s-ci-robot commented Nov 15, 2024

Uh oh!

dmitsh commented Nov 15, 2024

Uh oh!

dmitsh commented Nov 15, 2024

Uh oh!

k8s-ci-robot commented Nov 15, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmitsh Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aojea commented Nov 18, 2024

Uh oh!

aojea Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aojea Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aojea Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Dec 11, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmitsh Nov 18, 2024 •

edited

Loading

aojea Nov 18, 2024 •

edited

Loading

aojea Nov 19, 2024 •

edited

Loading

aojea Nov 18, 2024 •

edited

Loading

sftim commented Mar 9, 2025 •

edited

Loading

resker commented May 11, 2026 •

edited

Loading