feat: Add quota handling via NPE by Aryan-sharma11 · Pull Request #2593 · kubearmor/KubeArmor

Aryan-sharma11 · 2026-05-12T05:45:07Z

Purpose of PR?:

Fixes #2164

Does this PR introduce a breaking change?

If the changes in this PR are manually verified, list down the scenarios covered::

Additional information for reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Checklist:

Bug fix. Fixes #
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update
PR Title follows the convention of <type>(<scope>): <subject>
Commit has unit tests
Commit has integration tests

AryanBakliwal

Please update getting-started/network_security_policy_examples.md and getting-started/network_security_policy_specification.md also

AryanBakliwal · 2026-06-04T10:55:27Z

nsp test is failing in ci-test-ginkgo with some policy validation issue, can you please check

AryanBakliwal · 2026-06-04T13:56:08Z

@Aryan-sharma11 I tested these scenarios

1. Pod level quota ✅

Created a nginx deployment with 3 replicas

Tested pod level network quota with following policy

apiVersion: security.kubearmor.com/v1
kind: KubeArmorNetworkPolicy
metadata:
  name: nsp-quota-pod-nginx
spec:
  selector:
    matchLabels:
      app: nginx
  action: Block
  level: Pod
  ingress:
  - limit: "1KB"
    duration: "20m"

Received the expected alert for each pod, and network got blocked afterwards

$ ./karmor logs --gRPC :32767
Created a gRPC client (:32767)
Checked the liveness of the gRPC server
Started to watch alerts
== Alert / 2026-06-04 11:53:05.441994 ==
ClusterName: default
HostName: ubuntu
NamespaceName: default
PodName: nginx-deployment-59f86b59ff-mm5b5
Labels: app=nginx
Type: MatchedNetworkPolicy
PolicyName: nsp-quota-pod-nginx
Resource: INGRESS
Operation: NetworkFirewall
Action: Block
Data: SourceIP=8.8.8.8 SourcePort=0 DestinationIP=10.42.0.37 DestinationPort=0 Protocol=ICMP QuotaLevel=pod QuotaLimit=1KB
EventData: map[DestinationIP:10.42.0.37 DestinationPort:0 Protocol:ICMP QuotaLevel:pod QuotaLimit:1KB SourceIP:8.8.8.8 SourcePort:0]
Enforcer: NetworkPolicyEnforcer
Result: Permission denied
ExecEvent: map[]
PPID: 0
UID: 0
== Alert / 2026-06-04 11:53:54.210777 ==
ClusterName: default
HostName: ubuntu
NamespaceName: default
PodName: nginx-deployment-59f86b59ff-vc44n
Labels: app=nginx
Type: MatchedNetworkPolicy
PolicyName: nsp-quota-pod-nginx
Resource: INGRESS
Operation: NetworkFirewall
Action: Block
Data: SourceIP=8.8.8.8 SourcePort=0 DestinationIP=10.42.0.38 DestinationPort=0 Protocol=ICMP QuotaLevel=pod QuotaLimit=1KB
EventData: map[DestinationIP:10.42.0.38 DestinationPort:0 Protocol:ICMP QuotaLevel:pod QuotaLimit:1KB SourceIP:8.8.8.8 SourcePort:0]
Enforcer: NetworkPolicyEnforcer
Result: Permission denied
ExecEvent: map[]
PPID: 0
UID: 0
== Alert / 2026-06-04 11:54:26.721846 ==
ClusterName: default
HostName: ubuntu
NamespaceName: default
PodName: nginx-deployment-59f86b59ff-sk87x
Labels: app=nginx
Type: MatchedNetworkPolicy
PolicyName: nsp-quota-pod-nginx
Resource: INGRESS
Operation: NetworkFirewall
Action: Block
Data: SourceIP=8.8.8.8 SourcePort=0 DestinationIP=10.42.0.36 DestinationPort=0 Protocol=ICMP QuotaLevel=pod QuotaLimit=1KB
EventData: map[DestinationIP:10.42.0.36 DestinationPort:0 Protocol:ICMP QuotaLevel:pod QuotaLimit:1KB SourceIP:8.8.8.8 SourcePort:0]
Enforcer: NetworkPolicyEnforcer
Result: Permission denied
ExecEvent: map[]
PPID: 0
UID: 0

2. Policy level quota ✅

With the same nginx deployment havign 3 replicas

Tested pod level network quota with following policy

apiVersion: security.kubearmor.com/v1
kind: KubeArmorNetworkPolicy
metadata:
  name: nsp-quota-pod-nginx
spec:
  selector:
    matchLabels:
      app: nginx
  action: Block
  level: Policy
  ingress:
  - limit: "1KB"
    duration: "20m"

Got the alert, with network blocked for all 3 pods

$ ./karmor logs --gRPC :32767
Created a gRPC client (:32767)
Checked the liveness of the gRPC server
Started to watch alerts
== Alert / 2026-06-04 13:30:40.183343 ==
ClusterName: default
HostName: ubuntu
NamespaceName: default
PodName: nginx-deployment-59f86b59ff-sk87x
Labels: app=nginx
Type: MatchedNetworkPolicy
PolicyName: nsp-quota-pod-nginx
Resource: INGRESS
Operation: NetworkFirewall
Action: Block
Data: SourceIP=8.8.8.8 SourcePort=0 DestinationIP=10.42.0.36 DestinationPort=0 Protocol=ICMP QuotaLevel=policy QuotaLimit=1KB
EventData: map[DestinationIP:10.42.0.36 DestinationPort:0 Protocol:ICMP QuotaLevel:policy QuotaLimit:1KB SourceIP:8.8.8.8 SourcePort:0]
Enforcer: NetworkPolicyEnforcer
Result: Permission denied
ExecEvent: map[]
NodeID: ab58f2d4e6794592afd5e8d2312ad6e9
PPID: 0
UID: 0

3. Host quota (Audit) ✅

On Ubuntu 24.04 LTS machine,

With host policy

apiVersion: security.kubearmor.com/v1
kind: KubeArmorNetworkPolicy
metadata:
  name: nsp-quota-host
spec:
  nodeSelector:
    matchLabels:
      kubernetes.io/hostname: ubuntu
  action: Audit
  ingress:
  - limit: "5M"
    duration: "20m"

Got expected alert

$ ./karmor logs --gRPC :32767
Created a gRPC client (:32767)
Checked the liveness of the gRPC server
Started to watch alerts
== Alert / 2026-06-04 13:36:52.698707 ==
ClusterName: default
HostName: ubuntu
Type: MatchedNetworkPolicy
PolicyName: nsp-quota-host
Resource: INGRESS
Operation: NetworkFirewall
Action: Audit
Data: SourceIP=2600:9000:2656:4e00:9:4855:aac0:93a1 SourcePort=443 DestinationIP=2405:201:5c00:68da:46e5:9e2a:c8a0:53be DestinationPort=38856 Protocol=TCP QuotaLimit=5M
EventData: map[DestinationIP:2405:201:5c00:68da:46e5:9e2a:c8a0:53be DestinationPort:38856 Protocol:TCP QuotaLimit:5M SourceIP:2600:9000:2656:4e00:9:4855:aac0:93a1 SourcePort:443]
Enforcer: NetworkPolicyEnforcer
Result: Passed
ExecEvent: map[]
PPID: 0
UID: 0

4. Host quota (Block) ❌

On Ubuntu 24.04 LTS machine

With host policy

apiVersion: security.kubearmor.com/v1
kind: KubeArmorNetworkPolicy
metadata:
  name: nsp-quota-host
spec:
  nodeSelector:
    matchLabels:
      kubernetes.io/hostname: ubuntu
  action: Block
  ingress:
  - limit: "1M"
    duration: "20m"

Did not get the alert because all connections (including kubearmor-karmor, kubearmor-k8s, kubearmor-CRI) were broken. This needs to be discussed, and if this is a limitation then it needs to be documented well

5. Host quota + CIDR, Port policy ❌

On Ubuntu 24.04 LTS machine

Using following policy

apiVersion: security.kubearmor.com/v1
kind: KubeArmorNetworkPolicy
metadata:
  name: nsp-egress-block-dns
spec:
  nodeSelector:
    matchLabels:
      kubernetes.io/hostname: "ubuntu"
  egress:
  - limit: "1M"
    duration: "20m"
  - to:
    - ipBlock:
        cidr: "8.8.8.8/32"
    ports:
    - port: "dns"
      protocol: "UDP"
  severity: 5
  action: Audit

Expected 2 but got only 1 alert

$ ./karmor logs --gRPC :32767
Created a gRPC client (:32767)
Checked the liveness of the gRPC server
Started to watch alerts
== Alert / 2026-06-04 13:50:50.294545 ==
ClusterName: default
HostName: ubuntu
Type: MatchedNetworkPolicy
PolicyName: nsp-egress-block-dns
Severity: 5
Resource: EGRESS
Operation: NetworkFirewall
Action: Audit
Data: SourceIP=2405:201:5c00:68da:46e5:9e2a:c8a0:53be SourcePort=46983 DestinationIP=2405:201:5c00:68da::c0a8:1d01 DestinationPort=53 Protocol=UDP
EventData: map[DestinationIP:2405:201:5c00:68da::c0a8:1d01 DestinationPort:53 Protocol:UDP SourceIP:2405:201:5c00:68da:46e5:9e2a:c8a0:53be SourcePort:46983]
Enforcer: NetworkPolicyEnforcer
Result: Passed
ExecEvent: map[]
PPID: 0
UID: 0

AryanBakliwal

@Aryan-sharma11 I tested with this policy again

apiVersion: security.kubearmor.com/v1
kind: KubeArmorNetworkPolicy
metadata:
  name: nsp-egress-block-dns
spec:
  nodeSelector:
    matchLabels:
      kubernetes.io/hostname: "ubuntu"
  egress:
  - limit: "1M"
    duration: "20m"
  - to:
    - ipBlock:
        cidr: "8.8.8.8/32"
    ports:
    - port: "dns"
      protocol: "UDP"
  severity: 5
  action: Audit

and got the expected 2 alerts ✅

== Alert / 2026-07-01 04:21:02.676996 ==
ClusterName: default
HostName: kubearmor-dev-next
Type: MatchedNetworkPolicy
PolicyName: nsp-cidr-limit
Severity: 5
Resource: EGRESS
Operation: NetworkFirewall
Action: Audit
Data: SourceIP=10.0.2.15 SourcePort=50376 DestinationIP=18.154.206.69 DestinationPort=443 Protocol=TCP QuotaLimit=host
EventData: map[DestinationIP:18.154.206.69 DestinationPort:443 Protocol:TCP QuotaLimit:host SourceIP:10.0.2.15 SourcePort:50376]
Enforcer: NetworkPolicyEnforcer
Result: Passed
ExecEvent: map[]
PPID: 0
UID: 0
UserName: root
== Alert / 2026-07-01 04:22:49.416128 ==
ClusterName: default
HostName: kubearmor-dev-next
Type: MatchedNetworkPolicy
PolicyName: nsp-cidr-limit
Severity: 5
Resource: EGRESS
Operation: NetworkFirewall
Action: Audit
Data: SourceIP=10.0.2.15 SourcePort=40661 DestinationIP=8.8.8.8 DestinationPort=53 Protocol=UDP QuotaLimit=host
EventData: map[DestinationIP:8.8.8.8 DestinationPort:53 Protocol:UDP QuotaLimit:host SourceIP:10.0.2.15 SourcePort:40661]
Enforcer: NetworkPolicyEnforcer
Result: Passed
ExecEvent: map[]
PPID: 0
UID: 0
UserName: root

why we have QuotaLimit=host and not QuotaLimit=20M in quota alert ?
CIDR alert also has QuotaLimit=host
Same behavior with policy having only quota rules

AryanBakliwal

Correct alerts received ✅

== Alert / 2026-07-01 06:58:48.532683 ==
ClusterName: default
HostName: kubearmor-dev-next
Type: MatchedNetworkPolicy
PolicyName: nsp-cidr-limit
Severity: 5
Resource: EGRESS
Operation: NetworkFirewall
Action: Audit
Data: SourceIP=10.0.2.15 SourcePort=52695 DestinationIP=8.8.8.8 DestinationPort=53 Protocol=UDP
EventData: map[DestinationIP:8.8.8.8 DestinationPort:53 Protocol:UDP SourceIP:10.0.2.15 SourcePort:52695]
Enforcer: NetworkPolicyEnforcer
Result: Passed
ExecEvent: map[]
NodeID: bf5cc365450545fcbbbda42e4623a367
PPID: 0
UID: 0
UserName: root
== Alert / 2026-07-01 07:00:01.172655 ==
ClusterName: default
HostName: kubearmor-dev-next
Type: MatchedNetworkPolicy
PolicyName: nsp-cidr-limit
Severity: 5
Resource: EGRESS
Operation: NetworkFirewall
Action: Audit
Data: SourceIP=10.0.2.15 SourcePort=48204 DestinationIP=44.205.212.179 DestinationPort=443 Protocol=TCP QuotaLevel=host QuotaLimit=1M
EventData: map[DestinationIP:44.205.212.179 DestinationPort:443 Protocol:TCP QuotaLevel:host QuotaLimit:1M SourceIP:10.0.2.15 SourcePort:48204]
Enforcer: NetworkPolicyEnforcer
Result: Passed
ExecEvent: map[]
PPID: 0
UID: 0
UserName: root

Signed-off-by: Aryan-sharma11 <[email protected]>

Aryan-sharma11 force-pushed the network-quota branch 3 times, most recently from 6abc7f6 to 065d8c3 Compare May 21, 2026 03:01

Aryan-sharma11 force-pushed the network-quota branch 3 times, most recently from 7d42685 to 70b075c Compare May 25, 2026 05:48

Aryan-sharma11 marked this pull request as ready for review May 25, 2026 05:54

Aryan-sharma11 force-pushed the network-quota branch 2 times, most recently from b07a10c to cb52b93 Compare May 25, 2026 05:59

rksharma95 requested review from AryanBakliwal, achrefbensaad, charankamal20 and rksharma95 May 25, 2026 06:15

Aryan-sharma11 force-pushed the network-quota branch 2 times, most recently from ef64989 to cd61fd6 Compare May 31, 2026 14:05

Aryan-sharma11 mentioned this pull request Jun 2, 2026

feat: add network quota monitor #2186

Closed

AryanBakliwal reviewed Jun 2, 2026

View reviewed changes

Comment thread KubeArmor/networkPolicyEnforcer/networkPolicyEnforcer.go Outdated

Aryan-sharma11 force-pushed the network-quota branch 2 times, most recently from bbc7e19 to a80e11f Compare June 4, 2026 04:30

Aryan-sharma11 force-pushed the network-quota branch 2 times, most recently from 5247d23 to 1675b2d Compare June 9, 2026 09:25

charankamal20 reviewed Jun 16, 2026

View reviewed changes

Comment thread pkg/KubeArmorController/api/security.kubearmor.com/v1/zz_generated.deepcopy.go

charankamal20 previously approved these changes Jun 16, 2026

View reviewed changes

Aryan-sharma11 dismissed charankamal20’s stale review via 60e395f June 19, 2026 05:42

Aryan-sharma11 force-pushed the network-quota branch from 1675b2d to 60e395f Compare June 19, 2026 05:42

Aryan-sharma11 force-pushed the network-quota branch 3 times, most recently from 73446a5 to 440ee77 Compare June 28, 2026 08:47

Aryan-sharma11 force-pushed the network-quota branch 2 times, most recently from 89ddd2d to a7067d1 Compare June 28, 2026 17:13

Aryan-sharma11 requested review from AryanBakliwal and charankamal20 June 29, 2026 06:16

AryanBakliwal reviewed Jul 1, 2026

View reviewed changes

Comment thread KubeArmor/networkPolicyEnforcer/networkPolicyEnforcer.go Outdated

Aryan-sharma11 force-pushed the network-quota branch from a7067d1 to 8500ccb Compare July 1, 2026 05:49

Aryan-sharma11 requested a review from AryanBakliwal July 1, 2026 06:36

AryanBakliwal approved these changes Jul 1, 2026

View reviewed changes

Aryan-sharma11 force-pushed the network-quota branch from 8500ccb to 25218b9 Compare July 1, 2026 07:22

achrefbensaad approved these changes Jul 1, 2026

View reviewed changes

Aryan-sharma11 added 5 commits July 1, 2026 16:42

handle network bandwidth quota for pod and host

9577066

Signed-off-by: Aryan-sharma11 <[email protected]>

Add policy level ( POD | POLICY )

e418f9b

Signed-off-by: Aryan-sharma11 <[email protected]>

refactor + documentation for quota

4b14b6b

Signed-off-by: Aryan-sharma11 <[email protected]>

fix CI

46e1e3d

Signed-off-by: Aryan-sharma11 <[email protected]>

handle edge cases

1736c4b

Signed-off-by: Aryan-sharma11 <[email protected]>

Aryan-sharma11 force-pushed the network-quota branch from 25218b9 to 1736c4b Compare July 1, 2026 11:12

Aryan-sharma11 merged commit 9b136f6 into kubearmor:main Jul 1, 2026
42 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add quota handling via NPE#2593

feat: Add quota handling via NPE#2593
Aryan-sharma11 merged 5 commits into
kubearmor:mainfrom
Aryan-sharma11:network-quota

Aryan-sharma11 commented May 12, 2026 •

edited

Loading

Uh oh!

AryanBakliwal left a comment

Uh oh!

Uh oh!

AryanBakliwal commented Jun 4, 2026

Uh oh!

AryanBakliwal commented Jun 4, 2026

Uh oh!

Uh oh!

AryanBakliwal left a comment •

edited

Loading

Uh oh!

Uh oh!

AryanBakliwal left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

Aryan-sharma11 commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AryanBakliwal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AryanBakliwal commented Jun 4, 2026

Uh oh!

AryanBakliwal commented Jun 4, 2026

1. Pod level quota ✅

2. Policy level quota ✅

3. Host quota (Audit) ✅

4. Host quota (Block) ❌

5. Host quota + CIDR, Port policy ❌

Uh oh!

Uh oh!

AryanBakliwal left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AryanBakliwal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Aryan-sharma11 commented May 12, 2026 •

edited

Loading

AryanBakliwal left a comment •

edited

Loading