feat(agentgateway): scope inference-gateway LB to allowed source ranges by yuanchen8911 · Pull Request #1138 · NVIDIA/aicr

yuanchen8911 · 2026-06-01T22:11:54Z

Summary

Adds an agentgateway.allowedSourceRanges option that scopes the inference-gateway LoadBalancer to operator-supplied source CIDRs (rendered into the generated Service's spec.loadBalancerSourceRanges). One portable setting locks the gateway down on AWS, GCP, Azure, and OCI.

Motivation / Context

The agentgateway controller materializes the inference-gateway Gateway into a type: LoadBalancer Service, which every cloud provisions as an internet-facing load balancer open to 0.0.0.0/0 — exposing the plaintext, unauthenticated inference endpoint to the public internet. A security scan flagged exactly this on an AICR-provisioned cluster. There was previously no first-class way to restrict that exposure from a recipe.

Fixes: N/A
Related: AICR inference-gateway public-exposure finding (internal security review)

Type of Change

New feature (non-breaking change that adds functionality)
Documentation update

Component(s) Affected

Recipe engine / data (recipes/components/agentgateway)
Docs/examples (docs/)
Other: tests/chainsaw (bundle-template test)

Implementation Notes

The CIDRs flow through the existing AgentgatewayParameters CR (spec.service.spec.loadBalancerSourceRanges), which the controller merges onto the generated Service — so no new mechanism is introduced.
Default is empty, which is non-breaking (no source-range restriction is rendered; behavior is unchanged). It is intentionally not defaulted to an NVIDIA-specific CIDR: a baked-in range would firewall every downstream deployment to one network and lock other operators out of their own gateway. Operators scope it to their own trusted networks.
Because list values cannot be passed via the scalar-only --set, the knob is set through a recipe overlay / componentRef override (documented in the component catalog).

Alternative considered: ClusterIP-by-default

A stronger, secure-by-default alternative was prototyped: default the gateway Service to ClusterIP (no public load balancer at all — reach it in-cluster or via kubectl port-forward), with LoadBalancer + source ranges / internal-scheme annotations as an explicit opt-in. It removes the public surface entirely rather than narrowing it, and verified end-to-end (default ClusterIP, opt-in LoadBalancer+ranges, internal-LB-via-annotations all render correctly).

This PR takes the less disruptive path — keep the existing LoadBalancer behavior and add scoping — because ClusterIP-by-default changes out-of-the-box behavior (no external ingress unless explicitly enabled) and is a broader product decision. The ClusterIP-default variant can be adopted as a follow-up if the team prefers secure-by-default; the allowedSourceRanges knob composes with it.

Testing

Recipes-YAML / docs / chainsaw-test only — no Go changes — so per the repo's infra-only verification guidance, scoped checks were run in place of full make qualify (Go unit tests, -race, e2e, and golangci-lint cannot regress from non-code changes):

# chainsaw bundle-template test (default renders no source ranges; override renders the scoped CIDR)
AICR_BIN=.../aicr chainsaw test --no-cluster --test-dir tests/chainsaw/bundle-templates/agentgateway   # PASS
yamllint recipes/components/agentgateway/{values.yaml,manifests/inference-gateway.yaml} \
         tests/chainsaw/bundle-templates/agentgateway/chainsaw-test.yaml                                # clean
make bom-docs   # docs/user/container-images.md unchanged (no image impact)

Also verified on a live EKS cluster: with allowedSourceRanges set, the generated ELB security group admitted only the configured CIDR (no 0.0.0.0/0); requests from out-of-range sources were dropped at the ELB.

CRD field-path validity (resolves the cross-review open question)

AgentgatewayParameters.spec.service.spec.loadBalancerSourceRanges is accepted by the installed CRD and is merged onto the generated Service:

# CRD accepts the field (server-side schema validation against the live CRD)
kubectl apply --dry-run=server -f - <<'YAML'
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayParameters
metadata: {name: dryrun-fieldcheck, namespace: agentgateway-system}
spec: {service: {spec: {loadBalancerSourceRanges: ["1.2.3.4/32"]}}}
YAML
# -> agentgatewayparameters.agentgateway.dev/dryrun-fieldcheck created (server dry run)

# Controller merge confirmed live: CR field is reflected on the Service
kubectl get agentgatewayparameters system-proxy -n agentgateway-system \
  -o jsonpath='{.spec.service.spec.loadBalancerSourceRanges}'   # ["216.228.127.128/30"]
kubectl get svc inference-gateway -n agentgateway-system \
  -o jsonpath='{.spec.loadBalancerSourceRanges}'                # ["216.228.127.128/30"]

Review follow-up: `--set` wording hardened

CodeRabbit correctly flagged that the docs understated the --set risk. Confirmed empirically: --set agentgateway:allowedSourceRanges=1.2.3.4/32 exits 0 but renders loadBalancerSourceRanges: 1.2.3.4/32 (a bare string, not a list) — a type-invalid Service. The values.yaml comment and the component-catalog doc now warn against --set for this key and point to the overlay/componentRef override (which renders a correct list). Also hardened the chainsaw assertions with set -e so each check fails closed (without it, only the last command's status counted).

Risk Assessment

Low — Isolated, additive change; empty default preserves current behavior; easy to revert.

Rollout notes: No migration needed. The default (empty) renders no source-range restriction, identical to prior behavior. Operators opt into scoping via an overlay/componentRef override.

Checklist

Tests pass locally — chainsaw bundle-template test passes (make test/-race N/A: no Go changes)
Linter passes — yamllint clean (golangci-lint N/A: no Go changes)
I did not skip/disable tests to make CI green
I added/updated tests for new functionality (chainsaw default + override cases)
I updated docs (component catalog: "Inference Gateway Network Exposure")
Changes follow existing patterns in the codebase
Commits are cryptographically signed (git commit -S)

github-actions · 2026-06-01T22:12:49Z

🌿 Preview your docs: https://nvidia-preview-feat-agentgateway-lb-source-ranges.docs.buildwithfern.com/aicr

coderabbitai · 2026-06-01T22:17:52Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: dec5f603-b60e-4713-9690-beb0e2d1a6bc

📥 Commits

Reviewing files that changed from the base of the PR and between 825162a and d9adba3.

📒 Files selected for processing (4)

docs/user/component-catalog.md
recipes/components/agentgateway/manifests/inference-gateway.yaml
recipes/components/agentgateway/values.yaml
tests/chainsaw/bundle-templates/agentgateway/chainsaw-test.yaml

📝 Walkthrough

Walkthrough

This PR adds a new Helm value agentgateway.allowedSourceRanges (empty list default), conditionally renders Service.spec.loadBalancerSourceRanges in the agentgateway manifest when that list is non-empty, updates user docs describing default open LoadBalancer behavior and how to restrict it via recipe overlays, and extends Chainsaw tests to assert both default (no source ranges) and configured (CIDR present) rendering; test scripts were hardened with set -e.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'feat(agentgateway): scope inference-gateway LB to allowed source ranges' directly and concisely summarizes the main change: adding a feature to restrict the inference-gateway LoadBalancer to allowed source CIDR ranges.
Description check	✅ Passed	The description comprehensively explains the feature addition, motivation, implementation details, testing approach, and risk assessment—all directly related to the changeset across documentation, Helm templates, values, and tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/user/component-catalog.md`:
- Line 70: Update the sentence that first uses "CIDRs" to define the acronym on
first use by expanding it to "Classless Inter-Domain Routing (CIDR)" before
using the short form; specifically modify the line mentioning
agentgateway.allowedSourceRanges and spec.loadBalancerSourceRanges so it reads
something like "set agentgateway.allowedSourceRanges to a list of Classless
Inter-Domain Routing (CIDR) blocks (CIDRs)" and then continue using "CIDRs"
afterward.

In `@recipes/components/agentgateway/values.yaml`:
- Around line 49-51: Update the comment that currently states "list values
cannot be passed via `--set` (scalar only)" to avoid being incorrect: reword it
to acknowledge that Helm 3 supports setting lists via `--set` (e.g., `{a,b}` or
indexed form `servers[0]=...`) and `--set-json`, but recommend using a recipe
overlay override for better maintainability and clarity; edit the comment block
in values.yaml where the current guidance about `--set` and recipe overlays
appears so it suggests overlays as the preferred approach rather than the only
option.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 97e23cbf-6960-4278-894e-38bfea62c6e1

📥 Commits

Reviewing files that changed from the base of the PR and between 3f7584d and ea6486a.

📒 Files selected for processing (4)

docs/user/component-catalog.md
recipes/components/agentgateway/manifests/inference-gateway.yaml
recipes/components/agentgateway/values.yaml
tests/chainsaw/bundle-templates/agentgateway/chainsaw-test.yaml

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@recipes/components/agentgateway/values.yaml`:
- Line 59: The template condition in manifests/inference-gateway.yaml uses if
$agw.allowedSourceRanges so an empty list (allowedSourceRanges: []) is treated
the same as unset; update the test or the values to make the presence
detectable: either change the template to test for the key’s existence (e.g.,
check if hasKey .Values "agentgateway" and index existence or use default nil
distinction) or update recipes/components/agentgateway/values.yaml to set a
non-empty default for allowedSourceRanges when you want the “present” assertion
(replace [] with a sentinel like ["0.0.0.0/0"] or another non-empty override
used in tests); refer to the $agw variable,
.Values.agentgateway.allowedSourceRanges, and
service.spec.loadBalancerSourceRanges when making the fix.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: ce0efd1e-9ee9-4bdf-b480-706c5abd93af

📥 Commits

Reviewing files that changed from the base of the PR and between ea6486a and 0c24cc5.

📒 Files selected for processing (4)

docs/user/component-catalog.md
recipes/components/agentgateway/manifests/inference-gateway.yaml
recipes/components/agentgateway/values.yaml
tests/chainsaw/bundle-templates/agentgateway/chainsaw-test.yaml

The agentgateway controller materializes the inference-gateway Gateway into a type=LoadBalancer Service, which every cloud provisions as an internet-facing load balancer open to 0.0.0.0/0 — exposing the plaintext, unauthenticated inference endpoint to the public internet. Add an `agentgateway.allowedSourceRanges` value rendered into the generated Service's spec.loadBalancerSourceRanges via the AgentgatewayParameters CR. This field is honored by the AWS, GCP, Azure, and OCI cloud load balancers, so one setting locks the gateway down on every platform. The default is intentionally empty (unrestricted) so deployments work out of the box regardless of network; a baked-in CIDR would firewall every downstream deployment to one network. Operators scope it via a recipe/componentRef override (lists cannot be passed through scalar-only --set). Extends the agentgateway bundle-template chainsaw test to cover both the open-by-default and configured-source-range paths, and documents the exposure and the knob in the component catalog.

…exposure The agentgateway inference-gateway is provisioned as a public LoadBalancer open to 0.0.0.0/0 by default (NVIDIA#1138). The default is defensible, but the exposure was silent — nothing in bundle output or validation flagged that the inference endpoint is internet-facing. Add two guardrails that keep open-by-default while removing the silence. 1. Bundle-time warning (pkg/bundler): when a bundle includes agentgateway with an empty allowedSourceRanges, emit a non-blocking warning that the inference-gateway will be open to 0.0.0.0/0, with remediation. Mirrors the existing storageClassName PVC warning and inspects the merged component values, so it catches the empty state regardless of override mechanism. 2. Conformance finding (validators/conformance): extend the existing inference-gateway check (already wired into every inference overlay) to assess the gateway's LoadBalancer Service and record its exposure as evidence — scoped source ranges, or an explicit open-to-0.0.0.0/0 finding. Open-by-default stays a non-fatal warning; AICR_REQUIRE_SCOPED_INFERENCE_GATEWAY=true escalates it to a check failure (fail-closed policy). Docs: component-catalog gains an "Exposure guardrails" subsection; cli-reference documents the bundle warning; validation.md and the validator env table cover the conformance finding and the enforcement env var. Closes NVIDIA#1160

…exposure The agentgateway inference-gateway is provisioned as a public LoadBalancer open to 0.0.0.0/0 by default (NVIDIA#1138). The default is defensible, but the exposure was silent — nothing in bundle output or validation flagged that the inference endpoint is internet-facing. Add two guardrails that keep open-by-default while removing the silence. 1. Bundle-time warning (pkg/bundler): when a bundle includes agentgateway with an unscoped allowedSourceRanges, emit a non-blocking warning that the inference-gateway will be open to 0.0.0.0/0, with remediation. Mirrors the existing storageClassName PVC warning and inspects the merged component values, so it catches the open state regardless of override mechanism. 2. Conformance finding (validators/conformance): extend the existing inference-gateway check (already wired into every inference overlay) to assess the gateway's LoadBalancer Service and record its exposure as evidence — scoped source ranges, or an explicit open-to-0.0.0.0/0 finding. Open-by-default stays a non-fatal warning; AICR_REQUIRE_SCOPED_INFERENCE_GATEWAY=true escalates it to a check failure (fail-closed policy). An "unscoped" source-range list is empty OR contains an any-source CIDR (0.0.0.0/0 or ::/0) — a length-only check would let an explicit ["0.0.0.0/0"] pass enforcement. The exposure assessment filters to the inference-gateway Service by name (mirroring the EndpointSlice readiness filter) so a co-located LoadBalancer is neither mislabeled nor able to fail enforce mode. The InferenceGatewayCheckName constant is exported and locked by TestEmbeddedCatalog_InferenceGatewayEntryExists so a catalog rename can't silently no-op enforcement forwarding. Docs: component-catalog gains an "Exposure guardrails" subsection; cli-reference documents the bundle warning; validation.md and the validator env table cover the conformance finding and the enforcement env var. Closes NVIDIA#1160

yuanchen8911 requested review from a team as code owners June 1, 2026 22:11

yuanchen8911 added enhancement labels Jun 1, 2026

github-actions Bot added area/recipes area/tests area/docs size/M labels Jun 1, 2026

coderabbitai Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread docs/user/component-catalog.md Outdated

Comment thread recipes/components/agentgateway/values.yaml Outdated

yuanchen8911 force-pushed the feat/agentgateway-lb-source-ranges branch 2 times, most recently from 0c24cc5 to 9e2e31e Compare June 1, 2026 22:27

coderabbitai Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread recipes/components/agentgateway/values.yaml

yuanchen8911 force-pushed the feat/agentgateway-lb-source-ranges branch 2 times, most recently from 50ee459 to 825162a Compare June 1, 2026 22:30

yuanchen8911 force-pushed the feat/agentgateway-lb-source-ranges branch from 825162a to d9adba3 Compare June 1, 2026 22:34

yuanchen8911 requested a review from mchmarny June 1, 2026 22:54

mchmarny approved these changes Jun 1, 2026

View reviewed changes

yuanchen8911 merged commit cfb0cb0 into NVIDIA:main Jun 1, 2026
200 of 202 checks passed

yuanchen8911 deleted the feat/agentgateway-lb-source-ranges branch June 1, 2026 23:00

This was referenced Jun 1, 2026

agentgateway inference-gateway provisions a public LoadBalancer open to 0.0.0.0/0 #1139

Closed

agentgateway: surface open inference-gateway exposure (bundle warning + conformance check) #1160

Closed

coderabbitai Bot mentioned this pull request Jun 2, 2026

feat(cli): add --set-json/--set-file for list and object bundle overrides #1162

Merged

13 tasks

yuanchen8911 mentioned this pull request Jun 2, 2026

feat(bundler): warn on open agentgateway inference-gateway exposure #1163

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(agentgateway): scope inference-gateway LB to allowed source ranges#1138

feat(agentgateway): scope inference-gateway LB to allowed source ranges#1138
yuanchen8911 merged 1 commit into
NVIDIA:mainfrom
yuanchen8911:feat/agentgateway-lb-source-ranges

yuanchen8911 commented Jun 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 1, 2026

Uh oh!

coderabbitai Bot commented Jun 1, 2026 •

edited

Loading

Walkthrough

Estimated code review effort

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

yuanchen8911 commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation / Context

Type of Change

Component(s) Affected

Implementation Notes

Alternative considered: ClusterIP-by-default

Testing

CRD field-path validity (resolves the cross-review open question)

Review follow-up: --set wording hardened

Risk Assessment

Checklist

Uh oh!

github-actions Bot commented Jun 1, 2026

Uh oh!

coderabbitai Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Estimated code review effort

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yuanchen8911 commented Jun 1, 2026 •

edited

Loading

Review follow-up: `--set` wording hardened

coderabbitai Bot commented Jun 1, 2026 •

edited

Loading