atenet-router xDS server sets ads=true on SnapshotCache, deadlocking Envoy SDS startup for 15s on every router boot #3268

Summary & Root Cause

NewXdsServer creates the go-control-plane snapshot cache with ads=true. In ADS mode the cache only answers a watch once the request names every resource of that type held in the snapshot. UpdateSnapshot puts four secrets in the SecretType table: the two listener secrets (https_serving_cert, ingress_client_trust_bundle) and the two cluster secrets (upstream_client_cert, upstream_trust_bundle). Envoy receives CDS first and, while warming clusters (cm init), subscribes over SDS only to the two cluster secrets. It does not request LDS (and so the listener secrets) until cm init completes. The cache withholds the partial SDS response, Envoy waits, and the deadlock only breaks when Envoy's default 15s initial_fetch_timeout fires. Because the router runs a single replica, every boot or restart is a 15s+ ingress outage.

Affected Code Locations

• cmd/atenet/internal/router/xds.go:286 (NewXdsServer: cachev3.NewSnapshotCache(true, cachev3.IDHash{}, nil))
• cmd/atenet/internal/router/xds.go:574-585 (UpdateSnapshot: listener secrets and upstream secrets go into one SecretType table)
• cmd/atenet/internal/router/xds.go:588-593 (snapshot built with ClusterType, RouteType, ListenerType, SecretType)
• cmd/atenet/internal/router/xds.go:86, 94-95, 100 (secret names https_serving_cert, upstream_client_cert, upstream_trust_bundle, ingress_client_trust_bundle)
• cmd/atenet/internal/router/xds.go:760-791 (upstream transport socket references upstream_client_cert / upstream_trust_bundle over SDS)
• cmd/atenet/internal/router/xds.go:1639-1646 (adsConfigSource: SDS over ADS, no InitialFetchTimeout set, so Envoy's 15s default applies)
• manifests/ate-install/atenet-router.yaml:182 (replicas: 1)

Detailed Reproduction Guide

1. Deploy ate-system on a fresh cluster, or restart the router:

   kubectl delete pod -n ate-system -l app=atenet-router

2. Watch readiness events and Envoy logs on the new pod:

   kubectl describe pod -n ate-system -l app=atenet-router
   kubectl logs -n ate-system -l app=atenet-router -c envoy | grep -E 'cds: added|initial fetch timed out|cm init'

Expected vs. Actual Behavior

• Expected: Envoy initializes clusters and listeners within about a second of connecting to the local xDS server.
• Actual: Every boot stalls for exactly 15.0s between cds: added/updated 6 cluster(s) and cm init: all clusters initialized, with two initial fetch timed out ... tls.v3.Secret warnings. Kubelet readiness probes fail (:9902/ready: connect: connection refused) for the whole window.

Observed Output & Logs

  # first boot
  [2026-10-27 00:48:34.265] cds: added/updated 6 cluster(s)
  [2026-10-27 00:48:49.263][warning] gRPC config: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret   (x2)
  [2026-10-27 00:48:49.263] cm init: all clusters initialized
  [2026-10-27 00:48:49.276] all dependencies initialized. starting workers
  # after pod delete
  [2026-10-27 00:50:34.828] cds: added/updated 6 cluster(s)
  [2026-10-27 00:50:49.824][warning] gRPC config: initial fetch timed out for ...tls.v3.Secret   (x2)
  [2026-10-27 00:50:49.824] cm init: all clusters initialized
  Warning  Unhealthy  pod/atenet-router-65c8b6cb47-9xrc7  Readiness probe failed: Get "http://10.64.0.23:9902/ready": dial tcp 10.64.0.23:9902: connect: connection refused

  02:01:58.705  cds: added/updated 6 cluster(s)
  02:02:13.703  gRPC config: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret   (x2)
  02:02:13.703  cm init: all clusters initialized
  Readiness probe failed: ... :9902/ready: dial tcp ...: connect: connection refused