Skip to content

Commit 73a2a05

Browse files
authored
feat(push): support platform-managed gateway runtime pods (#8113)
Platform-owned service meshes need runtime pod identity and a replacement network policy without patching rendered gateway manifests. Add runtime-only pod labels, service account and shutdown-grace settings, plus an explicit named replacement-policy mode. Migration isolation remains independent; the default render is unchanged. Chart 0.3.4 publication and live deployment verification follow in the separate release and deployment work. ### Validation Added tests cover runtime-only settings, reserved selector labels, minimum shutdown grace, replacement-policy acknowledgement and conflicting settings. The complete chart render suite and release-contract tests pass. Default rendering matches the current main chart byte-for-byte. Repository-wide `just ci` reached unit tests but failed in existing buzz-acp tests outside this chart change. Live mesh behavior has not been exercised. --------- Signed-off-by: Tom Brow <[email protected]>
1 parent 001290a commit 73a2a05

8 files changed

Lines changed: 533 additions & 10 deletions

File tree

‎deploy/charts/buzz-push-gateway/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,6 @@ apiVersion: v2
33
# branches (see docs/push-gateway-deployment.md, "Gateway chart release").
44
name: buzz-push-gateway
55
description: Public capability-gated APNs last-hop gateway for Buzz
6-
version: 0.3.3
6+
version: 0.3.4
77
appVersion: "0.1.0"
88
type: application
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Buzz push gateway chart
2+
3+
### Platform-managed runtime integration
4+
5+
`podLabels`, `serviceAccountName`, and `terminationGracePeriodSeconds` configure
6+
only runtime pods. Defaults retain the existing pod specification; the grace
7+
period cannot be shorter than 60 seconds. Additional labels cannot override the
8+
chart's name, instance, or component selectors. Migration settings remain separate.
9+
10+
A parent chart that supplies its own runtime NetworkPolicy can set
11+
`networkPolicy.enabled=false` and `networkPolicy.externalPolicyName` to that
12+
policy's name. The name is an explicit acknowledgement, not a cluster existence
13+
check: the parent must render and validate a policy selecting the runtime pods
14+
before deployment. Supplying a replacement name while the upstream policy is
15+
enabled is rejected. The migration NetworkPolicy remains enabled independently.
16+
17+
External-policy releases must run `tests/check-external-policy.rb` on the
18+
**complete parent render including hooks**, before install or upgrade. Helm 3
19+
excludes hooks from its post-renderer input, so `--post-renderer` alone is not a
20+
sufficient gate. Render without `--no-hooks` or resource filtering, using the
21+
same chart, release name, namespace, values and capabilities as the deployment:
22+
23+
```sh
24+
set -o pipefail
25+
helm template RELEASE PARENT_CHART --namespace NAMESPACE -f VALUES.yaml \
26+
| ruby tests/check-external-policy.rb --expect DEPLOYMENT POLICY NAMESPACE > validated-release.yaml
27+
```
28+
29+
`--expect` is mandatory for external-policy deployment and CI invocations.
30+
Provide the intended Deployment name, replacement policy name and release namespace
31+
from deployment configuration, not by extracting the annotations being checked.
32+
Repeat it for each external-policy Deployment. Missing Deployments or removed or
33+
overwritten markers then fail closed. No-argument mode is only for renders that
34+
intentionally have no external-policy requirement, such as default-mode tests.
35+
36+
For upgrade preflight, include `--is-upgrade`. Deployment automation must stop if
37+
this pipeline fails and must not change the chart or rendering inputs between
38+
validation and deployment. Run the post-renderer additionally during Helm
39+
install/upgrade to check ordinary resources, but retain the full-render preflight
40+
to detect hook collisions. GitOps render pipelines must validate the complete
41+
hook-inclusive output before submitting it to their reconciler.
42+
The Deployment records the required policy name. The gate rejects a missing,
43+
misspelled, duplicate, wrong-namespace or non-selecting replacement, and requires
44+
both ingress and egress isolation. It emits no manifests on failure and preserves
45+
successful output byte-for-byte. Parent CI must run this same gate on its complete
46+
render; a subchart-only render intentionally fails in external-policy mode.
47+
The deployment owner must separately validate the allowed traffic and rolling drain.

‎deploy/charts/buzz-push-gateway/templates/deployment.yaml‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ kind: Deployment
33
metadata:
44
name: {{ include "push.name" . }}
55
labels: {{- include "push.runtimeLabels" . | nindent 4 }}
6+
{{- if not .Values.networkPolicy.enabled }}
7+
annotations:
8+
buzz.block.xyz/release-namespace: {{ .Release.Namespace | quote }}
9+
buzz.block.xyz/external-network-policy: {{ .Values.networkPolicy.externalPolicyName | quote }}
10+
{{- end }}
611
spec:
712
replicas: {{ .Values.replicaCount }}
813
strategy: { type: RollingUpdate, rollingUpdate: { maxSurge: 1, maxUnavailable: 0 } }
@@ -11,12 +16,18 @@ spec:
1116
template:
1217
metadata:
1318
labels: {{- include "push.runtimeLabels" . | nindent 8 }}
19+
{{- with .Values.podLabels }}
20+
{{- toYaml . | nindent 8 }}
21+
{{- end }}
1422
{{- with .Values.podAnnotations }}
1523
annotations: {{- toYaml . | nindent 8 }}
1624
{{- end }}
1725
spec:
1826
automountServiceAccountToken: false
19-
terminationGracePeriodSeconds: 60
27+
{{- with .Values.serviceAccountName }}
28+
serviceAccountName: {{ . | quote }}
29+
{{- end }}
30+
terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
2031
securityContext: { runAsNonRoot: true, runAsUser: 65532, runAsGroup: 65532, fsGroup: 65532, seccompProfile: { type: RuntimeDefault } }
2132
{{- with .Values.image.pullSecrets }}
2233
imagePullSecrets: {{ toYaml . | nindent 8 }}
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
#!/usr/bin/env ruby
2+
# Validate the complete release; strict external callers supply --expect.
3+
require 'yaml'
4+
require 'date'
5+
# Strict callers provide expectations independently of rendered annotations.
6+
expectations = []
7+
until ARGV.empty?
8+
abort 'usage: check-external-policy.rb [--expect DEPLOYMENT POLICY NAMESPACE]...' unless ARGV.shift == '--expect' && ARGV.length >= 3
9+
expectations << ARGV.shift(3)
10+
end
11+
input = STDIN.read
12+
resources = YAML.parse_stream(input).children.map do |doc|
13+
stream = Psych::Nodes::Stream.new
14+
stream.children << doc
15+
YAML.safe_load(stream.to_yaml, permitted_classes: [Date, Time], aliases: true)
16+
end.compact
17+
# Match Kubernetes Unstructured.IsList: an items array, including typed lists.
18+
# https://github.com/kubernetes/apimachinery/blob/v0.34.1/pkg/apis/meta/v1/unstructured/unstructured.go
19+
def flatten_resources(resource, enclosing_hooks = {})
20+
annotations = resource.dig('metadata', 'annotations') || {}
21+
hooks = enclosing_hooks.merge(annotations.select { |k, _| %w[helm.sh/hook argocd.argoproj.io/hook].include?(k) })
22+
if resource['items'].is_a?(Array)
23+
resource.fetch('items').flat_map { |item| flatten_resources(item, hooks) }
24+
else
25+
unless hooks.empty?
26+
resource = Marshal.load(Marshal.dump(resource))
27+
resource['metadata'] ||= {}
28+
resource['metadata']['annotations'] = annotations.merge(hooks)
29+
end
30+
[resource]
31+
end
32+
end
33+
resources = resources.flat_map { |resource| flatten_resources(resource) }
34+
def label_value?(value)
35+
value.is_a?(String) && value.length <= 63 && (value.empty? || /\A[A-Za-z0-9](?:[-_.A-Za-z0-9]*[A-Za-z0-9])?\z/.match?(value))
36+
end
37+
38+
def label_key?(key)
39+
return false unless key.is_a?(String)
40+
parts = key.split('/', -1)
41+
return false unless [1, 2].include?(parts.length) && !parts.last.empty? && label_value?(parts.last)
42+
return true if parts.length == 1
43+
prefix = parts.first
44+
prefix.length <= 253 && /\A[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\.[a-z0-9](?:[-a-z0-9]*[a-z0-9])?)*\z/.match?(prefix)
45+
end
46+
annotation = 'buzz.block.xyz/external-network-policy'
47+
expectations.each do |deployment_name, policy_name, namespace|
48+
candidates = resources.select do |r|
49+
r['kind'] == 'Deployment' && r.dig('metadata', 'name') == deployment_name &&
50+
(r.dig('metadata', 'namespace').to_s.empty? ? namespace : r.dig('metadata', 'namespace')) == namespace
51+
end
52+
abort "external policy #{policy_name}: expected exactly one deployment #{deployment_name}" unless candidates.length == 1
53+
annotations = candidates.first.dig('metadata', 'annotations') || {}
54+
abort "external policy #{policy_name}: missing or overwritten ownership marker" unless annotations[annotation] == policy_name && annotations['buzz.block.xyz/release-namespace'] == namespace
55+
end
56+
resources.select { |r| r['kind'] == 'Deployment' }.each do |deployment|
57+
name = deployment.dig('metadata', 'annotations', annotation)
58+
next unless name
59+
release_namespace = deployment.dig('metadata', 'annotations', 'buzz.block.xyz/release-namespace')
60+
abort "external policy #{name}: missing release namespace" if !release_namespace.is_a?(String) || release_namespace.empty?
61+
namespace = deployment.dig('metadata', 'namespace')
62+
namespace = release_namespace if namespace.nil? || namespace.empty?
63+
policies = resources.select do |r|
64+
r['apiVersion'] == 'networking.k8s.io/v1' && r['kind'] == 'NetworkPolicy' &&
65+
r.dig('metadata', 'name') == name &&
66+
(r.dig('metadata', 'namespace').to_s.empty? ? release_namespace : r.dig('metadata', 'namespace')) == namespace
67+
end
68+
abort "external policy #{name}: expected exactly one replacement in #{namespace}" unless policies.length == 1
69+
abort "external policy #{name}: replacement must not be a Helm hook" if policies.first.dig('metadata', 'annotations', 'helm.sh/hook')
70+
abort "external policy #{name}: replacement must not be an Argo CD hook" if policies.first.dig('metadata', 'annotations', 'argocd.argoproj.io/hook')
71+
spec = policies.first.fetch('spec')
72+
labels = deployment.dig('spec', 'template', 'metadata', 'labels')
73+
selector = spec.fetch('podSelector')
74+
# Require the immutable runtime identity explicitly, not a namespace-wide policy.
75+
identity = deployment.dig('spec', 'selector', 'matchLabels')
76+
match = selector.fetch('matchLabels', {})
77+
expressions = selector.fetch('matchExpressions', [])
78+
identity_bound = identity.all? do |k, v|
79+
match[k] == v || expressions.any? { |e| e['key'] == k && e['operator'] == 'In' && e['values'] == [v] }
80+
end
81+
abort "external policy #{name}: missing runtime identity selector" unless identity_bound
82+
abort "external policy #{name}: selector does not match runtime" unless match.all? { |k, v| labels[k] == v }
83+
selector.fetch('matchExpressions', []).each do |expression|
84+
key = expression.fetch('key')
85+
values = expression.fetch('values', [])
86+
operator = expression.fetch('operator')
87+
valid_values = values.is_a?(Array) && values.all? { |v| label_value?(v) }
88+
valid_values &&= %w[In NotIn].include?(operator) ? !values.empty? : (%w[Exists DoesNotExist].include?(operator) && values.empty?)
89+
abort "external policy #{name}: invalid selector expression" unless label_key?(key) && valid_values
90+
matches = case operator
91+
when 'In' then labels.key?(key) && values.include?(labels[key])
92+
when 'NotIn' then !labels.key?(key) || !values.include?(labels[key])
93+
when 'Exists' then labels.key?(key)
94+
when 'DoesNotExist' then !labels.key?(key)
95+
else false
96+
end
97+
abort "external policy #{name}: expression does not match runtime" unless matches
98+
end
99+
types = spec['policyTypes']
100+
if types.nil? || types.empty?
101+
types = ['Ingress']
102+
types << 'Egress' unless spec.fetch('egress', []).empty?
103+
end
104+
abort "external policy #{name}: must isolate ingress and egress" unless %w[Ingress Egress].all? { |t| types.include?(t) }
105+
end
106+
# Preserve Helm output byte-for-byte only after every deployment passes.
107+
STDOUT.write(input)

‎deploy/charts/buzz-push-gateway/tests/render.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
#!/usr/bin/env bash
22
set -euo pipefail
3+
bash deploy/charts/buzz-push-gateway/tests/runtime-options.sh
34
bash deploy/charts/buzz-push-gateway/tests/grant-lifetime.sh
45
bash deploy/charts/buzz-push-gateway/tests/migration-annotations.sh
56
out=$(mktemp); production_out=$(mktemp); route_out=$(mktemp); datadog_out=$(mktemp)

0 commit comments

Comments
 (0)