Skip to content

Commit 8ad5dc3

Browse files
committed
Follow the amended design on owners, Lost addresses and pre-bindings
- A claim with any controller owner deletes its addresses on deletion, not only a Service-owned one. - A Lost address keeps its family bound, so the claim no longer binds a second address of that family; status still reports Lost. - Claim reclaim leaves an address in Conflict or Lost in its phase. - A pre-binding is accepted only if it matches the claim's class, a requested family, and a family the claim does not hold yet; otherwise it is refused with an event and the claim's reclaim skips it. - Auto-claim skips a Service carrying a MetalLB pool, pin or sharing annotation, so driver placeholders are excluded without the opt-out. Assisted-By: LLM Signed-off-by: Timofei Larkin <[email protected]>
1 parent 530646e commit 8ad5dc3

6 files changed

Lines changed: 330 additions & 70 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -90,12 +90,13 @@ nothing:
9090
`local.sdn.cozystack.io/ip-address-claim` annotation; the driver takes it
9191
from there.
9292
- Deleting the Service deletes the claim and its addresses, whatever the
93-
class's `reclaimPolicy`. To keep the address, remove the claim's owner
94-
reference; it is an ordinary claim from then on.
93+
class's `reclaimPolicy`. This holds for any claim with a controller
94+
owner. To keep the address, remove the claim's owner reference; it is an
95+
ordinary claim from then on.
9596
- Skipped: Services that already name a claim, set
9697
`local.sdn.cozystack.io/auto-claim: "false"`, set
97-
`spec.loadBalancerClass` or `spec.loadBalancerIP`, or already have an
98-
address.
98+
`spec.loadBalancerClass` or `spec.loadBalancerIP`, carry a MetalLB pool,
99+
pin or sharing annotation, or already have an address.
99100

100101
Apart from auto-claim, the core controller never touches Services. It
101102
never parses class parameters and never puts a packet on a wire.

‎docs/design.md‎

Lines changed: 36 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -191,17 +191,24 @@ idempotent; the pass re-derives everything from the cluster state.
191191
is bound to an earlier, deleted claim that happened to have the same
192192
name — a stale binding. It is never adopted; the address reconciler
193193
reclaims it (§5 step 6).
194-
4. **Complete pre-bindings.** For collected addresses with an empty
195-
`claimRef.uid` (a driver pre-bound them at creation), write the claim's
196-
UID. This is the core's acceptance of the driver's provisioning.
197-
5. **Anything missing?** Expand `spec.family` into concrete families
198-
(`Dual` → v4 + v6); a family is satisfied if some collected address of
199-
that family exists and is not in phase `Lost`. **If every family is
200-
satisfied, skip straight to status (step 8) — the class is not
201-
consulted at all.** Binding state comes strictly before class state:
194+
4. **Accept pre-bindings.** For collected addresses with an empty
195+
`claimRef.uid` (a driver or an admin pre-bound them), write the claim's
196+
UID if the address is of the claim's class, of a requested family, and
197+
of a family the claim does not already hold. This is the core's
198+
acceptance of the driver's provisioning. A claim that has not recorded
199+
a class resolves it first (step 6). A mismatched pre-binding is refused
200+
with a `PreBindingRefused` event and is not collected, as the PV
201+
controller leaves a pre-bound volume that does not satisfy its claim.
202+
5. **Anything unbound?** Expand `spec.family` into concrete families
203+
(`Dual` → v4 + v6); a family is unbound if no collected address of that
204+
family exists, in any phase. A `Lost` address keeps its family: the
205+
claim waits for it to be restored or deleted and is never given a
206+
second address of that family. **If no family is unbound, skip
207+
straight to status (step 8) — the class is not consulted at all.** Binding state comes strictly before class state:
202208
a fully bound claim needs nothing from its class, so a deleted class
203209
never disturbs existing bindings (see *Class deletion* below).
204-
6. **Class resolution** (only reached with families missing). Resolve a
210+
6. **Class resolution** (only reached with families unbound, or with a
211+
pre-binding to judge). Resolve a
205212
class name, trying in order:
206213
1. `spec.className`, if set (an explicit spec always wins);
207214
2. `status.className` — the sticky record of a previous resolution, so
@@ -222,7 +229,7 @@ idempotent; the pass re-derives everything from the cluster state.
222229
the stamp follows. This annotation is the driver's watch key; it is
223230
how a driver knows a claim is its to serve without ever resolving
224231
classes itself.
225-
7. **Match, per missing family.** For each unsatisfied family:
232+
7. **Match, per unbound family.** For each unbound family:
226233
- candidate set: if `spec.addressName` is set, only that object;
227234
otherwise every address in phase `Available` (so: no `claimRef`, and
228235
already accepted by the address reconciler) with the resolved class
@@ -235,11 +242,11 @@ idempotent; the pass re-derives everything from the cluster state.
235242
- no candidate → the family stays unsatisfied and the claim waits for
236243
its driver to provision.
237244
8. **Status.** Recompute from scratch:
238-
- all families satisfied → `Bound`;
239-
- previously `Bound` (or `Lost`) and now unsatisfied → `Lost` — a
240-
binding degraded, which is surfaced, never silently re-provisioned
241-
around; the claim returns to `Bound` by itself if the address comes
242-
back;
245+
- every family has a collected address not in phase `Lost` → `Bound`;
246+
- previously `Bound` (or `Lost`) and now not → `Lost` — a binding
247+
degraded, which is surfaced, never silently re-provisioned around;
248+
the claim returns to `Bound` by itself if the address comes back,
249+
and binds a new one once an admin deletes the `Lost` address;
243250
- otherwise `Pending`, with the `Bound` condition explaining what it
244251
waits for (`WaitingForProvisioning` names the stamped provisioner).
245252
`status.addresses` lists every collected address (name + IP), sorted
@@ -312,18 +319,22 @@ Runs behind the claim's protection finalizer, so it always runs before
312319
the claim disappears:
313320

314321
- For every address bound to the claim (same collection rule as §4 step
315-
3), apply **the address's** `reclaimPolicy` — copied from the class at
322+
3, less pre-bindings step 4 would refuse), apply **the address's**
323+
`reclaimPolicy` — copied from the class at
316324
provisioning time, so a later class edit does not retroactively change
317325
the fate of existing addresses:
318326
- `Retain` → phase `Released`. The `claimRef` **survives** — the
319327
released address is not reusable until an admin clears it (§5 step 5).
328+
An address in `Conflict` or `Lost` keeps its phase: both belong to the
329+
driver (§3).
320330
- `Delete` → delete the `IPAddress` object. The core does not tear down
321331
any backend state; the driver's own finalizer on the address
322332
intercepts the deletion and deallocates first (§7 obligation 4).
323-
- Exception: a claim whose controller is a Service (an auto-claim, §10)
324-
lives and dies with that Service, so its addresses are deleted
325-
whatever their `reclaimPolicy`. Removing the controller reference turns
326-
it into an ordinary claim, and the policy applies again.
333+
- Exception: a claim with a controller owner — a Service for an
334+
auto-claim (§10), or any controller that mints claims for a tenant —
335+
lives and dies with that owner, so its addresses are deleted whatever
336+
their `reclaimPolicy`. Removing the controller reference turns it into
337+
an ordinary claim, and the policy applies again.
327338
- Remove the finalizer; the claim goes away.
328339

329340
**What each policy actually reclaims.** "The address" is up to three
@@ -455,6 +466,11 @@ A Service qualifies when it is a live `type: LoadBalancer` Service that:
455466
stamp this on Services they create for their own bookkeeping),
456467
- sets neither `spec.loadBalancerClass` (another implementation's) nor
457468
`spec.loadBalancerIP` (asks for a specific address),
469+
- carries none of the backend's pool, pin or sharing annotations: for
470+
MetalLB, `loadBalancerIPs`, `address-pool` and `allow-shared-ip` under
471+
`metallb.io/` or `metallb.universe.tf/`. Such a Service has chosen its
472+
address source. A driver's placeholders name their pool this way, so
473+
this rule excludes them even without the opt-out,
458474
- has no `status.loadBalancer.ingress` yet. This is what keeps a Service
459475
addressed by other means from being moved to a new address.
460476

‎internal/controller/ipaddressclaim_controller.go‎

Lines changed: 107 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,10 @@ limitations under the License.
1717
package controller
1818

1919
import (
20+
"cmp"
2021
"context"
2122
"fmt"
23+
"slices"
2224
"sort"
2325

2426
apierrors "k8s.io/apimachinery/pkg/api/errors"
@@ -82,18 +84,28 @@ func (r *IPAddressClaimReconciler) Reconcile(ctx context.Context, req ctrl.Reque
8284
if err != nil {
8385
return ctrl.Result{}, err
8486
}
85-
if err := r.completePreBindings(ctx, claim, bound); err != nil {
87+
88+
// Accepting a pre-binding needs the claim's class, so a claim that has
89+
// not recorded one resolves it first.
90+
className, resolved := "", false
91+
if knownClassName(claim) == "" && hasPendingPreBinding(bound) {
92+
if className, err = r.resolveClass(ctx, claim); err != nil {
93+
return ctrl.Result{}, err
94+
}
95+
resolved = true
96+
}
97+
if bound, err = r.acceptPreBindings(ctx, claim, cmp.Or(knownClassName(claim), className), bound); err != nil {
8698
return ctrl.Result{}, err
8799
}
88100

89-
className := ""
90-
if len(missingFamilies(claim, bound)) > 0 {
91-
className, err = r.resolveClass(ctx, claim)
92-
if err != nil {
93-
return ctrl.Result{}, err
101+
if unbound := unboundFamilies(claim, bound); len(unbound) > 0 {
102+
if !resolved {
103+
if className, err = r.resolveClass(ctx, claim); err != nil {
104+
return ctrl.Result{}, err
105+
}
94106
}
95107
if className != "" {
96-
for _, family := range missingFamilies(claim, bound) {
108+
for _, family := range unbound {
97109
matched, err := r.bindAvailableAddress(ctx, claim, className, family)
98110
if err != nil {
99111
return ctrl.Result{}, err
@@ -123,29 +135,36 @@ func (r *IPAddressClaimReconciler) finalizeClaim(ctx context.Context, claim *loc
123135
if err != nil {
124136
return err
125137
}
126-
// A claim still controlled by a Service lives and dies with it, so its
138+
// A claim with a controller owner lives and dies with it, so its
127139
// addresses are deleted whatever their reclaimPolicy. Removing the
128140
// controller reference is how such a claim is retained.
129-
ownedByService := controlledByService(claim)
141+
owner := metav1.GetControllerOf(claim)
130142
for i := range addrs {
131143
addr := &addrs[i]
132144
if !addr.DeletionTimestamp.IsZero() {
133145
continue
134146
}
147+
// A pre-binding the claim never accepted is not the claim's to
148+
// reclaim; the address controller handles it once the claim is gone.
149+
if addr.Spec.ClaimRef.UID == "" && preBindingMismatch(claim, knownClassName(claim), nil, addr) != "" {
150+
continue
151+
}
135152
switch {
136-
case ownedByService || addr.Spec.ReclaimPolicy == localv1alpha1.ReclaimDelete:
153+
case owner != nil || addr.Spec.ReclaimPolicy == localv1alpha1.ReclaimDelete:
137154
// The driver's own finalizer tears down the backend allocation
138155
// before the object goes away.
139156
if err := r.Delete(ctx, addr); err != nil && !apierrors.IsNotFound(err) {
140157
return err
141158
}
142-
if ownedByService {
159+
if owner != nil {
143160
r.Recorder.Eventf(claim, "Normal", "AddressDeleted",
144-
"IPAddress %s deleted with the claim's owning Service", addr.Name)
161+
"IPAddress %s deleted with the claim's owner %s %s", addr.Name, owner.Kind, owner.Name)
145162
} else {
146163
r.Recorder.Eventf(claim, "Normal", "AddressDeleted",
147164
"IPAddress %s deleted per reclaimPolicy Delete", addr.Name)
148165
}
166+
case addr.Status.Phase == localv1alpha1.IPAddressConflict || addr.Status.Phase == localv1alpha1.IPAddressLost:
167+
// Conflict and Lost belong to the driver and stay as recorded.
149168
default: // Retain
150169
if addr.Status.Phase != localv1alpha1.IPAddressReleased {
151170
addr.Status.Phase = localv1alpha1.IPAddressReleased
@@ -161,10 +180,10 @@ func (r *IPAddressClaimReconciler) finalizeClaim(ctx context.Context, claim *loc
161180
return r.Update(ctx, claim)
162181
}
163182

164-
// controlledByService reports whether a claim's controller is a Service.
165-
func controlledByService(claim *localv1alpha1.IPAddressClaim) bool {
166-
ref := metav1.GetControllerOf(claim)
167-
return ref != nil && ref.APIVersion == "v1" && ref.Kind == serviceKind
183+
// knownClassName is the class the claim names or has already resolved to,
184+
// or "" if neither is recorded yet.
185+
func knownClassName(claim *localv1alpha1.IPAddressClaim) string {
186+
return cmp.Or(claim.Spec.ClassName, claim.Status.ClassName)
168187
}
169188

170189
// resolveClass resolves the claim's class name — spec first, then the sticky
@@ -251,19 +270,67 @@ func (r *IPAddressClaimReconciler) markUnresolved(claim *localv1alpha1.IPAddress
251270
})
252271
}
253272

254-
// completePreBindings accepts driver pre-bound addresses by writing the
255-
// claim's UID into claimRefs that carry none yet.
256-
func (r *IPAddressClaimReconciler) completePreBindings(ctx context.Context, claim *localv1alpha1.IPAddressClaim, addrs []localv1alpha1.IPAddress) error {
273+
// hasPendingPreBinding reports whether any address is pre-bound to the claim
274+
// by name only, without the claim's UID.
275+
func hasPendingPreBinding(addrs []localv1alpha1.IPAddress) bool {
276+
for _, addr := range addrs {
277+
if addr.Spec.ClaimRef.UID == "" {
278+
return true
279+
}
280+
}
281+
return false
282+
}
283+
284+
// acceptPreBindings completes pre-bindings — addresses whose claimRef names
285+
// the claim but carries no UID yet — by writing the claim's UID, as the PV
286+
// controller does for a pre-bound volume that satisfies its claim. A
287+
// pre-binding of another class, of a family the claim does not request, or
288+
// of a family the claim already holds is refused with an event and left
289+
// out of the returned list. Without a resolved class nothing is accepted.
290+
func (r *IPAddressClaimReconciler) acceptPreBindings(ctx context.Context, claim *localv1alpha1.IPAddressClaim, className string, addrs []localv1alpha1.IPAddress) ([]localv1alpha1.IPAddress, error) {
291+
accepted := make([]localv1alpha1.IPAddress, 0, len(addrs))
292+
for _, addr := range addrs {
293+
if addr.Spec.ClaimRef.UID != "" {
294+
accepted = append(accepted, addr)
295+
}
296+
}
257297
for i := range addrs {
258298
addr := &addrs[i]
259-
if addr.Spec.ClaimRef.UID == "" {
260-
addr.Spec.ClaimRef.UID = claim.UID
261-
if err := r.Update(ctx, addr); err != nil {
262-
return err
263-
}
299+
if addr.Spec.ClaimRef.UID != "" || className == "" {
300+
continue
264301
}
302+
if reason := preBindingMismatch(claim, className, accepted, addr); reason != "" {
303+
r.Recorder.Eventf(claim, "Warning", "PreBindingRefused",
304+
"IPAddress %s is pre-bound to this claim but %s", addr.Name, reason)
305+
continue
306+
}
307+
addr.Spec.ClaimRef.UID = claim.UID
308+
if err := r.Update(ctx, addr); err != nil {
309+
return nil, err
310+
}
311+
accepted = append(accepted, *addr)
265312
}
266-
return nil
313+
return accepted, nil
314+
}
315+
316+
// preBindingMismatch reports why a pre-bound address cannot bind to the
317+
// claim, or "" if it can. held lists the addresses the claim already holds.
318+
func preBindingMismatch(claim *localv1alpha1.IPAddressClaim, className string, held []localv1alpha1.IPAddress, addr *localv1alpha1.IPAddress) string {
319+
family := familyOf(addr.Spec.Address)
320+
switch {
321+
case className == "":
322+
return "the claim's class is not resolved"
323+
case addr.Spec.ClassName != className:
324+
return fmt.Sprintf("its class %q is not the claim's class %q", addr.Spec.ClassName, className)
325+
case !slices.Contains(requestedFamilies(claim), family):
326+
return fmt.Sprintf("the claim does not request family %q", family)
327+
}
328+
for _, h := range held {
329+
if familyOf(h.Spec.Address) == family {
330+
return fmt.Sprintf("the claim already holds %s address %s", family, h.Name)
331+
}
332+
}
333+
return ""
267334
}
268335

269336
// addressesBoundTo lists live addresses whose claimRef names this claim and
@@ -302,6 +369,21 @@ func requestedFamilies(claim *localv1alpha1.IPAddressClaim) []localv1alpha1.Addr
302369
}
303370
}
304371

372+
// unboundFamilies reports which requested families no bound address covers,
373+
// in any phase. A Lost address still covers its family: the claim keeps it
374+
// and waits for it to be restored or deleted rather than binding another.
375+
func unboundFamilies(claim *localv1alpha1.IPAddressClaim, addrs []localv1alpha1.IPAddress) []localv1alpha1.AddressFamily {
376+
var unbound []localv1alpha1.AddressFamily
377+
for _, family := range requestedFamilies(claim) {
378+
if !slices.ContainsFunc(addrs, func(addr localv1alpha1.IPAddress) bool {
379+
return familyOf(addr.Spec.Address) == family
380+
}) {
381+
unbound = append(unbound, family)
382+
}
383+
}
384+
return unbound
385+
}
386+
305387
// missingFamilies reports which requested families no bound address
306388
// satisfies. An address in phase Lost does not satisfy its family.
307389
func missingFamilies(claim *localv1alpha1.IPAddressClaim, addrs []localv1alpha1.IPAddress) []localv1alpha1.AddressFamily {

0 commit comments

Comments
 (0)