Skip to content

Implementation of Ready status condition - #26

Merged
Kirill Ilin (sircthulhu) merged 3 commits into
mainfrom
24-impl-ready-check
Mar 18, 2024
Merged

Kirill Ilin (sircthulhu) merged 3 commits into
mainfrom
24-impl-ready-check

Conversation

@sircthulhu

@sircthulhu Kirill Ilin (sircthulhu) commented Mar 15, 2024 •

Copy link
Copy Markdown
Collaborator

Update cluster Ready status according to StatefulSet status and update ConfigMap cluster state to existing after first time STS is ready
fixes #24

@sircthulhu Kirill Ilin (sircthulhu) linked an issue Mar 15, 2024 that may be closed by this pull request

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@sircthulhu Kirill Ilin (sircthulhu) added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Mar 18, 2024
@sircthulhu
Kirill Ilin (sircthulhu) deleted the 24-impl-ready-check branch March 27, 2024 08:33
Terry Tang (saymololy) pushed a commit to saymololy/etcd-operator that referenced this pull request Nov 2, 2026
…tack#26)

discoverMemberID built its endpoint list from voter peers and then
*unconditionally* appended the member's own client URL as a "self
last" fallback. The comment claimed the learner rejection "only fires
when we route past a voter to a learner", but clientv3's balancer
does not honor endpoint order — it round-robins. So when the member
being discovered is itself a freshly-added learner and a voter peer
is present, the balancer can land MemberList on our own etcd, which
returns "rpc not supported for learner". With a 5s context budget
that wedges discovery: MemberID never gets populated, MemberReady
never flips true, the cluster controller's allMembersReady gate never
opens, the scale-up stalls past progressDeadline, and the next member
is never added.

Observed on dev4 during the cert-manager TLS smoke: a 3-replica
cluster stuck at 2 members (one voter + one perpetual learner) with
the operator logging MemberList against the learner's own endpoint.
Not TLS-specific — any multi-member scale-up where a learner's pod
restarts before promotion (re-triggering discovery) can hit it.

Fix: make self a true fallback — append it only when no voter peer is
available (single-node bootstrap, or no other voter Ready yet),
mirroring memberEndpoints' voter-or-fallback shape. When a voter peer
exists, discovery dials only voters and resolves the learner's ID
from their MemberList by name / peer URL.

Tests: TestDiscoverMemberID_ExcludesSelfWhenVoterAvailable reproduces
the wedge (target IS the learner, one voter peer) and asserts self is
excluded — it fails on the old code with the exact two-endpoint list.
TestDiscoverMemberID_FallsBackToSelfWhenNoVoter pins the single-node
fallback the tightening must preserve.

Signed-off-by: Timofei Larkin <[email protected]>
Co-authored-by: Claude Opus 4.7 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Ready status condition

4 participants