Skip to content

skip nil values in Memberlist WatchPrefix#7429

Merged
SungJin1212 merged 2 commits into
cortexproject:masterfrom
SungJin1212:fix-ha-tracker-ReplicaDesc-nil
Apr 17, 2026
Merged

skip nil values in Memberlist WatchPrefix#7429
SungJin1212 merged 2 commits into
cortexproject:masterfrom
SungJin1212:fix-ha-tracker-ReplicaDesc-nil

Conversation

@SungJin1212

Copy link
Copy Markdown
Member

This PR adds a skip in WatchPrefix when a key is deleted, preventing a panic in the HA tracker caused by a failed type assertion on a nil interface value.

Related panic log

panic: interface conversion: interface {} is nil, not *ha.ReplicaDesc

goroutine 284 [running]:
github.com/cortexproject/cortex/pkg/ha.(*HATracker).loop.func3({0xc000c68ebb?, 0xc2703e459cea4e8f?}, {0x0?, 0x0?})
/__w/cortex/cortex/pkg/ha/ha_tracker.go:437 +0x629
github.com/cortexproject/cortex/pkg/ring/kv.(*prefixedKVClient).WatchPrefix.func1({0xc000c68eb0?, 0x41?}, {0x0?, 0x0?})
/__w/cortex/cortex/pkg/ring/kv/prefix.go:52 +0xb1
github.com/cortexproject/cortex/pkg/ring/kv/memberlist.(*KV).WatchPrefix(0xc000607808, {0x415fa48, 0xc000708e40}, {0xc000938060, 0xb}, {0x416ee40, 0xc00097f320}, 0xc000bf6090)
/__w/cortex/cortex/pkg/ring/kv/memberlist/memberlist_client.go:841 +0x4c7
github.com/cortexproject/cortex/pkg/ring/kv/memberlist.(*Client).WatchPrefix(0xc00097f338, {0x415fa48, 0xc000708e40}, {0xc000938060, 0xb}, 0xc000bf6090)
/__w/cortex/cortex/pkg/ring/kv/memberlist/memberlist_client.go:113 +0x6a

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
  • docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags

Signed-off-by: SungJin1212 <[email protected]>
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 16, 2026
@SungJin1212
SungJin1212 merged commit 3c89123 into cortexproject:master Apr 17, 2026
62 of 65 checks passed
Comment thread pkg/ring/kv/memberlist/memberlist_client.go
friedrichg pushed a commit that referenced this pull request Apr 17, 2026
* skip nil values in Memberlist WatchPrefix

Signed-off-by: SungJin1212 <[email protected]>

* fix lint

Signed-off-by: SungJin1212 <[email protected]>

---------

Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>
friedrichg added a commit that referenced this pull request Apr 17, 2026
* Memberlist cas error code false positive (#7408)

* use errors.As in getCasErrorCode to unwrap memberlist errors

Signed-off-by: SungJin1212 <[email protected]>

* fix test

Signed-off-by: SungJin1212 <[email protected]>

---------

Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>

* Fix nil when ingesterQueryMaxAttempts > 1 (#7369)

* Trigger nil with test

Signed-off-by: Friedrich Gonzalez <[email protected]>

* Fix nil results

Signed-off-by: Friedrich Gonzalez <[email protected]>

* fix changelog

Signed-off-by: Friedrich Gonzalez <[email protected]>

---------

Signed-off-by: Friedrich Gonzalez <[email protected]>

* fix: alertmanager user config disappearing when ring is unreachable  (#7372)

* Fix multitenant alertmanager user config disappearing when ring is unreachable

Signed-off-by: Kishore K G <[email protected]>

* Add change log

Signed-off-by: Kishore K G <[email protected]>

* format multitenant

Signed-off-by: Kishore K G <[email protected]>

* fix pr number

Signed-off-by: Kishore K G <[email protected]>

* use ErrNotFound for error validation in unit test

Signed-off-by: kishorekg1999 <[email protected]>

---------

Signed-off-by: Kishore K G <[email protected]>
Signed-off-by: kishorekg1999 <[email protected]>
Signed-off-by: kishorekg1999 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>

* Clean Symbol Tables (#7373)

Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>

* Fix root cause of nil return in queryWithRetry and labelsWithRetry (#7375)

Signed-off-by: Friedrich Gonzalez <[email protected]>

* fix regex resolver match 0 or 1 tenant bug (#7424)

* fix regex resolver match 0 or 1 tenant bug

Signed-off-by: SungJin1212 <[email protected]>

* fix test

Signed-off-by: SungJin1212 <[email protected]>

---------

Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>

* skip nil values in Memberlist WatchPrefix (#7429)

* skip nil values in Memberlist WatchPrefix

Signed-off-by: SungJin1212 <[email protected]>

* fix lint

Signed-off-by: SungJin1212 <[email protected]>

---------

Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>

* Remove duplicate CHANGELOG entry for #7373

Signed-off-by: Friedrich Gonzalez <[email protected]>

* Fix integration test flag name for release-1.21

The cherry-pick of #7424 brought the master flag name
-limits.query-ingesters-within, but release-1.21 still uses
-querier.query-ingesters-within (renamed in #7160, master-only).

Signed-off-by: Friedrich Gonzalez <[email protected]>

---------

Signed-off-by: SungJin1212 <[email protected]>
Signed-off-by: Friedrich Gonzalez <[email protected]>
Signed-off-by: Kishore K G <[email protected]>
Signed-off-by: kishorekg1999 <[email protected]>
Signed-off-by: kishorekg1999 <[email protected]>
Co-authored-by: SungJin1212 <[email protected]>
Co-authored-by: kishorekg1999 <[email protected]>
CharlieTLe added a commit to CharlieTLe/cortex that referenced this pull request Apr 17, 2026
CI run 24576043912: all jobs passed after merging upstream/master
(cortexproject#7424 regex resolver fix, cortexproject#7429 memberlist WatchPrefix fix).

Signed-off-by: Charlie Le <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Signed-off-by: Charlie Le <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants