Skip to content

Conversation

@pprovenzano
Copy link
Contributor

@pprovenzano pprovenzano commented May 24, 2023

Create lists of all the objects in the snapshot and add to the lists the objects that
are only in ZK. Then reverse migrate as appropriate.

Added test to validate for SCRAM and Quota

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.

Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

…the objects that

are only in ZK. Then reverse migrate as appropriate
@mumrah
Copy link
Member

mumrah commented May 26, 2023

The jira for this is KAFKA-15017 btw

@pprovenzano pprovenzano changed the title [WIP] Fix snapshot load during dual write. KAFKA-15017 Fix snapshot load in dual write mode for Quotas and SCRAM May 31, 2023
@mumrah mumrah added the kraft label May 31, 2023
Copy link
Member

@mumrah mumrah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pprovenzano, LGTM

@mumrah mumrah merged commit 731c8c9 into apache:trunk May 31, 2023
mumrah pushed a commit that referenced this pull request Jun 1, 2023
… SCRAM (#13757)

This patch fixes the case where a ClientQuota or SCRAM credential was added in KRaft, but not written back to ZK. This missed write only occurred when handling a KRaft snapshot. If the changed quota was processed in a metadata delta (which is the typical case), it would be written to ZK.

Reviewers: David Arthur <[email protected]>
emissionnebula added a commit to confluentinc/kafka that referenced this pull request Jun 16, 2023
…tream-3.5

* commit 'c2f6f29ca6e1306ac77ec726bac4cd09bd1aa80b': (76 commits)
  KAFKA-15019: Improve handling of broker heartbeat timeouts (apache#13759)
  KAFKA-15003: Fix ZK sync logic for partition assignments (apache#13735)
  MINOR: Add 3.5 upgrade steps for ZK and KRaft (apache#13792)
  KAFKA-15010 ZK migration failover support (apache#13758)
  KAFKA-15017 Fix snapshot load in dual write mode for ClientQuotas and SCRAM  (apache#13757)
  MINOR: Update LICENSE-binary following snappy upgrade (apache#13791)
  Upgrade to snappy v1.1.10.0 (apache#13786)
  KAFKA-15004: Fix configuration dual-write during migration (apache#13767)
  KAFKA-8713: JsonConverter replace.null.with.default should prevent emitting default for Struct fields (apache#13781)
  KAFKA-14996: Handle overly large user operations on the kcontroller (apache#13742)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants