Skip to content

Fixed ListConsumerGroupOffsets not fetching offsets for all the topic s in a group with Apache Kafka version below 2.4.0.#4346

Merged
Emanuele Sabellico (emasab) merged 2 commits intomasterfrom
dev_fix-list-consumer-group-offsets-below-kafka-2.3.0
Aug 14, 2023
Merged

Fixed ListConsumerGroupOffsets not fetching offsets for all the topic s in a group with Apache Kafka version below 2.4.0.#4346
Emanuele Sabellico (emasab) merged 2 commits intomasterfrom
dev_fix-list-consumer-group-offsets-below-kafka-2.3.0

Conversation

@pranavrth
Copy link
Copy Markdown
Member

@pranavrth Pranav Rathi (pranavrth) commented Jul 11, 2023

@pranavrth Pranav Rathi (pranavrth) force-pushed the dev_fix-list-consumer-group-offsets-below-kafka-2.3.0 branch from b21d4c3 to 95a0bd9 Compare August 7, 2023 14:03
@pranavrth
Fixed ListConsumerGroupOffsets not fetching offsets for all the topic…
734b2e4 
…s in a group with Apache Kafka version below 2.4.0
@pranavrth Pranav Rathi (pranavrth) force-pushed the dev_fix-list-consumer-group-offsets-below-kafka-2.3.0 branch from 95a0bd9 to 734b2e4 Compare August 8, 2023 10:58
emasab
Emanuele Sabellico (emasab) requested changes Aug 8, 2023
View reviewed changes
Comment thread .gitignore Outdated
@emasab Emanuele Sabellico (emasab) deleted the dev_fix-list-consumer-group-offsets-below-kafka-2.3.0 branch August 14, 2023 13:01
@gkousouris
Copy link
Copy Markdown

Georgios Kousouris (gkousouris) commented Sep 21, 2023

Hi Pranav Rathi (@pranavrth) Emanuele Sabellico (@emasab), thanks for this! Just stumbled on this error and this made it clear what the problem is. Is there a plan on releasing this ? I do not see version v2.2.1 currently under Releases.

Milind L (milindl) added a commit that referenced this pull request Sep 27, 2023
@milindl
[admin] KIP-430: Add authorized operations to describe API
dfbacdc 
commit bee8d6a80b81a8f31c823750e3e504537cd954ac
Author: Milind L <[email protected]>
Date:   Wed Sep 27 11:40:35 2023 +0530

    Squashed commit of the following:

    commit 73cf87c
    Author: Emanuele Sabellico <[email protected]>
    Date:   Tue Sep 26 14:48:57 2023 +0200

        Fix segfault if assignor state is NULL, (#4381)

        e.g. a destroy happens before the first assignment.
        Only affects the cooperative-sticky assignor

    commit 1cb8009
    Author: Emanuele Sabellico <[email protected]>
    Date:   Thu Sep 21 10:04:07 2023 +0200

        Add destroy call when leader epoch is stale (#4429)

        and partition is in state active

    commit 49f180a
    Author: Pranav Rathi <[email protected]>
    Date:   Tue Aug 15 23:47:28 2023 +0530

        Added new environment variable to tests - TESTS_SKIP_BEFORE (#4317)

    commit ff1aaf5
    Author: Pranav Rathi <[email protected]>
    Date:   Mon Aug 14 18:31:48 2023 +0530

        Fix ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0. (#4346)

commit f6f0a62
Author: Milind L <[email protected]>
Date:   Mon Sep 25 17:00:05 2023 +0530

    Fix case where broker RPC might not contain authorized operations

commit 63e0748
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:14:47 2023 +0530

    Fix possible initialization issue

commit 7205354
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:02:37 2023 +0530

    Differentiate between 0 and unrequested authorized ops

commit adc0c4f
Author: Milind L <[email protected]>
Date:   Mon Sep 25 09:35:36 2023 +0530

    Avoid malloc(0)

commit 4257dc9
Author: Milind L <[email protected]>
Date:   Fri Sep 22 10:35:33 2023 +0530

    Fix CI issues

commit 0e05dea
Author: Milind L <[email protected]>
Date:   Fri Sep 22 07:45:53 2023 +0530

    Change TopicCollection create method name

commit 74d98dc
Author: Milind L <[email protected]>
Date:   Thu Sep 21 15:02:51 2023 +0530

    Fix warning and add rack to cluster nodes

commit fd14d24
Author: Milind L <[email protected]>
Date:   Fri Sep 15 12:38:56 2023 +0530

    Retain topic ordering for DescribeTopics

commit d90bb98
Author: Milind L <[email protected]>
Date:   Thu Sep 14 12:41:10 2023 +0530

    Change rack_id to rack in Node public API

commit 0233c0f
Author: Milind L <[email protected]>
Date:   Wed Sep 6 15:50:37 2023 +0530

    Fix typo in result type

commit e6d4d63
Author: Milind L <[email protected]>
Date:   Mon Sep 4 17:26:44 2023 +0530

    Add brokers_sorted to internal metadata

commit 2c7f5e1
Author: Milind L <[email protected]>
Date:   Mon Sep 4 16:53:20 2023 +0530

    Address comments: AuthorizedOperations_parse uses array

commit 9a6ef10
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:47:41 2023 +0530

    Address comments: cosmetic and copyright changes

commit beccb0c
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:10:54 2023 +0530

    Address comments: Cosmetic changes

commit 6239372
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:59:20 2023 +0530

    Address comment and RD_IF_FREE fixes

commit bdd8a4b
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:55:26 2023 +0530

    Fix style and refactor MetadataRequest into op/cb

commit 53cd983
Author: Milind L <[email protected]>
Date:   Thu Aug 31 17:47:54 2023 +0530

    Address DescribeCluster/DescribeConsumerGroup changes, fix memleak

commit 59ab5bb
Author: Milind L <[email protected]>
Date:   Thu Aug 31 13:57:51 2023 +0530

    Address comments for DescribeTopics API

commit 85f7f91
Author: Milind L <[email protected]>
Date:   Fri Aug 11 15:08:51 2023 +0530

    Get rid of stray conflict marker

commit bd7f5a7
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:27:39 2023 +0530

    Style fixes

commit ef3f480
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:26:48 2023 +0530

    Fix examples

commit 0ed7250
Merge: 61840ab 07262c4
Author: Milind L <[email protected]>
Date:   Wed Aug 9 14:08:09 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 61840ab
Author: Milind L <[email protected]>
Date:   Wed Aug 9 13:53:33 2023 +0530

    Address review comments: Part 2

commit 45780f8
Author: Milind L <[email protected]>
Date:   Wed Aug 9 10:13:23 2023 +0530

    Address review comments: Part 1

commit 0bc97b3
Author: Milind L <[email protected]>
Date:   Wed Aug 2 11:31:37 2023 +0530

    Break into functions, update Supported KIPS

commit 6f8aa0d
Author: Milind L <[email protected]>
Date:   Wed Aug 2 10:40:36 2023 +0530

    Fix 0080 after addressing comments

commit 8d8a473
Author: Milind L <[email protected]>
Date:   Tue Aug 1 19:59:39 2023 +0530

    Address review comments: Refactor MetadataRequest into two parts

commit 28615cd
Author: Milind L <[email protected]>
Date:   Tue Aug 1 18:10:46 2023 +0530

    Address review comments - Part 2

commit e01ad41
Author: Milind L <[email protected]>
Date:   Tue Aug 1 15:09:46 2023 +0530

    Address review comments - Part 1

commit 3215a90
Author: Milind L <[email protected]>
Date:   Tue Aug 1 09:50:59 2023 +0530

    Fixed changelog

commit 4a4f503
Author: Milind L <[email protected]>
Date:   Mon Jul 31 20:41:57 2023 +0530

    Remove fanout from DescribeTopics

commit 9787d6c
Merge: 8c2b4d4 c07a335
Author: Milind L <[email protected]>
Date:   Mon Jul 31 14:47:43 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 8c2b4d4
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:44:24 2023 +0530

    Fix cluster_id allocation not on tmpabuf

commit 4a4d9df
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:29:26 2023 +0530

    Revert to macro to fix msvc compilation

commit 7fff6eb
Merge: 3a9a340 aa50e52
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:18:39 2023 +0530

    Remove warnings and add CHANGELOG

commit 3a9a340
Author: Milind L <[email protected]>
Date:   Sat Jun 24 15:36:22 2023 +0530

    Change API for common stuff

commit 6ddbaba
Merge: f497307 e52aa3b
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:24:15 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit f497307
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:22:39 2023 +0530

    Fix tests and change API to be similar to existing ones

commit 3fec50a
Author: Milind L <[email protected]>
Date:   Thu Jun 15 09:52:33 2023 +0530

    Fix style and docs issues

commit 99a368d
Merge: 7c15746 865cf60
Author: Ruchir Jain <[email protected]>
Date:   Wed Jun 14 18:52:29 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 7c15746
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:49:32 2023 +0530

    bug fix: request NULL, force_racks in MetadataReq

commit 88707a0
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:11:47 2023 +0530

    bug fixes and include auth ops false default

commit 060d694
Merge: 0e16f98 966b63d
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 15:54:10 2023 +0530

    Merge 'master' of librdkafka into dev_kip430_cp

commit 0e16f98
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 10:04:40 2023 +0530

    style-fix

commit b9fea3c
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 09:33:57 2023 +0530

    ClusterDescription destroy having double free bug removed

commit 7d512bf
Merge: 830d9b7 2c6dd0a
Author: Ruchir Jain <[email protected]>
Date:   Tue Apr 11 09:32:39 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 830d9b7
Author: jainruchir <[email protected]>
Date:   Thu Mar 30 14:36:25 2023 +0530

    bug fix for adminoptions

commit 0bfbf67
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 16:05:22 2023 +0530

    dev_KIP430 branch merged

commit c28152a
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 15:05:36 2023 +0530

    changes to describeCG tests

commit d2c902b
Author: jainruchir <[email protected]>
Date:   Tue Feb 21 15:55:50 2023 +0530

    added test for include auth ops in 0081 initial commit

commit 3a79ed6
Author: jainruchir <[email protected]>
Date:   Fri Feb 17 14:25:37 2023 +0530

    KIP-430 initial changes without checking tests

commit d0e2156
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 10:01:08 2023 +0530

    bug fix when include authorized ops is 0 desc Topics/Cluster

commit a542f4d
Author: jainruchir <[email protected]>
Date:   Wed Mar 15 10:25:40 2023 +0530

    integration tests for describecluster

commit 8df6f62
Author: jainruchir <[email protected]>
Date:   Tue Mar 14 14:04:44 2023 +0530

    describe topics and cluster

commit 069b8e8
Author: jainruchir <[email protected]>
Date:   Fri Feb 24 14:32:49 2023 +0530

    include authorized topic/cluster ops initial draft
Milind L (milindl) added a commit that referenced this pull request Sep 27, 2023
@milindl
[admin] KIP-430: Add authorized operations to describe API
efbb966 
commit bee8d6a80b81a8f31c823750e3e504537cd954ac
Author: Milind L <[email protected]>
Date:   Wed Sep 27 11:40:35 2023 +0530

    Squashed commit of the following:

    commit 73cf87c
    Author: Emanuele Sabellico <[email protected]>
    Date:   Tue Sep 26 14:48:57 2023 +0200

        Fix segfault if assignor state is NULL, (#4381)

        e.g. a destroy happens before the first assignment.
        Only affects the cooperative-sticky assignor

    commit 1cb8009
    Author: Emanuele Sabellico <[email protected]>
    Date:   Thu Sep 21 10:04:07 2023 +0200

        Add destroy call when leader epoch is stale (#4429)

        and partition is in state active

    commit 49f180a
    Author: Pranav Rathi <[email protected]>
    Date:   Tue Aug 15 23:47:28 2023 +0530

        Added new environment variable to tests - TESTS_SKIP_BEFORE (#4317)

    commit ff1aaf5
    Author: Pranav Rathi <[email protected]>
    Date:   Mon Aug 14 18:31:48 2023 +0530

        Fix ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0. (#4346)

commit f6f0a62
Author: Milind L <[email protected]>
Date:   Mon Sep 25 17:00:05 2023 +0530

    Fix case where broker RPC might not contain authorized operations

commit 63e0748
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:14:47 2023 +0530

    Fix possible initialization issue

commit 7205354
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:02:37 2023 +0530

    Differentiate between 0 and unrequested authorized ops

commit adc0c4f
Author: Milind L <[email protected]>
Date:   Mon Sep 25 09:35:36 2023 +0530

    Avoid malloc(0)

commit 4257dc9
Author: Milind L <[email protected]>
Date:   Fri Sep 22 10:35:33 2023 +0530

    Fix CI issues

commit 0e05dea
Author: Milind L <[email protected]>
Date:   Fri Sep 22 07:45:53 2023 +0530

    Change TopicCollection create method name

commit 74d98dc
Author: Milind L <[email protected]>
Date:   Thu Sep 21 15:02:51 2023 +0530

    Fix warning and add rack to cluster nodes

commit fd14d24
Author: Milind L <[email protected]>
Date:   Fri Sep 15 12:38:56 2023 +0530

    Retain topic ordering for DescribeTopics

commit d90bb98
Author: Milind L <[email protected]>
Date:   Thu Sep 14 12:41:10 2023 +0530

    Change rack_id to rack in Node public API

commit 0233c0f
Author: Milind L <[email protected]>
Date:   Wed Sep 6 15:50:37 2023 +0530

    Fix typo in result type

commit e6d4d63
Author: Milind L <[email protected]>
Date:   Mon Sep 4 17:26:44 2023 +0530

    Add brokers_sorted to internal metadata

commit 2c7f5e1
Author: Milind L <[email protected]>
Date:   Mon Sep 4 16:53:20 2023 +0530

    Address comments: AuthorizedOperations_parse uses array

commit 9a6ef10
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:47:41 2023 +0530

    Address comments: cosmetic and copyright changes

commit beccb0c
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:10:54 2023 +0530

    Address comments: Cosmetic changes

commit 6239372
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:59:20 2023 +0530

    Address comment and RD_IF_FREE fixes

commit bdd8a4b
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:55:26 2023 +0530

    Fix style and refactor MetadataRequest into op/cb

commit 53cd983
Author: Milind L <[email protected]>
Date:   Thu Aug 31 17:47:54 2023 +0530

    Address DescribeCluster/DescribeConsumerGroup changes, fix memleak

commit 59ab5bb
Author: Milind L <[email protected]>
Date:   Thu Aug 31 13:57:51 2023 +0530

    Address comments for DescribeTopics API

commit 85f7f91
Author: Milind L <[email protected]>
Date:   Fri Aug 11 15:08:51 2023 +0530

    Get rid of stray conflict marker

commit bd7f5a7
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:27:39 2023 +0530

    Style fixes

commit ef3f480
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:26:48 2023 +0530

    Fix examples

commit 0ed7250
Merge: 61840ab 07262c4
Author: Milind L <[email protected]>
Date:   Wed Aug 9 14:08:09 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 61840ab
Author: Milind L <[email protected]>
Date:   Wed Aug 9 13:53:33 2023 +0530

    Address review comments: Part 2

commit 45780f8
Author: Milind L <[email protected]>
Date:   Wed Aug 9 10:13:23 2023 +0530

    Address review comments: Part 1

commit 0bc97b3
Author: Milind L <[email protected]>
Date:   Wed Aug 2 11:31:37 2023 +0530

    Break into functions, update Supported KIPS

commit 6f8aa0d
Author: Milind L <[email protected]>
Date:   Wed Aug 2 10:40:36 2023 +0530

    Fix 0080 after addressing comments

commit 8d8a473
Author: Milind L <[email protected]>
Date:   Tue Aug 1 19:59:39 2023 +0530

    Address review comments: Refactor MetadataRequest into two parts

commit 28615cd
Author: Milind L <[email protected]>
Date:   Tue Aug 1 18:10:46 2023 +0530

    Address review comments - Part 2

commit e01ad41
Author: Milind L <[email protected]>
Date:   Tue Aug 1 15:09:46 2023 +0530

    Address review comments - Part 1

commit 3215a90
Author: Milind L <[email protected]>
Date:   Tue Aug 1 09:50:59 2023 +0530

    Fixed changelog

commit 4a4f503
Author: Milind L <[email protected]>
Date:   Mon Jul 31 20:41:57 2023 +0530

    Remove fanout from DescribeTopics

commit 9787d6c
Merge: 8c2b4d4 c07a335
Author: Milind L <[email protected]>
Date:   Mon Jul 31 14:47:43 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 8c2b4d4
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:44:24 2023 +0530

    Fix cluster_id allocation not on tmpabuf

commit 4a4d9df
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:29:26 2023 +0530

    Revert to macro to fix msvc compilation

commit 7fff6eb
Merge: 3a9a340 aa50e52
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:18:39 2023 +0530

    Remove warnings and add CHANGELOG

commit 3a9a340
Author: Milind L <[email protected]>
Date:   Sat Jun 24 15:36:22 2023 +0530

    Change API for common stuff

commit 6ddbaba
Merge: f497307 e52aa3b
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:24:15 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit f497307
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:22:39 2023 +0530

    Fix tests and change API to be similar to existing ones

commit 3fec50a
Author: Milind L <[email protected]>
Date:   Thu Jun 15 09:52:33 2023 +0530

    Fix style and docs issues

commit 99a368d
Merge: 7c15746 865cf60
Author: Ruchir Jain <[email protected]>
Date:   Wed Jun 14 18:52:29 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 7c15746
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:49:32 2023 +0530

    bug fix: request NULL, force_racks in MetadataReq

commit 88707a0
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:11:47 2023 +0530

    bug fixes and include auth ops false default

commit 060d694
Merge: 0e16f98 966b63d
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 15:54:10 2023 +0530

    Merge 'master' of librdkafka into dev_kip430_cp

commit 0e16f98
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 10:04:40 2023 +0530

    style-fix

commit b9fea3c
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 09:33:57 2023 +0530

    ClusterDescription destroy having double free bug removed

commit 7d512bf
Merge: 830d9b7 2c6dd0a
Author: Ruchir Jain <[email protected]>
Date:   Tue Apr 11 09:32:39 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 830d9b7
Author: jainruchir <[email protected]>
Date:   Thu Mar 30 14:36:25 2023 +0530

    bug fix for adminoptions

commit 0bfbf67
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 16:05:22 2023 +0530

    dev_KIP430 branch merged

commit c28152a
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 15:05:36 2023 +0530

    changes to describeCG tests

commit d2c902b
Author: jainruchir <[email protected]>
Date:   Tue Feb 21 15:55:50 2023 +0530

    added test for include auth ops in 0081 initial commit

commit 3a79ed6
Author: jainruchir <[email protected]>
Date:   Fri Feb 17 14:25:37 2023 +0530

    KIP-430 initial changes without checking tests

commit d0e2156
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 10:01:08 2023 +0530

    bug fix when include authorized ops is 0 desc Topics/Cluster

commit a542f4d
Author: jainruchir <[email protected]>
Date:   Wed Mar 15 10:25:40 2023 +0530

    integration tests for describecluster

commit 8df6f62
Author: jainruchir <[email protected]>
Date:   Tue Mar 14 14:04:44 2023 +0530

    describe topics and cluster

commit 069b8e8
Author: jainruchir <[email protected]>
Date:   Fri Feb 24 14:32:49 2023 +0530

    include authorized topic/cluster ops initial draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@emasab Emanuele Sabellico (emasab) emasab approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

3 participants

@pranavrth @gkousouris @emasab