Skip to content

Kafka with num_consumers > available partitions in topic = performance degradation #11218

@filimonov

Description

@filimonov

Consuming speed of topic with single partition:
num_consumers = 1: 262144 rows / sec.
num_consumers = 2: 43018 rows / sec. (6 times slower!)

It's a misconfiguration actually, but it seems possible to avoid that.

Metadata

Metadata

Assignees

Labels

comp-message-queuesMessage queue integrations (Kafka, RabbitMQ, NATS table engines for stream ingestion/egress).performance

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions