Skip to content

fix: make kafka producer usable in cluster setup#3189

Merged
limingxinleo merged 1 commit intohyperf:masterfrom
Reasno:kafka
Jan 24, 2021
Merged

fix: make kafka producer usable in cluster setup#3189
limingxinleo merged 1 commit intohyperf:masterfrom
Reasno:kafka

Conversation

@Reasno
Copy link
Copy Markdown
Member

@Reasno Reasno commented Jan 23, 2021

简单解释一下这次变更;

  1. 在多副本集群环境下,生产消息必须发送给分区的leader,而原API要求用户手写broker,否则就随机,那么当随机到的broker不是leader是就会报错。修改为:无需用户指定broker,自动根据分区和meta信息推断。作为一个生产者,如何确定正确的brokerId? swoole/phpkafka#15
  2. 原API发送信息时默认指定分区0,我认为不妥,修改为默认按照key或value取模。
  3. 原API不能当做单例使用,在server环境下每次使用都需要重新拉取meta信息,消耗较大。修改为单例模式。
  4. 原SwooleSocket会在关闭连接时抛出致命错误,这里fork并处理了一下。socket关闭时程序异常退出。 swoole/phpkafka#16

@Reasno Reasno requested a review from limingxinleo January 23, 2021 15:55
@limingxinleo limingxinleo merged commit d8a4b1f into hyperf:master Jan 24, 2021
@Reasno Reasno deleted the kafka branch January 24, 2021 10:34
@Yurunsoft
Copy link
Copy Markdown
Contributor

现在 longyan/phpkafka 已经支持了多 broker 下生产和消费,并且内置支持了分区策略。

参数可以看这边:https://github.com/longyan/phpkafka/blob/master/doc/producer.md

默认分区策略:

如果指定了分区,则使用指定的分区;

如果没有指定分区,但指定了 key,会根据 key 的哈希值(crc32)选择分区;

如果没有指定分区,也没有指定 key,会使用轮询策略。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants