1

我正在玩 Kafka 主题和 rest-proxy,遇到了我不理解的行为。

我有 1 个 zookeeper、1 个 kafka 代理、1 个模式注册表和 1 个 rest-proxy 实例。然后创建具有 1 个分区和 2 个从中读取的消费者的主题。

我使用以下命令创建了一个主题:

kafka-topics --create --if-not-exists --zookeeper localhost:32181 --partitions 4 --replication-factor 1 --topic my-ttopic

然后我试图从两个消费者那里阅读。在创建第二个消费者并尝试使用它进行读取时,触发了一些重新平衡并且读取只是挂起。Kafka Rest 依次产生大量的日志,如下所示:

[2020-10-11 09:29:30,645] INFO [Consumer clientId=consumer-grps-2, groupId=grps] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2020-10-11 09:29:30,649] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Join group failed with org.apache.kafka.common.errors.MemberIdRequiredException: The group member needs to have a valid member id before actually entering a consumer group. (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2020-10-11 09:29:30,649] INFO [Consumer clientId=consumer-grps-2, groupId=grps] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2020-10-11 09:29:30,650] INFO [GroupCoordinator 2]: Preparing to rebalance group grps in state PreparingRebalance with old generation 1 (__consumer_offsets-14) (reason: Adding new member consumer-grps-2-0b19385b-df80-4d11-8eaa-7c313354a131 with group instance id None) (kafka.coordinator.group.GroupCoordinator)
[2020-10-10 20:44:51,713] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:44:54,726] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:44:57,739] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:00,752] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:03,766] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:06,780] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:09,793] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:12,806] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:15,818] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:18,829] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:21,841] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:24,854] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)                                                                                                                                                                                                    
[2020-10-10 20:45:27,867] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2020-10-10 20:45:30,880] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2020-10-10 20:45:33,893] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2020-10-10 20:45:36,905] INFO [Consumer clientId=consumer-grps-2, groupId=grps] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)

你能就问题可能是什么提供任何建议吗?

4

0 回答 0