apache-kafka - How to set the max queue time in Kafka 0.9 producer?

Question

How to setup the max queue time in Kafka v0.9?

My understanding is that the Kafka producer will try to buffer the messages in a queue to a certain size and then send the messages in the queue as batch to broker. However in Kafka v0.8, the configuration of "queue.buffering.max.ms" will send the messages in the queue when it reaches the time limit, even if they are smaller than the buffer size.

The batching can be configured to accumulate no more than a fixed number of messages and to wait no longer than some fixed latency bound (say 64k or 10 ms).

from Kafka 0.9 document: http://kafka.apache.org/documentation.html#design_asyncsend. (This section is the same as Kafka 0.8.2)

But I didn't find the configuration of "queue.buffering.max.ms" or equivalent in the v0.9 producer configuration section http://kafka.apache.org/documentation.html#producerconfigs

Is there still a way to configure this in Kafka v0.9 or I have to call KafkaProducer.close() or KafkaProducer.flush() to do so?

Answer 1

对于该配置使用batch.size和linger.ms

Answer 2

您需要使用batch.sizeand queue.time，而不是linger.ms如接受的答案中所述。

linger.ms是发送一批请求之前的延迟，但默认为零，而不是发送一批之前的最长时间。

有关详细信息，请参阅：https ://kafka.apache.org/documentation。