0

如果我将最大的 Json 发送到 Kafka 服务器，它会显示这种错误，我该如何增加message.max.bytes=15728640和replica.fetch.max.bytes=15728640在 Kafka 中。我试图增加字节级别，如下所示它不起作用

套接字服务器使用的发送缓冲区（SO_SNDBUF）

socket.send.buffer.bytes=15728640

套接字服务器使用的接收缓冲区（SO_RCVBUF）

socket.receive.buffer.bytes=15728640

错误：=>

[2022-01-06 12:36:51,281] [9015] [ERROR] [^-App]: Crashed reason=ProducerSendError("Error while sending: MessageSizeTooLargeError('The message is 6677420 bytes when serialized which is larger than the maximum request size you have configured with the max_request_size configuration',)",) 
Traceback (most recent call last):
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/transport/drivers/aiokafka.py", line 1059, in send
    transactional_id=transactional_id,
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/aiokafka/producer/producer.py", line 310, in send
    key_bytes, value_bytes = self._serialize(topic, key, value)
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/aiokafka/producer/producer.py", line 231, in _serialize
    " max_request_size configuration" % message_size)
kafka.errors.MessageSizeTooLargeError: [Error 10] MessageSizeTooLargeError: The message is 6677420 bytes when serialized which is larger than the maximum request size you have configured with the max_request_size configuration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/mode/services.py", line 779, in _execute_task
    await task
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/app/base.py", line 941, in _wrapped
    return await task()
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/app/base.py", line 991, in around_timer
    await fun(*args)
  File "/home/twilightuser/faust_library/producer.py", line 14, in my_send
    await topic.send(value=value)
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/topics.py", line 193, in send
    callback=callback,
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/channels.py", line 303, in _send_now
    schema, key_serializer, value_serializer, callback))
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/topics.py", line 417, in publish_message
    headers=headers,
  File "/home/twilightuser/faust_library/venv/lib/python3.6/site-packages/faust/transport/drivers/aiokafka.py", line 1062, in send
    raise ProducerSendError(f'Error while sending: {exc!r}') from exc
faust.exceptions.ProducerSendError: Error while sending: MessageSizeTooLargeError('The message is 6677420 bytes when serialized which is larger than the maximum request size you have configured with the max_request_size configuration',)
4

0 回答 0