0

Java 模块写入单行日志：

{"timestamp":"2020-09-29 10:46:18.761","level":"INFO","message":"status: OK","logger":"thrift.handler.CheckTokenThriftHandler","system":"-","service":"AUTH BE","conf_item":"-"}{"timestamp":"2020-09-29 10:49:11.338","level":"INFO","message":"status: OK","logger":"thrift.handler.CheckTokenThriftHandler","system":"-","service":"AUTH BE","conf_item":"-"}...

每个新事件都粘贴到行尾。在 fluentd 设置中，我使用标准解析器：

[PARSER]
    Name         java-system
    Format       json
    Time_Key     time
    Time_Format  %Y-%m-%d %H:%M:%S.%L
    Time_Keep    On

如果我将日志中的事件分成几行，代理会毫无问题地将所有内容发送到弹性。如果所有内容都在一行中，则它不会发送任何内容。它不会在日志中写入任何信息。

Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [trace] [router] input=tail.0 tag=gw_system
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [router] match rule tail.0:es.0
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [trace] [router] input=tail.1 tag=calc-system
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [router] match rule tail.1:es.3
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [trace] [router] input=tail.2 tag=push-system
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [router] match rule tail.2:es.2
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [trace] [router] input=tail.3 tag=gw_integration
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [router] match rule tail.3:es.1
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [ info] [sp] stream processor started
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.0] file=/opt/java_services/gateway/log/system.log read=35484 lines=0
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.1] file=/opt/java_services/calculator/log/system.log read=336548 lines=0
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.2] file=/opt/java_services/push/log/system_temp.log read=12302 lines=0
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.3] file=/opt/java_services/gateway/log/integration.log promote to TAIL_EVENT
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.0] file=/opt/java_services/gateway/log/system.log promote to TAIL_EVENT
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.1] file=/opt/java_services/calculator/log/system.log promote to TAIL_EVENT
Sep 29 15:11:41  td-agent-bit: [2020/09/29 15:11:41] [debug] [input:tail:tail.2] file=/opt/java_services/push/log/system_temp.log promote to TAIL_EVENT

写了一个单独的解析器

[PARSER]
    Name    java_push_system
    Format  regex
    Regex   (?<date>(?<={"timestamp":")(\d+\-\d+\-\d+\s+\d+:\d+:\d+.\d+))",(?<message>[^}]*)
    Time_Key    time
    Time_Format %Y-%m-%d %H:%M:%S.%L
    Time_Keep    On

在 regex101.com 网站上，我的字符串被解析，但代理仍然不发送任何内容，直到它被分成单独的行。

/etc/td-agent-bit/conf]# cat push-system*
[FILTER]
    Name   record_modifier
    Match  push-system
    Record hostname ${HOSTNAME}
    Record environment DEV
    Record path /opt/java_services/push/log/system.log
[INPUT]
    Name   tail
    Tag    push-system
    Parser java-system
    Path   /opt/java_services/push/log/system.log
    DB     /etc/td-agent-bit/system-push.db
    Buffer_Max_Size 32MB
    Buffer_Chunk_Size 8MB
[OUTPUT]
    Name        es
    Match       push-system
    Host        10.*
    Port        9200
    HTTP_User   *
    HTTP_Passwd *
    Index       push

请帮忙理解！

4

0 回答 0