0

我有两个配置文件分别指向数据库的不同表,但是在elasticsearch中,我将两个表合并,因此tablea和tableb项目将全部复制为typea,tablea和tableb项目将全部复制为typeb。

但我将其配置为将 tablea 作为 typea 并将 tableb 作为 typeb。

这是我的配置文件a

input {

    jdbc {
        jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://db:3306/nombase"
        jdbc_user => "root"
        jdbc_password => "root"
        schedule => "* * * * *"
        statement => "SELECT * FROM `tablecandelete`"
    }

}


## Add your filters / logstash plugins configuration here


output {

    stdout { codec => json_lines }

    if [deleted] == 1 {
        elasticsearch {
            hosts => "elasticsearch:9200"
            index => "nombase"
            document_type => "tablecandelete"

            ## TO PREVENT DUPLICATE ITEMS
            document_id => "tablecandelete-%{id}"
            action => "delete"
        }
    } else {
        elasticsearch {
            hosts => "elasticsearch:9200"
            index => "nombase"
            document_type => "tablecandelete"

            ## TO PREVENT DUPLICATE ITEMS
            document_id => "tablecandelete-%{id}"
        }
    }

}

这是我的配置文件

input {

    jdbc {
        jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://db:3306/nombase"
        jdbc_user => "root"
        jdbc_password => "root"
        schedule => "* * * * *"
        statement => "SELECT * FROM `nomtable`"
    }

}


## Add your filters / logstash plugins configuration here


output {

    stdout { codec => json_lines }

    elasticsearch {
        hosts => "elasticsearch:9200"
        index => "nombase"
        document_type => "nomtable"

        ## TO PREVENT DUPLICATE ITEMS
        document_id => "nomtable-%{id}"
    }

}

有人有想法吗?感谢您的任何回复/帮助

4

1 回答 1

1

虽然您确实有两个不同的配置文件,但logstash 不会将它们视为独立的——您可以将所有输入放在一个文件中,将所有输出放在另一个文件中。

要将事情分开,您必须添加type => something_unique每个输入,然后将配置文件中的剩余代码与

if [type] == 'something_unique' { 
  // config here 
}
于 2017-07-13T19:21:24.240 回答