0

我正在尝试从文件中读取并将每个事件输出到控制台（或文件）。我希望能够添加到文件中，并让 Logstash 拾取它并重复管道。但是，Logstash 似乎只读取和执行管道一次，即使它看到文件已更改。

我正在 OS X Yosemite 上开发。

这是我的 LogStash 配置

input {  
      file {
          path => "/Users/Justin/logstash-1.5.2/testFile"
          sincedb_path => "/Users/Justin/logstash-1.5.2/.sincedb"
          start_position => "beginning"
      }
}


output {  
   stdout {

   }
}

这是日志（同样，用 sudo 尝试过，没有运气）：

    Justins-MacBook-Pro-2:logstash-1.5.2 justin$ bin/logstash agent --debug -vf myConfig
Reading config file {:file=>"logstash/agent.rb", :level=>:debug, :line=>"295", :method=>"local_config"}
Compiled pipeline code:
        @inputs = []
        @filters = []
        @outputs = []
        @periodic_flushers = []
        @shutdown_flushers = []

          @input_file_1 = plugin("input", "file", LogStash::Util.hash_merge_many({ "path" => ("/Users/Justin/logstash-1.5.2/testFile") }, { "sincedb_path" => ("/Users/Justin/logstash-1.5.2/.sincedb") }, { "start_position" => ("beginning") }))

          @inputs << @input_file_1

          @output_stdout_2 = plugin("output", "stdout")

          @outputs << @output_stdout_2

  def filter_func(event)
    events = [event]
    @logger.debug? && @logger.debug("filter received", :event => event.to_hash)
    events
  end
  def output_func(event)
    @logger.debug? && @logger.debug("output received", :event => event.to_hash)
    @output_stdout_2.handle(event)

  end {:level=>:debug, :file=>"logstash/pipeline.rb", :line=>"28", 
    :method=>"initialize"}
    Plugin not defined in namespace, checking for plugin file {:type=>"input", :name=>"file", :path=>"logstash/inputs/file", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
    Plugin not defined in namespace, checking for plugin file {:type=>"codec", :name=>"plain", :path=>"logstash/codecs/plain", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
    config LogStash::Codecs::Plain/@charset = "UTF-8" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@path = ["/Users/Justin/logstash-1.5.2/testFile"] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@sincedb_path = "/Users/Justin/logstash-1.5.2/.sincedb" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@start_position = "beginning" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@debug = false {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@codec = <LogStash::Codecs::Plain charset=>"UTF-8"> {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@add_field = {} {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@stat_interval = 1 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@discover_interval = 15 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@sincedb_write_interval = 15 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Inputs::File/@delimiter = "\n" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    Plugin not defined in namespace, checking for plugin file {:type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
    Plugin not defined in namespace, checking for plugin file {:type=>"codec", :name=>"line", :path=>"logstash/codecs/line", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
    config LogStash::Codecs::Line/@charset = "UTF-8" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Outputs::Stdout/@type = "" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Outputs::Stdout/@tags = [] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Outputs::Stdout/@exclude_tags = [] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::Line charset=>"UTF-8"> {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    config LogStash::Outputs::Stdout/@workers = 1 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"112", :method=>"config_init"}
    Registering file input {:path=>["/Users/Justin/logstash-1.5.2/testFile"], :level=>:info, :file=>"logstash/inputs/file.rb", :line=>"76", :method=>"register"}
    Pipeline started {:level=>:info, :file=>"logstash/pipeline.rb", :line=>"86", :method=>"run"}
    Logstash startup completed
    _sincedb_open: reading from /Users/Justin/logstash-1.5.2/.sincedb {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"220", :method=>"_sincedb_open"}
    _sincedb_open: setting ["11809694", 1, 4] to 1089 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"224", :method=>"_sincedb_open"}
    _sincedb_open: setting ["11850406", 1, 4] to 15 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"224", :method=>"_sincedb_open"}
    _discover_file_glob: /Users/Justin/logstash-1.5.2/testFile: glob is: ["/Users/Justin/logstash-1.5.2/testFile"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"132", :method=>"_discover_file"}
    _discover_file: /Users/Justin/logstash-1.5.2/testFile: new: /Users/Justin/logstash-1.5.2/testFile (exclude is []) {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"141", :method=>"_discover_file"}
    _open_file: /Users/Justin/logstash-1.5.2/testFile: opening {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"119", :method=>"_open_file"}
    /Users/Justin/logstash-1.5.2/testFile: initial create, no sincedb, seeking to beginning of file {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"156", :method=>"_open_file"}
    Received line {:path=>"/Users/Justin/logstash-1.5.2/testFile", :text=>"Hello World", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"137", :method=>"run"}
    Received line {:path=>"/Users/Justin/logstash-1.5.2/testFile", :text=>"Testing", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"137", :method=>"run"}
    output received {:event=>{"message"=>"Hello World", "@version"=>"1", "@timestamp"=>"2015-07-21T22:58:13.460Z", "host"=>"Justins-MacBook-Pro-2.local", "path"=>"/Users/Justin/logstash-1.5.2/testFile"}, :level=>:debug, :file=>"(eval)", :line=>"21", :method=>"output_func"}
    writing sincedb (delta since last write = 1437519493) {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"196", :method=>"_read_file"}2015-07-21T22:58:13.460Z Justins-MacBook-Pro-2.local Hello World

    output received {:event=>{"message"=>"Testing", "@version"=>"1", "@timestamp"=>"2015-07-21T22:58:13.464Z", "host"=>"Justins-MacBook-Pro-2.local", "path"=>"/Users/Justin/logstash-1.5.2/testFile"}, :level=>:debug, :file=>"(eval)", :line=>"21", :method=>"output_func"}
    2015-07-21T22:58:13.464Z Justins-MacBook-Pro-2.local Testing
    /Users/Justin/logstash-1.5.2/testFile: file grew, old size 0, new size 20 {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"96", :method=>"each"}
    _discover_file_glob: /Users/Justin/logstash-1.5.2/testFile: glob is: ["/Users/Justin/logstash-1.5.2/testFile"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"132", :method=>"_discover_file"}

在我更改文件（在本例中为 testFile）后，它会显示出来，但没有其他内容：

/Users/Justin/logstash-1.5.2/testFile: file grew, old size 20, new size 29 {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"96", :method=>"each"}
_discover_file_glob: /Users/Justin/logstash-1.5.2/testFile: glob is: ["/Users/Justin/logstash-1.5.2/testFile"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"132", :method=>"_discover_file"}

此外，它似乎没有崩溃或任何东西。它看到文件已更改，但没有运行其输出功能。

4

1 回答 1

0

logstash 配置文件中有选项。

 input {
 file {
 discover_interval => ... # number (optional), default: 15
 stat_interval => ... # number (optional), default: 1
 sincedb_path => ... # string (optional)
 sincedb_write_interval => ... # number (optional), default: 15
 }
}

在这里discover_interval * stat_interval = seconds 等待 15 秒。如果仍然面临同样的问题，请删除 .sincedb_* 文件。并重新启动logstash 服务。sincedb 存储所有已处理文件的详细信息。我希望它会奏效。

于 2015-07-22T03:45:39.740 回答