在 AWS 中构建了一个运行在 RHEL 7.2 上的 ELK 服务器。计划是让它从 S3 存储桶中摄取 cloudtrail 日志，然后在 kibana 前端执行魔法，但它不起作用，我已经失去了几天的生命试图解决它，因此我为什么要寻求一些帮助. 我可以从我的 logstash.log 中看到它正在读取 S3 存储桶中的文件，但仅此而已。似乎没有其他事情发生。
我的设置：
RHEL 7.2
kibana-4.5.0-1.x86_64
logstash-2.3.2-1.noarch
elasticsearch-2.3.3-1.noarch
nginx-1.6.3-8.el7.x86_64 (reverse proxy kibana to port 80)
This is what my /etc/logstash/conf.d looks like :
-rw-r--r-- 1 root root 574 May 31 14:55 02-cloudtrail-input.conf
-rw-r--r-- 1 root root 432 May 31 15:04 30-elasticsearch-output.conf
root@elk conf.d]# 猫 *
input {
s3
bucket => "xyz..cloudtrail"
access_key_id => 'XYZ'
secret_access_key => 'ABC'
delete => false
codec => "cloudtrail"
prefix => "cloudtrail/AWSLogs/xxxxx/CloudTrail/ap-southeast-2/2016/"
type => "cloudtrail"
interval => 10 # seconds
region => "ap-southeast-2"
sincedb_path => "/data/logstash/cloudtrail/db/sincedb"
}
}
output {
#stdout {}
stdout { codec => rubydebug }
elasticsearch {
hosts => "localhost:9200"
sniffing => true
#codec => "cloudtrail"
#index => "cloudtrail"
index => "logstash-%{+YYYY.MM.dd}"
#index => "%{[@metadata][cloudtrail]}-%{+YYYY.MM.dd}"
#index => "cloudtrail-%{+YYYY.MM.dd}"
action => create
manage_template => false
workers => 4
}
}
installed plugins :
logstash-codec-cloudtrail
logstash-input-s3
logstash-output-s3
我不知道应该如何在 logstash 输出中定义索引，但我可以在 kibana 前端搜索它并选择 3 个时间范围选项之一，但不知道这意味着什么。我应该有 sniffing = true 吗？我应该定义动作 => 创建吗？我应该在 logstash 输入和输出中都定义 cloudtrail 编解码器吗？
谁能告诉我“插件未在命名空间中定义，检查插件文件”是什么意思？即使安装了所有必需的插件，在我看来它也找不到插件（我认为）
这是我手动启动logstash时的输出......但这对我来说意义不大......
Reading config file {:config_file=>"/etc/logstash/conf.d/02-cloudtrail-input.conf", :level=>:debug, :file=>"logstash/config/loader.rb", :line=>"69", :method=>"local_config"}
Reading config file {:config_file=>"/etc/logstash/conf.d/30-elasticsearch-output.conf", :level=>:debug, :file=>"logstash/config/loader.rb", :line=>"69", :method=>"local_config"}
Plugin not defined in namespace, checking for plugin file {:type=>"input", :name=>"s3", :path=>"logstash/inputs/s3", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
Plugin not defined in namespace, checking for plugin file {:type=>"codec", :name=>"cloudtrail", :path=>"logstash/codecs/cloudtrail", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
config LogStash::Codecs::CloudTrail/@spool_size = 50 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@bucket = "abcdbase-cloudtrail" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@access_key_id = "XYZ" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@secret_access_key = "ABC" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@delete = false {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@codec = <LogStash::Codecs::CloudTrail spool_size=>50> {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@prefix = "abcdbase-trail/AWSLogs/554658506446/CloudTrail/ap-southeast-2/2016/" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@type = "cloudtrail" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@interval = 10 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@region = "ap-southeast-2" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@sincedb_path = "/data/logstash/cloudtrail/db/sincedb" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@add_field = {} {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@use_ssl = true {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@credentials = [] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@backup_to_bucket = nil {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@backup_add_prefix = nil {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@backup_to_dir = nil {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@exclude_pattern = nil {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
config LogStash::Inputs::S3/@temporary_directory = "/tmp/logstash" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"153", :method=>"config_init"}
Plugin not defined in namespace, checking for plugin file {:type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
Plugin not defined in namespace, checking for plugin file {:type=>"output", :name=>"elasticsearch", :path=>"logstash/outputs/elasticsearch", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
当logstash.log一直说method =>“list_new_files”时是什么意思，这是否意味着它正忙于重新读取S3存储桶（每次logstash停止/启动）？
我知道这是很多问题，但我想在将这个设置扔进垃圾箱之前我会尝试寻求一些帮助。
谢谢