1

上下文：我想使用 XPACK 来控制哪些用户只能使用免费版本查看哪个 Dasboard。

我从这里下载了 Kibana 7.7.0 zip ，安装了它，我可以看到创建用户/角色的安全选项。事实上，我创建了一个索引、一个用户和一个角色，并在我的 Windows 中使用这个安装的 Elastic/Kibana 成功地将索引设置为这个角色。

只有从 docker 开始的 Elastic/Kibana 才会出现此问题。我从 docker 启动 Kibana 7.7.0，但在管理页面下看不到安全面板。谷歌搜索我发现我必须使用基本版本而不是开源。据我所见， docker-compose bellow 正在下载基本版本，因为最后没有“sso”。我还必须使用 Apache 的 Elastic instad 提供的安装程序。好吧，据我所知，它正在拉取与 Apache 无关的图像。

我不确定问题是否仅与 Kibana 相关，因为我可以在 Elastic 上启用 xpack 安全性并在弹性 docker 容器内交互式运行 elasticsearch-setup-passwords。我可以使用 Elastic 用户登录 Kibana，但在管理下看不到安全选项卡。

此外，即使我设置了 logstash_system，我也遇到了 LogStash 尝试连接到 ElasticSearch 的问题（请参阅下面的 logstash.conf）。

你可以看到我在 ElasticSearch 上启用了 xpack.security.enabled=true。

码头工人-compose.yml

version: '3.2'
services:

  zoo1:
    image: elevy/zookeeper:latest
    environment:
      MYID: 1
      SERVERS: zoo1
    ports:
      - "2181:2181"

  kafka1:
    image: wurstmeister/kafka
    command: [start-kafka.sh]
    depends_on:
      - zoo1
    links:
      - zoo1
    ports:
      - "9092:9092"
    environment:
      KAFKA_LISTENERS: PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      KAFKA_BROKER_ID: 1
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_LOG_RETENTION_HOURS: "168"
      KAFKA_LOG_RETENTION_BYTES: "100000000"
      KAFKA_ZOOKEEPER_CONNECT:  zoo1:2181
      KAFKA_CREATE_TOPICS: "log:1:1"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'

  filebeat:
    image: docker.elastic.co/beats/filebeat:7.7.0
    command: filebeat -e -strict.perms=false
    volumes:
      - "//c/Users/my-comp/docker_folders/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"
      - "//c/Users/my-comp/docker_folders/sample-logs:/sample-logs"
    links:
      - kafka1
    depends_on:
      - kafka1

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.enabled=true
      - discovery.type=single-node
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
     - "//c/Users/my-comp/docker_folders/esdata:/usr/share/elasticsearch/data"
    ports:
      - "9200:9200"

  kibana:
    image: docker.elastic.co/kibana/kibana:7.7.0
    volumes:
      - "//c/Users/my-comp/docker_folders/kibana.yml:/usr/share/kibana/config/kibana.yml"
    restart: always
    environment:
    - SERVER_NAME=kibana.localhost
    - ELASTICSEARCH_HOSTS=http://x.x.x.x:9200
    ports:
      - "5601:5601"
    links:
      - elasticsearch
    depends_on:
      - elasticsearch

  logstash:
    image: docker.elastic.co/logstash/logstash:7.7.0
    volumes:
      - "//c/Users/my-comp/docker_folders/logstash.conf:/config-dir/logstash.conf"
    restart: always
    command: logstash -f /config-dir/logstash.conf
    ports:
      - "9600:9600"
      - "7777:7777"
    links:
      - elasticsearch
      - kafka1

kibana.yml

server.name: kibana
server.host: "0"
xpack.monitoring.ui.container.elasticsearch.enabled: false
elasticsearch.ssl.verificationMode: none
elasticsearch.username: "kibana"
elasticsearch.password: "k12345"

logstash.conf

input{
  kafka{
    codec => "json"
    bootstrap_servers => "kafka1:9092"
    topics => ["app_logs","request_logs"]
    tags => ["myapp"]
  }
}

filter {    
*** not relevant
}


output {
  elasticsearch {
    hosts => ["http://x.x.x.x:9200"]
    index => "%{[fields][topic_name]}-%{+YYYY.MM.dd}"
    user => "logstash_system" 
    password => "l12345" 
  }
}

值得一提的是，LogStash 无法使用此日志连接到 ElasticSearch，正如您从 logstash.conf 中看到的那样，我设置了 logstash_system（从 elasticsearch-setup-passwords 交互创建的用户）

logstash_1       | [2020-05-19T20:18:45,559][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
logstash_1       | [2020-05-19T20:19:13,815][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/_xpack'"}

所以，我的直截了当的问题是：为了在 Kibana 上启用安全性，我是否缺少一些额外的配置？周围的问题是：来自 docker 的 Kibana/Elastic 与 zip 文件中的不同吗？我是否缺少一些额外的配置以允许 Logstash 连接到 ElasticSearch

*** 已编辑

在我更改为后，LogStash 仍然无法连接到 ElasticSearch

logstash.conf

...
output {
  elasticsearch {
    #hosts => [ "${ELASTIC_HOST1}", "${ELASTIC_HOST2}", "${ELASTIC_HOST3}" ]
    #hosts => ["http://192.168.99.100:9200"]
    index => "%{[fields][topic_name]}-%{+YYYY.MM.dd}"
    xpack.monitoring.elasticsearch.hosts: ["http://192.168.99.100:9200"]
    xpack.monitoring.elasticsearch.username: "logstash_system"
    xpack.monitoring.elasticsearch.password: => "l12345" 
  }
}

日志是

logstash_1       | WARNING: All illegal access operations will be denied in a future release
logstash_1       | Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
logstash_1       | [2020-05-20T13:39:05,095][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
logstash_1       | [2020-05-20T13:39:05,120][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.7.0"}
logstash_1       | [2020-05-20T13:39:06,134][WARN ][logstash.monitoringextension.pipelineregisterhook] xpack.monitoring.enabled has not been defined, but found elasticsearch configuration. Please explicitly set `xpack.monitoring.enabled: true` in logstash.yml
logstash_1       | [2020-05-20T13:39:06,150][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
logstash_1       | Please configure Metricbeat to monitor Logstash. Documentation can be found at:
logstash_1       | https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html
logstash_1       | [2020-05-20T13:39:08,008][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
logstash_1       | [2020-05-20T13:39:08,408][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
logstash_1       | [2020-05-20T13:39:08,506][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/_xpack'"}
filebeat_1       | 2020-05-20T13:38:53.069Z     INFO    log/harvester.go:297    Harvester started for file: /sample-logs/request-2019-11-17F.log
logstash_1       | [2020-05-20T13:39:08,611][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
logstash_1       | [2020-05-20T13:39:11,449][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"=>\" at line 86, column 7 (byte 2771) after output {\r\n  elasticsearch {\r\n    #hosts => [ \"${ELASTIC_HOST1}\", \"${ELASTIC_HOST2}\", \"${ELASTIC_HOST3}\" ]\r\n\t#hosts => [\"http://192.168.99.100:9200\"]\r\n    index => \"%{[fields][topic_name]}-%{+YYYY.MM.dd}\"\r\n\txpack", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge_state'"]}

我想这个日志最相关的部分是：

logstash_1       | [2020-05-20T13:39:08,008][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
logstash_1       | [2020-05-20T13:39:08,408][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
logstash_1       | [2020-05-20T13:39:08,506][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/_xpack'"}

请注意，它因“在 URL ' http://elasticsearch:9200/_xpack '上联系 Elasticsearch 的“得到响应代码 '401' ”错误而失败。我猜在我的特定 docker 设置中，它需要是 Docker 机器 IP，它在我的情况下是192.168.99.100。有没有办法用这个IP替换elasticsearch？

4

0 回答 0