2

我有一个“ElasticSearch - Logstash - Kibana”部署，其中单个 logstash 实例通过专用的 ElasticSearch 客户端将数据推送到 ElasticSearch 集群。该集群有 3 个数据主节点和一个专用客户端节点。所有机器都有 64 GB RAM、32 核、2TB 磁盘和运行 Debian 8。

我在我的 logstash 日志中看到很多警告说"Failed action with response of 400, dropping action"。在弹性搜索日志中，我看到相应的错误为failed to execute bulk item (index) index. 这并非每次都发生。

logstash 输出配置如下：-

output {
    elasticsearch {         
      index => "jab-%{env}-%{app}-%{iver}-%{[@metadata][app_log_time]}"
      cluster => "elasticsearch"
      host => "es-client"
      port => "9300"
      protocol => "transport"
    }
    file {
      path => "/var/log/shop/%{env}/%{app}/%{app}_%{host}_%{[@metadata][app_log_time]}.log"
    }      
}

elasticsearch的完整日志如下：-

elasticsearch.log:[2015-09-16 01:22:37,954][DEBUG][action.bulk              ] [ESMasterData2] [jab-prod-reincarnation-v1-2015-09-16][0] failed to execute bulk item (index) index {[jab-prod-reincarnation-v1-2015-09-16][lumberjack][AU_UmY2o7YuwugZ18KCl], source[{"timestamp":"2015-09-16T10:51:54.854+05:30","level":"error","reqId":"39624368-71de-4d47-8103-9f40a4973ba9","sessionId":"cef145fe40d48fefef4430f1270dfaeaf28a12615aca2b01c72c2c508e970a38","content":{"message":"TRACKING DATA POPULATION FAILED","loggedIn":"0","clientRequest":{"url":"/river-island-Black-Free-Item-651511.html","path":"/pdp","method":"GET","originalUrl":"/river-island-Black-Free-Item-651511.html","query":{},"bodyExists":false,"headers":{"connection":"upgrade","host":"www.jab.com","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","user-agent":"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36","accept-language":"en-US,en;q=0.8","cookie":"SITEID=new_site_25; __sonar=18274130205066917707; __utma=93690246.789214969.1442344107.1442344107.1442344107.1; __utmz=93690246.1442344107.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _vz=viz_55f72a1a2a061; referrer=direct","x-waf-network":"DmUF7sMn","jade-true-client-ip":"122.162.248.254","x-akamai-config-log-detail":"true","accept-encoding":"gzip","akamai-origin-hop":"2","via":"1.0 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 akamai.net(ghost) (AkamaiGHost)","x-forwarded-for":"122.162.248.254, 96.17.182.144, 59.144.112.204, 127.0.0.1","cache-control":"max-age=0","x-akamai-prefetched-object":"1"}},"exception":{"realm":"TypeError","stack":"TypeError: Cannot read property 'name' of undefined\n    at Object.PDP (/var/www/html/reincarnation/api/pojo/viewObjects/Tracking/Doubleclick/PDP.js:18:42)\n    at /var/www/html/reincarnation/api/models/Tracking.js:45:87\n    at Function.forEach (/var/www/html/reincarnation/node_modules/lodash/dist/lodash.js:3298:15)\n    at Object.Tracking.getProviderData (/var/www/html/reincarnation/api/models/Tracking.js:40:11)\n    at /var/www/html/reincarnation/api/controllers/PDPController.js:50:56\n    at tryCatcher (/var/www/html/reincarnation/node_modules/bluebird/js/main/util.js:26:23)\n    at Promise._settlePromiseFromHandler (/var/www/html/reincarnation/node_modules/bluebird/js/main/promise.js:503:31)\n    at Promise._settlePromiseAt (/var/www/html/reincarnation/node_modules/bluebird/js/main/promise.js:577:18)\n    at Async._drainQueue (/var/www/html/reincarnation/node_modules/bluebird/js/main/async.js:128:12)\n    at Async._drainQueues (/var/www/html/reincarnation/node_modules/bluebird/js/main/async.js:133:10)\n    at Immediate.Async.drainQueues (/var/www/html/reincarnation/node_modules/bluebird/js/main/async.js:15:14)\n    at Immediate.wrapped [as _onImmediate] (/var/www/html/reincarnation/node_modules/newrelic/lib/transaction/tracer/index.js:157:28)\n    at processImmediate [as _immediateCallback] (timers.js:358:17)","errorCode":{"key":"JBxDefault","value":"Opps! Something went wrong."},"message":"Cannot read property 'name' of undefined"}},"@version":"1","@timestamp":"2015-09-16T05:21:56.108Z","type":"lumberjack","file":"/var/log/reincarnation/reincarnation.log.2015-09-16","host":"reincarnation10","offset":"11700676","app":"reincarnation","env":"prod","iver":"v1"}]}

有人可以让我知道出了什么问题以及如何调试吗？

4

0 回答 0