2

在弹性搜索上的函数运行范围查询之后，当我在多线程环境中重复使用此函数时，内存急剧增加，直到应用程序崩溃。

有没有人可以解决这个问题？

/**
  RestHighLevelClient restHighLevelClient = new RestHighLevelClient( //
                RestClient.builder(new HttpHost("localhost",9200,"http")));
**/
public List<Map<String, Object>> getAllDocumentsInRange(long startTime, long endTime,
       RestHighLevelClient restHighLevelClient) {
    try {
        QueryBuilder queryBuilder = QueryBuilders//
                .rangeQuery("date")//
                .gte(startTime)//
                .lte(endTime);


        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        searchSourceBuilder.query(queryBuilder);

        searchSourceBuilder.sort(new FieldSortBuilder("date").order(SortOrder.ASC));

        searchSourceBuilder.size(10000);

        searchSourceBuilder.timeout(new TimeValue(20, TimeUnit.SECONDS));

        List<Map<String, Object>> docsMap = new ArrayList<>();

        SearchRequest searchRequest = new SearchRequest(new String[] { "mainIndex" }, searchSourceBuilder);

        Scroll scroll = new Scroll(TimeValue.timeValueSeconds(30));
        searchRequest.scroll(scroll);


        RestHighLevelClient restHighLevelClient = new RestHighLevelClient( //
                RestClient.builder(new HttpHost("localhost",9200,"http")));

        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        String scrollId = searchResponse.getScrollId();

        SearchHit[] searchHits = searchResponse.getHits().getHits();

        while (searchHits != null && searchHits.length > 0) {

            for (SearchHit hit : searchHits) {
                Map<String, Object> elasticDocVersion = hit.getSourceAsMap();
                docsMap.add(elasticDocVersion);
            }

            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            scrollRequest.scroll(scroll);

            searchResponse = restHighLevelClient.scroll(scrollRequest, RequestOptions.DEFAULT);

            scrollId = searchResponse.getScrollId();

            searchHits = searchResponse.getHits().getHits();
        }

        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = //
                restHighLevelClient.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
        boolean succeeded = clearScrollResponse.isSucceeded();
        logger.trace("search scroll clearation:{}", succeeded);
    } catch (Exception e) {
        logger.error("error in creaing QueryBuilder class: {}", e.getMessage());
    }
    return new ArrayList<>();
}

这是我运行应用程序后的内存使用情况图片

内存使用情况图片

我尝试了不同的解决方案，例如同步上述功能，但都没有解决问题！

4

1 回答 1

1

不确定您使用的是什么 elasticsearch 客户端版本，但您可能对这个elasticsearch rest 客户端内存泄漏感兴趣。

另外，我不确定restHighLevelClient传入的是哪一个，还是在方法中创建的那个。只有一个是有效的，但无论如何，我建议restHighLevelClient.close()在与客户端完成后在 try-with-resources 中调用或使用它。

try (RestHighLevelClient restHighLevelClient = 
   new RestHighLevelClient(RestClient.builder(new HttpHost("localhost",9200,"http"))) {
...
}
于 2020-10-08T18:00:09.720 回答