elasticsearch - Elasticsearch：在指定的时间范围之间滚动

Question

我在弹性搜索中有一些数据。如图所示

我使用下面的链接示例进行滚动

https://gist.github.com/drorata/146ce50807d16fd4a6aa

page = es.search(
    index = INDEX_NAME,
    scroll = '1m',
    size = 1000,
    body={"query": {"match_all": {}}})
    sid = page['_scroll_id']
    scroll_size = page['hits']['total']

    # Start scrolling

    print( "Scrolling...")
    while (scroll_size > 0):


        print("Page: ",count)
        page = es.scroll(scroll_id = sid, scroll = '10m')
        # Update the scroll ID
        sid = page['_scroll_id']

        for hit in page['hits']['hits']:
            #some code processing here

目前我的要求是我想滚动但想指定开始时间戳和结束时间戳需要帮助如何使用滚动来做到这一点。

Answer 1

只需更换

body={"query": {"match_all": {}}})

经过

body={"query": {"range": {"timestamp":{"gte":"2018-08-05T05:30:00Z", "glte":"2018-08-06T05:30:00Z"}}}})

Answer 2

示例代码。时间范围应该在es查询中。此外，您应该处理第一个查询结果。

es_query_dict = {"query": {"range": {"timestamp":{
    "gte":"2018-08-00T00:00:00Z", "lte":"2018-08-17T00:00:00Z"}}}}


def get_es_logs():
    es_client = Elasticsearch([source_es_ip], port=9200, timeout=300)

    total_docs = 0
    page = es_client.search(scroll=scroll_time,
                            size=scroll_size,
                            body=json.dumps(es_query_dict))
    while True:
        sid = page['_scroll_id']
        details = page["hits"]["hits"]
        doc_count = len(details)
        if len(details) > 0:
            total_docs += doc_count
            print("scroll size: " + str(doc_count))
            print("start bulk index docs")
            # index_bulk(details)
            print("end success")
        else:
            break
        page = es_client.scroll(scroll_id=sid, scroll=scroll_time)

    print("total docs: " + str(total_docs))

Answer 3

还要看看elasticsearch.helpers.scan你已经为你实现了循环逻辑的地方，只需传递它query={"query": {"range": {"timestamp": {"gt": ..., "lt": ...}}}}