2

我目前正在使用使用 mongodb 返回器的 saltstack 实现，并且我正在尝试查询存储在其中的结果以用于报告目的。不幸的是，返回者将信息写入数据库的方式如下所示：

db.hostname21.find({"20140421035007474763" : {$exists : 1}}, {_id: 0}).pretty()

{
     "fun" : "state.sls",
     "20140421035007474763" : {
            "file_|-Get-Logins-Stats_|-/scripts/server_logins_stats_|-managed" : {
                    "comment" : "File /scripts/server_logins_stats is in the correct state",
                    "__run_num__" : 2,
                    "changes" : {

                    },
                    "name" : "/scripts/server_logins_stats",
                    "result" : true
            },
            "service_|-Logger_|-rsyslog_|-running" : {
                    "comment" : "Service rsyslog is already enabled, and is in the desired state",
                    "__run_num__" : 1,
                    "changes" : {

                    },
                    "name" : "rsyslog",
                    "result" : true
            },
            "cmd_|-Run_Script_|-/scripts/server_logins_stats_|-run" : {
                    "comment" : "Command \"/scripts/server_logins_stats\" run",
                    "__run_num__" : 4,
                    "changes" : {
                            "pid" : 20899,
                            "retcode" : 0,
                            "stderr" : "",
                            "stdout" : "0"
                    },
                    "name" : "/scripts/server_logins_stats",
                    "result" : true
            },
            "cron_|-Schedule_Run_|-/scripts/server_logins_stats_|-present" : {
                    "comment" : "Cron /scripts/server_logins_stats already present",
                    "__run_num__" : 3,
                    "changes" : {

                    },
                    "name" : "/scripts/server_logins_stats",
                    "result" : true
            },
            "pkg_|-nc_|-nc_|-installed" : {
                    "comment" : "Package nc is already installed",
                    "__run_num__" : 0,
                    "changes" : {

                    },
                    "name" : "nc",
                    "result" : true
            }
    }
}

正如您在此处看到的，密钥不断变化，而不是拥有一个标识每个脚本的密钥。但是，我发现故障的格式非常一致，除了它们没有任何键来标识它是故障，只是一个字符串数组：

       "20140421041507478163" : [
            "Pillar failed to render with the following messages:",
            "Specified SLS 'globals' in environment 'Production' is not available on the salt master"
       ],
       "fun" : "state.sls"

所以最终我想要做的是能够通过识别工作、主机和失败的性质以及随着时间的推移成功和失败的总数来报告每个失败。如果您注意到每个主机（minion）都创建了自己的集合。所以我有一个 python 脚本，它将遍历集合以确定作业是否在该主机上实际执行：

import datetime
import pymongo


#hosts = ["mongoDBStaging", "mongoDBUAT"]
hosts = ["mongodbuat"]

for host in hosts:
    conn = pymongo.Connection(host)
    mdb = conn['salt']
    collections = set(mdb.collection_names())


    hosts = []
    jids = []

    # for every collection, identify whether it is a host or a job
    for c in collections:   
        # if the collection is a host add it to the host array
        if not (len(c) == 20 and int(c)):
            #print "{0} is a host".format(c)
            hosts.append(c)
        # other  wise add it to the job array
        else:
            #print "{0} is a jid".format(c)
            jids.append(c)


    for h in hosts:
        # for every job in a host connect to that collection 
        # and search for the job id to see if it exists 
        # and what its return was so we can report on that info
        for j in jids:
            coll = conn['salt'][h]
            #print "%s collection, %s jid" % (coll, j)
            for doc in coll.find({j: {'$exists': True}, "fun": "state.sls"}, {"_id": 0}):
                print "{0}".format(coll)
                print "{0} is a doc".format(doc)

但我正在努力查询结果以实际确定它是否成功。通过能够提取返回的文档元素以读取每个文档的结果。

如果有人对我如何始终如一地查询键不断变化的结果以获得真正有用​​的结果有任何建议。

4

1 回答 1

1

万一有人想知道。我通过使用以下 python 代码解决了我自己的问题。绝对不是最好的性能或充分利用 mongodb，但它确实有效。我可能会建议更新 salt returner 以更好地使用 mongodb，因为 shell 中的查询非常有限。

import datetime
import pymongo
import json
import re

hosts = ["mongodbuat"]

# initialize failures and successes
failures = 0
successes = 0

for host in hosts:
    conn = pymongo.Connection(host)
    mdb = conn['salt']
    collections = set(mdb.collection_names())

    hosts = []
    jids = []

    # for every collection, identify whether it is a host or a job
    for c in collections:   
        # if the collection is a host add it to the host array
        if not (len(c) == 20 and int(c)):
            hosts.append(c)
        # otherwise add it to the job array
        else:
            jids.append(c)


    for h in hosts:
        # for every job in a host connect to that collection 
        # and search for the job id to see if it exists 
        # and what its return was so we can report on that info
        for j in jids:
            coll = conn['salt'][h]

            # search for the json documents returned from mongodb
            # if the jobid exists in that host
            for doc in coll.find({j: {'$exists': True}, "fun": "state.sls"}, {"_id": 0}):

                # if the jobid exists find the host name in a readable format
                c = str(coll)
                thishost = ''
                match = re.search('(\w+)\.spottrading\.com',c)
                if match:
                    thishost = match.group(1)

                # search the document returned in the form of a dictionary for
                # the states you want to report on
                for jobid, states in doc.iteritems():
                    if re.search('\d+', jobid):
                        print '\njob id =', jobid
                        if isinstance(states, list):
                            print states
                            failures += 1
                        elif isinstance(states, dict):
                            for job, data in states.iteritems():
                                print '\tjob: {0}, result: {1}'.format(job, data[u'result'])
                            successes += 1

print "{0} successes, {1} failures".format(successes, failures)
于 2014-04-25T17:54:24.723 回答