2

在Kubernetes-client Java api中，我可以使用以下方式获取给定应用程序的可用和总部署 pod 实例数：

ApiClient defaultClient = Configuration.getDefaultApiClient();
AppsV1beta1Api apiInstance = new AppsV1beta1Api();
...
try {
    AppsV1beta1DeploymentList result = apiInstance.listDeploymentForAllNamespaces(_continue, fieldSelector, includeUninitialized, labelSelector, limit, pretty, resourceVersion, timeoutSeconds, watch);
    foreach(ExtensionsV1beta1Deployment extensionsDeployment : result.getItems() ) {
        Map<String, String> labels = extensionsDeployment.getMetadata().getLabels();
        String appName = labels.getOrDefault("app", "");
        ExtensionsV1beta1DeploymentStatus status = extensionsDeployment.getStatus();
        int availablePods = status.getAvailableReplicas();
        int deployedPods = status.getReplicas();
        if ( availablePods != deployedPods) {
             // Generate an alert
        }
    }
} catch (ApiException e) {
    System.err.println("Exception when calling AppsV1beta1Api#listDeploymentForAllNamespaces");
    e.printStackTrace();
}

在上面的示例中，我将availablePods与进行比较deployedPods，如果它们不匹配，我会生成警报。

如何使用 Prometheus 使用警报规则和/或 Alertmanager 配置复制此逻辑，它检查给定应用程序或作业的可用 pod 实例数量，如果它与指定数量的实例不匹配，它将触发警报？

指定的阈值可以是总阈值deployedPods，也可以来自另一个配置文件或模板。

4

1 回答 1

3

我不知道如何为所有命名空间执行此操作，但对于一个命名空间，它看起来像：

curl -k -s 'https://prometheus-k8s/api/v1/query?query=(sum(kube_deployment_spec_replicas%7Bnamespace%3D%22default%22%7D)%20without%20(deployment%2C%20instance%2C%20pod))%20-%20(sum(kube_deployment_status_replicas_available%7Bnamespace%3D%22default%22%7D)%20without%20(deployment%2C%20instance%2C%20pod))'

这是对默认命名空间的 curl 请求。

警报配置将如下所示：

groups:
- name: example
  rules:

  # Alert for any instance that is unreachable for >5 minutes.
  - alert: availablePods!=deployedPods
    expr: (sum(kube_deployment_spec_replicas{namespace="$Name_of_namespace"}) without (deployment, instance, pod)) - (sum(kube_deployment_status_replicas_available{namespace="$Name_of_namespace"}) without (deployment, instance, pod)) != 0
    for: 15m
    labels:
      severity: page
    annotations:
      summary: "availablePods are not equal deployedPods"
      description: "In namespace $Name_of_namespace more than 15 minutes availablePods are not equal deployedPods. "

不要忘记将变量更改$Name_of_namespace为要检查的名称空间名称。

于 2018-06-27T11:25:33.460 回答