1

我从 gvisor-containerd-shim (Shim V1) 迁移到 containerd-shim-runsc-v1 (Shim V2)。在 gvisor-containerd-shim 的情况下，指标服务器和 Horizo​​ntal Pod Autoscaler 过去工作得很好。

但是现在，使用 containerd-shim-runsc-v1，我不断获得节点和 runc pod 的 CPU 和内存指标，但我只获得 runc (gvisor) pod 的内存指标。

例如，我使用 containerd-shim-runsc-v1 在 gvisor pod 中部署了一个 PHP 服务器。我得到以下指标：

kubectl get hpa
NAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
php-apache   Deployment/php-apache   0%/50%    1         10        1          68s


kubectl top nodes
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
snf-877559   549m         13%    2327Mi          39%


kubectl top pods
NAME                                 CPU(cores)   MEMORY(bytes)
php-apache-gvisor-6f7bb6cf84-28qdk   0m           52Mi

在向 php-apache-gvisor pod 发送一些负载后，我可以看到节点和 runc pod（负载生成器）的 CPU 和内存使用增量。我还可以看到 php-apache-gvisor 的内存从 52 Mi 增加到 72 Mi，但它的 CPU 使用率保持在 0%。为什么cpu使用率一直保持在0%？

我也尝试了不同的容器图像，但我一直得到相同的结果。

随着负载，我得到以下指标：

kubectl get hpa
NAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
php-apache   Deployment/php-apache   0%/50%    1         10        1          68s


kubectl top nodes
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
snf-877559   946m         23%    2413Mi          41%


kubectl top pods
NAME                             CPU(cores)   MEMORY(bytes)
load-generator-7d549cd44-xmbqw   3m           1Mi
php-apache-gvisor-6f7bb6cf84-28qdk      0m           72Mi

更多信息：

kubeadm，kubernetes 1.15.3，containerd 1.3.3，runsc nightly/2019-09-18，法兰绒

kubectl logs metrics-server-74657b4dc4-8nlzn -n kube-system
I0728 09:33:42.449921       1 serving.go:312] Generated self-signed cert (/tmp/apiserver.crt, /tmp/apiserver.key)
I0728 09:33:44.153682       1 secure_serving.go:116] Serving securely on [::]:4443
E0728 09:35:24.579804       1 reststorage.go:160] unable to fetch pod metrics for pod default/php-apache-gvisor-6f7bb6cf84-28qdk: no metrics known for pod
E0728 09:35:39.940417       1 reststorage.go:160] unable to fetch pod metrics for pod default/php-apache-gvisor-6f7bb6cf84-28qdk: no metrics known for pod

/etc/containerd/config.toml (containerd-shim-runsc-v1)

subreaper = true
oom_score = -999
disabled_plugins = ["restart"]


[debug]
    level = "debug"

[metrics]
    address = "127.0.0.1:1338"

[plugins.linux]
    runtime = "runc"
    shim_debug = true


[plugins.cri.containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

/etc/containerd/config.toml (gvisor-containerd-shim)

subreaper = true
oom_score = -999
disabled_plugins = ["restart"]


[debug]
    level = "debug"

[metrics]
    address = "127.0.0.1:1338"

[plugins.linux]
    runtime = "runc"
    shim_debug = true
    shim = "/usr/local/bin/gvisor-containerd-shim"


[plugins.cri.containerd.runtimes.runsc]
  runtime_type = "io.containerd.runtime.v1.linux"
  runtime_engine = "/usr/local/bin/runsc"
  runtime_root = "/run/containerd/runsc"

指标服务器 yaml 基于https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml，具有以下参数

....
      containers:
      - name: metrics-server
        image: k8s.gcr.io/metrics-server-amd64:v0.3.6
        imagePullPolicy: IfNotPresent
        args:
          - --kubelet-preferred-address-types=InternalIP
          - --kubelet-insecure-tls
          - --cert-dir=/tmp
          - --secure-port=4443
....

当前部署具有以下资源部分

  resources:
    limits:
      cpu: 500m
    requests:
      cpu: 200m
4

1 回答 1

0

gVisor 目前只报告每个 Pod 的内存和 Pid。请参阅：https ://github.com/google/gvisor/blob/add40fd/runsc/boot/events.go#L62-L68

我们计划导出更多统计数据，跟踪该工作的问题在这里： https ://gvisor.dev/issue/172

于 2020-09-01T14:03:59.353 回答