1

我最近通过https://github.com/tarosky/k8s-redis-ha设置了 redis设置了 redis ,这个 repo 包含一个 init 容器,并且我包含了一个额外的 init 容器以获取密码等设置。

我看到一些奇怪的(而且似乎没有记录)行为,即 init 容器在 redis 容器启动之前按预期运行,但是随后它们每小时运行一次,接近一个小时。我已经在部署和 statefulset 上使用busybox init 容器(什么都不做)测试了这种行为,并体验了相同的行为,因此它不是特定于这个redis pod。

我已经用 k8s 1.6 和 1.8 在裸机上对此进行了测试,结果相同,但是当将 init 容器应用于 GKE(k8s 1.7)时,不会发生这种行为。我看不到 GKE 的任何标志kubelet来指示这种行为。

请参阅下文,以kubectl describe pod显示在主 pod 尚未退出/崩溃时运行 init 容器。

Name:           redis-sentinel-1
Namespace:      (redacted)
Node:           (redacted)/(redacted)
Start Time:     Mon, 12 Mar 2018 06:20:55 +0000
Labels:         app=redis-sentinel
                controller-revision-hash=redis-sentinel-7cc557cf7c
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"StatefulSet","namespace":"(redacted)","name":"redis-sentinel","uid":"759a3a3b-25bd-11e8-a8ce-0242ac110...
                security.alpha.kubernetes.io/unsafe-sysctls=net.core.somaxconn=1024
Status:         Running
IP:             (redacted)
Controllers:    StatefulSet/redis-sentinel
Init Containers:
  redis-ha-server:
    Container ID:       docker://557d777a7c660b062662426ebe9bbf6f9725fb9d88f89615a8881346587c1835
    Image:              tarosky/k8s-redis-ha:sentinel-3.0.1
    Image ID:           docker-pullable://tarosky/k8s-redis-ha@sha256:98e09ef5fbea5bfd2eb1858775c967fa86a92df48e2ec5d0b405f7ca3f5ada1c
    Port:
    State:              Terminated
      Reason:           Completed
      Exit Code:        0
      Started:          Tue, 13 Mar 2018 03:01:12 +0000
      Finished:         Tue, 13 Mar 2018 03:01:12 +0000
    Ready:              True
    Restart Count:      0
    Environment:        <none>
    Mounts:
      /opt from opt (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hkj6d (ro)
  -redis-init:
    Container ID:       docker://18c4e353233a6827999ae4a16adf1f408754a21d80a8e3374750fdf9b54f9b1a
    Image:              gcr.io/(redacted)/redis-init
    Image ID:           docker-pullable://gcr.io/(redacted)/redis-init@sha256:42042093d58aa597cce4397148a2f1c7967db689256ed4cc8d9f42b34d53aca2
    Port:
    State:              Terminated
      Reason:           Completed
      Exit Code:        0
      Started:          Tue, 13 Mar 2018 03:01:25 +0000
      Finished:         Tue, 13 Mar 2018 03:01:25 +0000
    Ready:              True
    Restart Count:      0
    Environment:        <none>
    Mounts:
      /opt from opt (rw)
      /secrets/redis-password from redis-password (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hkj6d (ro)
Containers:
  redis-sentinel:
    Container ID:       docker://a54048cbb7ec535c841022c543a0d566c9327f37ede3a6232516721f0e37404d
    Image:              redis:3.2
    Image ID:           docker-pullable://redis@sha256:474fb41b08bcebc933c6337a7db1dc7131380ee29b7a1b64a7ab71dad03ad718
    Port:               26379/TCP
    Command:
      /opt/bin/k8s-redis-ha-sentinel
    Args:
      /opt/sentinel.conf
    State:              Running
      Started:          Mon, 12 Mar 2018 06:21:02 +0000
    Ready:              True
    Restart Count:      0
    Readiness:          exec [redis-cli -p 26379 info server] delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SERVICE:          redis-server
      SERVICE_PORT:     redis-server
    Mounts:
      /opt from opt (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hkj6d (ro)
  redis-sword:
    Container ID:       docker://50279448bbbf175b6f56f96dab59061c4652c2117452ed15b3a5380681c7176f
    Image:              tarosky/k8s-redis-ha:sword-3.0.1
    Image ID:           docker-pullable://tarosky/k8s-redis-ha@sha256:2315c7a47d9e47043d030da270c9a1252c2cfe29c6e381c8f50ca41d3065db6d
    Port:
    State:              Running
      Started:          Mon, 12 Mar 2018 06:21:03 +0000
    Ready:              True
    Restart Count:      0
    Environment:
      SERVICE:          redis-server
      SERVICE_PORT:     redis-server
      SENTINEL:         redis-sentinel
      SENTINEL_PORT:    redis-sentinel
    Mounts:
      /opt from opt (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hkj6d (ro)
Conditions:
  Type          Status
  Initialized   True
  Ready         True
  PodScheduled  True
Volumes:
  opt:
    Type:       HostPath (bare host directory volume)
    Path:       /store/redis-sentinel/opt
  redis-password:
    Type:       Secret (a volume populated by a Secret)
    SecretName: redis-password
    Optional:   false
  default-token-hkj6d:
    Type:       Secret (a volume populated by a Secret)
    SecretName: default-token-hkj6d
    Optional:   false
QoS Class:      BestEffort
Node-Selectors: <none>
Tolerations:    <none>
Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath                           Type            Reason  Message
  ---------     --------        -----   ----                    -------------                           --------        ------  -------
  20h           30m             21      kubelet, 10.1.3.102     spec.initContainers{redis-ha-server}    Normal          Pulling pulling image "tarosky/k8s-redis-ha:sentinel-3.0.1"
  21h           30m             22      kubelet, 10.1.3.102     spec.initContainers{redis-ha-server}    Normal          Started Started container
  21h           30m             22      kubelet, 10.1.3.102     spec.initContainers{redis-ha-server}    Normal          Created Created container
  20h           30m             21      kubelet, 10.1.3.102     spec.initContainers{redis-ha-server}    Normal          Pulled  Successfully pulled image "tarosky/k8s-redis-ha:sentinel-3.0.1"
  21h           30m             22      kubelet, 10.1.3.102     spec.initContainers{redis-init}         Normal          Pulling pulling image "gcr.io/(redacted)/redis-init"
  21h           30m             22      kubelet, 10.1.3.102     spec.initContainers{redis-init}         Normal          Pulled  Successfully pulled image "gcr.io/(redacted)/redis-init"
  21h           30m             22      kubelet, 10.1.3.102     spec.initContainers{redis-init}         Normal          Created Created container
  21h           30m             22      kubelet, 10.1.3.102     spec.initContainers{redis-init}         Normal          Started Started container

请注意 pod 中的 Containers 开始于Mon, 12 Mar 2018 06:21:02 +0000(0 次重启)和 Init Containers 开始于Tue, 13 Mar 2018 03:01:12 +0000. 这些似乎每隔一小时几乎每小时重新运行一次。

我们的裸机必须在某个地方为 init 容器配置错误?任何人都可以阐明这种奇怪的行为吗?

4

1 回答 1

0

如果您正在修剪退出的容器,则可能是容器修剪/移除的原因。在我的测试中,似乎从 Docker 引擎(每小时或以其他方式)中删除的已退出初始化容器,例如使用“docker system prune -f”将导致 Kubernetes 重新启动初始化容器。如果这仍然存在,这是您的问题吗?

另外,请参阅https://kubernetes.io/docs/concepts/cluster-administration/kubelet-garbage-collection/获取 Kubelet 垃圾收集文档,该文档似乎支持这些类型的任务(而不是需要自己实现)

于 2018-08-19T20:46:25.463 回答