docker - 在 kubeadm 集群中升级 calico-node 的问题

Question

我将按照“单独升级组件”的链接升级 Calico 节点和 cni

方向非常清楚（我将封锁每个节点并为 and 执行步骤calico/cni）calico/node，但我不确定是什么意思

Update the image in your process management to reference the new version

wrt升级calico/node容器。

否则，我看不到其他问题。我们的环境是一个 k8s kubeadm 集群。

我想真正的问题是：我在哪里告诉 k8s 使用更新版本的calico/node图像？

编辑

回答上述问题：

我只是kubectl delete -f对两者都做了一个calico.yaml，rbac-kdd.yaml然后kubectl create -f对这些文件的最新版本做了一个。

现在一切似乎都在 3.3.2 版本，但我现在在所有 calico-node pod 上都收到此错误：

Warning Unhealthy 84s (x181 over 31m) kubelet, thalia4 Readiness probe failed: calico/node is not ready: BIRD is not ready: BGP not established with <node IP addresses here

我跑了calicoctl nodd status，得到了

Calico process is running.

IPv4 BGP status
+---------------+-------------------+-------+----------+--------------------------------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |              INFO              |
+---------------+-------------------+-------+----------+--------------------------------+
| 134.x.x.163 | node-to-node mesh | start | 02:36:29 | Connect                        |
| 134.x.x.164 | node-to-node mesh | start | 02:36:29 | Connect                        |
| 134.x.x.165 | node-to-node mesh | start | 02:36:29 | Connect                        |
| 134.x.x.168 | node-to-node mesh | start | 02:36:29 | Active Socket: Host is         |
|             |                   |       |          | unreachable                    |
+---------------+-------------------+-------+----------+--------------------------------+

IPv6 BGP status
No IPv6 peers found.

我认为 134.xx168 无法访问是我收到上述健康检查警告的原因。

虽然不完全确定该怎么做。该节点在 k8s 集群中可用（这是 node thalia4）：

[gms@thalia0 calico]$ kubectl get nodes
NAME                  STATUS   ROLES    AGE   VERSION
thalia0               Ready    master   87d   v1.13.1
thalia1               Ready    <none>   48d   v1.13.1
thalia2               Ready    <none>   30d   v1.13.1
thalia3               Ready    <none>   87d   v1.13.1
thalia4               Ready    <none>   48d   v1.13.1

编辑 2

calicoctl node status在 thalia4 上给了

[sudo] password for gms:
Calico process is running.

IPv4 BGP status
+---------------+-------------------+-------+----------+---------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |  INFO   |
+---------------+-------------------+-------+----------+---------+
| 134.xx.xx.162 | node-to-node mesh | start | 02:36:29 | Connect |
| 134.xx.xx.163 | node-to-node mesh | start | 02:36:29 | Connect |
| 134.xx.xx.164 | node-to-node mesh | start | 02:36:29 | Connect |
| 134.xx.xx.165 | node-to-node mesh | start | 02:36:29 | Connect |
+---------------+-------------------+-------+----------+---------+

虽然kubectl describe node thalia4给了

Name:               thalia4.domain
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    dns=dns4
                    kubernetes.io/hostname=thalia4
                    node_name=thalia4
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 134.xx.xx.168/26
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Mon, 03 Dec 2018 14:17:07 -0600
Taints:             <none>
Unschedulable:      false
Conditions:
  Type             Status    LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------    -----------------                 ------------------                ------                       -------
  OutOfDisk        Unknown   Fri, 21 Dec 2018 11:58:38 -0600   Sat, 12 Jan 2019 16:44:10 -0600   NodeStatusUnknown            Kubelet stopped posting node status.
  MemoryPressure   False     Mon, 21 Jan 2019 20:54:38 -0600   Sat, 12 Jan 2019 16:50:18 -0600   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False     Mon, 21 Jan 2019 20:54:38 -0600   Sat, 12 Jan 2019 16:50:18 -0600   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False     Mon, 21 Jan 2019 20:54:38 -0600   Sat, 12 Jan 2019 16:50:18 -0600   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True      Mon, 21 Jan 2019 20:54:38 -0600   Sun, 20 Jan 2019 20:27:10 -0600   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  134.xx.xx.168
  Hostname:    thalia4
Capacity:
 cpu:                4
 ephemeral-storage:  6878Mi
 hugepages-1Gi:      0
 hugepages-2Mi:      0
 memory:             8009268Ki
 pods:               110
Allocatable:
 cpu:                4
 ephemeral-storage:  6490895145
 hugepages-1Gi:      0
 hugepages-2Mi:      0
 memory:             7906868Ki
 pods:               110
System Info:
 Machine ID:                 c011569a40b740a88a672a5cc526b3ba
 System UUID:                42093037-F27E-CA90-01E1-3B253813B904
 Boot ID:                    ffa5170e-da2b-4c09-bd8a-032ce9fca2ee
 Kernel Version:             3.10.0-957.1.3.el7.x86_64
 OS Image:                   Red Hat Enterprise Linux
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://1.13.1
 Kubelet Version:            v1.13.1
 Kube-Proxy Version:         v1.13.1
PodCIDR:                     192.168.4.0/24
Non-terminated Pods:         (3 in total)
  Namespace                  Name                        CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                  ----                        ------------  ----------  ---------------  -------------  ---
  kube-system                calico-node-8xqbs           250m (6%)     0 (0%)      0 (0%)           0 (0%)         24h
  kube-system                coredns-786f4c87c8-sbks2    100m (2%)     0 (0%)      70Mi (0%)        170Mi (2%)     47h
  kube-system                kube-proxy-zp4fk            0 (0%)        0 (0%)      0 (0%)           0 (0%)         31d
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests   Limits
  --------           --------   ------
  cpu                350m (8%)  0 (0%)
  memory             70Mi (0%)  170Mi (2%)
  ephemeral-storage  0 (0%)     0 (0%)
Events:              <none>

我认为这是一个防火墙问题，但我在 Slack 频道上被告知“如果你不使用主机端点，那么我们不会弄乱你主机的连接。听起来你有东西阻塞了端口 179在那台主机上。”

不知道那会在哪里？iptables 规则在所有节点上看起来都一样。

score 1 · Accepted Answer

我弄清楚了这个问题。我必须在所有节点上为cali-failsafe-in链的 iptables 添加显式规则。sudo iptables -A cali-failsafe-in -p tcp --match multiport --dport 179 -j ACCEPT

现在，所有节点上的一切似乎都可以正常工作：

IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+---------------+-------------------+-------+----------+-------------+
| 134.xx.xx.163 | node-to-node mesh | up    | 19:33:58 | Established |
| 134.xx.xx.164 | node-to-node mesh | up    | 19:33:40 | Established |
| 134.xx.xx.165 | node-to-node mesh | up    | 19:35:07 | Established |
| 134.xx.xx.168 | node-to-node mesh | up    | 19:35:01 | Established |
+---------------+-------------------+-------+----------+-------------+

score 0 · Accepted Answer

--network-plugin=cni 指定我们使用 cni 网络插件，实际 CNI 插件二进制文件位于 --cni-bin-dir（默认 /opt/cni/bin）和 CNI 插件配置位于 --cni-conf-目录（默认 /etc/cni/net.d）。

例如

--network-plugin=cni

--cni-bin-dir=/opt/cni/bin #可能有多个cni bin，比如calico/weave...，你可以使用命令'/opt/cni/bin/calico -v'来显示calico版本

--cni-conf-dir=/etc/cni/net.d #define详细cni插件配置，如下：

{
  "name": "calico-network",
  "cniVersion": "0.3.1",
  "plugins": [
    {
      "type": "calico",
      "mtu": 8950,
      "policy": {
        "type": "k8s"
      },
      "ipam": {
        "type": "calico-ipam",
        "assign_ipv6": "false",
        "assign_ipv4": "true"
      },
      "etcd_endpoints": "https://172.16.1.5:2379,https://172.16.1.9:2379,https://172.16.1.15:2379",
      "etcd_key_file": "/etc/etcd/ssl/etcd-client-key.pem",
      "etcd_cert_file": "/etc/etcd/ssl/etcd-client.pem",
      "etcd_ca_cert_file": "/etc/etcd/ssl/ca.pem",
      "kubernetes": {
        "kubeconfig": "/etc/kubernetes/cluster-admin.kubeconfig"
      }
    }
  ]
}

docker - 在 kubeadm 集群中升级 calico-node 的问题

2 回答 2

Related

Reference