1

我已经使用 kubeadm 设置了 Kubernetes 集群,它运行良好。我能够部署 pod 和服务。我曾尝试重新启动集群,它过去没有任何问题。突然,几天后,我不得不重新启动我的机器(在 vms 上托管集群)并且集群开始没有出现。无法理解为什么现在开始发生这种情况。有人可以帮我解决这个问题吗?

这是我的集群的外观:

 Node                         Hostname         IP 
 master                       master-kube      192.168.50.10
 worker-node-1                node-1-kube      192.168.50.11
 worker-node-2                node-2-kube      192.168.50.12  

所有节点(ubuntu 仿生)都托管在一个虚拟盒子上。

有关集群的更多详细信息:

kubelet = 1.16.0-00
kubeadm = 1.16.0-00
kubectl = 1.16.0-00
docker = 18.06.2-ce, build 6d37f41
pod-network = calico-latest
command used to init cluster = kubeadm init --apiserver-advertise-address="192.168.50.10" --apiserver-cert-extra-sans="192.168.50.10" --pod-network-cidr=172.16.0.0/16

我想强调一下我目前在重新启动 kubelet 时遇到的错误:


kubelet[19946]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubel
kubelet[19946]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet
kubelet[19946]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubel
kubelet[19946]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet
kubelet[19946]: I1221 05:37:49.631489   19946 server.go:410] Version: v1.16.0
kubelet[19946]: I1221 05:37:49.631636   19946 plugins.go:100] No cloud provider specified.
kubelet[19946]: I1221 05:37:49.631649   19946 server.go:773] Client rotation is on, will bootstrap in background
kubelet[19946]: I1221 05:37:49.633624   19946 certificate_store.go:129] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
kubelet[19946]: I1221 05:37:49.658641   19946 server.go:644] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
kubelet[19946]: I1221 05:37:49.659118   19946 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
kubelet[19946]: I1221 05:37:49.659269   19946 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime
kubelet[19946]: I1221 05:37:49.659508   19946 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
kubelet[19946]: I1221 05:37:49.659604   19946 container_manager_linux.go:305] Creating device plugin manager: true
kubelet[19946]: I1221 05:37:49.659735   19946 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} <nil> {{} [0 0 0]} 0x
kubelet[19946]: I1221 05:37:49.659850   19946 state_mem.go:36] [cpumanager] initializing new in-memory state store
kubelet[19946]: I1221 05:37:49.660030   19946 state_mem.go:84] [cpumanager] updated default cpuset: ""
kubelet[19946]: I1221 05:37:49.660127   19946 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
kubelet[19946]: I1221 05:37:49.660220   19946 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{{0 0} 0x799d338 10000000000 0xc0009ee9c0 <nil> <nil> <nil> <nil> map[memory:{{1
kubelet[19946]: I1221 05:37:49.660363   19946 kubelet.go:287] Adding pod path: /etc/kubernetes/manifests
kubelet[19946]: I1221 05:37:49.660502   19946 kubelet.go:312] Watching apiserver
kubelet[19946]: E1221 05:37:49.663227   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
kubelet[19946]: E1221 05:37:49.663829   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
kubelet[19946]: I1221 05:37:49.665047   19946 client.go:75] Connecting to docker on unix:///var/run/docker.sock
kubelet[19946]: I1221 05:37:49.665066   19946 client.go:104] Start docker client with request timeout=2m0s
kubelet[19946]: E1221 05:37:49.665772   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
kubelet[19946]: W1221 05:37:49.667035   19946 docker_service.go:563] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
kubelet[19946]: I1221 05:37:49.667144   19946 docker_service.go:240] Hairpin mode set to "hairpin-veth"
kubelet[19946]: I1221 05:37:49.694036   19946 docker_service.go:255] Docker cri networking managed by cni
kubelet[19946]: I1221 05:37:49.701825   19946 docker_service.go:260] Docker Info: &{ID:OF2B:RLPC:I6RY:MIMU:H5S4:3GBP:QB6T:IXQF:TMTB:7JBZ:XY2O:ICQO Containers:28 ContainersRunning:6 ContainersPaused:0 Containers
kubelet[19946]: I1221 05:37:49.702214   19946 docker_service.go:273] Setting cgroupDriver to cgroupfs
kubelet[19946]: I1221 05:37:49.711797   19946 remote_runtime.go:59] parsed scheme: ""
 kubelet[19946]: I1221 05:37:49.711813   19946 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
kubelet[19946]: I1221 05:37:49.711835   19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0  <nil>}] <nil>}
kubelet[19946]: I1221 05:37:49.711841   19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: I1221 05:37:49.711856   19946 remote_image.go:50] parsed scheme: ""
kubelet[19946]: I1221 05:37:49.711860   19946 remote_image.go:50] scheme "" not registered, fallback to default scheme
Dec 21 05:37:49 master-kube kubelet[19946]: I1221 05:37:49.711866   19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0  <nil>}] <nil>}
kubelet[19946]: I1221 05:37:49.711870   19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: E1221 05:37:49.916339   19946 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
kubelet[19946]:         For verbose messaging see aws.Config.CredentialsChainVerboseErrors
 kubelet[19946]: I1221 05:37:49.920329   19946 kuberuntime_manager.go:207] Container runtime docker initialized, version: 18.06.2-ce, apiVersion: 1.38.0
kubelet[19946]: I1221 05:37:49.925361   19946 server.go:1065] Started kubelet
Dec 21 05:37:49 master-kube kubelet[19946]: E1221 05:37:49.925583   19946 kubelet.go:1302] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data in memory ca
kubelet[19946]: E1221 05:37:49.927007   19946 event.go:246] Unable to write event: 'Post https://192.168.50.10:6443/api/v1/namespaces/default/events: dial tcp 192.168.50.10:6443: connect: connection refused' (m
kubelet[19946]: I1221 05:37:49.928287   19946 server.go:145] Starting to listen on 0.0.0.0:10250
kubelet[19946]: I1221 05:37:49.931484   19946 server.go:354] Adding debug handlers to kubelet server.
kubelet[19946]: I1221 05:37:49.935058   19946 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
kubelet[19946]: I1221 05:37:49.935868   19946 status_manager.go:156] Starting to sync pod status with apiserver
kubelet[19946]: I1221 05:37:49.936434   19946 kubelet.go:1822] Starting kubelet main sync loop.
kubelet[19946]: I1221 05:37:49.936965   19946 kubelet.go:1839] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
kubelet[19946]: I1221 05:37:49.943766   19946 volume_manager.go:249] Starting Kubelet Volume Manager
kubelet[19946]: E1221 05:37:49.955051   19946 controller.go:135] failed to ensure node lease exists, will retry in 200ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: I1221 05:37:49.960391   19946 desired_state_of_world_populator.go:131] Desired state populator starts to run
kubelet[19946]: E1221 05:37:49.960555   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
kubelet[19946]: E1221 05:37:49.960939   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
kubelet[19946]: I1221 05:37:49.977365   19946 clientconn.go:104] parsed scheme: "unix"
kubelet[19946]: I1221 05:37:49.977587   19946 clientconn.go:104] scheme "unix" not registered, fallback to default scheme
kubelet[19946]: I1221 05:37:49.977801   19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}
kubelet[19946]: I1221 05:37:49.977941   19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: W1221 05:37:49.988778   19946 cni.go:328] CNI failed to retrieve network namespace path: **cannot find network namespace for the terminated container "2fe393eaa9c070dcc24651a27fa01e421ada7a8b20a4d**
kubelet[19946]: W1221 05:37:49.991163   19946 docker_sandbox.go:394] **failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod "coredns-5644d7b6d9-j8f7f_kube-system": CNI fai**
kubelet[19946]: W1221 05:37:49.994413   19946 docker_sandbox.go:394] failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod "coredns-5644d7b6d9-j8f7f_kube-system": CNI fai
kubelet[19946]: I1221 05:37:50.039352   19946 kubelet.go:1839] skipping pod synchronization - container runtime status check may not have completed yet
kubelet[19946]: E1221 05:37:50.045922   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.046091   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.047491   19946 kubelet_node_status.go:72] Attempting to register node master-kube
kubelet[19946]: E1221 05:37:50.047805   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
kubelet[19946]: I1221 05:37:50.048908   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.050080   19946 cpu_manager.go:161] [cpumanager] starting with none policy
kubelet[19946]: I1221 05:37:50.050184   19946 cpu_manager.go:162] [cpumanager] reconciling every 10s
kubelet[19946]: I1221 05:37:50.050283   19946 policy_none.go:42] [cpumanager] none policy: Start
kubelet[19946]: I1221 05:37:50.052514   19946 plugin_manager.go:116] Starting Kubelet Plugin Manager
kubelet[19946]: E1221 05:37:50.054259   19946 eviction_manager.go:246] eviction manager: failed to get summary stats: failed to get node info: node "master-kube" not found
kubelet[19946]: 2019-12-21 05:37:50.059 [INFO][20107] k8s.go 228: Using Calico IPAM
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.155714   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.156669   19946 controller.go:135] failed to ensure node lease exists, will retry in 400ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: I1221 05:37:50.254696   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.254929   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: E1221 05:37:50.257253   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.264235   19946 kubelet_node_status.go:72] Attempting to register node master-kube
kubelet[19946]: E1221 05:37:50.267581   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
kubelet[19946]: I1221 05:37:50.278488   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.289042   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.298498   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: W1221 05:37:50.307965   19946 pod_container_deletor.go:75] Container "6d39199a688542a9336b33ed6e7d370ea6320ad47a2b17da800b0954bc985bec" not found in pod's containers
kubelet[19946]: W1221 05:37:50.308035   19946 pod_container_deletor.go:75] Container "d97a9bcd4aa63b11db422d92fed678a90878ab94528b3e6c1c7fd99200c972a7" not found in pod's containers
kubelet[19946]: E1221 05:37:50.361156   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.362961   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa1
kubelet[19946]: I1221 05:37:50.363626   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f3
kubelet[19946]: I1221 05:37:50.364328   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/68aa12790236de85ba80bddfd434
kubelet[19946]: I1221 05:37:50.364504   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/68aa12790236de85ba80bddfd43
kubelet[19946]: I1221 05:37:50.364802   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa1279023
kubelet[19946]: I1221 05:37:50.365575   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91
kubelet[19946]: I1221 05:37:50.366108   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/4e1bd6e5b41d60d13135315758
kubelet[19946]: I1221 05:37:50.366614   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/f15776993ab
kubelet[19946]: I1221 05:37:50.367265   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/cf6cdea881ad085c694fbe5bf3
kubelet[19946]: I1221 05:37:50.367499   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-data" (UniqueName: "kubernetes.io/host-path/cf6cdea881ad085c694fbe5bf37
kubelet[19946]: I1221 05:37:50.368086   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f36
kubelet[19946]: I1221 05:37:50.368745   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f
kubelet[19946]: I1221 05:37:50.369349   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/f1577
kubelet[19946]: I1221 05:37:50.369922   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa12790236de85b
kubelet[19946]: E1221 05:37:50.461604   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.598762   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.598807   19946 controller.go:135] failed to ensure node lease exists, will retry in 800ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: E1221 05:37:50.667616   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.667813   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:50 master-kube kubelet[19946]: I1221 05:37:50.669813   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.670999   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:50 master-kube kubelet[19946]: I1221 05:37:50.678853   19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.680121   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.701314   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.802348   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.903471   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.994146   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.994146   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.004535   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.105943   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.207194   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.320505   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.401092   19946 controller.go:135] failed to ensure node lease exists, will retry in 1.6s, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-le
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.424687   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: I1221 05:37:51.485948   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:51 master-kube kubelet[19946]: I1221 05:37:51.493301   19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.493752   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.524956   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.625893   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.669061   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.671281   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.672457   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.726204   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.827363   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.929008   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.996179   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.997820   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.035831   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.136937   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.238053   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.338740   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.439243   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.549008   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.649594   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.672587   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.673694   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.674519   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.749925   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.850243   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.950601   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.997582   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.999199   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.002688   19946 controller.go:135] failed to ensure node lease exists, will retry in 3.2s, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-le
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.051016   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: I1221 05:37:53.094012   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:53 master-kube kubelet[19946]: I1221 05:37:53.097332   19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.098210   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.152044   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.253059   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.353765   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.454334   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.554929   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.655913   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.675733   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.678185   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.678624   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.756190   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.856636   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.965559   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.004697   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.005008   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.070989   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.171403   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.272286   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.372573   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.474665   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.574953   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.675738   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.677084   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.680293   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.681313   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.782388   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.883757   19946 kubelet.go:2267] node "master-kube" not found

主节点上的 docker ps 命令: 在此处输入图像描述

4

1 回答 1

1

为了更好地了解这篇文章,我将适用于此 OP 的解决方案发布为Community Wiki.

该问题已通过完成@suren谁建议删除 var/lib/kubelet并重新安装 kubelet 发布的提示得到解决。

一些额外的故障排除步骤可以在 Kubernetes 文档 -故障排除 kubeadm中找到。

在某些情况下,也重新启动kubelet并且docker service可能会有所帮助。

sudo service docker restart
sudo systemctl restart kubelet
于 2021-03-17T15:08:33.163 回答