我按照https://coreos.com/kubernetes/docs/latest/getting-started.html在 OpenStack 中设置了多节点 Kubernetes 集群（3 个 etcd，2 个主节点和 2 个节点）
所有虚拟机都有 CoreOS 1185.3.0
kubectl version
Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.4.3", GitCommit:"ae4550cc9c89a593bcda6678df201db1b208133b", GitTreeState:"clean", BuildDate:"2016-08-26T18:13:23Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.0+coreos.0", GitCommit:"278a1f7034bdba61cba443722647da1a8204a6fc", GitTreeState:"clean", BuildDate:"2016-09-26T20:48:37Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
kubectl get nodes返回集群是健康的
NAME STATUS AGE
172.29.0.157 Ready,SchedulingDisabled 1d
172.29.0.158 Ready,SchedulingDisabled 1d
172.24.0.120 Ready 1d
172.24.0.121 Ready 1d
kubectl get pods --namespace=kube-system返回 kube-dns 和 kubernetes-dashboard pod 状态为 CrashLoopBackOff
NAME READY STATUS RESTARTS AGE
heapster-v1.2.0-3646253287-xweg5 2/2 Running 0 2h
kube-apiserver-172.29.0.157 1/1 Running 2 1d
kube-apiserver-172.29.0.158 1/1 Running 1 1d
kube-controller-manager-172.29.0.157 1/1 Running 2 1d
kube-controller-manager-172.29.0.158 1/1 Running 1 1d
kube-dns-v19-h7qyh 2/3 CrashLoopBackOff 13 2h
kube-proxy-172.24.0.120 1/1 Running 2 36m
kube-proxy-172.24.0.121 1/1 Running 2 37m
kube-proxy-172.29.0.157 1/1 Running 2 1d
kube-proxy-172.29.0.158 1/1 Running 1 1d
kube-scheduler-172.29.0.157 1/1 Running 2 1d
kube-scheduler-172.29.0.158 1/1 Running 1 1d
kubernetes-dashboard-v1.4.0-t2lpu 0/1 CrashLoopBackOff 12 2h
有人可以告诉我如何在这里找出确切的问题吗？
更新：
我能够获取 kube-dns 和 kubernetes-dashboard 容器的日志。尝试调用 kubernetes api 时似乎是证书问题。我已重新创建所有证书并替换它们。
设置 master 和 worker 指令， https ://coreos.com/kubernetes/docs/latest/deploy-master.html https://coreos.com/kubernetes/docs/latest/deploy-workers.html
Master 前面有一个负载均衡器。
最后重启了kubernetes 2个master VM和2个node VM。但是问题仍然存在于 kube-dns 和 kubernetes-dashboard 中。
kube-dns 容器日志
docker logs c8c82e68cde9
I1111 16:28:25.097452 1 server.go:94] Using https://10.3.0.1:443 for kubernetes master, kubernetes API: <nil>
I1111 16:28:25.103598 1 server.go:99] v1.4.0-alpha.2.1652+c69e3d32a29cfa-dirty
I1111 16:28:25.103789 1 server.go:101] FLAG: --alsologtostderr="false"
I1111 16:28:25.103928 1 server.go:101] FLAG: --dns-port="10053"
I1111 16:28:25.104185 1 server.go:101] FLAG: --domain="cluster.local."
I1111 16:28:25.104301 1 server.go:101] FLAG: --federations=""
I1111 16:28:25.104465 1 server.go:101] FLAG: --healthz-port="8081"
I1111 16:28:25.104607 1 server.go:101] FLAG: --kube-master-url=""
I1111 16:28:25.104718 1 server.go:101] FLAG: --kubecfg-file=""
I1111 16:28:25.104831 1 server.go:101] FLAG: --log-backtrace-at=":0"
I1111 16:28:25.104945 1 server.go:101] FLAG: --log-dir=""
I1111 16:28:25.105056 1 server.go:101] FLAG: --log-flush-frequency="5s"
I1111 16:28:25.105188 1 server.go:101] FLAG: --logtostderr="true"
I1111 16:28:25.105302 1 server.go:101] FLAG: --stderrthreshold="2"
I1111 16:28:25.105412 1 server.go:101] FLAG: --v="0"
I1111 16:28:25.105520 1 server.go:101] FLAG: --version="false"
I1111 16:28:25.105632 1 server.go:101] FLAG: --vmodule=""
I1111 16:28:25.105853 1 server.go:138] Starting SkyDNS server. Listening on port:10053
I1111 16:28:25.106185 1 server.go:145] skydns: metrics enabled on : /metrics:
I1111 16:28:25.106367 1 dns.go:167] Waiting for service: default/kubernetes
I1111 16:28:25.108281 1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I1111 16:28:25.108469 1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
E1111 16:28:25.176270 1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Endpoints: the server has asked for the client to provide credentials (get endpoints)
I1111 16:28:25.176774 1 dns.go:173] Ignoring error while waiting for service default/kubernetes: the server has asked for the client to provide credentials (get services kubernetes). Sleeping 1s before retrying.
kubernetes-dashboard 容器日志
docker logs b1d3b0fa617a
Starting HTTP server on port 9090
Creating API server client for https://10.3.0.1:443
Error while initializing connection to Kubernetes apiserver. This most likely means that the cluster is misconfigured (e.g., it has invalid apiserver certificates or service accounts configuration) or the --apiserver-host param points to a server that does not exist. Reason: the server has asked for the client to provide credentials
Kubernetes 节点日志
journalctl -u kubelet -f
Failed to list *api.Node: Get https://{load_balancer_ip}/api/v1/nodes?fieldSelector=metadata.name%3D172.24.0.121&resourceVersion=0: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca")
我在生成证书时遵循了https://coreos.com/kubernetes/docs/latest/openssl.html 。
以下 openssl 配置生成的 API 服务器证书
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
IP.1 = ${K8S_SERVICE_IP}
IP.2 = ${LOAD_BALANCER_IP}
我在这里错过了什么吗？
谢谢