我在具有以下节点的裸机（使用 VMware 虚拟机）上安装了 Kubernetes 集群
master-01 Ready control-plane,master 5d3h v1.21.3
master-02 Ready control-plane,master 5d3h v1.21.3
master-03 Ready control-plane,master 5d3h v1.21.3
worker-01 Ready <none> 5d2h v1.21.3
worker-02 Ready <none> 5d2h v1.21.3
worker-03 Ready <none> 5d2h v1.21.3
Metallb 安装为集群的负载均衡器，calico 安装为 CNI
我还用 helm 安装了 nginx-ingress-controller
$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update
$ helm install ingress-controller nginx-stable/nginx-ingress
我部署了一个简单的 nginx 服务器进行测试
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx-app
spec:
replicas: 2
selector:
matchLabels:
app: nginx-app
template:
metadata:
labels:
app: nginx-app
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx-app
#type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-myapp
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-service
port:
number: 80
我的负载均衡器类型的部署从 metallb 获取 IP 并且工作正常，但是当我添加入口时，虽然分配了 IP，但我收到错误 502 bad gateway，如下所示：
防火墙已启用，但所需端口已打开
6443/tcp 2379-2380/tcp 10250-10252/tcp 179/tcp 7946/tcp 7946/udp 8443/tcp on master nodes
10250/tcp 30000-32767/tcp 7946/tcp 7946/udp 8443/tcp 179/tcp on worker nodes
我的服务和 Pod 运行良好
kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
ingress-controller-nginx-ingress LoadBalancer 10.101.17.180 10.1.210.100 80:31509/TCP,443:30004/TCP 33m app=ingress-controller-nginx-ingress
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d <none>
nginx-service ClusterIP 10.101.48.198 <none> 80/TCP 31m app=nginx-app
我的入口日志给了我错误，没有到内部 IP 的路由
2021/07/29 07:46:24 [error] 42#42: *8 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:07:46:24 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
W0729 07:50:16.416830 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET / HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://10.1.210.100/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
W0729 07:56:43.420282 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:05:28.422594 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:10:45.425329 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
2021/07/29 08:13:59 [error] 42#42: *12 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:13:59 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
2021/07/29 08:14:09 [error] 42#42: *14 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:14:09 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
请问有什么想法吗？
编辑：正如这里询问的服务和豆荚的描述
$ kubectl describe pod nginx-deployment-6f7d8d4d55-sncdr
Name: nginx-deployment-6f7d8d4d55-sncdr
Namespace: default
Priority: 0
Node: worker-01/10.1.210.63
Start Time: Thu, 29 Jul 2021 08:43:59 +0100
Labels: app=nginx-app
pod-template-hash=6f7d8d4d55
Annotations: cni.projectcalico.org/podIP: 192.168.171.17/32
cni.projectcalico.org/podIPs: 192.168.171.17/32
Status: Running
IP: 192.168.171.17
IPs:
IP: 192.168.171.17
Controlled By: ReplicaSet/nginx-deployment-6f7d8d4d55
Containers:
nginx:
Container ID: docker://fc61b73f8a833ad13b8956d8ce151b221b75a58a9a2fbae928464f3b0a77cca2
Image: nginx
Image ID: docker-pullable://nginx@sha256:8f335768880da6baf72b70c701002b45f4932acae8d574dedfddaf967fc3ac90
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 29 Jul 2021 08:44:01 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wkc48 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-wkc48:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 16m default-scheduler Successfully assigned default/nginx-deployment-6f7d8d4d55-sncdr to worker-01
Normal Pulling 16m kubelet Pulling image "nginx"
Normal Pulled 16m kubelet Successfully pulled image "nginx" in 1.51808376s
Normal Created 16m kubelet Created container nginx
Normal Started 16m kubelet Started container nginx
$ kubectl describe svc ingress-controller-nginx-ingress
Name: ingress-controller-nginx-ingress
Namespace: default
Labels: app.kubernetes.io/instance=ingress-controller
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-controller-nginx-ingress
helm.sh/chart=nginx-ingress-0.10.0
Annotations: meta.helm.sh/release-name: ingress-controller
meta.helm.sh/release-namespace: default
Selector: app=ingress-controller-nginx-ingress
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.101.17.180
IPs: 10.101.17.180
LoadBalancer Ingress: 10.1.210.100
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 31509/TCP
Endpoints: 192.168.37.202:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 30004/TCP
Endpoints: 192.168.37.202:443
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 31108
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal IPAllocated 18m metallb-controller Assigned IP "10.1.210.100"
Normal nodeAssigned 3m21s (x182 over 18m) metallb-speaker announcing from node "worker-02"
$ kubectl describe svc nginx-service
Name: nginx-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=nginx-app
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.101.48.198
IPs: 10.101.48.198
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 192.168.171.17:80
Session Affinity: None
Events: <none>
$ kubectl exec -it ingress-controller-nginx-ingress-dd5db86dc-gqdpm -- /bin/bash
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17:80
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 10.101.48.198
curl: (7) Failed to connect to 10.101.48.198 port 80: Connection timed out
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl nginx-deployment-6f7d8d4d55-sncdr
curl: (6) Could not resolve host: nginx-deployment-6f7d8d4d55-sncdr
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$
老实说，我不明白为什么 curl svcip 不再起作用了；昨天它起作用了。