我们的安全部门对出口流量的要求非常严格:POD 内的每个应用程序都必须通过使用应用程序专用证书的 mTLS 身份验证 (app-proxy) 代理。他们建议使用带隧道的 squid 来处理双重 mTLS(一个用于代理,另一个用于特定流量应用程序服务器),但后来我们强制应用程序支持 ssl。Istio 可以进来做这项工作,但我们不能使用开箱即用的 ISTIO_MUTUAL 模式(在 istio-proxy 和 egress gateway 之间)。
因此,我尝试使用示例为出口流量配置双向 TLS 发起,方法是稍微修改如下(用 #- 和 #+ 标记的更改):
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- my-nginx.mesh-external.svc.cluster.local
tls:
#mode: ISTIO_MUTUAL #-
mode: MUTUAL #+
credentialName: egress-gateway-credential #+
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: egressgateway-for-nginx
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: nginx
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
#mode: ISTIO_MUTUAL #-
mode: MUTUAL #+
credentialName: egress-app-credential #+
sni: my-nginx.mesh-external.svc.cluster.local
其中秘密已创建:
kubectl create -n istio-system secret generic egress-app-credential \
--from-file=tls.key=client.app.key \
--from-file=tls.crt=client.app.crt \
--from-file=ca.crt=some-root.crt
kubectl create -n istio-system secret generic egress-gateway-credential \
--from-file=tls.key=egress.key \
--from-file=tls.crt=egress.crt \
--from-file=ca.crt=some-root.crt
我认为这在逻辑上是正确的,但这可能不是因为我得到了错误:
kubectl exec "$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- curl -vsS http://my-nginx.mesh-external.svc.cluster.local
* Trying 10.98.10.231:80...
* Connected to my-nginx.mesh-external.svc.cluster.local (10.98.10.231) port 80 (#0)
> GET / HTTP/1.1
> Host: my-nginx.mesh-external.svc.cluster.local
> User-Agent: curl/7.77.0-DEV
> Accept: */*
>
upstream connect error or disconnect/reset before headers. reset reason: connection termination* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< content-length: 95
< content-type: text/plain
< date: Mon, 07 Jun 2021 11:01:08 GMT
< server: envoy
<
{ [95 bytes data]
* Connection #0 to host my-nginx.mesh-external.svc.cluster.local left intact
附加信息(上述请求的 istio-egressgateway 日志):
- ISTIO_MUTUAL(示例 - 标准 istio 代码)
客户端 pod 日志:
istio-proxy [2021-06-08T09:18:02.777Z] "GET / HTTP/1.1" 200 - via_upstream - "-" 0 612 2 1 "-" "curl/7.77.0-DEV" "148be8db-5675-40eb-a246-26f51a5c73d2" "my-nginx.mesh-external.svc.cluste │
│ r.local" "172.17.0.7:8443" outbound|443|nginx|istio-egressgateway.istio-system.svc.cluster.local 172.17.0.5:37858 10.111.175.215:80 172.17.0.5:50610 - -
出口 pod 日志:
[2021-06-07T11:20:52.907Z] "GET / HTTP/1.1" 200 - via_upstream - "-" 0 612 2 1 "172.17.0.5" "curl/7.77.0-DEV" "f163fbb1-8c9d-4960-9814-fc7bf11549ff" "my-nginx.mesh-external.svc.c
- 自定义 MUTUAL 设置(IP:172.17.0.8 是 istio-egress POD):
客户端 pod 日志:
[2021-06-07T12:02:20.626Z] "GET / HTTP/1.1" 503 UC upstream_reset_before_response_started{connection_termination} - "-" 0 95 1 - "-" "curl/7.77.0-DEV" "5fb31226-21fd-4c10-882c-f72bed3483e7" "my-nginx.mesh-external.svc.cluster.local" "172.17.0.8:8443" outbound|443|nginx|istio-egressgateway.istio-system.svc.cluster.local 172.17.0.5:49588 10.98.10.231:80 172.17.0.5:41028 - -
出口 pod 日志:
[2021-06-07T11:20:38.018Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 172.17.0.8:8443 172.17.0.5:44558 - -
任何帮助都会很有价值,因为我自己也在为此苦苦挣扎,也许我在某个地方犯了一个逻辑错误。
编辑:至于 8443 端口号:
istioctl x describe pod istio-egressgateway-79fcc9c54b-bnbzm -n istio-system
Pod: istio-egressgateway-79fcc9c54b-bnbzm
Pod Ports: 8080 (istio-proxy), 8443 (istio-proxy), 15090 (istio-proxy)
Suggestion: add 'version' label to pod for Istio telemetry.
--------------------
Service: istio-egressgateway
Port: http2 80/HTTP2 targets pod port 8080
Port: https 443/HTTPS targets pod port 8443
测试:
- 1.10
- 1.9.2