0

我目前有以下设置：

  • Cloudflare https 代理 dns 将所有流量路由到我的服务器（使用 CloudFlare Origin CA 进行端到端 [client->proxy 和 proxy->server] 加密）
  • Nginx 反向代理，充当我服务器的所有流量的 SSL 终止（我相信原始签名证书）
  • Nginx 将访问 kube.my-domain.com 的任何内容代理到我的 kubeapi 服务器，如下所示：proxy_pass https://127.0.0.1:6443
  • kubeapi 在我的服务器上运行

目前我可以通过在我的 kubeconfig 中指定来很好地连接到我的集群：

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: https://127.0.0.1:6443  <-----------------------------
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    namespace: test-namespace
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED

但是，如果我将 kubeconfig 中的服务器更改为指向我的域：server: https://kube.my-domain.com我收到以下错误：

user@master:~$ kubectl get deployments
Unable to connect to the server: x509: certificate signed by unknown authority

现在我假设这是因为我的流量必须通过 cloudflare 的代理，并且一路上有不同的证书。所以我尝试附加 --insecure-skip-tls-verify=true 并得到以下响应：

user@master:~$ kubectl get deployments --insecure-skip-tls-verify=true
error: the server doesn't have a resource type "deployments"

所以我查了一下，发现很多时候是因为 RBAC（即使我应该以 kubernetes-admin 身份进行身份验证，当我在本地连接时它工作......）

I0317 00:41:09.281404   31989 loader.go:375] Config loaded from file:  /home/user/.kube/config
I0317 00:41:09.281813   31989 round_trippers.go:423] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://kube.my-domain.com/api?timeout=32s'
I0317 00:41:09.493885   31989 round_trippers.go:443] GET https://kube.my-domain.com/api?timeout=32s 403 Forbidden in 212 milliseconds
I0317 00:41:09.493921   31989 round_trippers.go:449] Response Headers:
I0317 00:41:09.493970   31989 round_trippers.go:452]     Set-Cookie: REDACTED; expires=Thu, 16-Apr-20 04:41:09 GMT; path=/; domain=.my-domain.com; HttpOnly; SameSite=Lax; Secure
I0317 00:41:09.493991   31989 round_trippers.go:452]     Expect-Ct: max-age=604800, report-uri="REDACTED"
I0317 00:41:09.494002   31989 round_trippers.go:452]     Server: cloudflare
I0317 00:41:09.494027   31989 round_trippers.go:452]     Cf-Ray: REDACTED
I0317 00:41:09.494065   31989 round_trippers.go:452]     Date: Tue, 17 Mar 2020 04:41:09 GMT
I0317 00:41:09.494070   31989 round_trippers.go:452]     X-Content-Type-Options: nosniff
I0317 00:41:09.494091   31989 round_trippers.go:452]     Cf-Cache-Status: DYNAMIC
I0317 00:41:09.494112   31989 round_trippers.go:452]     Content-Type: application/json
I0317 00:41:09.498893   31989 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/api\"","reason":"Forbidden","details":{},"code":403}
I0317 00:41:09.503266   31989 round_trippers.go:423] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://kube.my-domain.com/apis?timeout=32s'
I0317 00:41:09.640948   31989 round_trippers.go:443] GET https://kube.my-domain.com/apis?timeout=32s 403 Forbidden in 137 milliseconds
I0317 00:41:09.640992   31989 round_trippers.go:449] Response Headers:
I0317 00:41:09.641000   31989 round_trippers.go:452]     Date: Tue, 17 Mar 2020 04:41:09 GMT
I0317 00:41:09.641006   31989 round_trippers.go:452]     Content-Type: application/json
I0317 00:41:09.641011   31989 round_trippers.go:452]     Cf-Cache-Status: DYNAMIC
I0317 00:41:09.641016   31989 round_trippers.go:452]     Expect-Ct: max-age=604800, report-uri="REDACTED"
I0317 00:41:09.641021   31989 round_trippers.go:452]     Set-Cookie: REDACTED; expires=Thu, 16-Apr-20 04:41:09 GMT; path=/; domain=.my-domain.com; HttpOnly; SameSite=Lax; Secure
I0317 00:41:09.641027   31989 round_trippers.go:452]     X-Content-Type-Options: nosniff
I0317 00:41:09.641031   31989 round_trippers.go:452]     Server: cloudflare
I0317 00:41:09.641056   31989 round_trippers.go:452]     Cf-Ray: REDACTED

我究竟做错了什么？此外，公开我的集群并确保其安全的最佳方式是什么？

我觉得这与 NGINX 的 SSL 终止有关，删除了 api-server 识别我所必需的东西。

谢谢。

更新：似乎 cloudflare 代理 pr NGINX 正在剥离身份验证证书和密钥，因此导致请求的用户变得匿名。还在研究这个。

4

1 回答 1

2

无法连接到服务器：x509：证书由未知机构签名

上述错误表明https://kube.my-domain.com端点提供的服务器证书未由您在 kubeconfig 文件certificate-authority-data部分中拥有的证书颁发机构签名。在 kubeconfig 中，您需要配置正确的证书颁发机构 (CA)。

于 2020-03-17T05:04:07.267 回答