1

I have a haproxy as a load balancer running in k8s with a route to a service with two running pods. I want the server naming inside haproxy to correspond to the pod names behind my service. If I'm not mistaken the following configmap / annotation value should do exactly this: https://haproxy-ingress.github.io/docs/configuration/keys/#backend-server-naming. But for me it doesn't and for the life of me I can't find out why. The relevant parts of my configuration look like this:

controller deployment:

kind: Deployment
metadata:
  labels:
    run: haproxy-ingress
  name: haproxy-ingress
  namespace: haproxy-controller
spec:
  replicas: 2
  selector:
    matchLabels:
      run: haproxy-ingress
  template:
    metadata:
      labels:
        run: haproxy-ingress
    spec:
      serviceAccountName: haproxy-ingress-service-account
      containers:
        - name: haproxy-ingress
          image: haproxytech/kubernetes-ingress
          args:
            - --configmap=haproxy-controller/haproxy-ingress
            - --configmap-errorfiles=haproxy-controller/errorfile-conf
            - --default-ssl-certificate=haproxy-controller/haproxy-tls
            - --ingress.class=haproxy

controller service:

kind: Service
metadata:
  labels:
    run: haproxy-ingress
  name: haproxy-ingress
  namespace: haproxy-controller
spec:
  selector:
    run: haproxy-ingress
  type: ClusterIP
  ports:
    - name: https
      port: 443
      protocol: TCP
      targetPort: 443

controller configmap:

kind: ConfigMap
metadata:
  name: haproxy-ingress
  namespace: haproxy-controller
data:
  server-ssl: "true"
  scale-server-slots: "2"
  cookie-persistence: "LFR_SRV"
  backend-server-naming: "pod"
  backend-config-snippet: |
    cookie LFR_SRV indirect nocache insert maxidle 10m httponly secure

backend server ingress:

kind: Ingress
metadata:
  name: liferay-dxp
  namespace: backend
  annotations:
    kubernetes.io/ingress.class: "haproxy"
spec:
  tls:
    - secretName: backend-tls
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: backend
                port:
                  number: 443

The generated backend part of the haproxy.conf looks like this:

  mode http
  balance roundrobin
  option forwardfor
  cookie LFR_SRV indirect nocache insert
  ###_config-snippet_### BEGIN                                                                                                                                          
  cookie LFR_SRV indirect nocache insert maxidle 10m httponly secure
  ###_config-snippet_### END
  server SRV_1 10.xx.xx.xx:443 check ssl alpn h2,http/1.1 weight 128 cookie SRV_1 verify none
  server SRV_2 10.xx.xx.xx:443 check ssl alpn h2,http/1.1 weight 128 cookie SRV_2 verify none

Everything works fine except backend-server-naming: "pod". I also can't get any of the session-cookie-* properties from here to work. Because of that I used the backend-config-snippet to overwrite the cookie line in the generated haproxy.conf with my custom one (I added maxidle 10m httponly secure). What am I doing wrong?

4

1 回答 1

0

这里有一些提示可以帮助您解决问题。

确保您知道 haproxy-ingress 控制器的确切版本：

查看您共享的清单文件，很难判断haproxy-ingress-controller您在集群中运行的容器的确切版本（顺便说一句，将其保留为无标签是违反生产环境中的最佳实践的，请在此处阅读更多信息）。

要使backend-server-naming配置密钥正常工作，至少v0.8.1需要（它是向后移植的）。

在继续进行故障排除之前，首先请仔细检查您的入口部署的兼容性。

我对“backend-server-naming=pod”行为的观察

配置动态更新：

如果我正确理解有关此配置键的官方文档，将后端的服务器命名设置为 pod 名称 ( backend-server-naming=pod) 而不是sequences，确实支持 haproxy 配置的动态重新加载，但目前不支持对 haproxy 运行的动态更新-在后端部分对服务器名称进行时间配置（haproxy-ingress 作者在此处和此处进行了解释）

这意味着您需要首先重新启动您的 haproxy-ingress 控制器实例，以便能够看到后端服务器名称的变化反映在 haproxy 配置中，例如出现新 Pod 副本或由于 Pod 崩溃而更改 POD_IP 的情况（预计添加/更新服务器基于序列命名的条目）。

入口类：

我已经成功地测试了（参见下面的测试）基于field 的分类backend-server-naming=podIngress 类型的设置，而不是 deprecated annotation ，如您的情况：v0.13.4ingressClassNamekubernetes.io/ingress.class

我并不是说您的配置不起作用（它也应该起作用），但重要的是要知道，配置的动态更新（包括对后端配置的更改）不会发生在未分类的 Ingress 资源或错误分类的资源上，除非你真的在运行v0.12或更新版本。

测试：

# Ingress class 
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: my-class
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"
spec:
  controller: haproxy-ingress.github.io/controller

# Demo Ingress resource
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    haproxy-ingress.github.io/backend-server-naming: "pod"
  name: echoserver
spec:
  ingressClassName: my-class
  rules:
  - http:
      paths:
      - backend:
          service:
            name: echoserver
            port:
              number: 8080
        path: /
        pathType: Prefix

带有注释的 HA 代理配置：

backend default_echoserver_8080
    mode http
    balance roundrobin
    acl https-request ssl_fc
    http-request set-header X-Original-Forwarded-For %[hdr(x-forwarded-for)] if { hdr(x-forwarded-for) -m found }
    http-request del-header x-forwarded-for
    option forwardfor
    http-response set-header Strict-Transport-Security "max-age=15768000" if https-request
    # pod name start
    server echoserver-75d6f584bb-jlwb8 172.17.0.2:8080 weight 1 check inter 2s
    # pod name end
    server srv002 127.0.0.1:1023 disabled weight 1 check inter 2s
    server srv003 127.0.0.1:1023 disabled weight 1 check inter 2s
    server srv004 127.0.0.1:1023 disabled weight 1 check inter 2s
    ...
于 2021-12-06T17:07:44.013 回答