是否有可能使用 Openshift Origin 1.5.0(kubernetes 1.5.2)自动扩展部署并为此使用自定义指标?
Kubernetes 文档指出,从 1.2 版本开始支持使用自定义指标进行自动缩放。这看起来是对的,因为 Openshift 水平 pod 自动缩放器 (HPA) 试图获取一些指标并计算所需的指标。但是我的配置未能成功执行此操作。伙计们,请帮助我找出我做错了什么。
那么,会发生什么:
我已经按照 Origin 最新文档中的建议设置了一个指标(所有步骤都已通过): https ://docs.openshift.org/latest/install_config/cluster_metrics.html ;
- 我有一个应用程序,它正在使用部署类对象进行部署;
- 此应用程序使用 http json 端点公开自定义指标;
- 正在收集和存储自定义指标 - 这显示在相应 pod 的 Metrics 选项卡中的 Openshift 原始 UI 中;
- 在我创建 HPA 之后 - 出现了一些关于收集自定义指标的警告,它会写出类似“收集自定义指标失败,没有收到任何准备好的 pod 的指标”之类的内容;
- 我使用 API 版本 1 创建 HPA 并包含注释
alpha/target.custom-metrics.podautoscaler.kubernetes.io: '{"items":[{"name":"requests_count", "value": "10"}]}'
; 如果我通过主代理请求部署的 heapster 应用程序,我会收到类似这样的信息
{“元数据”:{},“项目”:[{“元数据”:{“名称”:“resty-1722683747-kmbw0”,“命名空间”:“可用性演示”,“创建时间戳”:“2017-05- 24T09:50:24Z" }, "timestamp": "2017-05-24T09:50:00Z", "window": "1m0s", "containers": [ { "name": "resty", "usage": { “cpu”:“0”,“内存”:“2372Ki” } } ] } ] }
如您所见,确实没有自定义指标,而我的自定义指标名为
requests_count
.
我应该采取哪些步骤来成功实施自定义指标自动缩放?
通过 Openshift 控制台 UI 收集和公开自定义指标的屏幕截图
更新: 在 openshift 主日志警告看起来像这样:
I0524 10:17:47.537985 1 panics.go:76GET /apis/extensions/v1beta1/namespaces/availability-demo/deployments/resty/scale: (3.379724ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.543354 1 panics.go:76] GET /api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/apis/metrics/v1alpha1/namespaces/availability-demo/pods?labelSelector=app%3Dresty: (4.830135ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.553255 1 panics.go:76] GET /api/v1/namespaces/availability-demo/pods?labelSelector=app%3Dresty: (8.864864ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.559909 1 panics.go:76] GET /api/v1/namespaces/availability-demo/pods?labelSelector=app%3Dresty: (5.725342ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.560977 1 panics.go:76] PATCH /api/v1/namespaces/availability-demo/events/resty.14c14bbf8b89534c: (6.385846ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.565418 1 panics.go:76] GET /api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/api/v1/model/namespaces/availability-demo/pod-list/resty-1722683747-kmbw0/metrics/custom/requests_count?start=2017-05-24T10%3A12%3A47Z: (5.015336ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.569843 1 panics.go:76] GET /api/v1/namespaces/availability-demo/pods?labelSelector=app%3Dresty: (4.040029ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.575530 1 panics.go:76] PUT /apis/autoscaling/v1/namespaces/availability-demo/horizontalpodautoscalers/resty/status: (4.894835ms) 200 [[openshift/v1.5.2+43a9be4 (linux/amd64) kubernetes/43a9be4 system:serviceaccount:openshift-infra:hpa-controller] 10.105.8.81:33945]
I0524 10:17:47.575856 1 horizontal.go:438] Successfully updated status for resty
W0524 10:17:47.575890 1 horizontal.go:104] Failed to reconcile resty: failed to compute desired number of replicas based on Custom Metrics for Deployment/availability-demo/resty: failed to get custom metric value: did not recieve metrics for any ready pods
更新:发现什么请求 HPA 问题通过代理收集自定义指标。此请求始终返回空指标数组:
GET /api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/api/v1/model/namespaces/availability-demo/pod-list/availability-example-1694583826-55hqh/metrics/custom/requests_count?start=2017-05-25T13%3A14%3A24Z HTTP/1.1
Host: kubernetes-master:8443
Authorization: Bearer hpa-agent-token
它返回
{"items":[{"metrics":[],"latestTimestamp":"0001-01-01T00:00:00Z"}]}
更新:事实证明,HPA 通过代理请求 heapster,而 heapster - 反过来 - 请求“摘要”kubernetes api。那么问题是 - 为什么 kubernetes “summary” api 没有回答上述请求的指标,尽管指标存在?