aws-msk - KEDA 与 AWS MSK 集成

Question

我正在使用 AWS MSK，是否可以与 Keda 集成？我找不到 AWS MSK 的任何相关文档。

Answer 1

What do you exactly mean by integrate with AWS MSK? What do you want to scale? Cluster or Consumers?

In case you want to scale the cluster, the answer is no. That's not what Keda is meant for.

On the contrary, if you want to scale consumers, than the answer is yes, as MSK is just a managed (by AWS) Kafka cluster and it already exists a Kafka trigger (aka Scaler) for that whose description is here. That's based on consumer lag metric. After having deployed your consumer onto Kubernetes, all you have to do is apply a ScaledObject (Custom Resource), e.g.

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    deploymentName: myDeployment # the consumer deployment you want to scale
  pollingInterval: 30
  triggers:
  - type: kafka
    metadata:
      bootstrapServers: localhost:9092
      consumerGroup: my-group       # Make sure that this consumer group name is the same one as the one that is consuming topics
      topic: test-topic
      # Optional
      lagThreshold: "50"

Hope I helped.