我无法让制作人开始撰写关于 kafka 主题的文章。我的 kafka 服务已经用 srimzi 部署在 k8s 上。我的 k8s 集群在谷歌云服务上有 2 个节点。
正如我从 k8s 配置中看到的，所有服务都在：
这kafka-cluster-kafka-external-bootstrap是与 kafka 代理进行通信的服务 ( nodeport)。基本上它将请求从外部节点转发到内部代理服务。这里有一些细节：
按照指南（使用 minikube 作为集群示例），我提取了节点的 ip：
kubectl get nodes --output=jsonpath='{range .items[*]}{.status.addresses[?(@.type=="ExternalIP")].address}{"\n"}{end}'
35.xxx.xxx.xxx
34.xxx.xxx.xxx
（这里与指南的主要区别是我使用的是 "ExternalIP" 而不是 "InternalIP"，因为我正在远程做所有事情）
然后我搜索暴露服务的端口：
kubectl get service kafka-cluster-kafka-external-bootstrap -n xxxx-kafka -o=jsonpath='{.spec.ports[0].nodePort}{"\n"}'
30680
但是当我尝试使用本地的 apache-kafka 容器启动我的生产者时，我得到了这个：
sh kafka-console-producer.sh --broker-list 35.xxx.xxx.xxx:30680 --topic test
>[2020-02-11 16:37:18,388] WARN [Producer clientId=console-producer] Connection to node -1 (/35.xxx.xxx.xxx:30680) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
所以我尝试ping ip，看看它是否可以访问：
ping 35.xxx.xxx.xxx
PING 35.xxx.xxx.xxx (35.xxx.xxx.xxx) 56(84) bytes of data.
64 bytes from 35.xxx.xxx.xxx: icmp_seq=1 ttl=54 time=63.4 ms
64 bytes from 35.xxx.xxx.xxx: icmp_seq=2 ttl=54 time=51.4 ms
它是可达的，但端口不是：
telnet 35.xxx.xxx.xxx 30680
Trying 35.xxx.xxx.xxx...
telnet: Unable to connect to remote host: Connection timed out
这是我对 kafka 集群的 yaml 配置：
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
creationTimestamp: "2020-02-10T15:04:38Z"
generation: 1
name: kafka-cluster
namespace: xxxx-kafka
resourceVersion: "5868409"
selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/xxxx-kafka/kafkas/kafka-cluster
uid: 93d0d9b6-7c88-4e01-af9c-49f9fcaac1d1
spec:
entityOperator:
topicOperator: {}
userOperator: {}
kafka:
config:
offsets.topic.replication.factor: 1
transaction.state.log.min.isr: 1
transaction.state.log.replication.factor: 1
listeners:
external:
tls: false
type: nodeport
plain: {}
tls: {}
replicas: 1
storage:
type: jbod
volumes:
- deleteClaim: false
id: 0
size: 100Gi
type: persistent-claim
zookeeper:
replicas: 1
storage:
deleteClaim: false
size: 100Gi
type: persistent-claim
status:
conditions:
- lastTransitionTime: 2020-02-11T15:33:50+0000
status: "True"
type: Ready
listeners:
- addresses:
- host: kafka-cluster-kafka-bootstrap.xxxx-kafka.svc
port: 9092
type: plain
- addresses:
- host: kafka-cluster-kafka-bootstrap.xxxx-kafka.svc
port: 9093
type: tls
- addresses:
- host: <AnyNodeAddress>
port: 30680
type: external
observedGeneration: 1
有谁知道为什么会出现这个问题以及如何正确运行生产者？
编辑：
这是 yamlkafka-cluster-kafka-bootstrap
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2020-02-11T15:31:57Z"
labels:
app.kubernetes.io/instance: kafka-cluster
app.kubernetes.io/managed-by: strimzi-cluster-operator
app.kubernetes.io/name: strimzi
strimzi.io/cluster: kafka-cluster
strimzi.io/kind: Kafka
strimzi.io/name: kafka-cluster-kafka-bootstrap
name: kafka-cluster-kafka-bootstrap
namespace: xxxx-kafka
ownerReferences:
- apiVersion: kafka.strimzi.io/v1beta1
blockOwnerDeletion: false
controller: false
kind: Kafka
name: kafka-cluster
uid: 93d0d9b6-7c88-4e01-af9c-49f9fcaac1d1
resourceVersion: "5867902"
selfLink: /api/v1/namespaces/xxxx-kafka/services/kafka-cluster-kafka-bootstrap
uid: 3dfdbd89-fc97-43d3-8e10-4f6a23eae20e
spec:
clusterIP: 10.0.2.111
ports:
- name: replication
port: 9091
protocol: TCP
targetPort: 9091
- name: clients
port: 9092
protocol: TCP
targetPort: 9092
- name: clientstls
port: 9093
protocol: TCP
targetPort: 9093
selector:
strimzi.io/cluster: kafka-cluster
strimzi.io/kind: Kafka
strimzi.io/name: kafka-cluster-kafka
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
我也尝试从 GCP 节点进行内部连接，但我遇到了同样的问题。我提取了内部 Ips：
kubectl get nodes --output=jsonpath='{range .items[*]}{.status.addresses[?(@.type=="InternalIP")].address}{"\n"}{end}'
10.132.0.4
10.132.0.5
然后我尝试从 GCP 节点连接生产者，这就是我得到的：
sh kafka-console-producer.sh --broker-list 10.132.0.4:30680 --topic test_topic
>hello
[2020-02-12 15:31:15,033] ERROR Error when sending message to topic test_topic with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic test_topic not present in metadata after 60000 ms.
>[2020-02-12 15:32:24,629] WARN [Producer clientId=console-producer] Connection to node 0 (/34.xxx.xxx.xxx:30350) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
现在，在出现连接错误之前，我也得到了这个：
[2020-02-12 15:31:15,033] ERROR Error when sending message to topic test_topic with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic test_topic not present in metadata after 60000 ms.
这是kafka-cluster-kafka-brokersyaml：
apiVersion: v1
kind: Service
metadata:
annotations:
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
creationTimestamp: "2020-02-11T15:31:57Z"
labels:
app.kubernetes.io/instance: kafka-cluster
app.kubernetes.io/managed-by: strimzi-cluster-operator
app.kubernetes.io/name: strimzi
strimzi.io/cluster: kafka-cluster
strimzi.io/kind: Kafka
strimzi.io/name: kafka-cluster-kafka-brokers
name: kafka-cluster-kafka-brokers
namespace: xxxx-kafka
ownerReferences:
- apiVersion: kafka.strimzi.io/v1beta1
blockOwnerDeletion: false
controller: false
kind: Kafka
name: kafka-cluster
uid: 93d0d9b6-7c88-4e01-af9c-49f9fcaac1d1
resourceVersion: "5867905"
selfLink: /api/v1/namespaces/xxxx-kafka/services/kafka-cluster-kafka-brokers
uid: 42d7fa08-cf52-47e1-9746-89a45d65351b
spec:
clusterIP: None
ports:
- name: replication
port: 9091
protocol: TCP
targetPort: 9091
- name: clients
port: 9092
protocol: TCP
targetPort: 9092
- name: clientstls
port: 9093
protocol: TCP
targetPort: 9093
publishNotReadyAddresses: true
selector:
strimzi.io/cluster: kafka-cluster
strimzi.io/kind: Kafka
strimzi.io/name: kafka-cluster-kafka
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}