尝试使用 ALB 在 AWS ECS 环境中启动 Apache Kafka 和 zookeeper。启动后,docker 实例会在几秒钟后被杀死。Zookeeper 实例已链接到 kafka 实例。
启动 Kafka 容器时使用的环境变量:
KAFKA_ADVERTISED_HOST_NAME <AWS EC2 Instance Private IP Address > ( used to modify the advertised.hostname in server.properties file )
KAFKA_ZOOKEEPER_CONNECT zookeeper:2181
Host Port = 9092
Container port =9092
kafka 容器的 Docker 日志
2018-03-05 05:58:08,218] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:os.version=4.9.77-31.58.amzn1.x86_64 (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,218] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,219] INFO Initiating client connection, connectString=zookeeper:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@363ee3a2 (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:08,251] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
[2018-03-05 05:58:08,253] INFO Opening socket connection to server zookeeper/172.17.0.2:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2018-03-05 05:58:14,252] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2018-03-05 05:58:14,256] WARN Client session timed out, have not heard from server in 6005ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2018-03-05 05:58:14,368] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2018-03-05 05:58:14,368] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2018-03-05 05:58:14,371] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: **Unable to connect to zookeeper server 'zookeeper:2181' with timeout of 6000 ms**
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1233)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:157)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:131)
at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:115)
at kafka.utils.ZkUtils$.withMetrics(ZkUtils.scala:92)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:350)
at kafka.server.KafkaServer.startup(KafkaServer.scala:194)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:92)
at kafka.Kafka.main(Kafka.scala)
[2018-03-05 05:58:14,381] INFO shutting down (kafka.server.KafkaServer)
[2018-03-05 05:58:14,410] INFO shut down completed (kafka.server.KafkaServer)
[2018-03-05 05:58:14,411] FATAL Exiting Kafka. (kafka.server.KafkaServerStartable)
[2018-03-05 05:58:14,418] INFO shutting down (kafka.server.KafkaServer)
Zookeeper 容器的 Docker 日志:
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.9/bin/../conf/zoo.cfg
2018-03-05 05:58:05,694 [myid:] - INFO [main:QuorumPeerConfig@124] - Reading configuration from: /opt/zookeeper-3.4.9/bin/../conf/zoo.cfg
2018-03-05 05:58:05,703 [myid:] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2018-03-05 05:58:05,703 [myid:] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 1
2018-03-05 05:58:05,704 [myid:] - WARN [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running in standalone mode
2018-03-05 05:58:05,704 [myid:] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started.
2018-03-05 05:58:05,729 [myid:] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge task completed.
2018-03-05 05:58:05,732 [myid:] - INFO [main:QuorumPeerConfig@124] - Reading configuration from: /opt/zookeeper-3.4.9/bin/../conf/zoo.cfg
2018-03-05 05:58:05,732 [myid:] - INFO [main:ZooKeeperServerMain@96] - Starting server
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:host.name=5944c205a288
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:java.version=1.7.0_65
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:java.class.path=/opt/zookeeper-3.4.9/bin/../build/classes:/opt/zookeeper-3.4.9/bin/../build/lib/*.jar:/opt/zookeeper-3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper-3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper-3.4.9/bin/../lib/netty-3.10.5.Final.jar:/opt/zookeeper-3.4.9/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper-3.4.9/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.9/bin/../zookeeper-3.4.9.jar:/opt/zookeeper-3.4.9/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.9/bin/../conf:
2018-03-05 05:58:05,751 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2018-03-05 05:58:05,752 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler=<NA>
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:os.version=4.9.77-31.58.amzn1.x86_64
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:user.name=root
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/root
2018-03-05 05:58:05,754 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/opt/zookeeper-3.4.9
2018-03-05 05:58:05,757 [myid:] - INFO [main:ZooKeeperServer@815] - tickTime set to 2000
2018-03-05 05:58:05,757 [myid:] - INFO [main:ZooKeeperServer@824] - minSessionTimeout set to -1
2018-03-05 05:58:05,757 [myid:] - INFO [main:ZooKeeperServer@833] - maxSessionTimeout set to -1
2018-03-05 05:58:05,785 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181