1

我正在尝试将外部应用程序连接到在 mesos 集群上运行 dockerized 的 Cassandra。

这些是我在 mesos 上运行的应用程序：

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                        NAMES
137760ce852a        cassandra:latest    "/docker-entrypoint.s"   15 minutes ago      Up 15 minutes       7000-7001/tcp, 7199/tcp, 9160/tcp, 0.0.0.0:31634->9042/tcp   mesos-1b65f33a-3d36-4bf4-8a77-32077d8d234a-S1.0db174cc-2e0c-4790-9cd7-1f142d08c6e2
fec5fc93ccfd        cassandra:latest    "/docker-entrypoint.s"   22 minutes ago      Up 22 minutes       7000-7001/tcp, 7199/tcp, 9160/tcp, 0.0.0.0:31551->9042/tcp   mesos-1b65f33a-3d36-4bf4-8a77-32077d8d234a-S1.0022a3d2-d695-43c4-b22f-f5274cbd03ce
ca729ee628bb        tobilg/mesos-dns    "./bootstrap.sh"         About an hour ago   Up About an hour                                                                 mesos-1b65f33a-3d36-4bf4-8a77-32077d8d234a-S1.12593777-2295-42fa-a56d-1d3cc9fc70ff
3921002a8a5b        python:3            "/bin/sh -c 'env >env"   About an hour ago   Up About an hour    0.0.0.0:31295->8080/tcp                                      mesos-1b65f33a-3d36-4bf4-8a77-32077d8d234a-S1.b101ab59-2538-416f-80cf-29215794bd37

名为peek的应用程序仅用于测试提案。我可以毫无问题地通过 URL：http: //192.168.56.101 :10001访问它。

2 个 cassandra 实例是一个种子，另一个用于扩展；形成一个集群。

marathon上cassandra应用部署的json描述如下：

/cassandra-种子

{
    "id": "cassandra-seed",
    "constraints": [["hostname", "CLUSTER", "docker-sl-vm"]],
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "cassandra:latest",
            "network": "BRIDGE",
            "portMappings": [ {"containerPort": 9042,"hostPort": 0,"servicePort": 0,"protocol": "tcp"} ]
        }
    },
    "cpus": 0.5,
    "mem": 512.0,
    "instances": 1,
    "backoffSeconds": 1,
    "backoffFactor": 1.15,
    "maxLaunchDelaySeconds": 3600
}

/卡珊德拉

{
    "id": "cassandra",
    "constraints": [["hostname", "CLUSTER", "docker-sl-vm"]],
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "cassandra:latest",
            "network": "BRIDGE",
            "portMappings": [ {"containerPort": 9042,"hostPort": 0,"servicePort": 0,"protocol": "tcp"} ]
        }
    },
    "env": {
            "CASSANDRA_SEED_COUNT": "1",
        "CASSANDRA_SEEDS": "cassandra-seed.marathon.mesos"
    },
    "cpus": 0.5,
    "mem": 512.0,
    "instances": 1,
    "backoffSeconds": 1,
    "backoffFactor": 1.15,
    "maxLaunchDelaySeconds": 3600
}

haproxy 配置如下：

global
  daemon
  log 127.0.0.1 local0
  log 127.0.0.1 local1 notice
  maxconn 4096
  tune.ssl.default-dh-param 2048

defaults
  log               global
  retries           3
  maxconn           2000
  timeout connect   5s
  timeout client    50s
  timeout server    50s

listen stats
  bind 127.0.0.1:9090
  balance
  mode http
  stats enable
  stats auth admin:admin

frontend marathon_http_in
  bind *:80
  mode http

frontend marathon_http_appid_in
  bind *:81
  mode http

frontend marathon_https_in
  bind *:443 ssl crt /etc/ssl/xip.io/xip.io.pem
  mode http

frontend cassandra_10003
  bind *:10003
  mode tcp
  use_backend cassandra_10003

frontend cassandra-seed_10002
  bind *:10002
  mode tcp
  use_backend cassandra-seed_10002

frontend dns_10000
  bind *:10000
  mode tcp
  use_backend dns_10000

frontend peek_10001
  bind *:10001
  mode tcp
  use_backend peek_10001

backend cassandra_10003
  balance roundrobin
  mode tcp
  server docker-sl-vm_31634 192.168.56.102:31634

backend cassandra-seed_10002
  balance roundrobin
  mode tcp
  server docker-sl-vm_31551 192.168.56.102:31551

backend dns_10000
  balance roundrobin
  mode tcp
  server docker-sl-vm_31314 192.168.56.102:31314

backend peek_10001
  balance roundrobin
  mode tcp
  server docker-sl-vm_31295 192.168.56.102:31295

我试图连接到 Cassandra 的应用程序是一个 Play 应用程序。我是这样设置的：

akka.persistence {
  journal.plugin = "cassandra-journal"
  snapshot-store.plugin = "cassandra-snapshot-store"
}

cassandra-journal.contact-points = ["192.168.56.101:10003"]
cassandra-snapshot-store.contact-points = ["192.168.56.101:10003"]

该应用程序启动正常，但是当我尝试访问它时，出现以下错误：

! @6o380dcg9 - Internal server error, for (GET) [/issues/list] ->

play.api.Application$$anon$1: Execution exception[[TimeoutException: deadline passed]]
        at play.api.Application$class.handleError(Application.scala:296) ~[play_2.11-2.3.10.jar:2.3.10]
        at play.api.DefaultApplication.handleError(Application.scala:402) [play_2.11-2.3.10.jar:2.3.10]
        at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$14$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:205) [play_2.11-2.3.10.jar:2.3.10]
        at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$14$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:202) [play_2.11-2.3.10.jar:2.3.10]
        at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36) [scala-library-2.11.7.jar:na]
Caused by: java.util.concurrent.TimeoutException: deadline passed
        at akka.actor.dsl.Inbox$InboxActor$$anonfun$receive$1.applyOrElse(Inbox.scala:117) ~[akka-actor_2.11-2.4.0.jar:na]
        at scala.PartialFunction$AndThen.applyOrElse(PartialFunction.scala:189) ~[scala-library-2.11.7.jar:na]
        at akka.actor.Actor$class.aroundReceive(Actor.scala:480) ~[akka-actor_2.11-2.4.0.jar:na]
        at akka.actor.dsl.Inbox$InboxActor.aroundReceive(Inbox.scala:62) ~[akka-actor_2.11-2.4.0.jar:na]
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:525) ~[akka-actor_2.11-2.4.0.jar:na]
[error] c.d.d.c.Session - Error creating pool to /172.17.0.2:9042
com.datastax.driver.core.TransportException: [/172.17.0.2:9042] Cannot connect
        at com.datastax.driver.core.Connection.<init>(Connection.java:109) ~[cassandra-driver-core-2.1.5.jar:na]
        at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:32) ~[cassandra-driver-core-2.1.5.jar:na]
        at com.datastax.driver.core.Connection$Factory.open(Connection.java:586) ~[cassandra-driver-core-2.1.5.jar:na]
        at com.datastax.driver.core.SingleConnectionPool.<init>(SingleConnectionPool.java:76) ~[cassandra-driver-core-2.1.5.jar:na]
        at com.datastax.driver.core.HostConnectionPool.newInstance(HostConnectionPool.java:35) ~[cassandra-driver-core-2.1.5.jar:na]
Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: /172.17.0.2:9042
        at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:139) ~[netty-3.9.9.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) ~[netty-3.9.9.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[netty-3.9.9.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.9.9.Final.jar:na]
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[netty-3.9.9.Final.jar:na]
[error] c.d.d.c.Session - Error creating pool to /172.17.0.2:9042

有谁知道如何解决这一问题？我究竟做错了什么？

先感谢您...

更新==============================

有趣的是，我的应用程序的键空间已创建（akka，akka_snapshots）：

cqlsh> describe keyspaces;
akka_snapshot  system_auth  system  system_distributed  system_traces  akka

更新2 ==============================

我刚刚注意到我什至无法将应用程序直接连接到正在运行的 cassandra（不通过 h​​aproxy）。因此，我将 portMapping 更改为：

"portMappings": [ {"containerPort": 9042,"hostPort": 0,"servicePort": 9042,"protocol": "tcp"} ]

它奏效了。但是，由于 servicePort 声明，它只允许我启动一台机器。

问题出在端口映射中。有什么线索吗？

4

1 回答 1

2

我了解您正在使用集群haproxy的服务发现。Cassandra如果是这样，如果您没有在 Marathon 的任务更改（缩放等）后更新配置的机制，它将不会成功。

您的Cassandra节点无法相互通信的问题可能是该/cassandra应用程序没有引用/cassandra-seed.

根据Cassandra Docker 映像文档，您应该能够动态配置CASSANDRA_SEEDSenv 参数。

为了能够使用服务名称cassandra-seed.marathon.mesos，如果有必要首先将其解析为 IP 地址，恕我直言：

"CASSANDRA_SEEDS": "$(host cassandra-seed.marathon.mesos | awk '/has address/ { print $4 }')"

理论上可行（例如，如果您的应用程序只有一个实例）。

当您似乎使用 Mesos DNS 时，可能会出现问题，因为目前 (v0.4.0) 仅公布内部 IP 地址（请参阅问题）。你可能不得不回退到一个“真正的”Mesos DNS 客户端，它可以解析 SRV 记录以正确地将它们映射到 Mesos 从 IP 地址和端口。

或者，您可以自己解析dig结果并将其用作CASSANDRA_SEEDSenv 参数的输入：

dig _cassandra-seed._tcp.marathon.mesos SRV

请参阅Mesos DNS 文档。

mesosdns-cli可以处理这个问题，但需要在 Docker 容器中使用 Node.js 运行时。因此，您必须创建自己的cassandraDocker 映像派生。

于 2015-11-11T13:44:10.400 回答