0

我正在使用最新的 DCOS 版本 (0.8.0) 并创建了一个本地 vagrant 集群，其中包含 4 个私有代理节点、一个公共代理节点、一个主节点和一台引导机器

vagrant up m1 a1 a2 a3 a4 p1 boot

然后我使用以下命令使用默认配置安装了 kafka：

dcos package install kafka

在检查马拉松日志后，看起来 mesos 正在拒绝为 kafka 提供资源，因此无法部署所有三个代理。我怀疑问题在于资源提供的角色与所需的 kafka 角色之间的不匹配。

聚合集群统计信息：CPU：18 个共享内存：7GiB

这是马拉松日志的快照：

s.offer.OfferEvaluator: Satisfying resource requirement: name: "cpus" type: SCALAR scalar { value: 1.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "" } } }
with resource: name: "cpus" type: SCALAR scalar { value: 1.0 } role: "*"
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Reserves Resource
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Fulfilled resource: name: "cpus" type: SCALAR scalar { value: 1.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "f8468a5a-8166-4d5d-a594-f197a8b356cd" } } }
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.MesosResourcePool: Retrieving resource for reservation
WARN  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Failed to satisfy resource requirement: name: "mem" type: SCALAR scalar { value: 2304.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "" } } }
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.MesosResourcePool: Retrieving resource for reservation
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Satisfying resource requirement: name: "cpus" type: SCALAR scalar { value: 0.5 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "" } } }
with resource: name: "cpus" type: SCALAR scalar { value: 0.5 } role: "*"
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Reserves Resource
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Fulfilled resource: name: "cpus" type: SCALAR scalar { value: 0.5 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "37ba47a3-7c6e-47a2-972c-d35d130ae8df" } } }
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.MesosResourcePool: Retrieving resource for reservation
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Satisfying resource requirement: name: "mem" type: SCALAR scalar { value: 256.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "" } } }
with resource: name: "mem" type: SCALAR scalar { value: 256.0 } role: "*"
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Reserves Resource
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Fulfilled resource: name: "mem" type: SCALAR scalar { value: 256.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "8f937e43-c80a-4434-a858-08e6bdb02cc2" } } }
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.MesosResourcePool: Retrieving resource for reservation
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Satisfying resource requirement: name: "cpus" type: SCALAR scalar { value: 1.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "" } } }
with resource: name: "cpus" type: SCALAR scalar { value: 1.0 } role: "*"
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Reserves Resource
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Fulfilled resource: name: "cpus" type: SCALAR scalar { value: 1.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "b568cfaa-c216-4479-9853-5bc4e6ee2d9e" } } }
INFO  [2016-08-02 16:28:08,919] org.apache.mesos.offer.MesosResourcePool: Retrieving resource for reservation
WARN  [2016-08-02 16:28:08,919] org.apache.mesos.offer.OfferEvaluator: Failed to satisfy resource requirement: name: "mem" type: SCALAR scalar { value: 2304.0 } role: "kafka-role" reservation { principal: "kafka-principal" labels { labels { key: "resource_id" value: "" } } }
WARN  [2016-08-02 16:28:08,919] org.apache.mesos.scheduler.plan.DefaultStageScheduler: Unable to find any offers which fulfill requirement provided by block broker-0: org.apache.mesos.offer.OfferRequirement@2a28d9e[avoidAgents=[],colocateAgents=[],taskRequirements=[org.apache.mesos.offer.TaskRequirement@5c8e5b0f],executorRequirement=org.apache.mesos.offer.ExecutorRequirement@6179902f[executorInfo=executor_id {
  value: ""
}
resources {
  name: "cpus"
  type: SCALAR
  scalar {
    value: 0.5
  }
  role: "kafka-role"
  reservation {
    principal: "kafka-principal"
    labels {
      labels {
        key: "resource_id"
        value: ""
      }
    }
  }
}
resources {
  name: "mem"
  type: SCALAR
  scalar {
    value: 256.0
  }
  role: "kafka-role"
  reservation {
    principal: "kafka-principal"
    labels {
      labels {
        key: "resource_id"
        value: ""
      }
    }
  }
}
command {
  uris {
    value: "https://downloads.mesosphere.com/kafka/assets/jre-8u91-linux-x64.tar.gz"
  }
  uris {
    value: "https://downloads.mesosphere.com/kafka/assets/kafka_2.11-0.10.0.0.tgz"
  }
  uris {
    value: "https://downloads.mesosphere.com/kafka/assets/1.1.9-0.10.0.0/overrider.zip"
  }
  uris {
    value: "https://downloads.mesosphere.com/kafka/assets/1.1.9-0.10.0.0/executor.zip"
  }
  environment {
    variables {
      name: "JAVA_HOME"
      value: "jre1.8.0_91"
    }
  }
  value: "./executor/bin/kafka-executor -Dlogback.configurationFile=executor/conf/logback.xml"
}
framework_id {
  value: "2601d122-2b59-4823-83f8-84351376d53c-0001"
}
name: "broker-0"
,resourceRequirements=[org.apache.mesos.offer.ResourceRequirement@39ba5849, org.apache.mesos.offer.ResourceRequirement@2f62ec9d]]]
INFO  [2016-08-02 16:28:08,919] com.mesosphere.dcos.kafka.plan.KafkaUpdateBlock: broker-0: changed status from: Pending to: Pending
INFO  [2016-08-02 16:28:08,920] com.mesosphere.dcos.kafka.scheduler.KafkaRepairScheduler: Missing brokerIds: []
INFO  [2016-08-02 16:28:08,921] org.apache.mesos.offer.ResourceCleaner: 4 offers with 16 resources => 0 destroy and 0 unreserve operations
INFO  [2016-08-02 16:28:08,921] com.mesosphere.dcos.kafka.scheduler.KafkaScheduler: Accepted 0 of 4 offers: []
INFO  [2016-08-02 16:28:08,921] com.mesosphere.dcos.kafka.scheduler.KafkaScheduler: Declining offer: 2601d122-2b59-4823-83f8-84351376d53c-O1343
INFO  [2016-08-02 16:28:08,921] com.mesosphere.dcos.kafka.scheduler.KafkaScheduler: Declining offer: 2601d122-2b59-4823-83f8-84351376d53c-O1344
INFO  [2016-08-02 16:28:08,921] com.mesosphere.dcos.kafka.scheduler.KafkaScheduler: Declining offer: 2601d122-2b59-4823-83f8-84351376d53c-O1345
INFO  [2016-08-02 16:28:08,921] com.mesosphere.dcos.kafka.scheduler.KafkaScheduler: Declining offer: 2601d122-2b59-4823-83f8-84351376d53c-O1346
INFO  [2016-08-02 16:28:14,928]
4

2 回答 2

0

问题确实如日志报告的那样：没有足够的内存来使用默认设置来安排 kafka。我能够使用默认的 vagrant 配置和使用以下 kafka 配置的三个代理节点在 DCOS vagrant 上部署 kafka：

{
    "brokers": {
        "count": 1,
        "mem": 512,
        "disk": 1000
    }
}
于 2016-08-11T17:43:43.900 回答
0

DCOS 1.8 中的 Kafka 框架是一个有状态的服务。它专门设计用于为每个代理分配存储并维护代理实例与该存储之间的链接（因此不会将代理 0 重新部署到没有最初为代理 0 分配的存储的主机上）。我的经验是，当不满足存储限制时，围绕资源提供的消息不是很清楚。默认情况下，该软件包将为每个代理请求 5 GB 的专用存储空间。

我对 vagrant 没有太多经验，但可能是 vagrant 没有为 DC/OS 公开必要的“存储可用性”，以便能够满足代理资源请求。

于 2016-08-02T18:01:36.397 回答