0

我正在尝试部署一个 Nomad 作业，该作业具有从 Vault 获取一些秘密的模板。

我的问题是，由于我不明白为什么的约束，它不断给出这个放置失败：

Constraint ${attr.vault.version} version >= 0.6.1 filtered 1 node

游牧配置

datacenter = "dc1"
data_dir = "/var/lib/nomad"
advertise {
  # Defaults to the first private IP address.
  http = "10.134.43.195"
  rpc  = "10.134.43.195"
  serf = "10.134.43.195"
}

server {
  enabled = true
  bootstrap_expect = 1
  server_join {
    retry_join = ["provider=digitalocean api_token=[SECRET] tag_name=nomad_auto_join"],
  }
}
client {
  enabled = true
}
# Consul is installed locally and clustered
consul {
  address = "http://127.0.0.1:8500"
  server_auto_join = true
  client_auto_join = true
  auto_advertise = true
}
vault {
  enabled = true
  address = "http://vault.service.consul:8200"
  token = "[NOMAD_VAULT_TOKEN]"
  create_from_role = "nomad-cluster"
  task_token_ttl = "1h"
}
autopilot {
    cleanup_dead_servers = true
    last_contact_threshold = "200ms"
    max_trailing_logs = 250
    server_stabilization_time = "10s"
    enable_redundancy_zones = false
    disable_upgrade_migration = false
    enable_custom_upgrades = false
}
telemetry {
  publish_allocation_metrics = true
  publish_node_metrics = true
  prometheus_metrics = true
}

### Nomad Token to access Vault

NOMAD_VAULT_TOKEN使用以下命令生成：

vault token create -policy nomad-server -period 72h -orphan

保险库策略游牧服务器

nomad-server保险库政策是这样的：

# Allow creating tokens under "nomad-cluster" token role.Z
path "auth/token/create/nomad-cluster" {
  capabilities = ["update"]
}

# Allow looking up "nomad-cluster" token role.
path "auth/token/roles/nomad-cluster" {
  capabilities = ["read"]
}

# Allow looking up the token passed to Nomad to validate # the token has the
# proper capabilities. This is provided by the "default" policy.
path "auth/token/lookup-self" {
  capabilities = ["read"]
}

# Allow looking up incoming tokens to validate they have permissions to access
# the tokens they are requesting. This is only required if
# `allow_unauthenticated` is set to false.
path "auth/token/lookup" {
  capabilities = ["update"]
}

# Allow revoking tokens that should no longer exist. This allows revoking
# tokens for dead tasks.
path "auth/token/revoke-accessor" {
  capabilities = ["update"]
}

# Allow checking the capabilities of our own token. This is used to validate the
# token upon startup.
path "sys/capabilities-self" {
  capabilities = ["update"]
}

# Allow our own token to be renewed.
path "auth/token/renew-self" {
  capabilities = ["update"]
}

# This is where needed secretes are fetched from
path "kv/*" {
  capabilities = ["update", "read", "create"]
}

游牧工作定义

我的 Nomad 工作定义是：

job "api" {
  datacenters = ["dc1"]
  type = "service"



  group "api" {
    count = 1
    update {
      max_parallel = 1
      min_healthy_time = "30s"
      healthy_deadline = "10m"
      progress_deadline = "11m"
      auto_revert = true
    }
    task "api" {
      driver = "docker"
      config {
        image = "registry.gitlab.com/[GROUP]/[PROJECT]/${ENVIRONMENT}:${BUILD_NUMBER}"
        port_map {
          nginx = 80
        }
        auth {
          username = "${REGISTRY_USER}"
          password = "${REGISTRY_PASS}"
        }
        force_pull = true
        hostname = "api"
      }

      vault {
        policies = ["kv"]

        change_mode   = "signal"
        change_signal = "SIGINT"
        //    env = "false"
      }

      template {
        data        = <<EOT
APP_NAME={{ key "services/api/app/${ENVIRONMENT}/APP_NAME" }}
APP_ENV={{ key "services/api/app/${ENVIRONMENT}/APP_ENV" }}
APP_KEY={{with secret "kv/services/api/app/${ENVIRONMENT}"}}{{.Data.APP_KEY.value}}{{end}}
APP_DEBUG={{ key "services/api/app/${ENVIRONMENT}/APP_DEBUG" }}
APP_URL={{ key "services/api/app/${ENVIRONMENT}/APP_URL" }}

LOG_CHANNEL={{ key "services/api/log/${ENVIRONMENT}/LOG_CHANNEL" }}

DB_CONNECTION={{ key "services/api/db/${ENVIRONMENT}/DB_CONNECTION" }}
DB_HOST={{ key "services/api/db/${ENVIRONMENT}/DB_HOST" }}
DB_PORT={{ key "services/api/db/${ENVIRONMENT}/DB_PORT" }}
DB_DATABASE={{with secret "kv/services/api/db/${ENVIRONMENT}/DB_DATABASE"}}{{.Data.value}}{{end}}
DB_USERNAME={{with secret "kv/services/api/db/${ENVIRONMENT}/DB_USERNAME"}}{{.Data.value}}{{end}}
DB_PASSWORD={{with secret "kv/services/api/db/${ENVIRONMENT}/DB_PASSWORD"}}{{.Data.value}}{{end}}

BROADCAST_DRIVER={{ key "services/api/broadcast/${ENVIRONMENT}/BROADCAST_DRIVER" }}
CACHE_DRIVER={{ key "services/api/cache/${ENVIRONMENT}/CACHE_DRIVER" }}
SESSION_DRIVER={{ key "services/api/session/${ENVIRONMENT}/SESSION_DRIVER" }}
SESSION_LIFETIME={{ key "services/api/session/${ENVIRONMENT}/SESSION_LIFETIME" }}
QUEUE_DRIVER={{ key "services/api/queue/${ENVIRONMENT}/QUEUE_DRIVER" }}

REDIS_HOST={{ key "services/api/redis/${ENVIRONMENT}/REDIS_HOST" }}
REDIS_PORT={{ key "services/api/redis/${ENVIRONMENT}/REDIS_PORT" }}
REDIS_PASSWORD={{with secret "kv/services/api/redis/${ENVIRONMENT}/REDIS_PASSWORD"}}{{.Data.value}}{{end}}

MAIL_DRIVER={{ key "services/api/mail/${ENVIRONMENT}/MAIL_DRIVER" }}
MAIL_HOST={{ key "services/api/mail/${ENVIRONMENT}/MAIL_HOST" }}
MAIL_PORT={{ key "services/api/mail/${ENVIRONMENT}/MAIL_PORT" }}
MAIL_USERNAME={{ key "services/api/mail/${ENVIRONMENT}/MAIL_USERNAME" }}
MAIL_PASSWORD={{ key "services/api/mail/${ENVIRONMENT}/MAIL_PASSWORD" }}
MAIL_ENCRYPTION={{ key "services/api/mail/${ENVIRONMENT}/MAIL_ENCRYPTION" }}
MAIL_FROM_ADDRESS={{ key "services/api/mail/${ENVIRONMENT}/MAIL_FROM_ADDRESS" }}
MAIL_FROM_NAME={{ key "services/api/mail/${ENVIRONMENT}/MAIL_FROM_NAME" }}

MAILGUN_DOMAIN={{ key "services/api/mailgun/${ENVIRONMENT}/MAILGUN_DOMAIN" }}
MAILGUN_SECRET={{with secret "kv/services/api/mailgun/${ENVIRONMENT}/MAILGUN_SECRET"}}{{.Data.value}}{{end}}

DO_SPACES_KEY={{with secret "kv/services/api/spaces/${ENVIRONMENT}/DO_SPACES_KEY"}}{{.Data.value}}{{end}}
DO_SPACES_SECRET={{with secret "kv/services/api/spaces/${ENVIRONMENT}/DO_SPACES_SECRET"}}{{.Data.value}}{{end}}
DO_SPACES_ENDPOINT={{ key "services/api/spaces/${ENVIRONMENT}/DO_SPACES_ENDPOINT" }}
DO_SPACES_REGION={{ key "services/api/spaces/${ENVIRONMENT}/DO_SPACES_REGION" }}
DO_SPACES_BUCKET={{ key "services/api/spaces/${ENVIRONMENT}/DO_SPACES_BUCKET" }}

JWT_SECRET={{with secret "kv/services/api/jwt/${ENVIRONMENT}"}}{{.Data.JWT_SECRET}}{{end}}
EOT
        destination   = "custom/.env"
//        change_mode   = "signal"
//        change_signal = "SIGINT"
        env         = true
      }

      service {
        name = "api"
        tags = [
          "urlprefix-${ENVIRONMENT_URL}/"
        ]
        port = "nginx"
        check {
          type = "tcp"
          port = "nginx"
          interval = "10s"
          timeout = "2s"
        }
      }
      resources {
        cpu    = 500 
        memory = 256
        network {
          mbits = 100
          port "nginx" {}
        }
      }
    }
  }
}

游牧日志

在 Nomad 日志中，我可以检查它是否正确地从 Vault 获取令牌：

Oct 16 13:29:19 nomad-server-0 nomad: 2019-10-16T13:29:19.333Z [INFO ] client.fingerprint_mgr.vault: Vault is available
Oct 16 13:29:19 nomad-server-0 nomad: 2019-10-16T13:29:19.334Z [DEBUG] client.fingerprint_mgr: fingerprinting periodically: fingerprinter=vault period=15s
Oct 16 13:29:19 nomad-server-0 nomad: 2019-10-16T13:29:19.354Z [DEBUG] nomad.vault: starting renewal loop: creation_ttl=72h0m0s
Oct 16 13:29:19 nomad-server-0 nomad: 2019-10-16T13:29:19.375Z [DEBUG] client.fingerprint_mgr: detected fingerprints: node_attrs="[arch cgroup consul cpu host network nomad signal storage vault]"
Oct 16 13:29:19 nomad-server-0 nomad: 2019-10-16T13:29:19.380Z [DEBUG] nomad.vault: successfully renewed server token
Oct 16 13:29:19 nomad-server-0 nomad: 2019-10-16T13:29:19.380Z [INFO ] nomad.vault: successfully renewed token: next_renewal=35h59m59.999975432s

我被困在这里，如果有人能提供任何见解，我将不胜感激！

[编辑] 我正在使用Vault v1.2.0-rc1.

4

3 回答 3

1

有时此错误意味着保管库仍处于密封状态或在重新启动保管库后已被密封

这可以通过dig在保险库的 dns 地址上进行测试，例如 vault.service.consul

于 2021-04-14T18:37:26.927 回答
0

根据文档，Note: Vault integration requires Vault version 0.6.2 or higher.

您的错误消息支持了这一点 - Constraint ${attr.vault.version} version >= 0.6.1 filtered 1 node。通常，约束是您在 nomad 规范中指定的内容，但在这种情况下，它看起来像是来自 nomad。

我认为您需要将保管库升级到 0.6.2

于 2019-10-16T16:35:39.303 回答
0

我已将 Vault 升级到版本v1.2.3并开始工作。

于 2019-10-16T17:30:43.500 回答