13

更新：一直在努力解决这个问题。似乎无法获得带有两个子网和一个 SSH 堡垒的工作配置。为完整的 .tf 文件配置设置赏金： * 创建两个私有子网 * 创建一个堡垒 * 在通过堡垒配置的每个子网上旋转一个 ec2 实例（通过堡垒运行一些任意 shell 命令） * 已配置 Internet 网关 * 已私有子网上的主机的 nat 网关 * 具有相应配置的路由和安全组

原始帖子：我正在尝试学习 Terraform 并构建原型。我有一个通过 Terraform 配置的 AWS VPC。除了 DMZ 子网之外，我还有一个公共子网“web”，它接收来自 Internet 的流量。我有一个无法从 Internet 访问的私有子网“应用程序”。我正在尝试配置堡垒主机，以便 terraform 可以在私有“应用程序”子网上配置实例。我还不能让它工作。

当我 ssh 进入堡垒时，我无法从堡垒主机通过 SSH 连接到私有子网中的任何实例。我怀疑有路由问题。我一直在通过几个可用的示例和文档来构建这个原型。许多示例通过 aws 提供程序使用略有不同的技术和 terraform 路由定义。

有人可以提供理想或正确的方法来定义这三个子网（公共'web'、公共'dmz' w/堡垒和私有'app'），以便'web'子网上的实例可以访问'app'子网，并且 DMZ 中的堡垒主机可以在私有“应用”子网中配置实例？

我的配置片段如下：

resource "aws_subnet" "dmz" {
    vpc_id = "${aws_vpc.vpc-poc.id}"
    cidr_block = "${var.cidr_block_dmz}"
}

resource "aws_route_table" "dmz" {
    vpc_id = "${aws_vpc.vpc-poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        gateway_id = "${aws_internet_gateway.gateway.id}"
    }
}

resource "aws_route_table_association" "dmz" {
    subnet_id = "${aws_subnet.dmz.id}"
    route_table_id = "${aws_route_table.dmz.id}"
}

resource "aws_subnet" "web" {
    vpc_id = "${aws_vpc.vpc-poc.id}"
    cidr_block = "10.200.2.0/24"
}

resource "aws_route_table" "web" {
    vpc_id = "${aws_vpc.vpc-poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        instance_id = "${aws_instance.bastion.id}"
    }
}

resource "aws_route_table_association" "web" {
    subnet_id = "${aws_subnet.web.id}"
    route_table_id = "${aws_route_table.web.id}"
}

resource "aws_subnet" "app" {
    vpc_id = "${aws_vpc.vpc-poc.id}"
    cidr_block = "10.200.3.0/24"
}

resource "aws_route_table" "app" {
    vpc_id = "${aws_vpc.vpc-poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        instance_id = "${aws_instance.bastion.id}"
    }
}

resource "aws_route_table_association" "app" {
    subnet_id = "${aws_subnet.app.id}"
    route_table_id = "${aws_route_table.app.id}"
}
4

5 回答 5

6

这是一个可以帮助你的片段。这是未经测试的，但是是从我在私有子网中配置 VM 的一个 terraform 文件中提取的。我知道这适用于一个私有子网，我尝试在这里实现两个，就像你原来的问题一样。

我跳过我的 NAT 实例以使用 Terraform 访问和配置私有子网框。如果您的安全组设置正确，它确实可以工作。我做了一些实验。

/* VPC creation */
resource "aws_vpc" "vpc_poc" {
  cidr_block = "10.200.0.0/16"
}

/* Internet gateway for the public subnets */
resource "aws_internet_gateway" "gateway" {
  vpc_id = "${aws_vpc.vpc_poc.id}"
}

/* DMZ subnet - public */
resource "aws_subnet" "dmz" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    cidr_block = "10.200.1.0/24"
    /* may help to be explicit here */
    map_public_ip_on_launch = true
    /* this is recommended in the docs */
    depends_on = ["aws_internet_gateway.gateway"]
}

resource "aws_route_table" "dmz" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        gateway_id = "${aws_internet_gateway.gateway.id}"
    }
}

resource "aws_route_table_association" "dmz" {
    subnet_id = "${aws_subnet.dmz.id}"
    route_table_id = "${aws_route_table.dmz.id}"
}

/* Web subnet - public */
resource "aws_subnet" "web" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    cidr_block = "10.200.2.0/24"
    map_public_ip_on_launch = true
    depends_on = ["aws_internet_gateway.gateway"]
}

resource "aws_route_table" "web" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        /* your public web subnet needs access to the gateway */
        /* this was set to bastion before so you had a circular arg */
        gateway_id = "${aws_internet_gateway.gateway.id}"
    }
}

resource "aws_route_table_association" "web" {
    subnet_id = "${aws_subnet.web.id}"
    route_table_id = "${aws_route_table.web.id}"
}

/* App subnet - private */
resource "aws_subnet" "app" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    cidr_block = "10.200.3.0/24"
}

/* Create route for DMZ Bastion */
resource "aws_route_table" "app" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        /* this send traffic to the bastion to pass off */
        instance_id = "${aws_instance.nat_dmz.id}"
    }
}

/* Create route for App Bastion */
resource "aws_route_table" "app" {
    vpc_id = "${aws_vpc.vpc_poc.id}"
    route {
        cidr_block = "0.0.0.0/0"
        /* this send traffic to the bastion to pass off */
        instance_id = "${aws_instance.nat_web.id}"
    }
}

resource "aws_route_table_association" "app" {
    subnet_id = "${aws_subnet.app.id}"
    route_table_id = "${aws_route_table.app.id}"
}

/* Default security group */
resource "aws_security_group" "default" {
  name = "default-sg"
  description = "Default security group that allows inbound and outbound traffic from all instances in the VPC"
  vpc_id = "${aws_vpc.vpc_poc.id}"

  ingress {
    from_port   = "0"
    to_port     = "0"
    protocol    = "-1"
    self        = true
  }

  egress {
    from_port   = "0"
    to_port     = "0"
    protocol    = "-1"
    self        = true
  }
}

/* Security group for the nat server */
resource "aws_security_group" "nat" {
  name        = "nat-sg"
  description = "Security group for nat instances that allows SSH and VPN traffic from internet. Also allows outbound HTTP[S]"
  vpc_id      = "${aws_vpc.vpc_poc.id}"

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    /* this your private subnet cidr */
    cidr_blocks = ["10.200.3.0/24"]
  }
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    /* this is your private subnet cidr */
    cidr_blocks = ["10.200.3.0/24"]
  }
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = -1
    to_port     = -1
    protocol    = "icmp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    /* this is the vpc cidr block */
    cidr_blocks = ["10.200.0.0/16"]
  }
  egress {
    from_port   = -1
    to_port     = -1
    protocol    = "icmp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

/* Security group for the web */
resource "aws_security_group" "web" {
  name = "web-sg"
  description = "Security group for web that allows web traffic from internet"
  vpc_id = "${aws_vpc.vpc_poc.id}"

  ingress {
    from_port = 80
    to_port   = 80
    protocol  = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port = 443
    to_port   = 443
    protocol  = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

/* Install deploy key for use with all of our provisioners */
resource "aws_key_pair" "deployer" {
  key_name   = "deployer-key"
  public_key = "${file("~/.ssh/id_rsa")}"
}

/* Setup NAT in DMZ subnet */
resource "aws_instance" "nat_dmz" {
  ami               = "ami-67a54423"
  availability_zone = "us-west-1a"
  instance_type     = "m1.small"
  key_name          = "${aws_key_pair.deployer.id}"
  /* Notice we are assigning the security group here */
  security_groups   = ["${aws_security_group.nat.id}"]

  /* this puts the instance in your public subnet, but translate to the private one */
  subnet_id         = "${aws_subnet.dmz.id}"

  /* this is really important for nat instance */
  source_dest_check = false
  associate_public_ip_address = true
}

/* Give NAT EIP In DMZ */
resource "aws_eip" "nat_dmz" {
  instance  = "${aws_instance.nat_dmz.id}"
  vpc       = true
}

/* Setup NAT in Web subnet */
resource "aws_instance" "nat_web" {
  ami               = "ami-67a54423"
  availability_zone = "us-west-1a"
  instance_type     = "m1.small"
  key_name          = "${aws_key_pair.deployer.id}"
  /* Notice we are assigning the security group here */
  security_groups   = ["${aws_security_group.nat.id}"]

  /* this puts the instance in your public subnet, but translate to the private one */
  subnet_id         = "${aws_subnet.web.id}"

  /* this is really important for nat instance */
  source_dest_check = false
  associate_public_ip_address = true
}

/* Give NAT EIP In DMZ */
resource "aws_eip" "nat_web" {
  instance  = "${aws_instance.nat_web.id}"
  vpc       = true
}

/* Install server in private subnet and jump host to it with terraform */
resource "aws_instance" "private_box" {
  ami           = "ami-d1315fb1"
  instance_type = "t2.large"
  key_name      = "${aws_key_pair.deployer.id}"
  subnet_id     = "${aws_subnet.api.id}"
  associate_public_ip_address = false

  /* this is what gives the box access to talk to the nat */
  security_groups = ["${aws_security_group.nat.id}"]

  connection {
    /* connect through the nat instance to reach this box */
    bastion_host = "${aws_eip.nat_dmz.public_ip}"
    bastion_user = "ec2-user"
    bastion_private_key = "${file("keys/terraform_rsa")}"

    /* connect to box here */
    user = "ec2-user"
    host = "${self.private_ip}"
    private_key = "${file("~/.ssh/id_rsa")}"
  }
}
于 2016-04-26T05:43:12.123 回答
3

我没有看到堡垒主机的原因。

我使用 saltstack 有类似的东西，我只是使用 VPC 内的主服务器控制其余部分，为其分配特定的安全组以允许访问。 

CIDR X/24
subnetX.0/26- subnet for control server. <aster server ip EC2-subnet1/32
subnetX.64/26 - private minions 
subentX.128/26 - public minions
subnetX.192/26- private minions

然后为每个子网创建一个路由表，以满足您对隔离的热爱将每条路由附加到各个子网。例如

rt-1  - subnetX.0/26
rt-2  - subnetX.64/26
rt-3  - subnetX.128/26
rt-4  - subnetX.192/26

确保您的路由表有这样的东西，以便 rt-1 实例可以连接到每个人

destination: CIDR X/24  Target: local

然后通过安全组入站限制连接。例如允许来自 EC2-subnet1/32 的 SSH

一旦我完成了控制服务器的所有工作，我可以删除在我的公共子网中说 CIDR X/24 Target: local 的特定路由，因此它不再能够将流量路由到我的本地 CIDR。

我没有理由创建复杂的堡垒，因为我已经有权删除控制服务器中的路由。

于 2016-04-22T15:59:11.073 回答
3

您还没有提供完整的 Terraform，但您需要允许 SSH 从堡垒 IP 或堡垒主机的 CIDR 块进入您的“应用”VPC 实例，因此如下所示：

resource "aws_security_group" "allow_ssh" {
  name = "allow_ssh"
  description = "Allow inbound SSH traffic"

  ingress {
      from_port = 22
      to_port = 22
      protocol = "tcp"
      cidr_blocks = ["${aws_instance.bastion.private_ip}/32"]
  }
}

然后在您的“应用”实例资源中，您需要添加安全组：

...
vpc_security_group_ids = ["${aws_security_group.allow_ssh.id}"]
...

https://www.terraform.io/docs/providers/aws/r/security_group_rule.html

于 2016-04-20T02:03:22.873 回答
3

除非该堡垒主机还充当 NAT（我不建议您在同一实例上组合角色），否则 Web 和应用子网将没有任何出站 Web 访问，否则看起来很好的路由，因为 TF 会自动添加VPC 的本地路由记录。

只要您拥有覆盖您的 VPC 范围的本地路由记录，那么路由应该没问题。获取您的 Terraform 配置文件（并添加最少的必要资源）允许我在所有 3 个子网中创建一些基本实例并在它们之间成功路由，因此您可能会丢失其他内容，例如安全组或 NACL。

于 2016-03-06T17:50:34.357 回答
2

您应该使用 tcpdump 和其他调试工具检查网络问题。请检查：

  1. ips 可达且网络设置正确（如 10.200.2.X 可以到达堡垒主机的 ip）
  2. iptables/另一个防火墙不会阻止你的流量
  3. 一个 ssh 服务器正在侦听（从那些主机 ssh 到那些主机的 ip）
  4. 您拥有正确的主机安全组（您可以在 EC2 实例的属性中看到这一点）
  5. 尝试使用 tcpdump 嗅探流量
于 2016-04-20T20:41:44.643 回答