2

我正在使用Load S3 data into RDS MySql tableAWS Data Pipeline 中的模板将 csv 从 S3 存储桶导入我们的 RDS MySql。但是我（作为具有完全管理员权限的 IAM 用户）遇到了我无法解决的警告：

对象：Ec2Instance - 警告：无法验证角色的 S3 访问权限。请确保角色 ('DataPipelineDefaultRole') 具有 DataPipeline 的 s3:Get*、s3:List*、s3:Put* 和 sts:AssumeRole 权限。

谷歌告诉我不要对DataPipelineDefaultRoleand使用默认策略DataPipelineDefaultResourceRole。根据AWS Data Pipeline 的 IAM 角色文档和此 AWS 支持论坛上的主题，我使用了内联策略并编辑了这两个角色的信任关系。

政策DataPipelineDefaultRole：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:*",
                "datapipeline:DescribeObjects",
                "datapipeline:EvaluateExpression",
                "dynamodb:BatchGetItem",
                "dynamodb:DescribeTable",
                "dynamodb:GetItem",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:UpdateTable",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CancelSpotInstanceRequests",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DeleteTags",
                "ec2:Describe*",
                "ec2:ModifyImageAttribute",
                "ec2:ModifyInstanceAttribute",
                "ec2:RequestSpotInstances",
                "ec2:RunInstances",
                "ec2:StartInstances",
                "ec2:StopInstances",
                "ec2:TerminateInstances",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:DeleteSecurityGroup",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:DescribeNetworkInterfaces",
                "ec2:CreateNetworkInterface",
                "ec2:DeleteNetworkInterface",
                "ec2:DetachNetworkInterface",
                "elasticmapreduce:*",
                "iam:GetInstanceProfile",
                "iam:GetRole",
                "iam:GetRolePolicy",
                "iam:ListAttachedRolePolicies",
                "iam:ListRolePolicies",
                "iam:ListInstanceProfiles",
                "iam:PassRole",
                "rds:DescribeDBInstances",
                "rds:DescribeDBSecurityGroups",
                "redshift:DescribeClusters",
                "redshift:DescribeClusterSecurityGroups",
                "s3:CreateBucket",
                "s3:DeleteObject",
                "s3:Get*",
                "s3:List*",
                "s3:Put*",
                "sdb:BatchPutAttributes",
                "sdb:Select*",
                "sns:GetTopicAttributes",
                "sns:ListTopics",
                "sns:Publish",
                "sns:Subscribe",
                "sns:Unsubscribe",
                "sqs:CreateQueue",
                "sqs:Delete*",
                "sqs:GetQueue*",
                "sqs:PurgeQueue",
                "sqs:ReceiveMessage"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "iam:CreateServiceLinkedRole",
            "Resource": "*",
            "Condition": {
                "StringLike": {
                    "iam:AWSServiceName": [
                        "elasticmapreduce.amazonaws.com",
                        "spot.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

信任关系DataPipelineDefaultRole：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "ec2.amazonaws.com",
          "elasticmapreduce.amazonaws.com",
          "datapipeline.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

政策DataPipelineDefaultResourceRole：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:*",
                "datapipeline:*",
                "dynamodb:*",
                "ec2:Describe*",
                "elasticmapreduce:AddJobFlowSteps",
                "elasticmapreduce:Describe*",
                "elasticmapreduce:ListInstance*",
                "rds:Describe*",
                "redshift:DescribeClusters",
                "redshift:DescribeClusterSecurityGroups",
                "s3:*",
                "sdb:*",
                "sns:*",
                "sqs:*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

信任关系DataPipelineDefaultResourceRole：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

我尝试了几个选项/组合，但警告仍然存在。有谁知道如何解决这个权限问题？

4

2 回答 2

1

我回答这个问题可能有点晚了，但我刚刚发现您看到的警告信息可能具有误导性。如果您将管道配置为将日志放入 S3 存储桶，如果您仅指定存储桶的根目录而不是路径，则会出现警告。例如，如果我将配置字段“管道日志 Uri”（我在默认配置中找到）设置为，s3://bucket-name/那么我会看到警告。另一方面，如果我指定一个路径，例如s3://bucket-name/logs，警告就会消失。

AWS 论坛中的以下主题非常有助于解决这个问题：https ://forums.aws.amazon.com/thread.jspa?threadID=164635 。

于 2020-11-23T10:53:51.003 回答
1

我认为您的政策和角色的定义方式没有任何问题。一切看起来都不错。我唯一能想到的就是在定义角色之后你创建管道的速度有多快？

请记住IAM 策略是全球性的，而数据管道存在于特定区域中，因此在创建策略/角色和创建数据管道之间给它一些休眠时间，AWS 需要时间来复制所有区域中的 IAM 策略更改。

Ex. if you are using bash aws-cli to create/update role & then create/activate data-pipeline, insert `sleep Xs` between role & datapipeline creation.

Nitpick您不需要ec2.amazonaws.com信任关系DataPipelineDefaultRole。

于 2019-02-21T23:15:20.037 回答