0

我必须将我的 DynamoDb 表备份到 S3 中，但是当我启动此服务时，我在尝试三次后收到此错误：

private.com.amazonaws.AmazonServiceException：用户：arn:aws:sts::769870455028:assumed-role/DataPipelineDefaultResourceRole/i-3678d99c 无权执行：elasticmapreduce:ModifyInstanceGroups（服务：AmazonElasticMapReduce；状态代码：400；错误代码： AccessDeniedException；请求 ID：9065ea77-0f95-11e5-8f35-39a70915a1ef) 在 private.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1077) 在 private.com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java :725) 在 private.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:295) 在 private.com.amazonaws.services 的 private.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:460)。 elasticmapreduce.AmazonElasticMapReduceClient.invoke(AmazonElasticMapReduceClient.java:1391) 在 private.com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient.modifyInstanceGroups(AmazonElasticMapReduceClient.java:785) 在 sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 在 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)在 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 在 java.lang.reflect.Method.invoke(Method.java:606) 在 private.com.amazonaws.services.datapipeline.retrier.RetryProxy.invokeInternal(RetryProxy .java:36) 在private.com.amazonaws.services.datapipeline.retrier.RetryProxy.invoke(RetryProxy.java:48) 在 com.sun.proxy.$Proxy33.modifyInstanceGroups(Unknown Source) 在 amazonaws.datapipeline.cluster。 EmrUtil.acquireCoreNodes(EmrUtil.java:325) 在 amazonaws.datapipeline.activity.AbstractClusterActivity。resizeIfRequired(AbstractClusterActivity.java:47) at amazonaws.datapipeline.activity.AbstractHiveActivity.runActivity(AbstractHiveActivity.java:113) at amazonaws.datapipeline.objects.AbstractActivity.run(AbstractActivity.java:16) at amazonaws.datapipeline.taskrunner.TaskPoller .executeRemoteRunner(TaskPoller.java:132) at amazonaws.datapipeline.taskrunner.TaskPoller.executeTask(TaskPoller.java:101) at amazonaws.datapipeline.taskrunner.TaskPoller$1.run(TaskPoller.java:77) at private.com.amazonaws .services.datapipeline.poller.PollWorker.executeWork(PollWorker.java:76) 在 private.com.amazonaws.services.datapipeline.poller.PollWorker.run(PollWorker.java:53) 在 java.lang.Thread.run(Thread .java:745)runActivity(AbstractHiveActivity.java:113) at amazonaws.datapipeline.objects.AbstractActivity.run(AbstractActivity.java:16) at amazonaws.datapipeline.taskrunner.TaskPoller.executeRemoteRunner(TaskPoller.java:132) at amazonaws.datapipeline.taskrunner.TaskPoller .executeTask(TaskPoller.java:101) at amazonaws.datapipeline.taskrunner.TaskPoller$1.run(TaskPoller.java:77) at private.com.amazonaws.services.datapipeline.poller.PollWorker.executeWork(PollWorker.java:76)在 private.com.amazonaws.services.datapipeline.poller.PollWorker.run(PollWorker.java:53) 在 java.lang.Thread.run(Thread.java:745)runActivity(AbstractHiveActivity.java:113) at amazonaws.datapipeline.objects.AbstractActivity.run(AbstractActivity.java:16) at amazonaws.datapipeline.taskrunner.TaskPoller.executeRemoteRunner(TaskPoller.java:132) at amazonaws.datapipeline.taskrunner.TaskPoller .executeTask(TaskPoller.java:101) at amazonaws.datapipeline.taskrunner.TaskPoller$1.run(TaskPoller.java:77) at private.com.amazonaws.services.datapipeline.poller.PollWorker.executeWork(PollWorker.java:76)在 private.com.amazonaws.services.datapipeline.poller.PollWorker.run(PollWorker.java:53) 在 java.lang.Thread.run(Thread.java:745)TaskPoller.executeTask(TaskPoller.java:101) at amazonaws.datapipeline.taskrunner.TaskPoller$1.run(TaskPoller.java:77) at private.com.amazonaws.services.datapipeline.poller.PollWorker.executeWork(PollWorker.java:76 ) 在 private.com.amazonaws.services.datapipeline.poller.PollWorker.run(PollWorker.java:53) 在 java.lang.Thread.run(Thread.java:745)TaskPoller.executeTask(TaskPoller.java:101) at amazonaws.datapipeline.taskrunner.TaskPoller$1.run(TaskPoller.java:77) at private.com.amazonaws.services.datapipeline.poller.PollWorker.executeWork(PollWorker.java:76 ) 在 private.com.amazonaws.services.datapipeline.poller.PollWorker.run(PollWorker.java:53) 在 java.lang.Thread.run(Thread.java:745)

如何进行备份？有人有这个错误吗？谢谢

编辑：新政策

{“版本”：“2012-10-17”，“声明”：[{“效果”：“允许”，“操作”：[“s3：”，“dynamodb： ”，“ec2：Describe*”，“ elasticmapreduce:Describe*”、“elasticmapreduce:ListInstance*”、“elasticmapreduce:AddJobFlowSteps”、“elasticmapreduce: ”、“rds:Describe ”、 “datapipeline:” 、“cloudwatch:”、“redshift:DescribeClusters”、“redshift :DescribeClusterSecurityGroups"、"sdb: "、"sns: "、"sqs:" ], "资源": [ " " ] } ]

这是新的例外：

Error during job, obtaining debugging information... Examining task ID: task_1434014832347_0001_m_000008 (and more) from job job_1434014832347_0001 Examining task ID: task_1434014832347_0001_m_000013 (and more) from job job_1434014832347_0001 Examining task ID: task_1434014832347_0001_m_000005 (and more) from job job_1434014832347_0001 Examining task ID: task_1434014832347_0001_m_000034 (and more) from job job_1434014832347_0001 Examining task ID: task_1434014832347_0001_m_000044 (and more) from job job_1434014832347_0001 Examining task ID: task_1434014832347_0001_m_000004 (and more) from job job_1434014832347_0001 Task with the most failures(4): ----- Task ID: task_1434014832347_0001_m_000002网址：http://ip-10-37-138-149.eu-west-1.compute.internal:9026/taskdetails.jsp?jobid=job_1434014832347_0001&tipid=task_1434014832347_0001_m_000002 -----此任务的诊断消息：错误：Java堆space FAILED: 执行错误，从 org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs 返回代码 2

4

1 回答 1

1

在您的 EMR 集群上运行的 Datapipeline 代理 (TaskRunner) 正在尝试调整 EMR 集群的大小，但它失败了。您传递给 EMR 集群的资源角色无权调用以下 API AmazonElasticMapReduce::modifyInstanceGroups。

我刚刚查看了 DefaultResourceRolePolicy，它是使用控制台中的向导创建的（http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-iam-roles.html）这些是 emr 的允许策略：“elasticmapreduce:Describe*”、“elasticmapreduce:ListInstance*”、“elasticmapreduce:AddJobFlowSteps”

我发现它不允许 ModifyInstanceGroups。
请更新您的资源角色政策以允许这样做。例如，“弹性映射减少：*”

感谢报告此错误。同时，我们将致力于修复控制台向导生成的默认资源角色策略。

阿拉文·R。

于 2015-06-10T23:43:12.553 回答