0

是否可以调整 docker-workflow-plugin / docker-pipeline-plugin 的清理例程?也许离开容器并允许管道代码在重试块中处理它的删除?

我有一份工作,在 8 个 EC2 实例 ECS 集群上运行的 64 个节点上运行 30 个这些子句的串行组,一个在清理期间失败

                docker.image(selectedNodeLabel).inside {
                    build_kernel_module(version, distro, test, type)
                }

错误

java.io.IOException: Failed to rm container 'dd589a813fec46b7dc97fed273c4ddd09183a561ff8c9d584ea7b299d606d1fb'.
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.rm(DockerClient.java:191)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.stop(DockerClient.java:178)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep.destroy(WithContainerStep.java:109)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep.access$400(WithContainerStep.java:76)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Callback.finished(WithContainerStep.java:390)
at org.jenkinsci.plugins.workflow.steps.BodyExecutionCallback$TailCall.onSuccess(BodyExecutionCallback.java:118)
at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:377)
at com.cloudbees.groovy.cps.Outcome.resumeFrom(Outcome.java:73)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:166)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
4

1 回答 1

0

我们的案例似乎是由 jenkins ecs 清理作业驱动的,该作业定期清除失效的容器/卷等。我们运行它是因为我们有几个作业与 docker sock 交互,以便 ecs 主机启动他们自己的测试环境。如果作业被残忍地杀死,则清理不会运行。我们运行 docker system purges,删除停止的容器等。该作业应该清除超过 1 天的任何内容。

当停止该集群上的作业时,问题就消失了。

于 2020-06-25T14:52:20.457 回答