9
Jenkins version : 2.121.3

我在 Jenkins 中使用 k8s 插件,它帮助我在我的 k8s 集群上部署我的顺序和并行作业。

这是Jenkinsfile作业失败的部分

           parallel([
                    build: {
                        stage('check formatting')
                        // some code

                        stage('build')
                        // some code

                        stage('build image')
                        // some code

                        stage('push image') 
                        // some code
                    },
                    test: {
                        stage('test') 
                        // some code

                        stage('build e2e test image')
                        // some code

                        }

                        stage('push e2e test image') 
                        // some code
                    },
                   failFast: true])

在运行具有并行阶段的作业时,我遇到了错误

caught java.io.IOException: Connection was rejected, you should increase the Max connections to Kubernetes API

然后我转到管理 Jenkins --> 配置系统 --> 云 --> Kubernetes 并通过说.... 20 增加“与 Kubernetes API 的最大连接数”,然后重新运行该作业,它就可以工作了。如果在那之后我尝试重新运行该作业,它会失败并出现相同的错误,然后我必须再次增加限制。所以我有点在一个无限循环中做以下事情:

1)重启jenkins evrerytime

2) 提高 API Server 请求限制。

所以我不确定为什么会这样。有没有办法让我找出 API 服务器正在处理多少请求?他们的来源是什么?我怎样才能知道以前的工作是否有任何挥之不去的要求?

完整的错误日志:

Failed in branch build
[Pipeline] // parallel
[Pipeline] echo
caught java.io.IOException: Connection was rejected, you should increase the Max connections to Kubernetes API
[Pipeline] echo
org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:329)
org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:237)
hudson.Launcher$ProcStarter.start(Launcher.java:449)
org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:188)
org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:99)
org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:278)
org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
sun.reflect.GeneratedMethodAccessor646.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
WorkflowScript.run(WorkflowScript:109)
___cps.transform___(Native Method)
com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
sun.reflect.GeneratedMethodAccessor243.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:103)
com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
sun.reflect.GeneratedMethodAccessor243.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:60)
com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
sun.reflect.GeneratedMethodAccessor243.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
com.cloudbees.groovy.cps.Next.step(Next.java:83)
com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:836)
4

2 回答 2

2

看起来这是插件的官方错误

https://issues.jenkins-ci.org/browse/JENKINS-55392

您可以尝试评论此问题,以帮助社区解决问题。

于 2019-01-14T11:13:47.833 回答
1

Set the Max connections to Kubernetes API to 64. Sometimes, setting it too low will generate errors. Also, Set the Container Cleanup Timeout to 5.

于 2019-01-11T15:32:23.243 回答