2

我一直在尝试在我现有的 jenkins 设置（托管在 k8s 集群之外）上配置 jenkins kubernetes 云代理。我的 jenkins 托管在 Windows VM 中的 Google Cloud Platform 中。它暴露在互联网上，我们添加了一个 ssl 认证。

我们正在尝试使用 Jenkins Kubernetes 插件添加云代理。与 kubernetes 的连接有效（我已经通过测试连接验证了这一点。当我的工作尝试开始时，pods 容器也被添加到集群中）。我在我的 pod 模板中添加以下配置 - pod 容器在我的 kubernetes 引擎中启动。

问题 - 作业没有运行并不断创建新的 pod 并删除旧的 pod。在正确的方向上需要一些帮助。我在网上搜索了是否有人有类似的问题或设置。似乎每个人都在 k8s 中托管 jenkins 以及云代理。

我认为问题在于我们的詹金斯在我们的 Kubernetes 集群之外。

我现在使用默认的基本映像只是为了查看和检查作业是否运行。Job 在 Build shell 步骤中只有一个 echo 命令。我试过的图片

  • jnlp-slave-with-java-build-tools
  • 詹金斯/入站代​​理
  • 詹金斯/代理

对我的 kubernetes 集群的身份验证使用一个服务帐户 json，它具有所有访问权限。我正在使用自由式工作

谷歌堆栈驱动程序日志中的错误 -

SEVERE: Failed to connect to https://bflow.br.iq/tcpSlaveAgentListener/: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target`

`java.io.IOException: Failed to connect to https://bflow.br.iq/tcpSlaveAgentListener/: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:214)
    at hudson.remoting.Engine.innerRun(Engine.java:689)
    at hudson.remoting.Engine.run(Engine.java:514)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
    at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
    at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
    at sun.security.ssl.SSLTransport.decode(SSLTransport.java:149)
    at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1143)
    at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1054)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:394)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)
    at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:211)
    ... 2 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
    at sun.security.validator.Validator.validate(Validator.java:271)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:223)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
    ... 16 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
    ... 22 more
4

1 回答 1

0

设置细节：

之所以会出现这个问题，是因为最终用户必须使用带有 HTTPS 的 Jenkins 作为自签名证书。因此，当 Kubernetes 插件尝试启动基本 jenkins-inbound-agent 容器时，它不会识别主 Jenkins 证书。因此unable to find valid certification path to requested target错误。

解决方案： - 要解决此问题，请将主 Jenkins 证书导入 jenkins-inbound-agent 的 java truststore cacerts。这意味着在私有注册表上托管一个自定义 jenkins-inbound-agent。

  1. 获得詹金斯大师证书。
    $ openssl s_client -connect jenkins.my.domain.net:443 -showcerts > jenkins.crt
    depth=0 C = IN, ST = , L = Delhi, O = domain, OU = IT Operations, CN = *.jenkins.my.domain.net
    verify error:num=20:unable to get local issuer certificate
    verify return:1
    depth=0 C = IN, ST = , L = Delhi, O = domain, OU = IT Operations, CN = *.jenkins.my.domain.net
    verify error:num=21:unable to verify the first certificate
    verify return:1
    $ ls -lrth jenkins.crt
    -rw-rw-r--. 1 jenkins jenkins 3.3K Oct 20 09:52 jenkins.crt
    $
  1. 将此自签名证书注入 Jenkins-inbound-agent java cacerts。

在入口点脚本之前添加以下内容到您的Dockerfile中。

COPY jenkins.crt /tmp/jenkins.crt
RUN keytool -import -trustcacerts -keystore /opt/java/openjdk/jre/lib/security/cacerts -storepass ******* -noprompt -alias jenkins-master -file /tmp/jenkins.crt \
     && rm -Rf /tmp/jenkins.crt
  1. 构建jenkins-inbound-agent 的自定义 docker 镜像
$ docker build . -t myregistery.company.net:5000/company/jenkins-agent:latest
$ docker push myregistery.company.net:5000/company/jenkins-agent:latest
  1. 从私有注册表中提取自定义 jenkins-agent。
  • 在Manage Jenkins> Manage Nodes and Clouds> Configure Clouds>Kubernetes Cloud details
  • Advanced..>>Defaults Provider Template Name设定值default-java
  • 扩张Pod templates
    • 设置Name为default-java
  • 扩张Container tempalates
    • 设置Name为jnlp
    • 设置Docker Image为myregistery.company.net:5000/company/jenkins-agent:latest
    • 设置Always Pull Image为True
    • 设置Allocate pseudo-TTY为True

在此之后，创建一个示例管道项目并使用以下代码在 Kubernetes 集群上即时测试运行 jenkins-inbound-agent。

pipeline {
  agent {
    kubernetes {
      yaml '''
        apiVersion: v1
        kind: Pod
        '''
    }
  }
  stages {
    stage('Run') {
      steps {
            sh 'date'
            sh 'ls -lrth'
        
      }
    }
  }
}
于 2021-10-20T18:50:32.760 回答