2

我正在尝试使用包含 Java 工具的容器在 Kubernetes 作业中对 MySQL 数据库进行一些数据库迁移。

当我在 Docker 中本地运行容器时(使用同一网络中的 MySQL 容器),该工具按预期运行。如果我使用容器创建一个 Pod 并将命令参数设置为指向mysql在同一命名空间中运行的服务,它也可以。

但是,如果我将该 Pod 规范转换为作业,则创建的容器由于某种原因无法再连接到 MySQL 服务。

该容器基于amazoncorretto:8-al2-jdkJAR 并将其复制到/opt/.

MySQL DB 可通过mysql集群中的服务获得:

$ kubectl describe service mysql -n <namespace>
Name:              mysql
Namespace:         <namespace>
Labels:            app=mysql
Annotations:       <none>
Selector:          app=mysql
Type:              ClusterIP
IP Families:       <none>
IP:                <ip>
IPs:               <ip>
Port:              mysql  3306/TCP
TargetPort:        3306/TCP
Endpoints:         <ip>:3306
Session Affinity:  None
Events:            <none>

这些是 Pod 的规格:

apiVersion: v1
kind: Pod
metadata:
  name: java-tool-pod
spec:
  containers:
  - name: javatool
    image: <registry>/<image-name>:<version>
    command: [ "/bin/sh" ]
    args: [ "-x", "-c", "/usr/bin/java -jar /opt/<tool>.jar \"jdbc:mysql://mysql:3306/<db>\" -u <user> -p<password>" ]
  imagePullSecrets:
  - name: <secret>

将容器作为 Pod 运行:

$ kubectl apply -f /tmp/as-pod.yaml -n <namespace>
pod/java-tool-pod created

$ kubectl logs pod/java-tool-pod -n <namespace>
+ /usr/bin/java -jar /opt/<tool>.jar jdbc:mysql://mysql:3306/<db> -u <user> -p<password>
DB Migration Tool
Database Schema, 3.30.0.3300024390, built Wed Jul 14 12:13:52 UTC 2021
Driver class: com.mysql.jdbc.Driver
INFO Flyway 3.2.1 by Boxfuse
INFO Database: jdbc:mysql://mysql:3306/<db> (MySQL 5.7)
INFO Validated 721 migrations (execution time 00:00.253s)
INFO Current version of schema `<db>`: 3.29.0.10859.10
WARN outOfOrder mode is active. Migration of schema `<db>` may not be reproducible.
INFO Schema `<db>` is up to date. No migration necessary.

这些是作业的规格:

$ cat /tmp/as-job.yaml 
apiVersion: batch/v1
kind: Job
metadata:
  name: javatool-job
spec:
  template:
    spec:
      containers:
      - name: javatool
        image: <registry>/<image-name>:<version>
        command: [ "/bin/sh" ]
        args: [ "-x", "-c", "/usr/bin/java -jar /opt/<tool>.jar \"jdbc:mysql://mysql:3306/<db>\" -u <user -p<password>" ]
      imagePullSecrets:
      - name: <secret>
      restartPolicy: Never

将容器作为作业运行:

$ kubectl apply -f /tmp/as-job.yaml -n <namespace>
job.batch/javatool-job created

$ kubectl logs job.batch/javatool-job -n <namespace>
+ /usr/bin/java -jar /opt/<tool>.jar jdbc:mysql://mysql:3306/<db> -u <user> -p<password>
DB Migration Tool
Database Schema, 3.30.0.3300024390, built Wed Jul 14 12:13:52 UTC 2021
Driver class: com.mysql.jdbc.Driver
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:983)
    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:339)
    at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2252)
    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2285)
    at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2084)
    at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:795)
    at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
    at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400)
    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:208)
    at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:173)
    at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:164)
    at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:153)
    at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:119)
    at com.nordija.itv.db.FlywayMigrationSchemaData.isNotFlywaySchemaVersion(FlywayMigrationSchemaData.java:58)
[...]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:607)
    at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:214)
    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:298)
    ... 22 more
INFO Flyway 3.2.1 by Boxfuse
Unable to obtain Jdbc connection from DataSource
[...]

我没有看到正在创建的容器有任何显着差异。我唯一能想到的是某种字符编码问题,但我不明白为什么这只发生在为 Job 创建的 Pod 而不是直接创建的 Pod 中。

提前感谢您对此问题的任何帮助!

编辑:我忘了提到 Istio 在命名空间上处于活动状态,结果证明是导致问题的原因。

4

2 回答 2

1

问题是 Istio 不能很好地与 Kubernetes Jobs 配合使用(我忘了提到 Istio 在命名空间上是活跃的,抱歉)。

一旦我添加了一个短暂的延迟(sleep 5在启动 Java 工具之前),就可以建立连接。

但后来我又遇到了另一个问题:容器成功终止后,作业不会被标记为已完成。

原因又是 Istio。一旦所有Pod 都终止并且 Istio Sidecar 是一个不会终止的服务 Pod,作业就被认为是完成的。找到这篇文章后,我最终将他们的scuttle工具集成到容器中,现在可以成功完成作业。

于 2021-08-24T11:50:13.733 回答
0

错误消息是“通信链接故障”。所以我认为你应该看看你的网络政策。

于 2021-08-21T07:20:21.430 回答