mysql - mariadb 在具有主机路径卷的 Kubernetes pod 内崩溃

Question

我正在尝试将 linux 服务器上的许多 docker 容器移动到在不同的 linux 机器上运行的基于 kubernets 的测试部署，在该机器上我已经将 kubernetes 作为k3s实例安装在 vagrant 虚拟机中。

其中一个容器是 mariadb 容器实例，绑定卷映射

这是我正在使用的 docker-compose 的相关部分：

  academy-db:
    image: 'docker.io/bitnami/mariadb:10.3-debian-10'
    container_name: academy-db
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
      - MARIADB_USER=bn_moodle
      - MARIADB_DATABASE=bitnami_moodle
    volumes:
      - type: bind
        source: ./volumes/moodle/mariadb
        target: /bitnami/mariadb
    ports:
      - '3306:3306'

请注意，这可以正常工作。（该容器被另一个应用程序容器使用，该容器连接到它并从数据库中毫无问题地读取数据）。

然后我尝试将其转换为 kubernetes 配置，将卷文件夹复制到目标计算机并使用以下 kubernetes .yaml 部署文件。这包括部署 .yaml、持久卷声明和持久卷，以及使容器可访问的 NodePort 服务。对于数据卷，我使用了一个简单的 hostPath 卷，指向从 docker-compose 的绑定挂载复制的内容。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: academy-db
spec:
  replicas: 1
  selector:
    matchLabels:
      name: academy-db
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        name: academy-db
    spec:
      containers:
        - env:
            - name: ALLOW_EMPTY_PASSWORD
              value: "yes"
            - name: MARIADB_DATABASE
              value: bitnami_moodle
            - name: MARIADB_USER
              value: bn_moodle
          image: docker.io/bitnami/mariadb:10.3-debian-10
          name: academy-db
          ports:
            - containerPort: 3306
          resources: {}
          volumeMounts:
            - mountPath: /bitnami/mariadb
              name: academy-db-claim
      restartPolicy: Always
      volumes:
        - name: academy-db-claim
          persistentVolumeClaim:
            claimName: academy-db-claim
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: academy-db-pv
  labels:
    type: local
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "<...full path to deployment folder on the server...>/volumes/moodle/mariadb"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: academy-db-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: ""
  volumeName: academy-db-pv
---
apiVersion: v1
kind: Service
metadata:
  name: academy-db-service
spec:
  type: NodePort
  ports:
    - name: "3306"
      port: 3306
      targetPort: 3306
  selector:
    name: academy-db

应用部署后，一切似乎都运行良好，kubectl get ...因为 pod 和卷似乎运行正常

kubectl get pods

NAME                             READY   STATUS    RESTARTS   AGE
academy-db-5547cdbc5-65k79       1/1     Running   9          15d
.
.
.

kubectl get pv
NAME                    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                              STORAGECLASS   REASON   AGE
academy-db-pv           1Gi        RWO            Retain           Bound    default/academy-db-claim                                   15d
.
.
.

kubectl get pvc
NAME                       STATUS   VOLUME                  CAPACITY   ACCESS MODES   STORAGECLASS   AGE
academy-db-claim           Bound    academy-db-pv           1Gi        RWO                           15d
.
.
.

这是 pod 的日志：

kubectl logs pod/academy-db-5547cdbc5-65k79

mariadb 10:32:05.66
mariadb 10:32:05.66 Welcome to the Bitnami mariadb container
mariadb 10:32:05.66 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mariadb
mariadb 10:32:05.66 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mariadb/issues
mariadb 10:32:05.66
mariadb 10:32:05.67 INFO  ==> ** Starting MariaDB setup **
mariadb 10:32:05.68 INFO  ==> Validating settings in MYSQL_*/MARIADB_* env vars
mariadb 10:32:05.68 WARN  ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes. For safety reasons, do not use this flag in a production environment.
mariadb 10:32:05.69 INFO  ==> Initializing mariadb database
mariadb 10:32:05.69 WARN  ==> The mariadb configuration file '/opt/bitnami/mariadb/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file.
mariadb 10:32:05.70 INFO  ==> Using persisted data
mariadb 10:32:05.71 INFO  ==> Running mysql_upgrade
mariadb 10:32:05.71 INFO  ==> Starting mariadb in background

和描述 pod 命令：

Name:         academy-db-5547cdbc5-65k79
Namespace:    default
Priority:     0
Node:         zdmp-kube/192.168.33.99
Start Time:   Tue, 22 Dec 2020 13:33:43 +0000
Labels:       name=academy-db
              pod-template-hash=5547cdbc5
Annotations:  <none>
Status:       Running
IP:           10.42.0.237
IPs:
  IP:           10.42.0.237
Controlled By:  ReplicaSet/academy-db-5547cdbc5
Containers:
  academy-db:
    Container ID:   containerd://68af105f15a1f503bbae8a83f1b0a38546a84d5e3188029f539b9c50257d2f9a
    Image:          docker.io/bitnami/mariadb:10.3-debian-10
    Image ID:       docker.io/bitnami/mariadb@sha256:1d8ca1757baf64758e7f13becc947b9479494128969af5c0abb0ef544bc08815
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 07 Jan 2021 10:32:05 +0000
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 07 Jan 2021 10:22:03 +0000
      Finished:     Thu, 07 Jan 2021 10:32:05 +0000
    Ready:          True
    Restart Count:  9
    Environment:
      ALLOW_EMPTY_PASSWORD:  yes
      MARIADB_DATABASE:      bitnami_moodle
      MARIADB_USER:          bn_moodle
      MARIADB_PASSWORD:      bitnami
    Mounts:
      /bitnami/mariadb from academy-db-claim (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-x28jh (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  academy-db-claim:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  academy-db-claim
    ReadOnly:   false
  default-token-x28jh:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-x28jh
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason          Age                  From     Message
  ----    ------          ----                 ----     -------
  Normal  Pulled          15d (x8 over 15d)    kubelet  Container image "docker.io/bitnami/mariadb:10.3-debian-10" already present on machine
  Normal  Created         15d (x8 over 15d)    kubelet  Created container academy-db
  Normal  Started         15d (x8 over 15d)    kubelet  Started container academy-db
  Normal  SandboxChanged  18m                  kubelet  Pod sandbox changed, it will be killed and re-created.
  Normal  Pulled          8m14s (x2 over 18m)  kubelet  Container image "docker.io/bitnami/mariadb:10.3-debian-10" already present on machine
  Normal  Created         8m14s (x2 over 18m)  kubelet  Created container academy-db
  Normal  Started         8m14s (x2 over 18m)  kubelet  Started container academy-db

但后来，我注意到客户端应用程序在连接时出现问题。经过一番调查，我得出结论，虽然 pod 正在运行，但其中运行的 mariadb 进程可能在启动后就崩溃了。如果我进入容器kubectl exec并尝试运行例如我得到的mysql客户端：

kubectl  exec -it pod/academy-db-5547cdbc5-65k79 -- /bin/bash

I have no name!@academy-db-5547cdbc5-65k79:/$ mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/opt/bitnami/mariadb/tmp/mysql.sock' (2)

知道什么可能导致问题，或者我该如何进一步调查这个问题？（注：我不是 Kubernetes 方面的专家，但最近才开始学习它）

编辑：根据@Novo 的评论，我尝试删除卷文件夹并让 mariadb 从头开始重新创建部署。现在我的 pod 甚至都没有启动，就终止了CrashLoopBackOff！

通过比较 pod 日志，我注意到在之前的 mariabd 日志中有一条消息：

...
mariadb 10:32:05.69 WARN  ==> The mariadb configuration file '/opt/bitnami/mariadb/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file.
mariadb 10:32:05.70 INFO  ==> Using persisted data
mariadb 10:32:05.71 INFO  ==> Running mysql_upgrade
mariadb 10:32:05.71 INFO  ==> Starting mariadb in background

现在替换为

...
mariadb 14:15:57.32 INFO  ==> Updating 'my.cnf' with custom configuration
mariadb 14:15:57.32 INFO  ==> Setting user option
mariadb 14:15:57.35 INFO  ==> Installing database

是否该问题与主机 vagrant 机器中的卷文件夹的某些访问权限问题有关？

score 4 · Accepted Answer

默认情况下，hostPath 目录的创建权限为 755，由 kubelet 的用户和组拥有。要使用该目录，您可以尝试将以下内容添加到您的部署中：

  spec:
     securityContext:
       fsGroup: <gid>

其中 gid 是容器中进程使用的组。

此外，您可以通过更改要挂载到容器中的文件夹的权限来解决主机本身的问题：

chown-R <uid>:<gid> /path/to/volume

其中 uid 和 gid 是您应用中的 userId 和 groupId。

chmod -R 777 /path/to/volume

这应该可以解决您的问题。

但总的来说，在这种情况下，部署不是您想要创建的，因为部署不应该有状态。对于有状态的应用程序，Kubernetes 中有“StatefulSets”。将它们与“VolumeClaimTemplate”加上 spec.securityContext.fsgroup 一起使用，k3s 将使用它的默认存储类，即本地存储（在您的节点上）为您创建持久卷和持久卷声明。

mysql - mariadb 在具有主机路径卷的 Kubernetes pod 内崩溃

1 回答 1

Related

Reference