我需要帮助以使 XFS 配额在 DigitalOcean 上的 Kubernetes 中工作。
我的问题本质上是该xfs_quota
工具似乎仅在一个人还可以访问磁盘设备时才起作用,而不仅仅是对已安装的卷。但是,无论我尝试什么,我似乎都无法同时访问设备和安装。
我尝试了卷安装和原始块卷。
卷安装
这是我的存储类:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: block-storage-retain-xfs-prjquota
provisioner: dobs.csi.digitalocean.com
parameters:
fsType: xfs
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
mountOptions:
- prjquota
然后我声明一个新卷并将其添加到这样的 pod 中:
volumeClaimTemplates:
- metadata:
name: my-storage
namespace: my-namespace
spec:
accessModes:
- ReadWriteOnce
storageClassName: block-storage-retain-xfs-prjquota
resources:
requests:
storage: 1Gi
并安装它:
volumeMounts:
- name: my-storage
mountPath: "/var/www"
在 pod 中,一切都正确安装,我可以访问卷(我可以在其中创建内容)并且安装标志设置正确:
$ mount | grep -i www
/dev/disk/by-id/scsi-0DO_Volume_pvc-650ccba6-3177-45b5-9ffb-0ac2a931fddc on /var/www type xfs (rw,relatime,attr2,inode64,prjquota)
但是,磁盘设备在 pod 中不可用:
$ ls -la /dev/disk/by-id/scsi-0DO_Volume_pvc-650ccba6-3177-45b5-9ffb-0ac2a931fddc
ls: cannot access '/dev/disk/by-id/scsi-0DO_Volume_pvc-650ccba6-3177-45b5-9ffb-0ac2a931fddc': No such file or directory
(其实整个/dev/disk/
目录都不可用)
根据我的调查,无法访问设备是 XFS 工具失败的原因:
$ xfs_quota -x -c 'report -h' /var/www
xfs_quota: cannot setup path for mount /var/www: No such device or address
原始块卷
我还尝试改用原始块卷:
volumeClaimTemplates:
- metadata:
name: my-storage
namespace: my-namespace
spec:
accessModes:
- ReadWriteOnce
volumeMode: Block
storageClassName: block-storage-retain-xfs-prjquota
resources:
requests:
storage: 1Gi
并将其添加为:
volumeDevices:
- name: my-storage
devicePath: /dev/my-storage
这给了我设备,但由于某种原因我无法格式化/安装它(实际上既不是 XFS 也不是 ext4):
$ mkfs.xfs /dev/my-storage
mkfs.xfs: error - cannot set blocksize 512 on block device /dev/my-storage: Permission denied
$ mkfs.ext4 /dev/my-storage
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 18f07181-737c-4b68-a5fe-ccd7f2c50ff8
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
$ mount /dev/my-storage /var/www
mount: /var/www: cannot mount /dev/my-storage read-only.
使用SYS_ADMIN
Linux 功能,我实际上可以格式化它,但我仍然无法挂载它:
$ mkfs.xfs -f /dev/my-storage
meta-data=/dev/my-storage isize=512 agcount=4, agsize=65536 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
$ mount /dev/my-storage /var/www
mount: /var/www: cannot mount /dev/my-storage read-only.
(为什么磁盘设备是只读的?)
原始块卷 - 带分区
好的,所以我尝试创建一个分区并对其进行格式化。分区创建成功,但我无权访问分区设备:
$ fdisk -l /dev/my-storage
Disk /dev/my-storage: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk model: Volume
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb4a24142
Device Boot Start End Sectors Size Id Type
/dev/my-storage1 2048 2097151 2095104 1023M 83 Linux
但是,/dev/my-storage1
不存在:
$ ls -la /dev/my-storage*
brw-rw---- 1 root disk 8, 48 Oct 25 14:42 /dev/my-storage
我尝试运行容器,因为privileged
它使我可以访问更多设备/dev
,但是我根本没有看到我的原始块卷设备。
接下来是什么?
As I see that, any of those would work for me:
- Getting access to the underlying block device for volume mounts.
- Access to the partition device so that I can mount it.
- Ability to mount the raw block volume (e.g. by making it not read-only, whatever it means?).
- Making the
xfs_quota
tool NOT require the underlying device.
I believe I made it work a few months ago using raw block volumes with partitions, but either I forgot how or something changed on DigitalOcean and I can't seem to be able to create and access partitions anymore.
Any help is hugely appreciated, thank you!