我决定使用无根版本的 Buildkit 从 Kubernetes 的容器中构建 Docker 映像并将其推送到 GCR（Google Container Registry）。
我偶然发现了这个错误：
/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /home/user/.local/tmp/buildkit-mount859701112: [{Type:bind Source:/home/user/.local/share/buildkit/runc-native/snapshots/snapshots/2 Options:[rbind ro]}]: operation not permitted
我正在buildkitd作为deployment链接到buildkit 文档service指定的链接运行
这些资源在托管在 Google Kubernetes Engine 上的 Kubernetes 集群中运行。
我正在使用以下 YAML 进行部署和服务
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: buildkitd
name: buildkitd
spec:
replicas: 1
selector:
matchLabels:
app: buildkitd
template:
metadata:
labels:
app: buildkitd
annotations:
container.apparmor.security.beta.kubernetes.io/buildkitd: unconfined
container.seccomp.security.alpha.kubernetes.io/buildkitd: unconfined
spec:
containers:
- name: buildkitd
image: moby/buildkit:master-rootless
args:
- --addr
- unix:///run/user/1000/buildkit/buildkitd.sock
- --addr
- tcp://0.0.0.0:1234
- --oci-worker-no-process-sandbox
readinessProbe:
exec:
command:
- buildctl
- debug
- workers
initialDelaySeconds: 5
periodSeconds: 30
livenessProbe:
exec:
command:
- buildctl
- debug
- workers
initialDelaySeconds: 5
periodSeconds: 30
securityContext:
runAsUser: 1000
runAsGroup: 1000
ports:
- containerPort: 1234
---
apiVersion: v1
kind: Service
metadata:
labels:
app: buildkitd
name: buildkitd
spec:
ports:
- port: 1234
protocol: TCP
selector:
app: buildkitd
它与没有设置 TLS 证书的buildkit 文档相同。
然后，我在另一个 Pod 中使用以下命令联系 Buildkit Daemon：
./bin/buildctl \
--addr tcp://buildkitd:1234 \
build \
--frontend=dockerfile.v0 \
--local context=. \
--local dockerfile=. \
--output type=image,name=eu.gcr.io/$PROJECT_ID/test-image,push=true
buildkitd容器成功接收请求但抛出上述错误。
该buildctl命令的输出如下：
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.1s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 120B done
#2 DONE 0.1s
error: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /home/user/.local/tmp/buildkit-mount859701112: [{Type:bind Source:/home/user/.local/share/buildkit/runc-native/snapshots/snapshots/2 Options:[rbind ro]}]: operation not permitted
这是守护程序的错误。
令我印象深刻的是，我能够使用完全相同的 YAML 文件在集群buildkitd内进行容器化：minikube
NAME READY STATUS RESTARTS AGE
pod/buildkitd-5b46d94f5d-xvnbv 1/1 Running 0 36m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/buildkitd ClusterIP 10.100.72.194 <none> 1234/TCP 36m
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 36m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/buildkitd 1/1 1 1 36m
NAME DESIRED CURRENT READY AGE
replicaset.apps/buildkitd-5b46d94f5d 1 1 1 36m
我在内部部署服务和部署，minikube并使用以下命令转发服务端口，以便能够访问外部部署minikube。
kubectl port-forward service/buildkitd 2000:1234
通过该设置，我可以buildctl毫无问题地执行我的命令（图像构建并推送到 GCR）。
我想了解为什么它可以在 Google Kubernetes Engine 上运行minikube而不是在 Google Kubernetes Engine 上运行。
如果有任何帮助，这是容器启动日志
auto snapshotter: using native
NoProcessSandbox is enabled. Note that NoProcessSandbox allows build containers to kill (and potentially ptrace) an arbitrary process in the BuildKit host namespace. NoProcessSandbox should be enabled only when the BuildKit is running in a container as an unprivileged user.
found worker \"wdukby0uwmjyvf2ngj4e71s4m\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:buildkitd-5b46d94f5d-xvnbv org.mobyproject.buildkit.worker.snapshotter:native], platforms=[linux/amd64 linux/386]"
rootless mode is not supported for containerd workers. disabling containerd worker.
found 1 workers, default=\"wdukby0uwmjyvf2ngj4e71s4m\"
currently, only the default worker can be used.
TLS is not enabled for tcp://0.0.0.0:1234. enabling mutual TLS authentication is highly recommended
running server on /run/user/1000/buildkit/buildkitd.sock
running server on [::]:1234