kubernetes - Failed to install rook on k8s cluster

Question

I am trying to create a rook cluster inside k8s cluster.

Set up - 1 master node, 1 worker node

These are the steps I have followed

Master node: 
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
sudo sysctl net.bridge.bridge-nf-call-iptables=1
sudo sysctl net.bridge.bridge-nf-call-ip6tables=1
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/32a765fd19ba45b387fdc5e3812c41fff47cfd55/Documentation/kube-flannel.yml
kubeadm token create --print-join-command

Worker node: 
kubeadm join {master_ip_address}:6443 --token {token} --discovery-token-ca-cert-hash {hash} --apiserver-advertise-address={worker_private_ip}

Master node - Install rook - (reference - https://rook.github.io/docs/rook/master/ceph-quickstart.html):
kubectl create -f ceph/common.yaml
kubectl create -f ceph/operator.yaml
kubectl create -f ceph/cluster-test.yaml

Error while creating rook-ceph-operator pod:

(combined from similar events): Failed create pod sandbox: rpc error: code = 
Unknown desc = failed to set up sandbox container "4a901f12e5af5340f2cc48a976e10e5c310c01a05a4a47371f766a1a166c304f" 
network for pod "rook-ceph-operator-fdfbcc5c5-jccc9": networkPlugin cni failed to 
set up pod "rook-ceph-operator-fdfbcc5c5-jccc9_rook-ceph" network: failed to set bridge addr: 
"cni0" already has an IP address different from 10.244.1.1/24

Can anybody help me with this issue?

Answer 1

This issue start if you did kubeadm reset and after that kubeadm init reinitialize Kubernetes.

kubeadm reset
systemctl stop kubelet
systemctl stop docker
rm -rf /var/lib/cni/
rm -rf /var/lib/kubelet/*
rm -rf /etc/cni/
ifconfig cni0 down
ifconfig flannel.1 down
ifconfig docker0 down

After this start docker and kubelet and kubeadm again.

Work around

You can also try this way as simple easy solution

ip link delete cni0
ip link delete flannel.1

that depends on which network you are using inside k8s.