docker - Where is kube-apiserver located

Question

Base question: When I try to use kube-apiserver on my master node, I get command not found error. How I can install/configure kube-apiserver? Any link to example will help.

$ kube-apiserver --enable-admission-plugins DefaultStorageClass
-bash: kube-apiserver: command not found

Details: I am new to Kubernetes and Docker and was trying to create StatefulSet with volumeClaimTemplates. My problem is that the automatic PVs are not created and I get this message in the PVC log: "persistentvolume-controller waiting for a volume to be created". I am not sure if I need to define DefaultStorageClass and so needed kube-apiserver to define it.

Name:          nfs
Namespace:     default
StorageClass:  example-nfs
Status:        Pending
Volume:
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner=example.com/nfs
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
Events:
  Type    Reason                Age                  From                         Message
  ----    ------                ----                 ----                         -------
  Normal  ExternalProvisioning  3m (x2401 over 10h)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "example.com/nfs" or manually created by system administrator

Here is get pvc result:

$ kubectl get pvc
NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs       Pending                                       example-nfs    10h

And get storageclass:

$ kubectl describe storageclass example-nfs
Name:                  example-nfs
IsDefaultClass:        No
Annotations:           <none>
Provisioner:           example.com/nfs
Parameters:            <none>
AllowVolumeExpansion:  <unset>
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>

How can I troubleshoot this issue (e.g. logs for why the storage was not created)?

Answer 1

You are asking two different questions here, one about kube-apiserver configuration, one about troubleshooting your StorageClass.

Here's an answer for your first question:

kube-apiserver is running as a Docker container on your master node. Therefore, the binary is within the container, not on your host system. It is started by the master's kubelet from a file located at /etc/kubernetes/manifests. kubelet is watching this directory and will start any Pod defined here as "static pods".

To configure kube-apiserver command line arguments you need to modify /etc/kubernetes/manifests/kube-apiserver.yaml on your master.

Answer 2

I'll refer to the question regarding the location of the api-server.

Basic answer (specific to the question title):

The kube apiserver is located on the master node (known as the control plane).

It can be executed:

1 ) Via the host's init system (like systemd).

2 ) As a pod (I'll explain below).

In both cases it will be located on the control plane (left side below):

If its running under systemD you can run: systemctl status api-server to see the path to the configuration (drop-in) file.
If it is running as pod you can view it under the kube-system namespace with all other control panel components (plus kube-proxy and maybe network solution like weave below):

$ kubectl get pods -n kube-system
NAME                                      READY   STATUS    RESTARTS   AGE
coredns-f9fd979d6-lpdlc                   1/1     Running   1          2d22h
coredns-f9fd979d6-vcs7g                   1/1     Running   1          2d22h
etcd-my-master                            1/1     Running   1          2d22h
kube-apiserver-my-master                  1/1     Running   1          2d22h #<----Here
kube-controller-manager-my-master         1/1     Running   1          2d22h
kube-proxy-kh2lc                          1/1     Running   1          2d22h
kube-scheduler-my-master                  1/1     Running   1          2d22h
weave-net-59r5b                           2/2     Running   3          2d22h

You can run:

kubectl describe pod/kube-apiserver-my-master -n kube-system

In order to get more details regarding the pod.

A bit more advanced answer:

(regarding the location of /etc/kubernetes/manifests)

Lets say we have no idea where to find the relevant path for the kube-api-server config file.

But we need to remember two important things:

1 ) The kube-api-server is running on the master node.

2 ) The Kubelet isn't running as pod and when the control plane components (plus kube-proxy) are executed as static pods - it is done by the Kubelet on the master node.

So we can start our journey for reaching the manifests path by investigating the Kubelet logs.
If the Kubelet is running for a long time it will be a very large file and we'll need to dump it somewhere and go to the begging - or if Kubelet was started 5 minutes ago we can run:

sudo journalctl -u kubelet --since -5m >> kubelet_5_minutes.log

And a quick search for "api-server" will bring us to the 2 lines below where the path of the manifests in mentioned:

my-master kubelet[71..]: 00:03:21 kubelet.go:261] Adding pod path: /etc/kubernetes/manifests
my-master kubelet[71..]: 00:03:21 kubelet.go:273] Watching apiserver

And also we can see that the Kubelet is trying to create the kube-apiserver pod under my-master node and inside the kube-system namespace:

my-master kubelet[71..]: 00:03:29.05  kubelet.go:1576] ..
           Creating a  mirror pod for "kube-apiserver-my-master_kube-system

Answer 3

To make the storage class "example-nfs" default, you need to run the below command:

kubectl patch storageclass example-nfs -p '{"metadata": 
  {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'