# k8s: Kubernetes

meta-virtualization provides the recipes and packages suitable for a k8s
cluster instance.

For a kubernetes controller:

  - packagegroup-k8s-host

For a kubernetes worker/node:

  - packagegroup-k8s-node

If kernel issues or missing features are detected, consider adding the
"kernel-modules" package to your image (Since the configuration and RDEPENDS
may not be correct for your kernel + k8s version).

## CNI

The CNI base packages provide core support and are installed by default as
dependencies of the kubernetes packages. Minimal configuration and startup
are provided, but you will need to apply the CNI configuration of your choice
after boot (see below fo an example)

## Configure and initialize the host

A convenience script "k8s-init" is provided to do basic setup on the controller
node. After the contoller boots, run it for kubeadm setup and other basic
configuration.

Once the node is ready ('kubectl get nodes' to check), follow the instructions
for copying the token to your home directory, and apply the networking configuration
of choice (flannel in the example):

```shell
  % mkdir -p $HOME/.kube
  % cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  % chown $(id -u):$(id -g) $HOME/.kube/config

  % kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```

## Configuration and initialize the node/worker

Once the kubernetes node has booted, it is ready to join the cluster. Some
basica configuration is done via the packages and a systctl.d configuration
snippet.

Join the cluster (substitute your controller ip and token information):

```shell
kubeadm join <controller ip>:6443 --token cq8ngi.6m6mgqi9zf08ypc4 --discovery-token-ca-cert-hash sha256:6064ae531c8dad824f9eadff030f83ec84d00796fac75f1adbd343255eb34fd2
```

## Notes:

Memory:

  if running under qemu, the default of 256M of memory is not enough, k3s will
  OOM and exit.

  Boot with qemuparams="-m 2048" to boot with 2G of memory (or choose the
  appropriate amount for your configuration)

CPUs:

  Kubernetes needs at least two cpus, so ensure your qemuboot is smp of at
  least 2, and/or that your hardware has the required capabilties.

Disk:

  if using qemu and core-image* you'll need to add extra space in your disks
  to ensure containers can start. The following in your image recipe, or
  local.conf would add 2G of extra space to the rootfs:

```shell
IMAGE_ROOTFS_EXTRA_SPACE = "2097152"
```

## Example qemux86-64 boot line:

```shell
runqemu qemux86-64 nographic kvm slirp qemuparams="-m 2048"
```
k8s logs can be seen via:

```shell
% journalctl -u kubelet
```

or

```shell
% journalctl -xe
```

## Example output from qemux86-64:

If you've lost the join token, you can create a new one, or list existing
ones:

```shell
root@qemux86-64-7b:~# kubeadm token create --print-join-command
kubeadm join 10.10.10.117:6443 --token dr71zq.y5vi3s2n2antvcej --discovery-token-ca-cert-hash sha256:6064ae531c8dad824f9eadff030f83ec84d00796fac75f1adbd343255eb34fd2

root@qemux86-64-7b:~# kubeadm token list
TOKEN                     TTL         EXPIRES                USAGES                   DESCRIPTION                                                EXTRA GROUPS
cq8ngi.6m6mgqi9zf08ypc4   23h         2021-12-16T16:58:02Z   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
dr71zq.y5vi3s2n2antvcej   23h         2021-12-16T17:46:28Z   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token
```

```shell
root@qemux86-64:~# kubectl get nodes
NAME            STATUS   ROLES                  AGE   VERSION
qemux86-64-7b   Ready    control-plane,master   51m   v1.23.1-rc.0.1+dd1b0a12471310-dirty
qemux86-64-9d   Ready    <none>                 49m   v1.23.1-rc.0.1+dd1b0a12471310-dirty
```