Kubernetes on Bare Metal

Requirements

The following are the minimal requirements to setup a bare metal Kubernetes cluster:

• A compatible Linux host. The Kubernetes project provides generic instructions for Linux distributions based on Debian and Red Hat, and those distributions without a package manager.

• 2 GB or more of RAM per machine (any less will leave little room for your apps).

• 2 CPUs or more per machine.

• Full network connectivity between all machines in the cluster (public or private network is fine).

  Certain ports are open on your machines. See here for more details.

1. Set the Server Up

Before deploying Kubernetes on a bare metal server, it is important to configure the physical machines properly. Here are the steps you need to follow:

• Install the operating system: Choose a lightweight Linux distribution such as Ubuntu or any other distribution of your choice. In this tutorial, we go with Ubuntu since it’s widely spread and very well documented.

• Configure the network settings: After installing the operating system, make sure to go through the network configuration and assign a unique IP address to each machine.

• Assign proper hostnames: Edit the hostname file in the etc directory to assign descriptive names to each physical machine. For example, you can use names like “master node,” “worker1,” and “worker2.” Additionally, edit the hosts file to connect each hostname to its respective IP address.

Ports

We need different ports open for the different rules:

Control plane:

Although etcd ports are included in control plane section, you can also host your own etcd cluster externally or on custom ports.

Worker node(s):

By completing these steps, you will have the necessary operating system installed on each machine, along with unique names for easy management.

2. Install a Container Runtime

To ensure that the pods can operate on each node in your cluster, it’s necessary to install a container runtime on each one.

As of Kubernetes 1.29, a runtime that is compatible with the Container Runtime Interface (CRI) is required. For more details on CRI version support, refer to the relevant documentation.

There are several runtimes that can uses with Kubernetes 1.29:

• containerd
• CRI-O
• Docker Engine
• Mirantis Container Runtime

Since Kubernetes prefers Containerd over Docker, we go with Containerd as well.

Install Containerd

# update the system
sudo apt-get update
sudo apt-get upgrade

# install the dependencies
sudo apt-get install apt-transport-https ca-certificates curl gpg

# add Docker's GPG key
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# setup the repository
echo "deb [arch=$(dpkg --print-architecture) \
  signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Note: If you’re using an Ubuntu derivative distribution, like Linux Mint, you might need to use UBUNTU_CODENAME instead of VERSION_CODENAME.

# update the package index
sudo apt-get update
sudo apt-get install -y containerd.io

Configure Containerd

# make sure containerd starts using systemd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sudo sed -i 's/systemd_cgroup \= false/systemd_cgroup \= true/g' /etc/containerd/config.toml

# restart and enable the containerd service:
sudo systemctl restart containerd
sudo systemctl enable containerd

# check if the service started successfully
sudo systemctl status containerd

So, one down. The runtime is installed. Let’s proceed.

3. Disable the Swap File

Swap is a space on the hard drive the operating system uses to temporarily store the data when the physical RAM is full. It acts as an extension of the RAM and provides a way for the operating system to offload inactive data from the RAM.

However, Kubernetes and containerized applications are optimized for fast access to physical RAM. So, you must disable swap to keep Kubernetes from relying on slower swap space.

You can disable swap by entering the following code in your terminal:

sudo swapoff -a

Edit your fstab file in /etc, so that the swap file won’t be recreated as soon as the server reboots. Comment out the line saying, /swap...

4. Load the Kernel Modules and setup an Ipv4 Bridge

# load required kernel modules
sudo tee /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# configure kubernetes kernel paremeters
sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

# apply the changes
sudo sysctl --system

5. Install Kubernetes

Now, it’s time to install Kubernetes tools on all the physical machines in your cluster. The essential tools:

• kubeadm: For bootstrapping the Cluster and Nodes
• kubelet: The component that runs on all nodes, handles “stuff” like starting of pods, containers, report node’s status, etc.
• kubectl: The command line utility to talk to the cluster

# add Kubernetes' GPG key
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

WARNING: the next steps overwrites any existing configuration in /etc/apt/sources.list.d/kubernetes.list

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

# update the package index
sudo apt-get update

# update the apt package index, install kubelet, kubeadm and kubectl, and pin their version
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.

6. Initialize the master node

To initialize the master node in Kubernetes, we need to use the kubeadm init command. Specify the --pod-network-cidr flag with the appropriate CIDR value based on the chosen pod network. Since we’re using Flannel, the CIDR is 10.244.0.0/16.

Run the following command to initialize the master node:

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --upload-certs

After executing the command, we see an ouput in the form of kubeadm join <master-node-ip>:<master-node-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>. Save this since you will need it later on.

7. Set up kubectl on the master node

What we have to do next, is to save the config to be able to access the cluster with kubectl.

We can do so by entering the following commands in the terminal:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

To confirm if everything works, we run the following command to get the list of nodes:

kubectl get nodes

8. Install a pod network add-on

To assist communication across the nodes in the Kubernetes cluster, you’ll need a pod network add-on. You can find a list of add-on options on the Kubernetes website.

After choosing an add-on, you can install it by entering the following command:

kubectl apply -f <pod-network-addon>.yaml

Replace <pod-network-addon> with the URL for your chosen add-on. For instance, if you’re installing Flannel, you can enter the following command:

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

9. Join worker nodes to the cluster

After you have initialized the master node and configured the pod network, you can add the worker nodes to the cluster.

You can do so by running the following command on each node:

sudo kubeadm join <master-node-ip>:<master-node-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

Here, you must replace <master-node-ip>, <master-node-port>, <token> and <hash> with the values you saved at the end of step 6.

10. Verify cluster status

After adding all the worker nodes to the master node, it’s time to verify the status of the cluster.

You can do so by entering the below command in the master node:

kubectl get nodes

You should see an output with all the worker nodes and their status as “ready.”

11. Deploy applications

With the Kubernetes cluster set up, you can now deploy applications onto the cluster. For example, if you want to test with an NGINX application, you can do so by entering the following command:

kubectl create deployment nginxtest1 --image=nginx