Homegrown Kubernetes on ARM

When I was nascent in Cloud Native, I was told that Kubernetes is the next big thing that you will evidence for yourself. Looking back, I can see that quote was quite accurate.

medium
Raspberry Pi 3

Today, hundreds and thousands of companies have moved or are moving their production to K8s, and the reasons for this are obvious. Previously, we did not have much choice regarding the underlying hardware. Now it seems that virtually all of the hardware has native support for Kubernetes or vice versa. One of the types is ARM (Advanced RISC Machine), which has recently gained quite a lot of traction. There are a few reasons that this hardware is so popular:

  • Low power consumption
  • Reasonable performance
  • Cost efficient

ARM Architecture Families

Motivation

I have always been fond of small/miniature devices: they are a bit too sophisticated to carry huge tasks. As a sophomore, I was given the chance to get my hands on Arduino, a small micro-controller, embedded with an Atmega-328 microprocessor. It was quite fun to see my code in action and see the small devices doing fancy things. Very quickly, I developed a hobby in IoT, so a couple of months back I bought a few Raspberry Pi systems. I thought about creating my own Kubernetes cluster using all the Pi that would be at my disposal anytime. I was aware that Lucas already added ARM support in Kubernetes quite a ways back.

2-1
 

Hardware

1. Raspberry Pi 3 B+

3-7

2. 16 GB MicroSDHC cards

4-8

3. Switch

5-4

4. CAT5e LAN cables

6-4

5. Power adapters

Process

There are several methods available on the internet to get your ARM cluster up and running using kubeadm. However, I’ll go ahead and brief my experience.

Common Configuration

The first step was flashing all the SD cards with Raspbian lite OS using Etcher, and placing an empty ssh file in the boot directory.

 touch ssh

This will make the Raspberry headless, meaning that we can now access it using ssh. Now we no longer need any monitor or external display.

ssh pi@raspberrypi.local

Now disable swaps on all the Raspberry systems. Kubernetes requires swap to be disabled.

sudo swapoff

In my case, I had one master and two workers. I changed the hostname of all the Raspberry to k8s-master-01, k8s-worker-01, k8s-worker-02, and k8s-worker-03. To change the hostname, I ssh’ed inside one of the Pi and typed sudo raspi-config. Now,
Network Options > Hostname

7-2
sudo raspi-config

and changed the hostname. 

Next, I assigned a static IP to all the nodes.
vi /etc/dhcpcd.conf and append the following line at the end of the file.

interface eth0
static ip_address=x.x.x.y/24
static routers=x.x.x.1
static domain_name_servers=8.8.8.8

This will assign the static IP, reboot Pi sudo reboot.

Now we are ready to install the required binaries/softwares:

  1. Docker
  2. Kubeadm
  3. Kubectl

Master Node Configuration

We’ll do this the traditional way kubeadm does. That is, initializing kubeadm in master and joining the rest of nodes as workers.

1. Pre-pull all the images that are required by kubeadm.
sudo kubeadm config images

 2. Now initialize the kubeadm.

sudo kubeadm init --token-ttl=0

--token-ttl=0 will ensure your token won’t expire. This will run for around 20 mins and will give with a token. By default, kubeadm will take weave net as a network plugin. Now, once you get a token with command, save it where desired.

3. Copy the command; it should look like this:

kubeadm join --token 9e700f.7dc97f5e3a45c9e5 192.168.0.2:6443 --
discovery-token-ca-cert-hash sha256:<HASH>

Worker Node Configuration

Paste the above in all of the worker nodes. There will be a few more steps involved, such as running:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

Your Raspberry Pi cluster is now ready.

Now try running:

kubectl get nodes

The nodes might not be ready at the moment, but don’t worry and just hold on for a bit.

All set to sail!

Use Cases

Sumit Lalwani and I achieved success in running OpenEBS Node Disk Manager on Raspberry Pi k8s cluster. NDM, if you don’t know, is an OpenSource Go project by OpenEBS, a CNCF project, which helps you get underlying disks using CRD.

kubectl get disks

Since I have my ARM cluster, it’s important to perform benchmarking tests to compare between different storage options or different network plugins.

Links

  • Node Disk Manager on Raspberry Pi k8s cluster
  • Evaluating different CNI plugins on Raspberry Pi k8s cluster
  • [WIP] OpenEBS vs Rook, benchmarking on ARM

 

This article was first published on Jun 21, 2019 on MayaData's Medium Account

Murat Karslioglu
VP @OpenEBS & @MayaData_Inc. Murat Karslioglu is a serial entrepreneur, technologist, and startup advisor with over 15 years of experience in storage, distributed systems, and enterprise hardware development. Prior to joining MayaData, Murat worked at Hewlett Packard Enterprise / 3PAR Storage in various advanced development projects including storage file stack performance optimization and the storage management stack for HPE’s Hyper-converged solution. Before joining HPE, Murat led virtualization and OpenStack integration projects within the Nexenta CTO Office. Murat holds a Bachelor’s Degree in Industrial Engineering from the Sakarya University, Turkey, as well as a number of IT certifications. When he is not in his lab, he loves to travel, advise startups, and spend time with his family. Lives to innovate! Opinions my own!
Jeffry Molanus
Jeffry prior to being CTO at MayaData has worked at several other startups in the storage industry. He worked on several scale-out object storage products as well as traditional NAS and SAN storage solutions where he held technical leadership roles. At MayaData, his primary focus is to make sure the product is flexible and scalable. At the same time, robust enough to be integrated seamlessly into modern-day infrastructure where he believes, containers will have a dominant role. Jeffry holds a master's degree in electrical engineering with a focus on distributed control engineering from the University of Twente in the Netherlands. When he is not working with code, he practices martial arts.
Abhishek Raj
Abhishek is a Customer Success Engineer at Mayadata. He is currently working with Kubernetes and Docker.