I’ve seen Scaleway mentioned a few times on Twitter, mainly surrounding their cheap ARM64 hosting, so I thought I’d use the festive break to check them out and tinker with the latest version of kubeadm.
They’ve got some excellent pricing for their X86-64 instances too, for €2.99 per month you can get 2 cores, 2GB of RAM, and a local 50GB SSD which means for €11.96 (~£10.60) per month I could launch a X86-64 Kubernetes cluster (1 master and 3 nodes) with 8GB of RAM, 8 cores, and 150GB of storage to play with.
This post is not meant the be a replacement for the official documentation for installing Kubernetes with
kubeadm, but more a log of the steps I took and the issue(s) I encountered along the way.
So here we go…
I registered for a Scaleway account, provided some credit card details, and headed on over to the control panel. Before creating the servers I added an SSH key in the credentials section of my profile, once that was done I created four Ubuntu 16.04 LTS (Xenial) instances with the default settings and this is where I hit an issue.
The default Scaleway bootscript for the Ubuntu 16.04 LTS (Xenial) instance uses
x86_64 4.4.105 which doesn’t have the
xt_set Kernel module enabled, and this module is required for pod networking later on. So if, like I did, you manage to reach this point you’ll find some odd
iptables related errors that incurred some head scratching.
Thankfully, Scaleway have enabled this module in their
x86_64 4.10.8 bootscript, you can edit the server in the control panel and select this then reboot the server. You can check if the module is enabled with
zcat /proc/config.gz | grep XT_S which should return
CONFIG_NETFILTER_XT_SET=m - sadly you cannot select this bootscript in the the control panel when creating a server so there’s a bit of messing about to be done.
Once the servers were created (with the correct bootscript!) I SSH’d in to each server and ran the following commands:-
1 2 3 4 5 6 7 8 9 10 11 curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - echo "deb https://download.docker.com/linux/ubuntu xenial stable" > /etc/apt/sources.list.d/docker-ce.list export DEBIAN_FRONTEND=noninteractive apt-get install apt-transport-https apt-get update -yq --fix-missing && apt-get dist-upgrade -yq apt-get install -yq docker-ce=17.03.0~ce-0~ubuntu-xenial kubelet kubeadm kubectl kubernetes-cni apt-mark hold docker-ce
These commands install the Kubernetes and Docker repositories, update the base system, then installs the
docker-ce packages. Officially, Kubernetes only supports Docker
17.03 so I installed the
17.03.0~ce-0~ubuntu-xenial version supplied by the repository and used
apt-mark hold to prevent any updates/upgrades from automatically upgrading the
docker-ce package without manual intervention.
Once each server was updated and had all the required packages installed, I SSH’d in to the server which I was going to be the Kubernetes master. Once in, I ran
kubeadm init --pod-network-cidr=192.168.0.0/16 to bring up Kubernetes. I used the
--pod-network-cidr=192.168.0.0/16 option as I wanted to use Calico for pod networking and the documentation required this option.
You can see the steps
kubeadm makes when invoking
init in Lucas Käldström’s excellent “kubeadm Cluster Creation Internals: From Self-Hosting to Upgradability and HA” presentation.
kubedm completed and provided me with some instructions to configure
kubectl, and a command to run on each of the other servers to add them to this cluster, it was somewhat similar to the following:-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join --token 602414.e8ab1c628aecea66 10.4.123.75:6443 --discovery-token-ca-cert-hash sha256:df302c574bf82a6feb9b43f70d12cb323c661585f6fa59b10a9b78f0f035c67c
Before running the join command on the other servers I installed the Calico pod network by executing the following command on the master:-
1 kubectl --kubeconfig=/etc/kubernetes/admin.conf apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/kubeadm/1.7/calico.yaml
I then checked the status of the
kube-system pods by running
kubectl --kubeconfig=/etc/kubernetes/admin.conf -n kube-system get pods and waited for all the pods to be in the running state which will look something like:-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-etcd-sql6b 1/1 Running 0 2m kube-system calico-kube-controllers-d669cc78f-6hqwk 1/1 Running 0 2m kube-system calico-node-j2mw4 1/2 Running 0 46s kube-system calico-node-j4t9l 2/2 Running 0 1m kube-system calico-node-nd4f9 2/2 Running 0 2m kube-system etcd-k8s-master 1/1 Running 0 2m kube-system kube-apiserver-k8s-master 1/1 Running 0 1m kube-system kube-controller-manager-k8s-master 1/1 Running 0 2m kube-system kube-dns-6f4fd4bdf-cfptf 3/3 Running 0 2m kube-system kube-proxy-fx29m 1/1 Running 0 2m kube-system kube-proxy-s9mxx 1/1 Running 0 1m kube-system kube-proxy-tj26q 1/1 Running 0 46s kube-system kube-scheduler-k8s-master 1/1 Running 0 2m
After running the
kubeadm join ... command on each of the other servers, I SSH’d back in to the master and ran
kubectl --kubeconfig=/etc/kubernetes/admin.conf get nodes to confirm that they had joined, and for completion that looked like:-
1 2 3 4 5 NAME STATUS ROLES AGE VERSION k8s-master Ready master 2m v1.9.0 k8s-node-1 Ready <none> 1m v1.9.0 k8s-node-2 Ready <none> 52s v1.9.0 k8s-node-3 Ready <none> 46s v1.9.0
… and there we can have it, a Kubernetes cluster on Scaleway.
When picking up something new I tend to script it in some way, this helps me understand the required steps a little better and ensures that I haven’t missed anything. If you’re interested in how I did that, you can head on over to Deploying Kubernetes with the Scaleway CLI as this post is getting a little lengthy and you could probably stop here if you’re not interested.