What are the goals

This guide will walk you through the process of creating a self-hosted kubernetes cluster with the following features:

Because we are installing our cluster bare metal on servers exposed on the Internet, we’ll need a way to secure all of our network traffic around the critical parts of kubernetes. To do so, we’ll use OpenVPN to create a virtual secured network where all of our nodes will work. Moreover, this network will also contains MetalLB services when  configuring our bare metal load balancer .

Note : Even if this part is not required, you should not ignore it on dev environment and should really really REALLY not skip it for production. In fact, it can contain useful debug informations and security traces to see what is going on in your kubernetes cluster, and even on your whole server(s).

Create the cluster config file

We are now going to configure the cluster. For the sake of traceability, this configuration won’t be done via CLI flags, but via [a configuration file](()). The path of the cluster config file will later be referenced as the {{cluster.configFile}}, and should be inside /etc/kubernetes.

Start by creating traefik required resources. You can directly use resources from the  kubernetes/traefik templates: it does not contain variables. Those are taken from  traefik docs mixed up with  this PR for kubernetes 1.19 support and schemas.

Now that you have a router installed, you have to pass requests on your server to it. This setup use a single entry point directly binding some ports on the host server.

1. Make a static and previsible configuration

As you may have noticed in the step  Kickstart the cluster , the metallb configuration use only dynamic adresses. But for the reverse proxy to work, we’ll need to be sure that our traefik router has a constant IP in your VPN. For this, modify your metallb configuration using the new  kubernetes/metallb-configmap.yaml template. This new configuration declares a new address pool named frontend with a single IP in it.

As you may know, docker (and thus, kubernetes) does not persist anything by default. That means that everytime you restart a pod (container), it is in the exact same state as it was at its first execution, except for the mount points. Those mount points are real hard drive directories injected into your pod. Some apps we’ll setup later will require to persist data, and, more generally, when you’ll run real applications on your own, they will probably use a database or something.

Well, things are getting real and are on the point to become quite complex. So we’ll setup (super unsafe) dashboards to see what is going on easily. After all, we have nothing critical for now, but we might get troubles soon. And, don’t worry, we’ll make it safe just after that.

1. Traefik dashboard: monitoring routes

The traefik dashboard will help us in the diagnostics of our ingress routes and traefik-related stuff. For this, we need to:

Here is a graph of the RBAC setup we are going to implement:

1. Setup keycloak

We’ll use keycloak to proxy our authentication for all monitors, using a single realm. You may use several realms in real-life situations. This is probably the tough part, and you may tweak heavily the following guide. Moreover, I may forgot to write some instructions, or somes are heavily linked to your very own setup.

Create the realm and the client

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

REALM_URL="https://keycloak.{{cluster.baseHostName}}/auth/realms/{{apiServer.realmName}}"
# Log in
TOKEN_RESPONSE="$(curl \                           
        -d "grant_type=password" \                                
        -d "client_id={{apiServer.clientId}}" \
        -d "client_secret={{apiServer.clientSecret}}" \
        -d "username=admin-user" \
        -d "password=admin-user" \
        $REALM_URL/protocol/openid-connect/token)"
# Extract the access token
ACCESS_TOKEN="$(echo "$TOKEN_RESPONSE" | jq '.access_token' -r)"
# Check token
curl \
        --user "{{apiServer.clientId}}:{{apiServer.clientSecret}}" \
        -d "token=$ACCESS_TOKEN" \
        $REALM_URL/protocol/openid-connect/token/introspect -k

Set up certificates

Generate the certificates

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

mkdir certs
cd certs
# CA part (Certificate Authority)
# Generate the CA (Certificate Authority) private key
openssl genrsa -out ca.key 2048
# Generate the CA (Certificate Authority) certificate
openssl req -new -x509 \
        -subj "/C={{countryCodeIso3166_1_alpha_2}}/ST={{State}}/O={{companyName}}/CN={{cluster.baseHostName}}" \
        -addext "subjectAltName = DNS:{{cluster.baseHostName}}" \
        -key ca.key -out ca.crt
# # Import the CA (Certificate Authority) in the truststore, so that certificates signed by our authority are considered as trusted
# keytool -import -file ca.crt -keystore ca.truststore -keypass PASSWORD -storepass PASSWORD

# Keycloak part
# Generate the keycloak's private key
openssl genrsa -out keycloak.key 2048
# Generate the keycloak's CSR (Certificate Signing Request)
openssl req -new \
        -subj "/C={{countryCodeIso3166_1_alpha_2}}/ST={{State}}/O={{companyName}}/CN=kube-keycloak.{{cluster.baseHostName}}" \
        -addext "subjectAltName = DNS:kube-keycloak.{{cluster.baseHostName}}" \
        -key keycloak.key -out keycloak.csr
# Sign the CSR using our custom CA
openssl x509 -req \
        -days 3650 \
        -extfile <(printf "subjectAltName=DNS:kube-keycloak.{{cluster.baseHostName}}") \
        -CA ca.crt -CAkey ca.key \
        -in keycloak.csr -out keycloak.crt

Finally, inspect your keycloak’s certificate.