# Set up cert-manager in Kubernetes for TLS
# on Digital Ocean Kubernetes Service (DOKS)
🗓 February 15, 2020 | 👱 By: Hugh
Welcome to another part of my N part series on setting up a scalable Wordpress installation on Kubernetes, specifically a Digital Ocean cluster (DOKS).
We'll base this on the setup we had in the last tutorial when we installed nginx-ingress. In that version we didn't have any TLS, a.k.a. encryption on your communication, a.k.a. the little lock in the address bar.
You might not think it is a big deal, but consider that when you log in to your site you'll be sending your password in the clear over the public internet, and anyone with visibility of those packets can see your password being sent in the clear. So lets get started!
This is a bit of a cross between a couple of the official documentation pages (here and here), but I make some changes to the issuers that make things a bit easier to manage in a single tenant environment.
First up we need to configure DNS. The certificates we will generate/use will be tied to a domain name which we obviously need to have control of. It would be crazy if we could generate a certificate for google.com. So, get the IP of your load balancer in Digital Ocean either using the web UI, or by running the following
doctl compute load-balancer list --format Name,IP
You should see your load balancer name and IP, the name will probably be a random hex string, the IP will be...an IP.
Create an A record in you DNS management tool of choice pointing to your Load Balancer IP.
Update Ingress to include TLS
It is possible to add the TLS fields to our ingress before we have actually generated a certificate, the result will be that it will use a self signed certificate and you'll get a warning in your browser. At this stage it isn't an issue - we don't expect it to work.
We'll use the same manifest we did in the previous part, but we'll update the Ingress so that it looks like this:
apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx name: doks-example namespace: website-1 spec: tls: - hosts: - sample-app.goruncode.com secretName: website-1-kube-cert rules: - host: sample-app.goruncode.com http: paths: - backend: serviceName: doks-example servicePort: 80 path: /
There are four new lines here:
tls: - hosts: - sample-app.goruncode.com secretName: website-1-sample-app-cert
The domain should obviously match the domain you'll be using.
The secretName is just a name that will be used for the certificate. Make it whatever you want, but something that explains what it is is generally helpful.
Now apply the manifest to make the changes:
$ kubectl apply -f manifest.yaml
And inspect the Ingress to see that it is now listening on port 443:
$ kubectl get ingress -A
You'll notice that under ports 80 and 443 are now listed.
This is actually really easy, and can be done in two commands:
kubectl create namespace cert-manager kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.13.0/cert-manager.yaml
The first creates a namespace in which everything will live (nice!).
The second command creates everything required to run cert-manager in your cluster. It might be simple to install but a lot goes on when you run this command.
Check that the cert-manager pods are running:
kubectl get pods --namespace cert-manager
You should see 3 pods running, something along the lines of this:
NAME READY STATUS RESTARTS AGE cert-manager-6f57ae4565-x8bsg 1/1 Running 0 4d3h cert-manager-cainjector-75b9cc7b8b-fkn8p 1/1 Running 0 4d3h cert-manager-webhook-8444c4fa77-nw8lw 1/1 Running 0 4d3h
Create Certificate Issuers
Cert-manager needs to know how to get certificates issued, we'll be using LetsEncrypt, but you could be using some internal CA. We'll create two issuers, one for staging and one for prod. This is useful because LetsEncrypt has strict rate limiting on requests, so if you mess anything up it is better to do so with staging where you won't be blocked.
Note that this is where i diverge a bit from the official documentation. I prefer to create a
ClusterIssuer rather than an
Issuer. The main difference is the
ClusterIssuer doesn't belong to a specific namespace and can be referenced by certificates from multiple namespaces. If you need a specific
Issuer per namespace for some reason, then use an
Issuer. This could be, for example, because you issue certificates using one contact email in one namespace, and another contact email in another. If you will just be using LetsEncrypt, with a single contact email for all certificates, a
ClusterIssuer is probably the best choice.
So, to create your staging issuer, create a file called
staging-issuer.yaml with the following contents:
--- apiVersion: cert-manager.io/v1alpha2 kind: ClusterIssuer metadata: name: letsencrypt-staging spec: acme: # The ACME server URL server: https://acme-staging-v02.api.letsencrypt.org/directory # Email address used for ACME registration email: firstname.lastname@example.org # Name of a secret used to store the ACME account private key privateKeySecretRef: name: letsencrypt-staging # Enable the HTTP-01 challenge provider solvers: - http01: ingress: class: nginx
Be sure to update the email address, then create the
kubectl apply -f staging-issuer.yaml
Create a production issuer in the same way, but use the production LetsEncrypt API:
--- apiVersion: cert-manager.io/v1alpha2 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: # The ACME server URL server: https://acme-v02.api.letsencrypt.org/directory # Email address used for ACME registration email: email@example.com # Name of a secret used to store the ACME account private key privateKeySecretRef: name: letsencrypt-prod # Enable the HTTP-01 challenge provider solvers: - http01: ingress: class: nginx
kubectl apply -f staging-issuer.yaml
Everything should now be in place to generate and start using a TLS certificate!
Test the setup using staging
We're going to add another annotation to our Ingress now, this time it will cause cert-manager to do its thing and automate the TLS cert generation. This includes talking to nginx to do everything it needs to do to convince LetsEncrypt that we deserve a certificate.
The change is a single line:
A quick note, this again varies from the official doco in that we are using a cluster-issuer rather than an issuer. I have been caught out by this before so learn from my mistakes.
The whole ingress will now look like this:
--- apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx cert-manager.io/cluster-issuer: "letsencrypt-staging" name: doks-example namespace: website-1 spec: tls: - hosts: - sample-app.goruncode.com secretName: website-1-kube-cert rules: - host: sample-app.goruncode.com http: paths: - backend: serviceName: doks-example servicePort: 80 path: /
Apply the result:
kubectl apply -f manifest.yaml
A bunch of stuff will start happening in the background. A common issue i've experienced is DNS not being propogated, but it is generally pretty quick.
Have a look at what is going on. Rather than showing you, try running some of the following commands:
kubectl get certificates -n website-1 kubectl get secrets -n website-1 kubectl get certificaterequests -n website-1
kubectl describe ... to look at the details of each of the above, you should be able to see the request get made and certificates get generated.
Eventually, when you run
kubectl get certificates -n website-1 you should see the READY column switch to true. This means you've successfully been assigned a cert by the staging issuer. Note that this certificate doesn't work, but it does mean the setup is correct.
Create a proper certificate
Alright, you've installed everything and the test setup worked! Now lets create the real deal certificate! Update our ingress to change the cluster-issuer to letsencypt-prod. The result should be:
--- apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx cert-manager.io/cluster-issuer: "letsencrypt-prod" name: doks-example namespace: website-1 spec: tls: - hosts: - sample-app.goruncode.com secretName: website-1-kube-cert rules: - host: sample-app.goruncode.com http: paths: - backend: serviceName: doks-example servicePort: 80 path: /
I've run into issues directly applying the new config, so I recommend deleting the existing ingress, then re-applying the manifest. But I do tend to be risk averse. So feel free to skip the delete command...
kubectl delete ingress -n website-1 doks-example kubectl apply -f manifest.yaml
Once again watch as your certificate gets requested and generated, then eventually, if all goes well, browse to https://your.domain.com and enjoy a fully encrypted browsing experience!
Things will probably go wrong
As versions change and API's change with them, things constantly stop working. Try to figure it out, or write a comment and maybe I or someone else will be able to help. (I'm currently updating the site so comments might not work any more sorry.)