If you install an OCP cluster certificates for API and the wildcard domain will be signed by a self-signed CA. This is not a security but a convenience issue as you need to accept or ignore warnings regarding self-signed certificates.

Replacing the certificates by ones signed by a known certificate authority is well documented and works well. If you want to use Let’s Encrypt as the CA of your choice, you can request certificates there and provide the authorization information on your own, but since the availability of cert-manager Operator for Red Hat OpenShift tools for managing certificates automatically available.

Preparing for dns-01 challenge

If you have a working acme-dns, it can be used to issue both host and wildcard certificates with cert-manager and Let’s Encrypt.

  1. Register an account with your acme-dns service and save the account data to a file
  $ curl -XPOST https://auth.example.com/register | jq > acmedns.json
  $ curl -s -XPOST https://auth.example.com/update \
  -H "X-Api-User: <username>" \
  -H "X-Api-Key: <password>" \
  --data '{"subdomain": "<subdomain>", "txt": "___validation_token_received_from_the_ca___"}' | jq
  {
    "txt": "___validation_token_received_from_the_ca___"
  }

Replace the data between square brackets with the data from your registration.

  1. Configure your DNS zone to forward dns-01 challenge requests to your acme-dns service
  $ dig +noall +answer -t CNAME _acme-challenge.apps.ocp4.example.com @9.9.9.9
  _acme-challenge.apps.ocp4.example.com. 50 IN CNAME <subdomain>.auth.example.com.

Test whether a TXT record will be returned.

  $ dig +noall +answer -t TXT _acme-challenge.apps.ocp4.example.com @9.9.9.9
  _acme-challenge.apps.ocp4.example.com. 50 IN CNAME <subdomain>.auth.example.com.
  <subdomain>.auth.example.com. 1 IN TXT "___validation_token_received_from_the_ca___"

With these steps, the DNS setup is prepared for handling the dns-01 challenges.

cert-manager Operator

As the DNS side of the setup has finished, it is now time to install the cert-manager Operator for Red Hat OpenShift.

  1. Open the OpenShift web console. You need to log in as a cluster admin.
  2. Navigate to the OperatorHub. Operator → OperatorHub
  3. Search for cert-manager.
  4. Select the cert-manager Operator for Red Hat OpenShift and click Install. In the upcoming wizard, leave all values on its defaults and click Install.

Configure cert-manager

After installing the cert-manager Operator successfully, it is time to prepare it for issuing Let’s Encrypt certificates. To do so, we need

  1. Save the credentials json snippet from the registration process to a file with a key for all your domain that should be handled by the ClusterIssuer.

    As we want later replace the API and wildcard certificate with newly cert-manager created ones, the file should look like this.

     {
   "api.ocp4.example.com": {
   "username": "<username>",
   "password": "<password>",
   "fulldomain": "<subdomain>.acme-dns.adsfg.xyz",
   "subdomain": "<subdomain>",
   "allowfrom": []
   },
   "apps.ocp4.example.com": {
   "username": "<username>",
   "password": "<password>",
   "fulldomain": "<subdomain>.acme-dns.adsfg.xyz",
   "subdomain": "<subdomain>",
   "allowfrom": []
   }
 }

    Save this data in a file (e.g. acmedns.json) and create a secret in the cert-manager project

    If you want to use Issuer in favor of a ClusterIssuer the secret needs to be created in the same project as the Issue will be created.

    $ oc -n cert-manager create secret generic acme-dns-staging --from-file acmedns.json=acmedns.json
 secret/acme-dns-staging created

    This secret and the key (acmedns.json) needs to be placed in the ClusterIssuer manifest.

  2. Create a ClusterIssuer to handle the certificate issuing process

     $ cat <<EOF | oc create -f
 apiVersion: cert-manager.io/v1
 kind: ClusterIssuer
 metadata:
   name: letsencrypt-staging
 spec:
   acme:
     email: hostmaster@cexample.com
     preferredChain: ""
     privateKeySecretRef:
       name: letsencrypt-staging-private-key
     server: https://acme-staging-v02.api.letsencrypt.org/directory
     solvers:
     - dns01:
         acmeDNS:
           accountSecretRef:
             key: acmedns.json
             name: acme-dns-staging
           host: https://auth.example.com
 EOF

    Note: For this article, we use the staging instance of Let’s Encrypt. You should do it also this way to check if the setup is working properly. Only after issuing a certificate successfully, you should switch to the production ACME endpoint. Otherwise, you risk running into rate limiting and being blocked from the API if anything does not work as expected.

OCP certificates

To issue a certificate, it is needed to create a Certificate CR in the project where you need it. If you create a certificate, some corresponding custom resources will be created.

  • CertificateRequest, is used to request a signed certificate
  • Order, represents an order with an ACME server
  • Challenge, represents a challenge request with an ACME server

cert-manager flow

default ingress certificate

Unfortunately, it is not that easy to replace the ingress certificate as with upstream cert-manager and Kubernetes Ingress resources. In that case, only annotating the resource is needed to let the magic happen. See the original documentation for details.

Replacing the default ingress certificate for the *.apps subdomain is a common day-2 task. Combining it with the use of cert-manager, it is really comfortable to have this automated for future updates of the related certificate.

  1. Starting with creating a Certificate CR which uses the former configured ClusterIssuer to interact with the Let’s Encrypt API.

     $ cat <<EOF | oc create -f
 apiVersion: cert-manager.io/v1
 kind: Certificate
 metadata:
 name: letsencrypt-staging-wildcard
 namespace: openshift-ingress
 spec:
 secretName: letsencrypt-staging-wildcard
 secretTemplate:
     labels:
     stage: staging
 duration: 2160h # 90d
 renewBefore: 360h # 15d
 isCA: false
 privateKey:
     algorithm: RSA
     encoding: PKCS1
     size: 2048
 usages:
     - server auth
     - client auth
 dnsNames:
     - apps.ocp4.example.com
     - '*.apps.ocp4.example.com'
 issuerRef:
     name: letsencrypt-staging
     kind: ClusterIssuer
     group: cert-manager.io
 EOF

  2. After applying the manifest, the former discussed resources would be created, and you can monitor the status of the certificate by watching on it. As soon as the dns-01 challenge was solved the certificate will reach the ready state, and it can be used.

     $ oc -n openshift-ingress get certificates

 NAME                                                          READY   SECRET                            AGE
 certificate.cert-manager.io/letsencrypt-staging-wildcard      False   letsencrypt-staging-wildcard      9s

  3. Patching the ingress controller after the certificate was issued is done by the following command.

     $ oc -n openshift-ingress-operator patch --type=merge ingresscontrollers/default --patch '{"spec":{"defaultCertificate":{"name":"letsencrypt-staging-wildcard"}}}'
 ingresscontroller.operator.openshift.io/default patched

    After patching the resource, the router pods will be restarted and the certificate will be used.

API server certificate

To replace the default API server certificate, you need to run similar steps as with the default ingress certificate.

  1. A Certificate resource need to be created as followed.

     $ cat <<EOF | oc create -f
 apiVersion: cert-manager.io/v1
 kind: Certificate
 metadata:
   name: letsencrypt-staging-api
   namespace: openshift-config
 spec:
   dnsNames:
   - api.ocp4.example.com
   duration: 2160h0m0s
   issuerRef:
     group: cert-manager.io
     kind: ClusterIssuer
     name: letsencrypt-staging
   privateKey:
     algorithm: RSA
     encoding: PKCS1
     size: 2048
   renewBefore: 360h0m0s
   secretName: letsencrypt-staging-api
   secretTemplate:
     labels:
       stage: staging
   usages:
   - server auth
   - client auth
 EOF

  2. The status of the requested certificate can be monitored by watching the former created Certificate resource.

     $ oc -n openshift-config get certificates
 NAME                         READY   SECRET                       AGE
 letsencrypt-staging-api   True    letsencrypt-staging-api   122s

  3. To replace the API server certificate with the newly created one, the following command is enough. It will patch the Apiserver resource and a restart of the apiserver pods will be initiated.

     $ oc patch apiserver/cluster --type=merge -p '{"spec":{"servingCerts": {"namedCertificates": [{"names": ["api.ocp4.example.com"], "servingCertificate": {"name": "letsencrypt-staging-api"}}]}}}'
 apiserver.config.openshift.io/cluster patched