How to Encrypt Kubernetes Traffic with cert-manager, Let's Encrypt, and Internal TLS

How to Encrypt Kubernetes Traffic with cert-manager, Let's Encrypt, and Internal TLS

Destiny Erhabor

Most engineers assume their Kubernetes cluster encrypts all of its traffic. It doesn't. The commands you run with kubectl are encrypted — your client and the API server speak TLS. The API server talking to etcd is usually encrypted too, depending on how the cluster was provisioned. But traffic between your pods? Plaintext by default. Ingress traffic from the internet to your services? Only encrypted if you explicitly configure TLS. And certificates for internal services? You have to provision those yourself. This is not a Kubernetes oversight. It's a deliberate design choice — Kubernetes provides the primitives and leaves the implementation to you. The problem is that certificate management is notoriously painful. Certificates expire. Provisioning them manually doesn't scale. Forgetting to rotate them causes outages. cert-manager solves this. It runs as a controller inside your cluster, watches for Certificate resources, requests certificates from configured issuers, stores them in Kubernetes Secrets, and rotates them automatically before they expire. You declare what you want, cert-manager makes it happen and keeps it that way. In this article you'll work through how cert-manager's core model works, automate public Ingress TLS using Let's Encrypt, set up an internal Certificate Authority for service-to-service encryption, and understand how certificate rotation works so outages caused by expired certificates become a thing of the past. Prerequisites

A kind cluster with the nginx Ingress controller installed

Helm 3 installed

A domain name with DNS you control — needed for the Let's Encrypt demo

Basic understanding of TLS: you know what a certificate, a private key, and a CA are

All demo files are in the DevOps-Cloud-Projects GitHub repository. Table of Contents

What Is and Isn't Encrypted in Kubernetes

How cert-manager Works

The Four Core Resources

Issuers and ClusterIssuers

The Certificate Lifecycle

ACME Challenges: HTTP-01 vs DNS-01

Demo 1 — Install cert-manager and Issue a Let's Encrypt Certificate

How to Get a Wildcard Certificate with DNS-01

Demo 2 — Set Up an Internal CA for Service-to-Service TLS

How Certificate Rotation Works

Cleanup

Conclusion

What Is and Isn't Encrypted in Kubernetes? Before installing anything, it's worth being precise about what the cluster already protects and what it leaves open.

Traffic path Encrypted by default? Notes

kubectl → API server Yes TLS with the cluster CA

API server → etcd Usually Depends on cluster provisioner — verify with your setup

API server → kubelet Yes TLS, but kubelet cert verification depends on configuration

Pod → Pod (same cluster) No Plaintext unless you add a service mesh or mTLS

Internet → Ingress No Opt-in — requires TLS configuration on the Ingress resource

Pod → Kubernetes API Yes Via the service account token and cluster CA

The two gaps that matter most in practice are pod-to-pod traffic and Ingress TLS. This article covers both Ingress TLS with Let's Encrypt and internal service-to-service encryption using a private CA. How cert-manager Works cert-manager is a Kubernetes operator. It extends the Kubernetes API with custom resources that represent certificate requests and their configuration. When you create a Certificate resource, cert-manager's controller picks it up, requests a certificate from the configured issuer, and stores the resulting certificate and private key in a Kubernetes Secret. When the certificate approaches its expiry, cert-manager renews it automatically. This model means your application doesn't know or care about certificate management. It reads a Secret. cert-manager keeps that Secret fresh. The Four Core Resources cert-manager introduces four custom resources that you'll use regularly:

Resource What it represents

Issuer A certificate authority or ACME account — namespace-scoped

ClusterIssuer Same as Issuer, but available cluster-wide

Certificate A request for a certificate — describes what you want

CertificateRequest An individual signing request — created automatically by cert-manager, rarely touched directly

In practice you'll mostly deal with ClusterIssuer and Certificate. The ClusterIssuer defines where certificates come from. The Certificate defines what certificate you want and where to store it. Issuers and ClusterIssuers An Issuer can only issue certificates within its own namespace. A ClusterIssuer can issue certificates in any namespace. For shared infrastructure like Let's Encrypt, you almost always want a ClusterIssuer. For application-specific internal CAs, an Issuer scoped to that application's namespace is the safer choice. cert-manager supports several issuer types. The three you'll encounter most often are: ACME — for public certificates from Let's Encrypt or any ACME-compatible CA. Ownership of the domain is proven via an HTTP-01 or DNS-01 challenge. CA — for internal certificates signed by a CA whose private key is stored in a Kubernetes Secret. Used for service-to-service TLS within the cluster. Self-signed — generates self-signed certificates. Rarely useful on its own, but essential as the bootstrap step when creating an internal CA. The Certificate Lifecycle When you create a Certificate resource, cert-manager follows this sequence:

Creates a CertificateRequest with a CSR (Certificate Signing Request)

Passes the CSR to the configured issuer

For ACME issuers: creates a Challenge resource and fulfils it (more on this below)

Receives the signed certificate from the issuer

Stores the certificate and private key in the Kubernetes Secret named in spec.secretName

Monitors the certificate's expiry — by default, renews when 2/3 of the validity period has elapsed

Your application mounts the Secret. cert-manager updates it silently. Most applications that watch for file changes will pick up the new certificate without a restart. ACME Challenges: HTTP-01 vs DNS-01 Let's Encrypt needs proof that you control the domain before it issues a certificate. ACME defines two challenge types for this. HTTP-01 works by having cert-manager create a temporary HTTP endpoint at http:///.well-known/acme-challenge/. Let's Encrypt sends a request to that URL. If the response matches the expected token, the challenge passes. This requires your cluster to be reachable from the internet on port 80. DNS-01 works by having cert-manager create a temporary DNS TXT record at _acme-challenge.. Let's Encrypt checks for that record. This doesn't require inbound HTTP access, which makes it the right choice for private clusters, and it's the only way to get wildcard certificates (*.example.com). The trade-off: HTTP-01 is simpler to set up but only works for single domains and requires internet-accessible infrastructure. DNS-01 requires API access to your DNS provider but works for internal clusters and wildcards. Demo 1 — Install cert-manager and Issue a Certificate Using Pebble and Let's Encrypt Pebble is Let's Encrypt's local ACME test server. It runs inside your cluster, issues certificates using the same ACME protocol as Let's Encrypt, and requires no public domain or internet access. Using Pebble lets you test the full cert-manager flow — challenge, issuance, renewal — on a plain kind cluster. Once you understand the flow locally, switching to real Let's Encrypt is a one-line change: replace the ClusterIssuer server URL and point a DNS record at a publicly reachable cluster. The rest of the configuration is identical. You'll install cert-manager, create a ClusterIssuer for Let's Encrypt, deploy a sample application with an Ingress, and watch a real certificate be issued and stored automatically. Step 1: Install cert-manager cert-manager is now distributed via OCI Helm charts from quay.io/jetstack. The --set crds.enabled=true flag installs the Custom Resource Definitions as part of the chart: helm upgrade cert-manager oci://quay.io/jetstack/charts/cert-manager \ --install \ --create-namespace \ --namespace cert-manager \ --set crds.enabled=true \ --version v1.17.0 \ --wait

You also need the nginx Ingress controller — cert-manager routes HTTP-01 challenges through it. The controller.service.type=ClusterIP override is for kind specifically: the default LoadBalancer Service never gets an EXTERNAL-IP on kind (there's no cloud LB), which makes --wait hang forever. On a real cluster, drop the override and keep LoadBalancer. helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \ --namespace ingress-nginx \ --create-namespace \ --set controller.service.type=ClusterIP \ --wait

Confirm all four components are running: kubectl get pods -n cert-manager kubectl get pods -n ingress-nginx

NAME READY STATUS RESTARTS AGE cert-manager-76f84784c8-r4fx4 1/1 Running 0 6m45s cert-manager-cainjector-66fbf49587-gv25n 1/1 Running 0 6m45s cert-manager-webhook-577fddf86-l5wj4 1/1 Running 0 6m45s

NAME READY STATUS RESTARTS AGE ingress-nginx-controller-6c7cd85885-h7zgx 1/1 Running 0 3m34s

kind-specific gotcha — remove the nginx admission webhook now.** On kind, the nginx admission webhook serves with a self-signed certificate that the Kubernetes API server cannot verify. The first time you try to create any Ingress resource you'll see failed calling webhook "validate.nginx.ingress.kubernetes.io": ... x509: certificate signed by unknown authority. Delete the