Mutual TLS made easy with OpenShift Service Mesh, Part 2

Voravit L
4 min readFeb 22, 2021

In Part 2, We will extend our secure We will enable Mutual TLS to secure your application from clients outside your cluster.

Introduction

For the first part of this 2 part series, we discussed OpenShift Service Mesh setup and configure mutual TLS for service to service communication Mutual TLS (mTLS) made easy with OpenShift Service Mesh, Part 1

What is Mutual TLS?

Normally when we access a website with HTTPS. What happened in the background is the browser will verify that the server certificate is valid or not. (Signed by the authority that the browser knows).

Mutual authentication is extending this process by server will also verify the client’s certificate. In this case, client will send a certificate request with a list of Distinguished Name of root certificate authority (CA) then server will validate that this CA is included in their trusts CAs or not. If the verification is successful then the client is authenticated.

Prerequisites

Check my previous blog for setup Service Mesh with demo application. Mutual TLS (mTLS) made easy with OpenShift Service Mesh, Part 1

CA, Certificate and Private Key

For testing purposes, we will create our own CA and create a self-signed certificate for our functional company called Example Inc.

This certificate will be used for Istio Gateway for its TLS configuration.

create CA, CSR and self-signed certificate

Secure Istio Gateway with TLS

Create secret in control plane namespace to store certificate and private key for frontend application.

secrets for TLS

Enable TLS by updating Istio’s Gateway and use a secret that stored the frontend’s application private key and certificate.

SUBDOMAIN=$(oc whoami --show-console  | awk -F'apps.' '{print $2}')DOMAIN="apps.${SUBDOMAIN}"curl -s  https://raw.githubusercontent.com/voraviz/openshift-service-mesh-ingress-mtls/main/config/gateway-tls.yaml| sed 's/DOMAIN/'"$DOMAIN"'/' | oc apply -f -

Take a quick review at Gateway with TLS enabled.

Gateway with TLS enabled

Line 11–13 are configure for HTTPS with port 443

Line 15 is configure Gateway with TLS mode SIMPLE that is use TLS only not with mutual TLS

Line 16 is secret name

Test updated Gateway with TLS enabled.

SUBDOMAIN=$(oc whoami --show-console  | awk -F'apps.' '{print $2}')DOMAIN="apps.${SUBDOMAIN}"curl -kv frontend.$DOMAIN

Check verbose output from cURL for certificate information. Issuer is our fictional company Example Inc.

* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384* ALPN, server accepted to use h2* Server certificate:*  subject: CN=frontend.apps.cluster-9a88.9a88.example.opentlc.com; O=Great Department*  start date: Feb 21 12:31:42 2021 GMT*  expire date: Feb 21 12:31:42 2022 GMT*  issuer: O=example Inc.; CN=example.com

Secure Istio Gateway with mTLS

Let’s say that we want to enable mutual TLS to authenticate our business partner Acme Inc. We will create CA, private key and Certificate on behalf of Acme Inc. In the real world, you must not do this because Acme Inc will do this.

With mutual TLS we need to have a list of trusted CA. Then, we will update our secret that we created for the frontend gateway with Acme Inc’s certificate.

secrets with trusted CA

Enable Istio Gateway mutual TLS by change mTLS mode from SIMPLE to MUTUAL.

SUBDOMAIN=$(oc whoami --show-console  | awk -F'apps.' '{print $2}')DOMAIN="apps.${SUBDOMAIN}"curl -s  https://raw.githubusercontent.com/voraviz/openshift-service-mesh-ingress-mtls/main/config/gateway-mtls.yaml| sed 's/DOMAIN/'"$DOMAIN"'/' | oc apply -f -

Check Istio Gateway configuration.

Gateway with mTLS

Line 15, TLS mode is configured to MUTUAL

Use cURL to connect to the frontend application again. You will get handshake failure error

curl -k https://frontend.$DOMAIN# Sample outputcurl: (35) error:1401E410:SSL routines:CONNECT_CR_FINISHED:sslv3 alert handshake failure

Assume that you’re working for Acme Inc then you have their private key and CA. This time you can connect to the frontend application because we use a certificate that Gateway already trusted.

curl -k --cacert certs/acme.com.crt \--cert certs/great-partner.crt \--key certs/great-partner.key \https://frontend.$DOMAIN

Next another company named Pirate Inc and we’re not dealing with them trying to use our frontend application. Let’s create CA, private key and certificate for them first.

Use Pirate Inc CA and private key to connect to our frontend application. You will get unknown ca error

curl -k --cacert certs/pirate.com.crt \--cert certs/bad-partner.crt \--key certs/bad-partner.key \https://frontend.$DOMAIN#Sample outputcurl: (35) error:1401E418:SSL routines:CONNECT_CR_FINISHED:tlsv1 alert unknown ca

Next, our business department agreed to deal with Pirate Inc. We need to configure our Gateway to trust Pirate Inc CA by updating secret with Pirate Inc’s CA

First, create chained trusted certificates that contain both Pirate Inc and Acme Inc certificates.

cat certs/acme.com.crt > certs/trusted.crtcat certs/pirate.com.crt >> certs/trusted.crt

Update secret with chained certificates.

secrets with trusted chained CA

Test cURL with Pirate CA and private key again. This time you will not get unknown ca error anymore.

Notice that there is no need to restart any Gateway component when you change Gateway configuration. Just updating Gateway CRD or updating secrets.

Summary

With OpenShift Service Mesh (Istio), You can enable TLS and also mutual TLS with Istio Gateway. This can be done without any modification of your application and you can also update TLS configuration easily by just updating Gateway configuration and/or secret.

You can find YAML files and also shell script for automated setup all demo here.

Or you can learn Istio from this Online tutorial provided by Red Hat

Have fun with Service Mesh :)

--

--