Demo: Securing Communication Between Clients and Brokers Using SSL

The demo shows how to configure secure communication between Kafka clients (consumers and producers) and brokers using Transport Layer Security (TLS) (and its now-deprecated predecessor, Secure Sockets Layer (SSL)).

TLS is a newer variant of SSL, but because of the term SSL is used in many Kafka configuration properties the document will use SSL (over TLS) to refer to the protocol used for secure communication.

The demo is made up of the following steps:

Create Directory of SSL Files

Create a directory for SSL files.

mkdir -p /tmp/kafka-ssl-demo

All the following commands are executed in the directory.

cd /tmp/kafka-ssl-demo

Create Certificate Authority (CA)

In cryptography, a certificate authority or certification authority (CA) is an entity that issues digital certificates.

A digital certificate certifies the ownership of a public key by the named subject of the certificate.

A CA acts as a trusted third party — trusted both by the subject (owner) of the certificate and by the party relying upon the certificate. The format of these certificates is specified by the X.509 standard.

— Wikipedia

Simply put, a certificate authority is used to sign certificates (with public keys).

Generate a private key and a self-signed certificate for the CA.

// openssl genrsa -out ca.key
// openssl req -new -x509 -key ca.key -out ca.crt
$ openssl req \
  -new \
  -x509 \
  -days 365 \
  -keyout ca.key \
  -out ca.crt \
  -subj "/C=PL/L=Warsaw/CN=Certificate Authority" \
  -passout pass:1234
Generating a RSA private key
writing new private key to 'ca.key'

You should have the following files in the directory:

  • ca.key - the private key of the certificate authority

  • ca.crt - the certificate (with the public key) of the certificate authority

Use the following command to print certificate contents:

$ openssl x509 -text -noout -in ca.crt
        Version: 3 (0x2)
        Serial Number:
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = PL, L = Warsaw, CN = Certificate Authority
        Subject: C = PL, L = Warsaw, CN = Certificate Authority
... removed for clarity

Since ca.crt is a self-signed certificate, the Issuer and Subject sections use the same subject.

Generate SSL Keys and Certificate for Kafka Brokers

Generate the keys and certificate for every Kafka broker in a cluster. This time you’ll be using keytool utility that comes with Java Platform, Standard Edition (refer to keytool).

// openssl genrsa -out server.key
$ keytool \
  -genkey \
  -keystore server.keystore \
  -alias localhost \
  -dname CN=localhost \
  -keyalg RSA \
  -validity 365 \
  -ext san=dns:localhost \
  -storepass 123456

If you want to use a Subject Alternative Name (SAN) in your certificate, add the following options to the keytool command line:

  • Fully-qualified domain name (FQDN): –ext

  • IP address: –ext san=ip:

DistinguishedName or SubjectAltNames should be the fully-qualified domain name of a broker.

You should now have one more file in the directory:

  • server.keystore - the keystore with the private key and the certificate of the broker

Use keytool to print out the content of the keystore.

keytool -list -v -keystore server.keystore -storepass 123456

The keystore should contain 1 entry for the alias localhost.

Sign Broker Certificate (Using CA)

Create a certificate signing request (CSR).

Export the server certificate from server.keystore.

// openssl req -new -key server.key -out server.csr
$ keytool \
  -certreq \
  -keystore server.keystore \
  -alias localhost \
  -file server.unsigned.crt \
  -storepass 123456

Sign the certificate signing request (server.unsigned.crt) with the root CA.

$ openssl x509 \
  -req \
  -CA ca.crt \
  -CAkey ca.key \
  -in server.unsigned.crt \
  -out server.crt \
  -days 365 \
  -CAcreateserial \
  -passin pass:1234
Signature ok
subject=CN = localhost
Getting CA Private Key

You should have the following files in the directory:

  • server.unsigned.crt


  • server.crt - the signed certificate of the broker

Import Certificates to Broker Keystore

Create a SSL keystore for the Kafka broker. Each broker gets its own unique keystore.

Import the certificate of the CA into the broker keystore.

$ keytool \
  -import \
  -file ca.crt \
  -keystore server.keystore \
  -alias ca \
  -storepass 123456 \
Certificate was added to keystore

Import the signed certificate into the broker keystore. Make sure to use the same -alias as you used ealier.

$ keytool \
  -import \
  -file server.crt \
  -keystore server.keystore \
  -alias localhost \
  -storepass 123456 \
Certificate reply was installed in keystore

Use keytool to print out the certificates in the broker keystore.

keytool -list -v -keystore server.keystore -storepass 123456

There should be 2 entries (one for the CA and another for the broker itself).

Configure SSL on Kafka Broker

Create config/ (based on config/ and add the following configuration properties to enable SSL:


Start the broker(s).

./bin/ config/
Use export to debug SSL issues.

Verify the SSL configuration of the broker. The following uses the Cryptography and SSL/TLS Toolkit (OpenSSL) and the client tool.

openssl s_client -connect localhost:9093

The tool should print out the certificate chain of the broker (a chain of the subjects and the issuers). At the end, you should find the following Verify return code:

Verify return code: 19 (self signed certificate in certificate chain)

Enter Ctrl-C to close the session.

Use the client tool with -CAfile option to trust the CA certificate.

openssl s_client -connect localhost:9093 -CAfile /tmp/kafka-ssl-demo/ca.crt

With the change, you should find the following Verify return code:

Verify return code: 0 (ok)

Enter Ctrl-C to close the session.

Import CA Certificate to Client Truststore

Add the CA certificate ca.crt to a client truststore for the clients to trust this CA.

$ keytool \
  -import \
  -file ca.crt \
  -keystore client.truststore \
  -alias ca \
  -storepass 123456 \
Certificate was added to keystore

Use keytool to print out the certificates in the client keystore.

keytool -list -v -keystore client.truststore -storepass 123456

There should be 1 entry for the CA.

Configure SSL for Kafka Clients

Use the following as a minimal configuration of a Kafka client to use SSL:


Use utility to send records to Kafka brokers over SSL: \
  --broker-list :9093 \
  --topic ssl \
  --producer.config /tmp/kafka-ssl-demo/

That’s all for the demo.

results matching ""

    No results matching ""