Apache Kafka is a fault tolerant publish-subscribe streaming platform that lets you process streams of records as they occur. This post is a step by step guide of how to build a simple Apache Kafka Docker image. The original Dockerfile can be found here: https://github.com/nsonntag/docker-images/tree/master/kafka-quickstart.

The Dockerfile

This Dockerfile is very simple. It installs Java and wget, downloads and install Kafka, and starts the Zookeeper and Kafka servers (as part of another script).

FROM ubuntu:16.04

ENV KAFKA_HOME /usr/local/kafka
ADD ./start-kafka.sh /scripts/

# install java + others
RUN apt-get update && apt-get install -y \
  wget \
  openjdk-8-jdk

# install kafka
RUN wget http://apache.cs.utah.edu/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz && \
  tar -xzf kafka_2.11-0.10.2.0.tgz && \
  mv kafka_2.11-0.10.2.0 $KAFKA_HOME

CMD ["/scripts/start-kafka.sh"]

Starting Zookeeper and Kafka

In the Dockerfile above you can see CMD ["/scripts/start-kafka.sh"]. This command is used to start both Zookeeper and Kafka servers. Here are the contents of start-kafka.sh:

#!/bin/bash

# start zookeeper
$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties &

# start kafka broker
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties

Building the Kafka Image

Once we have our Dockerfile and start-kafka.sh files ready, we can build this docker image. Here is an example of building this image (in the same directory as our Dockerfile) as kafka-quickstart version 0.10.2.0.

docker build -t kafka-quickstart:0.10.2.0 .

Starting a Kafka Docker Container

Once the image is built we can create and run a new Kafka Docker container. The command below will create a container from our image and name it kafka-quickstart:

docker run --name kafka-quickstart -d kafka-quickstart:0.10.2.0