Apache Kafka is a fault tolerant publish-subscribe streaming platform that lets you process streams of records as they occur. This post is a step by step guide of how to build a simple Apache Kafka Docker image. The original Dockerfile can be found here: https://github.com/nsonntag/docker-images/tree/master/kafka-quickstart.
The Dockerfile
This Dockerfile is very simple. It installs Java and wget, downloads and install Kafka, and starts the Zookeeper and Kafka servers (as part of another script).
FROM ubuntu:16.04
ENV KAFKA_HOME /usr/local/kafka
ADD ./start-kafka.sh /scripts/
# install java + others
RUN apt-get update && apt-get install -y \
wget \
openjdk-8-jdk
# install kafka
RUN wget http://apache.cs.utah.edu/kafka/0.10.2.0/kafka_2.11-0.10.2.0.tgz && \
tar -xzf kafka_2.11-0.10.2.0.tgz && \
mv kafka_2.11-0.10.2.0 $KAFKA_HOME
CMD ["/scripts/start-kafka.sh"]
Starting Zookeeper and Kafka
In the Dockerfile above you can see CMD ["/scripts/start-kafka.sh"]
. This command is used to start both Zookeeper and Kafka servers. Here are the contents of start-kafka.sh
:
#!/bin/bash
# start zookeeper
$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties &
# start kafka broker
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties
Building the Kafka Image
Once we have our Dockerfile
and start-kafka.sh
files ready, we can build this docker image. Here is an example of building this image (in the same directory as our Dockerfile) as kafka-quickstart
version 0.10.2.0
.
docker build -t kafka-quickstart:0.10.2.0 .
Starting a Kafka Docker Container
Once the image is built we can create and run a new Kafka Docker container. The command below will create a container from our image and name it kafka-quickstart
:
docker run --name kafka-quickstart -d kafka-quickstart:0.10.2.0