Setting up Apache Kafka in Windows

Apache Kafka

It is a high throughput messaging system

1. Download Apache Kafka from http://kafka.apache.org/downloads.html
2. Extract the zip contents to a local folder
3. Go to the Kafka installation directory/bin/windows 
4. Run Zookeeper by the following command
C:\kafka_2.11-0.8.2.1\bin\windows> zookeeper-server-start.bat ..\..\config\zookeeper.properties
5. Now run Kafka by using command prompt
C:\kafka_2.11-0.8.2.1\bin\windows> kafka-server-start.bat ..\..\config\server.properties

Creating a Kafka Topic

1. Try to create a topic named  “test” 
2. The replication factor 1 since there is only one Kafka server running. If you have a cluster with more than 1 Kafka servers running, you can increase the replication-factor accordingly which will increase the data availability
3. Open a new command prompt in the location C:\kafka_2.11-0.8.2.1\bin\windows
4. Type the command 

kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Communication between Producer and Consumer

1. Open a new command prompt in the location C:\kafka_2.11-0.8.2.1\bin\windows
2. To start a producer type command “kafka-console-producer.bat --broker-list localhost:9092 --topic test”.
3. Again open a new command prompt in the same location as C:\kafka_2.11-0.8.2.1\bin\windows
4. Now start a consumer by typing command “kafka-console-consumer.bat --zookeeper localhost:2181 --topic test”.

Other Commonly used Commands

1. List Topics: kafka-topics.bat --list --zookeeper localhost:2181
2. Describe Topic: kafka-topics.bat --describe --zookeeper localhost:2181 --topic [Topic Name]
3. Read messages from beginning: kafka-console-consumer.bat --zookeeper localhost:2181 --topic [Topic Name] --from-beginning
4. Delete Topic: kafka-run-class.bat kafka.admin.TopicCommand --delete --topic [topic_to_delete] --zookeeper localhost:2181