Trending September 2023 # 6 Best Steps Of Integrating Kafka With Nodejs # Suggested October 2023 # Top 16 Popular

You are reading the article 6 Best Steps Of Integrating Kafka With Nodejs updated in September 2023 on the website Cersearch.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested October 2023 6 Best Steps Of Integrating Kafka With Nodejs

Introduction to Kafka Node

Throughout the years, Kafka has evolved tremendously in many ways. Kafka Node is nothing but a chúng tôi client for Apache Kafka versions of 0.9 and later. We will learn more about the Kafka Node and its examples further.

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

Let us learn more about Kafka Node:

Just to revise, Kafka is a distributed, partitioned and replicated pub-sub messaging system. Kafka stores messages in topics (partitioned and replicated across multiple brokers). Producers send messages to topics from which consumers or their consumer groups read. It is important, to begin with, the understanding of microservices before we address how Kafka-node operates. We are mainly in the Microservices Era of growth. Many businesses favour the use of Microservices to develop their applications.

This is one of the most common cases in which a Microservice through Kafka talks with another Microservice.

Integrating Kafka with NodeJS

Let’s build a NodeJS API that is going to serve as a Kafka producer. Then we will create another consumer in NodeJS who will consume the topic that we are going to create first. Let’s follow the steps needed to consume messages from the topic using the command line.

1. Starting Zookeeper

It doesn’t begin without a Zookeeper; Kafka needs a zookeeper. Start Zookeeper by using the command zkServer.sh.

Command:

$bin/zkServer.sh start

Output:

2. Start Kafka Server

Use the below command to start the Kafka server:

Command:

$ bin/kafka-server-start.sh -daemon $ config/server.properties

3. Topic Creation

Let’s create a topic called “example” with a single partition and a single replica:

Command:

$bin/kafka-topics.sh --create --zookeeper localhost:2181 -- replication-factor 1 --partitions 1 --topic example

The replication-factor determines how many copies of the data will be produced. As we’re working with a single instance, keep this value at 1. Set the partitions as the number of brokers you want your data to be divided between. As we’re running with a single broker, keep this value at 1.

4. Integrating Kafka with NodeJS

The next step is creating individual producers and consumers using chúng tôi But before that, let’s connect our Kafka with NodeJS.

npm install kafka-node –save

module.exports = {'example',kafka_server: 'localhost:2181',};

5. Creating Producer

We need a producer to instantiate to publish messages with Kafka-node. A message on the given topic is published using the following function.

Code:

}

6. Creating Consumer

We will instantiate a customer to listen to messages on a specific topic. The following functionality consumes from a topic.

Code:

}

Examples of Kafka Node

Let’s follow the below steps for creating a simple producer and consumer application in Node.js.

1. Create a producer using the code below in the node shell while the ‘mytopics123’ topic is already created.

2. Create a consumer using the code below in the node shell in the chúng tôi file.

consumer.js

All the available messages will be shown on the node terminal. Following is the output when we run the above consumer.js.

Running consumer.js:

If you don’t create chúng tôi and consume using chúng tôi then you will get the following output:

Let’s run the chúng tôi and consume all messages which the previous producer sent.

Running Kafka-console-consumer: