Open AccessDevelopment of Kafka Messaging System and its Performance Test Framework using Prometheus
Author(s) -
T L X X V,
Kiran V
Publication year - 2020
Publication title -
international journal of recent technology and engineering
Language(s) - English
Resource type - Journals
ISSN - 2277-3878
DOI - 10.35940/ijrte.a2516.059120
Subject(s) - computer science , the internet , container (type theory) , volume (thermodynamics) , process (computing) , throughput , middleware (distributed applications) , world wide web , database , operating system , engineering , wireless , mechanical engineering , physics , quantum mechanics
In today’s era of internet, communication plays a vital role in every aspect of daily activity, as the number of internet users increases an enormous volume of data is generated. This motivates to develop and use a messaging system that collects and analyzes large volume of generated data. Kafka is a distributed messaging system based on publish-subscribe model with high throughput and fault-tolerant mechanism. Around 40% of fortune 500 companies use Kafka messaging system for processing data. Kafka is used to stream real-time data and it is also used to process the data received from IOT sensors. This paper discusses about the procedure for developing and deploying the Kafka messaging system into the working environment based on container technology. The secure communication is enabled by configuring the Kafka with two security mechanisms. The performance of the developed system is evaluated considering QoS parameters for different test scenarios using Prometheus tool.