{"id":70467,"date":"2017-11-13T22:00:31","date_gmt":"2017-11-13T20:00:31","guid":{"rendered":"https:\/\/www.javacodegeeks.com\/?p=70467"},"modified":"2017-11-13T10:45:04","modified_gmt":"2017-11-13T08:45:04","slug":"introduction-apache-kafka","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html","title":{"rendered":"Introduction to Apache Kafka"},"content":{"rendered":"<h2>What is Apache Kafka?<\/h2>\n<p>Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an enterprise messaging system. It is highly fast, horizontally scalable and fault tolerant system. Kafka has four core APIs called,<\/p>\n<h3>Producer API:<\/h3>\n<p>This API allows the clients to connect to Kafka servers running in cluster and publish the stream of records to one or more Kafka topics .<\/p>\n<h3>Consumer API:<\/h3>\n<p>This API allows the clients to connect to Kafka servers running in cluster and consume the streams of records from one or more Kafka topics. Kafka consumers <em><strong>PULLS<\/strong> <\/em>the messages from Kafka topics.<\/p>\n<h3>Streams API:<\/h3>\n<p>This API allows the clients to act as stream processors by consuming streams from one or more topics and producing the streams to other output topics. This allows to transform the input and output streams.<\/p>\n<h3>Connector API:<\/h3>\n<p>This API allows to write reusable producer and consumer code. For example, if we want to read data from any RDBMS to publish the data to topic and consume data from topic and write that to RDBMS. With connector API we can create reusable source and sink connector components for various data sources.<\/p>\n<h2>What use cases Kafka used for?<\/h2>\n<p>Kafka is used for the below use cases,<\/p>\n<h3>Messaging System:<\/h3>\n<p>Kafka used as an enterprise messaging system to decouple the source and target systems to exchange the data. Kafka provides high throughput with partitions and fault tolerance with replication compared to JMS.<\/p>\n<p><a href=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache_kafka_messaging_system.png\"><img decoding=\"async\" class=\"aligncenter size-full wp-image-70489\" src=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache_kafka_messaging_system.png\" alt=\"\" width=\"690\" height=\"369\" srcset=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache_kafka_messaging_system.png 690w, https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache_kafka_messaging_system-300x160.png 300w\" sizes=\"(max-width: 690px) 100vw, 690px\" \/><\/a><\/p>\n<h3>Web Activity Tracking:<\/h3>\n<p>To track the user journey events on the website for analytics and offline data processing.<\/p>\n<h3>Log Aggregation:<\/h3>\n<p>To process the log from various systems. Especially in the distributed environments, with micro services architectures where the systems are deployed on various hosts. We need to aggregate the logs from various systems and make the logs available in a central place for analysis. Go through the article on distributed logging architecture where Kafka is used\u00a0<a href=\"https:\/\/www.javacodegeeks.com\/2017\/07\/distributed-logging-architecture-microservices.html\" target=\"_blank\" rel=\"noopener\">https:\/\/smarttechie.org\/2017\/07\/31\/distributed-logging-architecture-for-micro-services\/<\/a><\/p>\n<h3>Metrics Collector:<\/h3>\n<p>Kafka is used to collect the metrics from various systems and networks for operations monitoring. There are Kafka metrics reporters available for monitoring tools like <a href=\"http:\/\/ganglia.sourceforge.net\/\" target=\"_blank\" rel=\"noopener\">Ganglia<\/a>, <a href=\"https:\/\/graphiteapp.org\/\" target=\"_blank\" rel=\"noopener\">Graphite<\/a> etc\u2026<\/p>\n<p>Some references on this\u00a0<a href=\"https:\/\/github.com\/stealthly\/metrics-kafka\" target=\"_blank\" rel=\"noopener\">https:\/\/github.com\/stealthly\/metrics-kafka<\/a><div style=\"display:inline-block; margin: 15px 0;\"> <div id=\"adngin-JavaCodeGeeks_incontent_video-0\" style=\"display:inline-block;\"><\/div> <\/div><\/p>\n<h2>What is broker?<\/h2>\n<p>An instance in a Kafka cluster is called as broker. In a Kafka cluster if you connect to any one broker you will be able to access entire cluster. The broker instance which we connect to access cluster is also known as bootstrap server. Each broker is identified by a numeric id in the cluster. To start with Kafka cluster three brokers is a good number. But there are clusters which has hundreds of brokers in it.<\/p>\n<h2>What is Topic?<\/h2>\n<p>A topic is a logical name to which the records are published. Internally the topic is divided into partitions to which the data is published. These partitions are distributed across the brokers in cluster. For example if a topic has three partitions with 3 brokers in cluster each broker has one partition. The published data to partition is append only with the offset increment.<\/p>\n<p><a href=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/topic-partitions.png\"><img decoding=\"async\" class=\"aligncenter size-full wp-image-70490\" src=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/topic-partitions.png\" alt=\"\" width=\"854\" height=\"259\" srcset=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/topic-partitions.png 854w, https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/topic-partitions-300x91.png 300w, https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/topic-partitions-768x233.png 768w\" sizes=\"(max-width: 854px) 100vw, 854px\" \/><\/a><\/p>\n<p>Below are the couple of points we need to remember while working with partitions.<\/p>\n<ul>\n<li>Topics are identified by its name. We can have many topics in a cluster.<\/li>\n<li>The order of the messages is maintained at the partition level, not across topic.<\/li>\n<li>Once the data written to partition is not overridden. This is called immutability.<\/li>\n<li>The message in partitions are stored with key, value and timestamp. Kafka ensures to publish the message to same partition for a given key.<\/li>\n<li>From the Kafka cluster, each partition will have a leader which will take read\/write operations to that partition.<\/li>\n<\/ul>\n<p><a href=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache-kafka-partitions.png\"><img decoding=\"async\" class=\"aligncenter size-full wp-image-70491\" src=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache-kafka-partitions.png\" alt=\"\" width=\"797\" height=\"450\" srcset=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache-kafka-partitions.png 797w, https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache-kafka-partitions-300x169.png 300w, https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2017\/11\/apache-kafka-partitions-768x434.png 768w\" sizes=\"(max-width: 797px) 100vw, 797px\" \/><\/a><\/p>\n<p>In the above example, I have created a topic with three partitions with replication factor 3. In this case as the cluster is having 3 brokers, the three partitions are evenly distributed and the replicas of each partition is replicated over to another 2 brokers. As the replication factor is 3, there is no data loss even 2 brokers goes down. Always keep replication factor is greater than 1 and less than or equal to number of brokers in the cluster. You can not create topic with replication factor more then the number of brokers in a cluster.<\/p>\n<p>In the above diagram, for each partition there is a leader(glowing partition) and other in-sync replicas(gray out partitions) are followers. For partition 0, the broker-1 is leader and broker-2, broker-3 are followers. All the reads\/writes to partition 0 will go to broker-1 and the same will be copied to broker-2 and broker-3.<\/p>\n<p>Now let us create Kafka cluster with 3 brokers by following the below steps.<\/p>\n<h2>Step 1:<\/h2>\n<p>Download the <a href=\"https:\/\/www.apache.org\/dyn\/closer.cgi?path=\/kafka\/1.0.0\/kafka_2.11-1.0.0.tgz\" target=\"_blank\" rel=\"noopener\">Apache Kafka<\/a> latest version. In this example I am using 1.0 which is latest. Extract the folder and move into the bin folder. Start the Zookeeper which is essential to start with Kafka cluster. Zookeeper is the coordination service to manage the brokers, leader election for partitions and alerting the Kafka during the changes to topic ( delete topic, create topic etc\u2026) or brokers( add broker, broker dies etc \u2026). In this example I have started only one Zookeeper instance. In production environments we should have more Zookeeper instances to manage fail-over. <em><strong>With out Zookeeper Kafka cluster cannot work.<\/strong><\/em><\/p>\n<pre class=\"brush:bash\">.\/zookeeper-server-start.sh ..\/config\/zookeeper.properties<\/pre>\n<h2>Step 2:<\/h2>\n<p>Now start Kafka brokers. In this example we are going to start three brokers. Goto the config folder under Kafka root and copy the server.properties file 3 times and name it as server_1.properties, server_2.properties and server_3.properties. Change the below properties in those files.<\/p>\n<pre class=\"brush:bash\">#####server_1.properties#####\r\nbroker.id=1\r\nlisteners=PLAINTEXT:\/\/:9091\r\nlog.dirs=\/tmp\/kafka-logs-1\r\n\r\n#####server_2.properties######\r\nbroker.id=2\r\nlisteners=PLAINTEXT:\/\/:9092\r\nlog.dirs=\/tmp\/kafka-logs-2\r\n\r\n######server_3.properties#####\r\nbroker.id=3\r\nlisteners=PLAINTEXT:\/\/:9093\r\nlog.dirs=\/tmp\/kafka-logs-3M<\/pre>\n<p>Now run the 3 brokers with the below commands.<\/p>\n<pre class=\"brush:bash\">###Start Broker 1 #######\r\n.\/kafka-server-start.sh ..\/config\/server_1.properties\r\n\r\n###Start Broker 2 #######\r\n.\/kafka-server-start.sh ..\/config\/server_2.properties\r\n\r\n###Start Broker 3 #######\r\n.\/kafka-server-start.sh ..\/config\/server_3.properties<\/pre>\n<h2>Step 3:<\/h2>\n<p>Create topic with below command.<\/p>\n<pre class=\"brush:bash; wrap-lines:false\">.\/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic first_topic<\/pre>\n<h2>Step 4:<\/h2>\n<p>Produce some messages to the topic created in above step by using Kafka console producer. For console producer mention any one of the broker address. That will be the bootstrap server to gain access to the entire cluster.<\/p>\n<pre class=\"brush:bash\">.\/kafka-console-producer.sh --broker-list localhost:9091 --topic first_topic\r\n&gt;First message\r\n&gt;Second message\r\n&gt;Third message\r\n&gt;Fourth message\r\n&gt;<\/pre>\n<h2>Step 5:<\/h2>\n<p>Consume the messages using Kafka console consumer. For Kafka consumer mention any one of the broker address as bootstrap server. Remember while reading the messages you may not see the order. As the order is maintained at the partition level, not at the topic level.<\/p>\n<pre class=\"brush:bash; wrap-lines:false\">.\/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic first_topic --from-beginning<\/pre>\n<p>If you want you can describe the topic to see how partitions are distributed and the the leader\u2019s of each partition using below command.<\/p>\n<pre class=\"brush:bash\">.\/kafka-topics.sh --describe --zookeeper localhost:2181 --topic first_topic\r\n\r\n#### The Result for the above command#####\r\nTopic:first_topic\tPartitionCount:3\tReplicationFactor:3\tConfigs:\r\n\tTopic: first_topic\tPartition: 0\tLeader: 1\tReplicas: 1,2,3\tIsr: 1,2,3\r\n\tTopic: first_topic\tPartition: 1\tLeader: 2\tReplicas: 2,3,1\tIsr: 2,3,1\r\n\tTopic: first_topic\tPartition: 2\tLeader: 3\tReplicas: 3,1,2\tIsr: 3,1,2<\/pre>\n<p>In the above description, broker-1 is the leader for partition:0 and broker-1, broker-2 and broker-3 has replicas of each partition.<\/p>\n<p>In the next article we will see producer and consumer JAVA API. Till then, <strong><em>Happy Messaging!!!<\/em><\/strong><\/p>\n<div class=\"attribution\">\n<table>\n<tbody>\n<tr>\n<td>Published on Java Code Geeks with permission by Siva Janapati, partner at our <a href=\"http:\/\/www.javacodegeeks.com\/join-us\/jcg\/\" target=\"_blank\" rel=\"noopener\">JCG program<\/a>. See the original article here: <a href=\"https:\/\/smarttechie.org\/2017\/11\/12\/introduction-to-apache-kafka\/\" target=\"_blank\" rel=\"noopener\">Introduction to Apache Kafka<\/a><\/p>\n<p>Opinions expressed by Java Code Geeks contributors are their own.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an enterprise messaging system. It is highly fast, horizontally scalable and fault tolerant system. Kafka has four core APIs called, Producer API: This API allows the clients to connect to Kafka &hellip;<\/p>\n","protected":false},"author":978,"featured_media":112,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[940],"class_list":["post-70467","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-enterprise-java","tag-apache-kafka"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Introduction to Apache Kafka - Java Code Geeks<\/title>\n<meta name=\"description\" content=\"What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction to Apache Kafka - Java Code Geeks\" \/>\n<meta property=\"og:description\" content=\"What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html\" \/>\n<meta property=\"og:site_name\" content=\"Java Code Geeks\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/javacodegeeks\" \/>\n<meta property=\"article:published_time\" content=\"2017-11-13T20:00:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"150\" \/>\n\t<meta property=\"og:image:height\" content=\"150\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Siva Janapati\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@javacodegeeks\" \/>\n<meta name=\"twitter:site\" content=\"@javacodegeeks\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Siva Janapati\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html\"},\"author\":{\"name\":\"Siva Janapati\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/person\\\/a995648656c597041a6525c37ae2d9b9\"},\"headline\":\"Introduction to Apache Kafka\",\"datePublished\":\"2017-11-13T20:00:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html\"},\"wordCount\":1111,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"keywords\":[\"Apache Kafka\"],\"articleSection\":[\"Enterprise Java\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html\",\"name\":\"Introduction to Apache Kafka - Java Code Geeks\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"datePublished\":\"2017-11-13T20:00:31+00:00\",\"description\":\"What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#primaryimage\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"contentUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"width\":150,\"height\":150,\"caption\":\"java-interview-questions-answers\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2017\\\/11\\\/introduction-apache-kafka.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Java\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/category\\\/java\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Enterprise Java\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/category\\\/java\\\/enterprise-java\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Introduction to Apache Kafka\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#website\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\",\"name\":\"Java Code Geeks\",\"description\":\"Java Developers Resource Center\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\"},\"alternateName\":\"JCG\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.javacodegeeks.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\",\"name\":\"Exelixis Media P.C.\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2022\\\/06\\\/exelixis-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2022\\\/06\\\/exelixis-logo.png\",\"width\":864,\"height\":246,\"caption\":\"Exelixis Media P.C.\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/javacodegeeks\",\"https:\\\/\\\/x.com\\\/javacodegeeks\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/person\\\/a995648656c597041a6525c37ae2d9b9\",\"name\":\"Siva Janapati\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/c472924653e4c97c936632fe8e77ba988d9d456333caa686d9a110f4c4c25da7?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/c472924653e4c97c936632fe8e77ba988d9d456333caa686d9a110f4c4c25da7?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/c472924653e4c97c936632fe8e77ba988d9d456333caa686d9a110f4c4c25da7?s=96&d=mm&r=g\",\"caption\":\"Siva Janapati\"},\"description\":\"Siva Prasad Rao Janapati is an Architect. He has hands on experience on Java, JEE, Spring, Oracle Commerce, MOZU Commerce, Apache Solr, Apache Kafka, Node.js, JBoss, Hibernate, Memcached, MySql, Oracle, MongoDB, APIGEE, Cloud Native, BlockChain and other open source\\\/enterprise technologies. He loves to explore new technologies and trends.\",\"sameAs\":[\"http:\\\/\\\/smarttechie.org\"],\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/author\\\/siva-janapati\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introduction to Apache Kafka - Java Code Geeks","description":"What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html","og_locale":"en_US","og_type":"article","og_title":"Introduction to Apache Kafka - Java Code Geeks","og_description":"What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an","og_url":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html","og_site_name":"Java Code Geeks","article_publisher":"https:\/\/www.facebook.com\/javacodegeeks","article_published_time":"2017-11-13T20:00:31+00:00","og_image":[{"width":150,"height":150,"url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","type":"image\/jpeg"}],"author":"Siva Janapati","twitter_card":"summary_large_image","twitter_creator":"@javacodegeeks","twitter_site":"@javacodegeeks","twitter_misc":{"Written by":"Siva Janapati","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#article","isPartOf":{"@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html"},"author":{"name":"Siva Janapati","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/person\/a995648656c597041a6525c37ae2d9b9"},"headline":"Introduction to Apache Kafka","datePublished":"2017-11-13T20:00:31+00:00","mainEntityOfPage":{"@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html"},"wordCount":1111,"commentCount":0,"publisher":{"@id":"https:\/\/www.javacodegeeks.com\/#organization"},"image":{"@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#primaryimage"},"thumbnailUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","keywords":["Apache Kafka"],"articleSection":["Enterprise Java"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html","url":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html","name":"Introduction to Apache Kafka - Java Code Geeks","isPartOf":{"@id":"https:\/\/www.javacodegeeks.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#primaryimage"},"image":{"@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#primaryimage"},"thumbnailUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","datePublished":"2017-11-13T20:00:31+00:00","description":"What is Apache Kafka? Apache Kafka is a distributed streaming system with publish and subscribe the stream of records. In another aspect it is an","breadcrumb":{"@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#primaryimage","url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","contentUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","width":150,"height":150,"caption":"java-interview-questions-answers"},{"@type":"BreadcrumbList","@id":"https:\/\/www.javacodegeeks.com\/2017\/11\/introduction-apache-kafka.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.javacodegeeks.com\/"},{"@type":"ListItem","position":2,"name":"Java","item":"https:\/\/www.javacodegeeks.com\/category\/java"},{"@type":"ListItem","position":3,"name":"Enterprise Java","item":"https:\/\/www.javacodegeeks.com\/category\/java\/enterprise-java"},{"@type":"ListItem","position":4,"name":"Introduction to Apache Kafka"}]},{"@type":"WebSite","@id":"https:\/\/www.javacodegeeks.com\/#website","url":"https:\/\/www.javacodegeeks.com\/","name":"Java Code Geeks","description":"Java Developers Resource Center","publisher":{"@id":"https:\/\/www.javacodegeeks.com\/#organization"},"alternateName":"JCG","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.javacodegeeks.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.javacodegeeks.com\/#organization","name":"Exelixis Media P.C.","url":"https:\/\/www.javacodegeeks.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2022\/06\/exelixis-logo.png","contentUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2022\/06\/exelixis-logo.png","width":864,"height":246,"caption":"Exelixis Media P.C."},"image":{"@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/javacodegeeks","https:\/\/x.com\/javacodegeeks"]},{"@type":"Person","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/person\/a995648656c597041a6525c37ae2d9b9","name":"Siva Janapati","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/c472924653e4c97c936632fe8e77ba988d9d456333caa686d9a110f4c4c25da7?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c472924653e4c97c936632fe8e77ba988d9d456333caa686d9a110f4c4c25da7?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c472924653e4c97c936632fe8e77ba988d9d456333caa686d9a110f4c4c25da7?s=96&d=mm&r=g","caption":"Siva Janapati"},"description":"Siva Prasad Rao Janapati is an Architect. He has hands on experience on Java, JEE, Spring, Oracle Commerce, MOZU Commerce, Apache Solr, Apache Kafka, Node.js, JBoss, Hibernate, Memcached, MySql, Oracle, MongoDB, APIGEE, Cloud Native, BlockChain and other open source\/enterprise technologies. He loves to explore new technologies and trends.","sameAs":["http:\/\/smarttechie.org"],"url":"https:\/\/www.javacodegeeks.com\/author\/siva-janapati"}]}},"_links":{"self":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts\/70467","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/users\/978"}],"replies":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/comments?post=70467"}],"version-history":[{"count":0,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts\/70467\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/media\/112"}],"wp:attachment":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/media?parent=70467"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/categories?post=70467"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/tags?post=70467"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}