{"id":60830,"date":"2016-10-12T19:00:07","date_gmt":"2016-10-12T16:00:07","guid":{"rendered":"https:\/\/www.javacodegeeks.com\/?p=60830"},"modified":"2016-10-12T14:14:44","modified_gmt":"2016-10-12T11:14:44","slug":"getting-started-apache-flink-kafka","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html","title":{"rendered":"Getting started with Apache Flink and Kafka"},"content":{"rendered":"<h2>Introduction<\/h2>\n<p><a href=\"https:\/\/flink.apache.org\/\">Apache Flink<\/a> is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several APIs to create data streams oriented application.<\/p>\n<p>It is very common for Flink applications to use <a href=\"http:\/\/kafka.apache.org\/\">Apache Kafka<\/a> for data input and output. This article will guide you into the steps to use Apache Flink with Kafka.<\/p>\n<p><a href=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2016\/10\/flink-kafka.png\"><img decoding=\"async\" class=\"aligncenter size-full wp-image-60846\" src=\"http:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2016\/10\/flink-kafka.png\" alt=\"flink-kafka\" width=\"400\" height=\"180\" srcset=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2016\/10\/flink-kafka.png 400w, https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2016\/10\/flink-kafka-300x135.png 300w\" sizes=\"(max-width: 400px) 100vw, 400px\" \/><\/a><\/p>\n<p><a name=\"more\"><\/a><\/p>\n<h3>Prerequisites<\/h3>\n<ul>\n<li>Apache Kafka 0.9.x<\/li>\n<li>Git<\/li>\n<li>Maven 3.x or later<\/li>\n<\/ul>\n<h2>Create your Flink Streaming Project<\/h2>\n<p>The first step is to create an Java application, the easiest is to use the flink-quickstart-java archetype, that contains the core dependencies and packaging tasks. This article is similar with the <a href=\"https:\/\/ci.apache.org\/projects\/flink\/flink-docs-release-1.1\/quickstart\/run_example_quickstart.html\">Apache Flink Quick Start Example<\/a>, with a clear focus on data input and output with MapR Streams.<\/p>\n<p>In this application we will create two jobs:<\/p>\n<ul>\n<li><code>WriteToKafka<\/code> : that generates random string and post them to a MapR Streams Topic using the Kafka Flink Connector and its Producer API.<\/li>\n<li><code>ReadFromKafka<\/code> : that reads the same topic and print the messages in the standard output using the Kafka Flink Connector and its Consumer. API.<\/li>\n<\/ul>\n<p>The full project is available on GitHub:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/tgrall\/kafka-flink-101\">Flink and Kakfa Application<\/a><\/li>\n<\/ul>\n<p>Let\u2019s create the project using Apache Maven:<\/p>\n<pre class=\"brush:java\">mvn archetype:generate \\\r\n      -DarchetypeGroupId=org.apache.flink\\\r\n      -DarchetypeArtifactId=flink-quickstart-java \\\r\n      -DarchetypeVersion=1.1.2 \\\r\n      -DgroupId=com.grallandco.demos \\\r\n      -DartifactId=kafka-flink-101 \\\r\n      -Dversion=1.0-SNAPSHOT \\\r\n      -DinteractiveMode=false<\/pre>\n<p>Maven will create the following structure:<\/p>\n<pre class=\"brush:java\">tree kafka-flink-101\/\r\nkafka-flink-101\/\r\n\u251c\u2500\u2500 pom.xml\r\n\u2514\u2500\u2500 src\r\n    \u2514\u2500\u2500 main\r\n        \u251c\u2500\u2500 java\r\n        \u2502\u00a0\u00a0 \u2514\u2500\u2500 com\r\n        \u2502\u00a0\u00a0     \u2514\u2500\u2500 grallandco\r\n        \u2502\u00a0\u00a0         \u2514\u2500\u2500 demos\r\n        \u2502\u00a0\u00a0             \u251c\u2500\u2500 BatchJob.java\r\n        \u2502\u00a0\u00a0             \u251c\u2500\u2500 SocketTextStreamWordCount.java\r\n        \u2502\u00a0\u00a0             \u251c\u2500\u2500 StreamingJob.java\r\n        \u2502\u00a0\u00a0             \u2514\u2500\u2500 WordCount.java\r\n        \u2514\u2500\u2500 resources\r\n            \u2514\u2500\u2500 log4j.properties\r\n\r\n7 directories, 6 files<\/pre>\n<p>This project is configured to create a Jar file that contains your flink project code and also includes all dependencies needed to run it.<div style=\"display:inline-block; margin: 15px 0;\"> <div id=\"adngin-JavaCodeGeeks_incontent_video-0\" style=\"display:inline-block;\"><\/div> <\/div><\/p>\n<p>The project contains some other sample jobs, we do not need them for this article, you can either keep them to educational purposes or simply remove them from the project.<\/p>\n<h2>Add Kafka Connector<\/h2>\n<p>Open the <code>pom.xml<\/code> and add the following dependencies to your project:<\/p>\n<p>As a first step, we have to add the Flink Kafka connector as a dependency so that we can use the Kafka sink. Add this to the pom.xml file in the dependencies section:<\/p>\n<p>You must add now the Flink Kafka Connector dependency to use the Kafka sink. Add the following entry in the <code>&lt;dependencies&gt;<\/code> element:<\/p>\n<pre class=\"brush:xml\">&lt;dependency&gt;\r\n      &lt;groupId&gt;org.apache.flink&lt;\/groupId&gt;\r\n      &lt;artifactId&gt;flink-connector-kafka-0.9_2.10&lt;\/artifactId&gt;\r\n      &lt;version&gt;${flink.version}&lt;\/version&gt;\r\n &lt;\/dependency&gt;<\/pre>\n<p>The Flink project is now ready to use the DataStream using the Kafka Connector so you can send and receive messages from Apache Kafka.<\/p>\n<h2>Install and Start Kafka<\/h2>\n<p>Download Kafka, enter the following commands in your terminal:<\/p>\n<pre class=\"brush:java\">curl -O http:\/\/www.us.apache.org\/dist\/kafka\/0.9.0.0\/kafka_2.11-0.9.0.0.tgz\r\ntar -xzf kafka_2.11-0.9.0.0.tgz\r\ncd kafka_2.11-0.9.0.0<\/pre>\n<p>Kafka uses ZooKeeper, if you do not have Zookeeper running, you can start it using the following command:<\/p>\n<pre class=\"brush:java\">.\/bin\/zookeeper-server-start.sh config\/zookeeper.properties<\/pre>\n<p>Start a Kafka broker by running the following command in a new terminal:<\/p>\n<pre class=\"brush:java\">.\/bin\/kafka-server-start.sh config\/server.properties<\/pre>\n<p>In another terminal, run the following command to create a Kafka topic called <code>flink-demo<\/code>:<\/p>\n<pre class=\"brush:java\">.\/bin\/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic flink-demo<\/pre>\n<p>Use the Kafka tools to post and consume messages to the <code>flink-demo<\/code> topic.<\/p>\n<p>Producer<\/p>\n<pre class=\"brush:java\">.\/bin\/kafka-console-producer.sh --broker-list localhost:9092 --topic flink-demo<\/pre>\n<p>Consumer<\/p>\n<pre class=\"brush:java\">.\/bin\/kafka-console-consumer.sh --zookeeper localhost:2181 --topic flink-demo --from-beginning<\/pre>\n<p>In the producer window, you can post some messages and see them in the consumer windows. We will use these tools to follow the interactions between Kafka and Flink.<\/p>\n<h2>Write your Flink application<\/h2>\n<p>Let\u2019s now use the Flink Kafka Connector to send messages to Kafka and consume them.<\/p>\n<h3>Producer<\/h3>\n<p>The producer generates messages using the <code>SimpleStringGenerator()<\/code> class and send the string to the <code>flink-demo<\/code> topic.<\/p>\n<pre class=\"brush:java\">public static void main(String[] args) throws Exception {\r\n    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();\r\n\r\n    Properties properties = new Properties();\r\n    properties.setProperty(\"bootstrap.servers\", \u201clocalhost:9092\"); \r\n\r\n    DataStream&lt;String&gt; stream = env.addSource(new SimpleStringGenerator());\r\n    stream.addSink(new FlinkKafkaProducer09&lt;&gt;(\"flink-demo\", new SimpleStringSchema(), properties));\r\n\r\n    env.execute();\r\n  }<\/pre>\n<p>The <code>SimpleStringGenerator()<\/code> method code is available <a href=\"https:\/\/github.com\/tgrall\/kafka-flink-101\/blob\/master\/src\/main\/java\/com\/grallandco\/demos\/WriteToKafka.java#L45-L60\">here<\/a>.<\/p>\n<p>The main steps are:<\/p>\n<ul>\n<li>create a new <code>StreamExecutionEnvironment<\/code> the basis of any Flink application<\/li>\n<li>create a new <code>DataStream<\/code> in the application environment, the <code>SimpleStringGenerator<\/code> class implements the <a href=\"https:\/\/ci.apache.org\/projects\/flink\/flink-docs-release-1.1\/api\/java\/\">SourceFunction<\/a> the base interface for all streams data sources in Flink.<\/li>\n<li>add the <code>FlinkKafkaProducer09<\/code> sink to the topic.<\/li>\n<\/ul>\n<h3>Consumer<\/h3>\n<p>The consumer simply reads the messages from the <code>flink-demo<\/code> topic, and print them into the console.<\/p>\n<pre class=\"brush:java\">public static void main(String[] args) throws Exception {\r\n    \/\/ create execution environment\r\n    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();\r\n\r\n    Properties properties = new Properties();\r\n    properties.setProperty(\"bootstrap.servers\", \u201clocalhost:9092\");\r\n    properties.setProperty(\"group.id\", \"flink_consumer\");\r\n\r\n    DataStream&lt;String&gt; stream = env.addSource(new FlinkKafkaConsumer09&lt;&gt;(\r\n        \"flink-demo\", new SimpleStringSchema(), properties) );\r\n\r\n    stream.map(new MapFunction&lt;String, String&gt;() {\r\n      private static final long serialVersionUID = -6867736771747690202L;\r\n\r\n      @Override\r\n      public String map(String value) throws Exception {\r\n        return \"Stream Value: \" + value;\r\n      }\r\n    }).print();\r\n\r\n    env.execute();\r\n  }<\/pre>\n<p>The main steps are:<\/p>\n<ul>\n<li>create a new <code>StreamExecutionEnvironment<\/code> the basis of any Flink application<\/li>\n<li>create a set of properties with the consumer information, in this application we can only set the consumer <code>group.id<\/code>.<\/li>\n<li>use the <code>FlinkKafkaConsumer09<\/code> to get the message from the topic <code>flink-demo<\/code><\/li>\n<\/ul>\n<h2>Build and Run the application<\/h2>\n<p>Let\u2019s run the application directly from Maven (or from your favorite IDE).<\/p>\n<p>1- Build the project:<\/p>\n<pre class=\"brush:java\">$ mvn clean package<\/pre>\n<p>2- Run the Flink Producer Job<\/p>\n<pre class=\"brush:java\">$ mvn exec:java -Dexec.mainClass=com.mapr.demos.WriteToKafka<\/pre>\n<p>3- Run the Flink Consumer Job<\/p>\n<pre class=\"brush:java\">$ mvn exec:java -Dexec.mainClass=com.mapr.demos.ReadFromKafka<\/pre>\n<p>In the terminal, you should see the messages generated from the producer<\/p>\n<p>You can now deploy and execute this job on your Flink cluster.<\/p>\n<h2>Conclusion<\/h2>\n<p>In this article you have learned how to use Flink with kafka to write and read data streams.<\/p>\n<div class=\"attribution\">\n<table>\n<tbody>\n<tr>\n<td><span class=\"reference\">Reference: <\/span><\/td>\n<td><a href=\"http:\/\/tugdualgrall.blogspot.com\/2016\/10\/getting-started-with-apache-flink-and.html\">Getting started with Apache Flink and Kafka<\/a> from our <a href=\"http:\/\/www.javacodegeeks.com\/join-us\/jcg\/\">JCG partner<\/a> Tugdual Grall at the <a href=\"http:\/\/tugdualgrall.blogspot.com\/\">Tug&#8217;s Blog<\/a> blog.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several APIs to create data streams oriented application. It is very common for Flink applications to use Apache Kafka for data input and output. This article will guide you into the steps &hellip;<\/p>\n","protected":false},"author":211,"featured_media":112,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[1220,1349],"class_list":["post-60830","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-enterprise-java","tag-apache-flink","tag-kafka"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Getting started with Apache Flink and Kafka - Java Code Geeks<\/title>\n<meta name=\"description\" content=\"Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Getting started with Apache Flink and Kafka - Java Code Geeks\" \/>\n<meta property=\"og:description\" content=\"Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html\" \/>\n<meta property=\"og:site_name\" content=\"Java Code Geeks\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/javacodegeeks\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/tgrall\" \/>\n<meta property=\"article:published_time\" content=\"2016-10-12T16:00:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"150\" \/>\n\t<meta property=\"og:image:height\" content=\"150\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Tugdual Grall\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@http:\/\/twitter.com\/tgrall\" \/>\n<meta name=\"twitter:site\" content=\"@javacodegeeks\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Tugdual Grall\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html\"},\"author\":{\"name\":\"Tugdual Grall\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/person\\\/b52921fffd460d1dd6c706aab3408681\"},\"headline\":\"Getting started with Apache Flink and Kafka\",\"datePublished\":\"2016-10-12T16:00:07+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html\"},\"wordCount\":687,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"keywords\":[\"Apache Flink\",\"Kafka\"],\"articleSection\":[\"Enterprise Java\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html\",\"name\":\"Getting started with Apache Flink and Kafka - Java Code Geeks\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"datePublished\":\"2016-10-12T16:00:07+00:00\",\"description\":\"Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#primaryimage\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"contentUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"width\":150,\"height\":150,\"caption\":\"java-interview-questions-answers\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2016\\\/10\\\/getting-started-apache-flink-kafka.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Java\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/category\\\/java\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Enterprise Java\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/category\\\/java\\\/enterprise-java\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Getting started with Apache Flink and Kafka\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#website\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\",\"name\":\"Java Code Geeks\",\"description\":\"Java Developers Resource Center\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\"},\"alternateName\":\"JCG\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.javacodegeeks.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\",\"name\":\"Exelixis Media P.C.\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2022\\\/06\\\/exelixis-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2022\\\/06\\\/exelixis-logo.png\",\"width\":864,\"height\":246,\"caption\":\"Exelixis Media P.C.\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/javacodegeeks\",\"https:\\\/\\\/x.com\\\/javacodegeeks\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/person\\\/b52921fffd460d1dd6c706aab3408681\",\"name\":\"Tugdual Grall\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/af1b68cee45f3bf495c2cd03b3a14abbb56b53dcad9ef92afc7d387842ef9d72?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/af1b68cee45f3bf495c2cd03b3a14abbb56b53dcad9ef92afc7d387842ef9d72?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/af1b68cee45f3bf495c2cd03b3a14abbb56b53dcad9ef92afc7d387842ef9d72?s=96&d=mm&r=g\",\"caption\":\"Tugdual Grall\"},\"description\":\"Tugdual Grall, an open source advocate and a passionate developer, is a Chief Technical Evangelist EMEA at MapR. He currently works with the European developer communities to ease MapR, Hadoop, and NoSQL adoption. Before joining MapR, Tug was Technical Evangelist at MongoDB and Couchbase. Tug has also worked as CTO at eXo Platform and JavaEE product manager, and software engineer at Oracle. Tugdual is Co-Founder of the Nantes JUG (Java User Group) that holds since 2008 monthly meeting about Java ecosystem. Tugdual also writes a blog available at http:\\\/\\\/tgrall.github.io\\\/\",\"sameAs\":[\"http:\\\/\\\/tgrall.github.io\\\/\",\"https:\\\/\\\/www.facebook.com\\\/tgrall\",\"http:\\\/\\\/www.linkedin.com\\\/in\\\/tugdualgrall\",\"https:\\\/\\\/x.com\\\/http:\\\/\\\/twitter.com\\\/tgrall\"],\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/author\\\/tugdual-grall\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Getting started with Apache Flink and Kafka - Java Code Geeks","description":"Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html","og_locale":"en_US","og_type":"article","og_title":"Getting started with Apache Flink and Kafka - Java Code Geeks","og_description":"Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several","og_url":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html","og_site_name":"Java Code Geeks","article_publisher":"https:\/\/www.facebook.com\/javacodegeeks","article_author":"https:\/\/www.facebook.com\/tgrall","article_published_time":"2016-10-12T16:00:07+00:00","og_image":[{"width":150,"height":150,"url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","type":"image\/jpeg"}],"author":"Tugdual Grall","twitter_card":"summary_large_image","twitter_creator":"@http:\/\/twitter.com\/tgrall","twitter_site":"@javacodegeeks","twitter_misc":{"Written by":"Tugdual Grall","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#article","isPartOf":{"@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html"},"author":{"name":"Tugdual Grall","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/person\/b52921fffd460d1dd6c706aab3408681"},"headline":"Getting started with Apache Flink and Kafka","datePublished":"2016-10-12T16:00:07+00:00","mainEntityOfPage":{"@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html"},"wordCount":687,"commentCount":0,"publisher":{"@id":"https:\/\/www.javacodegeeks.com\/#organization"},"image":{"@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#primaryimage"},"thumbnailUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","keywords":["Apache Flink","Kafka"],"articleSection":["Enterprise Java"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html","url":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html","name":"Getting started with Apache Flink and Kafka - Java Code Geeks","isPartOf":{"@id":"https:\/\/www.javacodegeeks.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#primaryimage"},"image":{"@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#primaryimage"},"thumbnailUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","datePublished":"2016-10-12T16:00:07+00:00","description":"Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several","breadcrumb":{"@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#primaryimage","url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","contentUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","width":150,"height":150,"caption":"java-interview-questions-answers"},{"@type":"BreadcrumbList","@id":"https:\/\/www.javacodegeeks.com\/2016\/10\/getting-started-apache-flink-kafka.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.javacodegeeks.com\/"},{"@type":"ListItem","position":2,"name":"Java","item":"https:\/\/www.javacodegeeks.com\/category\/java"},{"@type":"ListItem","position":3,"name":"Enterprise Java","item":"https:\/\/www.javacodegeeks.com\/category\/java\/enterprise-java"},{"@type":"ListItem","position":4,"name":"Getting started with Apache Flink and Kafka"}]},{"@type":"WebSite","@id":"https:\/\/www.javacodegeeks.com\/#website","url":"https:\/\/www.javacodegeeks.com\/","name":"Java Code Geeks","description":"Java Developers Resource Center","publisher":{"@id":"https:\/\/www.javacodegeeks.com\/#organization"},"alternateName":"JCG","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.javacodegeeks.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.javacodegeeks.com\/#organization","name":"Exelixis Media P.C.","url":"https:\/\/www.javacodegeeks.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2022\/06\/exelixis-logo.png","contentUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2022\/06\/exelixis-logo.png","width":864,"height":246,"caption":"Exelixis Media P.C."},"image":{"@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/javacodegeeks","https:\/\/x.com\/javacodegeeks"]},{"@type":"Person","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/person\/b52921fffd460d1dd6c706aab3408681","name":"Tugdual Grall","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/af1b68cee45f3bf495c2cd03b3a14abbb56b53dcad9ef92afc7d387842ef9d72?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/af1b68cee45f3bf495c2cd03b3a14abbb56b53dcad9ef92afc7d387842ef9d72?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/af1b68cee45f3bf495c2cd03b3a14abbb56b53dcad9ef92afc7d387842ef9d72?s=96&d=mm&r=g","caption":"Tugdual Grall"},"description":"Tugdual Grall, an open source advocate and a passionate developer, is a Chief Technical Evangelist EMEA at MapR. He currently works with the European developer communities to ease MapR, Hadoop, and NoSQL adoption. Before joining MapR, Tug was Technical Evangelist at MongoDB and Couchbase. Tug has also worked as CTO at eXo Platform and JavaEE product manager, and software engineer at Oracle. Tugdual is Co-Founder of the Nantes JUG (Java User Group) that holds since 2008 monthly meeting about Java ecosystem. Tugdual also writes a blog available at http:\/\/tgrall.github.io\/","sameAs":["http:\/\/tgrall.github.io\/","https:\/\/www.facebook.com\/tgrall","http:\/\/www.linkedin.com\/in\/tugdualgrall","https:\/\/x.com\/http:\/\/twitter.com\/tgrall"],"url":"https:\/\/www.javacodegeeks.com\/author\/tugdual-grall"}]}},"_links":{"self":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts\/60830","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/users\/211"}],"replies":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/comments?post=60830"}],"version-history":[{"count":0,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts\/60830\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/media\/112"}],"wp:attachment":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/media?parent=60830"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/categories?post=60830"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/tags?post=60830"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}