{"id":110628,"date":"2021-06-30T10:00:00","date_gmt":"2021-06-30T07:00:00","guid":{"rendered":"https:\/\/www.javacodegeeks.com\/?p=110628"},"modified":"2021-06-28T14:18:51","modified_gmt":"2021-06-28T11:18:51","slug":"bigquery-storage-api-avro","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html","title":{"rendered":"BigQuery Storage API: Avro"},"content":{"rendered":"<p>Previously we had an <a href=\"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-arrow.html\">introduction<\/a> on the BigQuery Storage API and we proceeded reading data using the Arrow format. In this tutorial we shall read Data using the Avro format.<\/p>\n<p>What applied on the previous tutorial applies here too.<\/p>\n<p>We shall create a BigQuery Storage Client, create a ReadSession using the Avro format and iterate the data on each stream.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><a href=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2021\/06\/avro.png\"><img decoding=\"async\" width=\"267\" height=\"82\" src=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2021\/06\/avro.png\" alt=\"\" class=\"wp-image-110636\"\/><\/a><\/figure>\n<\/div>\n<p>Let\u2019s get started by importing the dependencies, we do import the Avro library needed.<\/p>\n<div>\n<div id=\"highlighter_76108\" class=\"syntaxhighlighter  xml\">\n<table border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n<tbody>\n<tr>\n<td class=\"gutter\">\n<div class=\"line number1 index0 alt2\">01<\/div>\n<div class=\"line number2 index1 alt1\">02<\/div>\n<div class=\"line number3 index2 alt2\">03<\/div>\n<div class=\"line number4 index3 alt1\">04<\/div>\n<div class=\"line number5 index4 alt2\">05<\/div>\n<div class=\"line number6 index5 alt1\">06<\/div>\n<div class=\"line number7 index6 alt2\">07<\/div>\n<div class=\"line number8 index7 alt1\">08<\/div>\n<div class=\"line number9 index8 alt2\">09<\/div>\n<div class=\"line number10 index9 alt1\">10<\/div>\n<div class=\"line number11 index10 alt2\">11<\/div>\n<div class=\"line number12 index11 alt1\">12<\/div>\n<div class=\"line number13 index12 alt2\">13<\/div>\n<div class=\"line number14 index13 alt1\">14<\/div>\n<div class=\"line number15 index14 alt2\">15<\/div>\n<div class=\"line number16 index15 alt1\">16<\/div>\n<div class=\"line number17 index16 alt2\">17<\/div>\n<div class=\"line number18 index17 alt1\">18<\/div>\n<div class=\"line number19 index18 alt2\">19<\/div>\n<div class=\"line number20 index19 alt1\">20<\/div>\n<div class=\"line number21 index20 alt2\">21<\/div>\n<div class=\"line number22 index21 alt1\">22<\/div>\n<div class=\"line number23 index22 alt2\">23<\/div>\n<\/td>\n<td class=\"code\">\n<div class=\"container\">\n<div class=\"line number1 index0 alt2\"><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">dependencyManagement<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number2 index1 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">dependencies<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number3 index2 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">dependency<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number4 index3 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">groupId<\/code><code class=\"xml plain\">&gt;com.google.cloud&lt;\/<\/code><code class=\"xml keyword\">groupId<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number5 index4 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">artifactId<\/code><code class=\"xml plain\">&gt;libraries-bom&lt;\/<\/code><code class=\"xml keyword\">artifactId<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number6 index5 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">version<\/code><code class=\"xml plain\">&gt;20.5.0&lt;\/<\/code><code class=\"xml keyword\">version<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number7 index6 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">type<\/code><code class=\"xml plain\">&gt;pom&lt;\/<\/code><code class=\"xml keyword\">type<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number8 index7 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">scope<\/code><code class=\"xml plain\">&gt;import&lt;\/<\/code><code class=\"xml keyword\">scope<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number9 index8 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;\/<\/code><code class=\"xml keyword\">dependency<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number10 index9 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;\/<\/code><code class=\"xml keyword\">dependencies<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number11 index10 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;\/<\/code><code class=\"xml keyword\">dependencyManagement<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number12 index11 alt1\">&nbsp;<\/div>\n<div class=\"line number13 index12 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">dependencies<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number14 index13 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">dependency<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number15 index14 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">groupId<\/code><code class=\"xml plain\">&gt;com.google.cloud&lt;\/<\/code><code class=\"xml keyword\">groupId<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number16 index15 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">artifactId<\/code><code class=\"xml plain\">&gt;google-cloud-bigquerystorage&lt;\/<\/code><code class=\"xml keyword\">artifactId<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number17 index16 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;\/<\/code><code class=\"xml keyword\">dependency<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number18 index17 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">dependency<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number19 index18 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">groupId<\/code><code class=\"xml plain\">&gt;org.apache.avro&lt;\/<\/code><code class=\"xml keyword\">groupId<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number20 index19 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">artifactId<\/code><code class=\"xml plain\">&gt;avro&lt;\/<\/code><code class=\"xml keyword\">artifactId<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number21 index20 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;<\/code><code class=\"xml keyword\">version<\/code><code class=\"xml plain\">&gt;1.9.2&lt;\/<\/code><code class=\"xml keyword\">version<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number22 index21 alt1\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;\/<\/code><code class=\"xml keyword\">dependency<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<div class=\"line number23 index22 alt2\"><code class=\"xml spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"xml plain\">&lt;\/<\/code><code class=\"xml keyword\">dependencies<\/code><code class=\"xml plain\">&gt;<\/code><\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<p>Our next step would be to create an Avro Data Reader for our rows that have the schema of col1:string, col2:int. In our case we shall just print the data through sys.out<\/p>\n<div>\n<div id=\"highlighter_798454\" class=\"syntaxhighlighter  java\">\n<table border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n<tbody>\n<tr>\n<td class=\"gutter\">\n<div class=\"line number1 index0 alt2\">01<\/div>\n<div class=\"line number2 index1 alt1\">02<\/div>\n<div class=\"line number3 index2 alt2\">03<\/div>\n<div class=\"line number4 index3 alt1\">04<\/div>\n<div class=\"line number5 index4 alt2\">05<\/div>\n<div class=\"line number6 index5 alt1\">06<\/div>\n<div class=\"line number7 index6 alt2\">07<\/div>\n<div class=\"line number8 index7 alt1\">08<\/div>\n<div class=\"line number9 index8 alt2\">09<\/div>\n<div class=\"line number10 index9 alt1\">10<\/div>\n<div class=\"line number11 index10 alt2\">11<\/div>\n<div class=\"line number12 index11 alt1\">12<\/div>\n<div class=\"line number13 index12 alt2\">13<\/div>\n<div class=\"line number14 index13 alt1\">14<\/div>\n<div class=\"line number15 index14 alt2\">15<\/div>\n<div class=\"line number16 index15 alt1\">16<\/div>\n<div class=\"line number17 index16 alt2\">17<\/div>\n<div class=\"line number18 index17 alt1\">18<\/div>\n<div class=\"line number19 index18 alt2\">19<\/div>\n<div class=\"line number20 index19 alt1\">20<\/div>\n<div class=\"line number21 index20 alt2\">21<\/div>\n<div class=\"line number22 index21 alt1\">22<\/div>\n<div class=\"line number23 index22 alt2\">23<\/div>\n<div class=\"line number24 index23 alt1\">24<\/div>\n<div class=\"line number25 index24 alt2\">25<\/div>\n<div class=\"line number26 index25 alt1\">26<\/div>\n<div class=\"line number27 index26 alt2\">27<\/div>\n<div class=\"line number28 index27 alt1\">28<\/div>\n<div class=\"line number29 index28 alt2\">29<\/div>\n<div class=\"line number30 index29 alt1\">30<\/div>\n<div class=\"line number31 index30 alt2\">31<\/div>\n<div class=\"line number32 index31 alt1\">32<\/div>\n<div class=\"line number33 index32 alt2\">33<\/div>\n<div class=\"line number34 index33 alt1\">34<\/div>\n<div class=\"line number35 index34 alt2\">35<\/div>\n<div class=\"line number36 index35 alt1\">36<\/div>\n<div class=\"line number37 index36 alt2\">37<\/div>\n<\/td>\n<td class=\"code\">\n<div class=\"container\">\n<div class=\"line number1 index0 alt2\"><code class=\"java keyword\">package<\/code> <code class=\"java plain\">com.gkatzioura.bigquery.storage.api.avro;<\/code><\/div>\n<div class=\"line number2 index1 alt1\">&nbsp;<\/div>\n<div class=\"line number3 index2 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">java.io.ByteArrayInputStream;<\/code><\/div>\n<div class=\"line number4 index3 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">java.io.IOException;<\/code><\/div>\n<div class=\"line number5 index4 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">java.io.InputStream;<\/code><\/div>\n<div class=\"line number6 index5 alt1\">&nbsp;<\/div>\n<div class=\"line number7 index6 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">org.apache.avro.Schema;<\/code><\/div>\n<div class=\"line number8 index7 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">org.apache.avro.generic.GenericDatumReader;<\/code><\/div>\n<div class=\"line number9 index8 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">org.apache.avro.generic.GenericRecord;<\/code><\/div>\n<div class=\"line number10 index9 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">org.apache.avro.io.BinaryDecoder;<\/code><\/div>\n<div class=\"line number11 index10 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">org.apache.avro.io.DecoderFactory;<\/code><\/div>\n<div class=\"line number12 index11 alt1\">&nbsp;<\/div>\n<div class=\"line number13 index12 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.AvroSchema;<\/code><\/div>\n<div class=\"line number14 index13 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.protobuf.ByteString;<\/code><\/div>\n<div class=\"line number15 index14 alt2\">&nbsp;<\/div>\n<div class=\"line number16 index15 alt1\"><code class=\"java keyword\">public<\/code> <code class=\"java keyword\">class<\/code> <code class=\"java plain\">AvroReader {<\/code><\/div>\n<div class=\"line number17 index16 alt2\">&nbsp;<\/div>\n<div class=\"line number18 index17 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">private<\/code> <code class=\"java keyword\">final<\/code> <code class=\"java plain\">GenericDatumReader&lt;GenericRecord&gt; datumReader;<\/code><\/div>\n<div class=\"line number19 index18 alt2\">&nbsp;<\/div>\n<div class=\"line number20 index19 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">public<\/code> <code class=\"java plain\">AvroReader(AvroSchema arrowSchema) {<\/code><\/div>\n<div class=\"line number21 index20 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">Schema schema = <\/code><code class=\"java keyword\">new<\/code> <code class=\"java plain\">Schema.Parser().parse(arrowSchema.getSchema());<\/code><\/div>\n<div class=\"line number22 index21 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">this<\/code><code class=\"java plain\">.datumReader = <\/code><code class=\"java keyword\">new<\/code> <code class=\"java plain\">GenericDatumReader&lt;&gt;(schema);<\/code><\/div>\n<div class=\"line number23 index22 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number24 index23 alt1\">&nbsp;<\/div>\n<div class=\"line number25 index24 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">public<\/code> <code class=\"java keyword\">void<\/code> <code class=\"java plain\">processRows(ByteString avroRows) <\/code><code class=\"java keyword\">throws<\/code> <code class=\"java plain\">IOException {<\/code><\/div>\n<div class=\"line number26 index25 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">try<\/code><code class=\"java plain\">(InputStream inputStream = <\/code><code class=\"java keyword\">new<\/code> <code class=\"java plain\">ByteArrayInputStream(avroRows.toByteArray())) {<\/code><\/div>\n<div class=\"line number27 index26 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">BinaryDecoder decoder =DecoderFactory.get().binaryDecoder(inputStream, <\/code><code class=\"java keyword\">null<\/code><code class=\"java plain\">);<\/code><\/div>\n<div class=\"line number28 index27 alt1\">&nbsp;<\/div>\n<div class=\"line number29 index28 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">while<\/code> <code class=\"java plain\">(!decoder.isEnd()) {<\/code><\/div>\n<div class=\"line number30 index29 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">GenericRecord item = datumReader.read(<\/code><code class=\"java keyword\">null<\/code><code class=\"java plain\">, decoder);<\/code><\/div>\n<div class=\"line number31 index30 alt2\">&nbsp;<\/div>\n<div class=\"line number32 index31 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">System.out.println(item.get(<\/code><code class=\"java string\">\"col1\"<\/code><code class=\"java plain\">)+<\/code><code class=\"java string\">\",\"<\/code><code class=\"java plain\">+item.get(<\/code><code class=\"java string\">\"col2\"<\/code><code class=\"java plain\">));<\/code><\/div>\n<div class=\"line number33 index32 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number34 index33 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number35 index34 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number36 index35 alt1\">&nbsp;<\/div>\n<div class=\"line number37 index36 alt2\"><code class=\"java plain\">}<\/code><\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<p>Then on to our main class which is the one with any BigQuery logic needed.<div style=\"display:inline-block; margin: 15px 0;\"> <div id=\"adngin-JavaCodeGeeks_incontent_video-0\" style=\"display:inline-block;\"><\/div> <\/div><\/p>\n<div>\n<div id=\"highlighter_313752\" class=\"syntaxhighlighter  java\">\n<table border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n<tbody>\n<tr>\n<td class=\"gutter\">\n<div class=\"line number1 index0 alt2\">01<\/div>\n<div class=\"line number2 index1 alt1\">02<\/div>\n<div class=\"line number3 index2 alt2\">03<\/div>\n<div class=\"line number4 index3 alt1\">04<\/div>\n<div class=\"line number5 index4 alt2\">05<\/div>\n<div class=\"line number6 index5 alt1\">06<\/div>\n<div class=\"line number7 index6 alt2\">07<\/div>\n<div class=\"line number8 index7 alt1\">08<\/div>\n<div class=\"line number9 index8 alt2\">09<\/div>\n<div class=\"line number10 index9 alt1\">10<\/div>\n<div class=\"line number11 index10 alt2\">11<\/div>\n<div class=\"line number12 index11 alt1\">12<\/div>\n<div class=\"line number13 index12 alt2\">13<\/div>\n<div class=\"line number14 index13 alt1\">14<\/div>\n<div class=\"line number15 index14 alt2\">15<\/div>\n<div class=\"line number16 index15 alt1\">16<\/div>\n<div class=\"line number17 index16 alt2\">17<\/div>\n<div class=\"line number18 index17 alt1\">18<\/div>\n<div class=\"line number19 index18 alt2\">19<\/div>\n<div class=\"line number20 index19 alt1\">20<\/div>\n<div class=\"line number21 index20 alt2\">21<\/div>\n<div class=\"line number22 index21 alt1\">22<\/div>\n<div class=\"line number23 index22 alt2\">23<\/div>\n<div class=\"line number24 index23 alt1\">24<\/div>\n<div class=\"line number25 index24 alt2\">25<\/div>\n<div class=\"line number26 index25 alt1\">26<\/div>\n<div class=\"line number27 index26 alt2\">27<\/div>\n<div class=\"line number28 index27 alt1\">28<\/div>\n<div class=\"line number29 index28 alt2\">29<\/div>\n<div class=\"line number30 index29 alt1\">30<\/div>\n<div class=\"line number31 index30 alt2\">31<\/div>\n<div class=\"line number32 index31 alt1\">32<\/div>\n<div class=\"line number33 index32 alt2\">33<\/div>\n<div class=\"line number34 index33 alt1\">34<\/div>\n<div class=\"line number35 index34 alt2\">35<\/div>\n<div class=\"line number36 index35 alt1\">36<\/div>\n<div class=\"line number37 index36 alt2\">37<\/div>\n<div class=\"line number38 index37 alt1\">38<\/div>\n<div class=\"line number39 index38 alt2\">39<\/div>\n<div class=\"line number40 index39 alt1\">40<\/div>\n<div class=\"line number41 index40 alt2\">41<\/div>\n<div class=\"line number42 index41 alt1\">42<\/div>\n<div class=\"line number43 index42 alt2\">43<\/div>\n<div class=\"line number44 index43 alt1\">44<\/div>\n<div class=\"line number45 index44 alt2\">45<\/div>\n<div class=\"line number46 index45 alt1\">46<\/div>\n<div class=\"line number47 index46 alt2\">47<\/div>\n<div class=\"line number48 index47 alt1\">48<\/div>\n<div class=\"line number49 index48 alt2\">49<\/div>\n<div class=\"line number50 index49 alt1\">50<\/div>\n<div class=\"line number51 index50 alt2\">51<\/div>\n<div class=\"line number52 index51 alt1\">52<\/div>\n<div class=\"line number53 index52 alt2\">53<\/div>\n<div class=\"line number54 index53 alt1\">54<\/div>\n<div class=\"line number55 index54 alt2\">55<\/div>\n<\/td>\n<td class=\"code\">\n<div class=\"container\">\n<div class=\"line number1 index0 alt2\"><code class=\"java keyword\">package<\/code> <code class=\"java plain\">com.gkatzioura.bigquery.storage.api.avro;<\/code><\/div>\n<div class=\"line number2 index1 alt1\">&nbsp;<\/div>\n<div class=\"line number3 index2 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">org.apache.arrow.util.Preconditions;<\/code><\/div>\n<div class=\"line number4 index3 alt1\">&nbsp;<\/div>\n<div class=\"line number5 index4 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.api.gax.rpc.ServerStream;<\/code><\/div>\n<div class=\"line number6 index5 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.BigQueryReadClient;<\/code><\/div>\n<div class=\"line number7 index6 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.CreateReadSessionRequest;<\/code><\/div>\n<div class=\"line number8 index7 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.DataFormat;<\/code><\/div>\n<div class=\"line number9 index8 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.ReadRowsRequest;<\/code><\/div>\n<div class=\"line number10 index9 alt1\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.ReadRowsResponse;<\/code><\/div>\n<div class=\"line number11 index10 alt2\"><code class=\"java keyword\">import<\/code> <code class=\"java plain\">com.google.cloud.bigquery.storage.v1.ReadSession;<\/code><\/div>\n<div class=\"line number12 index11 alt1\">&nbsp;<\/div>\n<div class=\"line number13 index12 alt2\"><code class=\"java keyword\">public<\/code> <code class=\"java keyword\">class<\/code> <code class=\"java plain\">AvroMain {<\/code><\/div>\n<div class=\"line number14 index13 alt1\">&nbsp;<\/div>\n<div class=\"line number15 index14 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">public<\/code> <code class=\"java keyword\">static<\/code> <code class=\"java keyword\">void<\/code> <code class=\"java plain\">main(String[] args) <\/code><code class=\"java keyword\">throws<\/code> <code class=\"java plain\">Exception {<\/code><\/div>\n<div class=\"line number16 index15 alt1\">&nbsp;<\/div>\n<div class=\"line number17 index16 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">String projectId = System.getenv(<\/code><code class=\"java string\">\"PROJECT_ID\"<\/code><code class=\"java plain\">);<\/code><\/div>\n<div class=\"line number18 index17 alt1\">&nbsp;<\/div>\n<div class=\"line number19 index18 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">try<\/code> <code class=\"java plain\">(BigQueryReadClient client = BigQueryReadClient.create()) {<\/code><\/div>\n<div class=\"line number20 index19 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">String parent = String.format(<\/code><code class=\"java string\">\"projects\/%s\"<\/code><code class=\"java plain\">, projectId);<\/code><\/div>\n<div class=\"line number21 index20 alt2\">&nbsp;<\/div>\n<div class=\"line number22 index21 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">String srcTable =<\/code><\/div>\n<div class=\"line number23 index22 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">String.format(<\/code><\/div>\n<div class=\"line number24 index23 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java string\">\"projects\/%s\/datasets\/%s\/tables\/%s\"<\/code><code class=\"java plain\">,<\/code><\/div>\n<div class=\"line number25 index24 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">projectId, System.getenv(<\/code><code class=\"java string\">\"DATASET\"<\/code><code class=\"java plain\">), System.getenv(<\/code><code class=\"java string\">\"TABLE\"<\/code><code class=\"java plain\">));<\/code><\/div>\n<div class=\"line number26 index25 alt1\">&nbsp;<\/div>\n<div class=\"line number27 index26 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">ReadSession.Builder sessionBuilder =<\/code><\/div>\n<div class=\"line number28 index27 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">ReadSession.newBuilder()<\/code><\/div>\n<div class=\"line number29 index28 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">.setTable(srcTable)<\/code><\/div>\n<div class=\"line number30 index29 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">.setDataFormat(DataFormat.AVRO);<\/code><\/div>\n<div class=\"line number31 index30 alt2\">&nbsp;<\/div>\n<div class=\"line number32 index31 alt1\">&nbsp;<\/div>\n<div class=\"line number33 index32 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">CreateReadSessionRequest.Builder builder =<\/code><\/div>\n<div class=\"line number34 index33 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">CreateReadSessionRequest.newBuilder()<\/code><\/div>\n<div class=\"line number35 index34 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">.setParent(parent)<\/code><\/div>\n<div class=\"line number36 index35 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">.setReadSession(sessionBuilder)<\/code><\/div>\n<div class=\"line number37 index36 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">.setMaxStreamCount(<\/code><code class=\"java value\">1<\/code><code class=\"java plain\">);<\/code><\/div>\n<div class=\"line number38 index37 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">ReadSession session = client.createReadSession(builder.build());<\/code><\/div>\n<div class=\"line number39 index38 alt2\">&nbsp;<\/div>\n<div class=\"line number40 index39 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">Preconditions.checkState(session.getStreamsCount() &gt; <\/code><code class=\"java value\">0<\/code><code class=\"java plain\">);<\/code><\/div>\n<div class=\"line number41 index40 alt2\">&nbsp;<\/div>\n<div class=\"line number42 index41 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">String streamName = session.getStreams(<\/code><code class=\"java value\">0<\/code><code class=\"java plain\">).getName();<\/code><\/div>\n<div class=\"line number43 index42 alt2\">&nbsp;<\/div>\n<div class=\"line number44 index43 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">ReadRowsRequest readRowsRequest =<\/code><\/div>\n<div class=\"line number45 index44 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">ReadRowsRequest.newBuilder().setReadStream(streamName).build();<\/code><\/div>\n<div class=\"line number46 index45 alt1\">&nbsp;<\/div>\n<div class=\"line number47 index46 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">ServerStream&lt;ReadRowsResponse&gt; stream = client.readRowsCallable().call(readRowsRequest);<\/code><\/div>\n<div class=\"line number48 index47 alt1\">&nbsp;<\/div>\n<div class=\"line number49 index48 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">for<\/code> <code class=\"java plain\">(ReadRowsResponse response : stream) {<\/code><\/div>\n<div class=\"line number50 index49 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java keyword\">new<\/code> <code class=\"java plain\">AvroReader(session.getAvroSchema()).processRows(response.getAvroRows().getSerializedBinaryRows());<\/code><\/div>\n<div class=\"line number51 index50 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number52 index51 alt1\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number53 index52 alt2\"><code class=\"java spaces\">&nbsp;&nbsp;&nbsp;&nbsp;<\/code><code class=\"java plain\">}<\/code><\/div>\n<div class=\"line number54 index53 alt1\">&nbsp;<\/div>\n<div class=\"line number55 index54 alt2\"><code class=\"java plain\">}<\/code><\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<p>A BigQuery client is created. Then we create a session request with a max number of streams. We did specify that the format to be used will be Avro.<br \/>Once we get a Response, the response will contain the initiated the Session, the Avro schema and the streams that we shall use to retrieve the Data.<br \/>For each stream there has to be a ReadRowsRequest in order to fetch the data.<br \/>Then we pass data to our Avro decoder.<\/p>\n<p>That\u2019s it we just read data from the BigQuery Storage API using Avro and Arrow!<\/p>\n<div class=\"attribution\">\n<table>\n<tbody>\n<tr>\n<td>\n<p>Published on Java Code Geeks with permission by Emmanouil Gkatziouras, partner at our <a href=\"\/\/www.javacodegeeks.com\/join-us\/jcg\/\" target=\"_blank\" rel=\"noopener\">JCG program<\/a>. See the original article here: <a href=\"https:\/\/egkatzioura.com\/2021\/06\/08\/bigquery-storage-api-avro\/\" target=\"_blank\" rel=\"noopener\">BigQuery Storage API: Avro<\/a><\/p>\n<p>Opinions expressed by Java Code Geeks contributors are their own.<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Previously we had an introduction on the BigQuery Storage API and we proceeded reading data using the Arrow format. In this tutorial we shall read Data using the Avro format. What applied on the previous tutorial applies here too. We shall create a BigQuery Storage Client, create a ReadSession using the Avro format and iterate &hellip;<\/p>\n","protected":false},"author":936,"featured_media":112,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[372],"class_list":["post-110628","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-enterprise-java","tag-big-data"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>BigQuery Storage API: Avro - Java Code Geeks<\/title>\n<meta name=\"description\" content=\"Interested to learn about Avro? Check our article explaining how to read Data using the Avro format.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"BigQuery Storage API: Avro - Java Code Geeks\" \/>\n<meta property=\"og:description\" content=\"Interested to learn about Avro? Check our article explaining how to read Data using the Avro format.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html\" \/>\n<meta property=\"og:site_name\" content=\"Java Code Geeks\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/javacodegeeks\" \/>\n<meta property=\"article:published_time\" content=\"2021-06-30T07:00:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"150\" \/>\n\t<meta property=\"og:image:height\" content=\"150\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Emmanouil Gkatziouras\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@javacodegeeks\" \/>\n<meta name=\"twitter:site\" content=\"@javacodegeeks\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Emmanouil Gkatziouras\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html\"},\"author\":{\"name\":\"Emmanouil Gkatziouras\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/person\\\/5eee031b356c7682e1fd24c8297561c6\"},\"headline\":\"BigQuery Storage API: Avro\",\"datePublished\":\"2021-06-30T07:00:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html\"},\"wordCount\":285,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"keywords\":[\"Big Data\"],\"articleSection\":[\"Enterprise Java\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html\",\"name\":\"BigQuery Storage API: Avro - Java Code Geeks\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"datePublished\":\"2021-06-30T07:00:00+00:00\",\"description\":\"Interested to learn about Avro? Check our article explaining how to read Data using the Avro format.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#primaryimage\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"contentUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2012\\\/10\\\/enterprise-java-logo.jpg\",\"width\":150,\"height\":150,\"caption\":\"java-interview-questions-answers\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/2021\\\/06\\\/bigquery-storage-api-avro.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Java\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/category\\\/java\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Enterprise Java\",\"item\":\"https:\\\/\\\/www.javacodegeeks.com\\\/category\\\/java\\\/enterprise-java\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"BigQuery Storage API: Avro\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#website\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\",\"name\":\"Java Code Geeks\",\"description\":\"Java Developers Resource Center\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\"},\"alternateName\":\"JCG\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.javacodegeeks.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#organization\",\"name\":\"Exelixis Media P.C.\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2022\\\/06\\\/exelixis-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.javacodegeeks.com\\\/wp-content\\\/uploads\\\/2022\\\/06\\\/exelixis-logo.png\",\"width\":864,\"height\":246,\"caption\":\"Exelixis Media P.C.\"},\"image\":{\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/javacodegeeks\",\"https:\\\/\\\/x.com\\\/javacodegeeks\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.javacodegeeks.com\\\/#\\\/schema\\\/person\\\/5eee031b356c7682e1fd24c8297561c6\",\"name\":\"Emmanouil Gkatziouras\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/5c6d031d211ab786ec335687ad6f3f076f93f47e24c92d78041d2f805ee6c291?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/5c6d031d211ab786ec335687ad6f3f076f93f47e24c92d78041d2f805ee6c291?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/5c6d031d211ab786ec335687ad6f3f076f93f47e24c92d78041d2f805ee6c291?s=96&d=mm&r=g\",\"caption\":\"Emmanouil Gkatziouras\"},\"description\":\"He is a versatile software engineer with experience in a wide variety of applications\\\/services.He is enthusiastic about new projects, embracing new technologies, and getting to know people in the field of software.\",\"sameAs\":[\"http:\\\/\\\/egkatzioura.wordpress.com\\\/\",\"https:\\\/\\\/gr.linkedin.com\\\/in\\\/gkatziourasemmanouil\"],\"url\":\"https:\\\/\\\/www.javacodegeeks.com\\\/author\\\/emmanouil-gkatziouras\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"BigQuery Storage API: Avro - Java Code Geeks","description":"Interested to learn about Avro? Check our article explaining how to read Data using the Avro format.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html","og_locale":"en_US","og_type":"article","og_title":"BigQuery Storage API: Avro - Java Code Geeks","og_description":"Interested to learn about Avro? Check our article explaining how to read Data using the Avro format.","og_url":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html","og_site_name":"Java Code Geeks","article_publisher":"https:\/\/www.facebook.com\/javacodegeeks","article_published_time":"2021-06-30T07:00:00+00:00","og_image":[{"width":150,"height":150,"url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","type":"image\/jpeg"}],"author":"Emmanouil Gkatziouras","twitter_card":"summary_large_image","twitter_creator":"@javacodegeeks","twitter_site":"@javacodegeeks","twitter_misc":{"Written by":"Emmanouil Gkatziouras","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#article","isPartOf":{"@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html"},"author":{"name":"Emmanouil Gkatziouras","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/person\/5eee031b356c7682e1fd24c8297561c6"},"headline":"BigQuery Storage API: Avro","datePublished":"2021-06-30T07:00:00+00:00","mainEntityOfPage":{"@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html"},"wordCount":285,"commentCount":0,"publisher":{"@id":"https:\/\/www.javacodegeeks.com\/#organization"},"image":{"@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#primaryimage"},"thumbnailUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","keywords":["Big Data"],"articleSection":["Enterprise Java"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html","url":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html","name":"BigQuery Storage API: Avro - Java Code Geeks","isPartOf":{"@id":"https:\/\/www.javacodegeeks.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#primaryimage"},"image":{"@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#primaryimage"},"thumbnailUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","datePublished":"2021-06-30T07:00:00+00:00","description":"Interested to learn about Avro? Check our article explaining how to read Data using the Avro format.","breadcrumb":{"@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#primaryimage","url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","contentUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2012\/10\/enterprise-java-logo.jpg","width":150,"height":150,"caption":"java-interview-questions-answers"},{"@type":"BreadcrumbList","@id":"https:\/\/www.javacodegeeks.com\/2021\/06\/bigquery-storage-api-avro.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.javacodegeeks.com\/"},{"@type":"ListItem","position":2,"name":"Java","item":"https:\/\/www.javacodegeeks.com\/category\/java"},{"@type":"ListItem","position":3,"name":"Enterprise Java","item":"https:\/\/www.javacodegeeks.com\/category\/java\/enterprise-java"},{"@type":"ListItem","position":4,"name":"BigQuery Storage API: Avro"}]},{"@type":"WebSite","@id":"https:\/\/www.javacodegeeks.com\/#website","url":"https:\/\/www.javacodegeeks.com\/","name":"Java Code Geeks","description":"Java Developers Resource Center","publisher":{"@id":"https:\/\/www.javacodegeeks.com\/#organization"},"alternateName":"JCG","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.javacodegeeks.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.javacodegeeks.com\/#organization","name":"Exelixis Media P.C.","url":"https:\/\/www.javacodegeeks.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2022\/06\/exelixis-logo.png","contentUrl":"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2022\/06\/exelixis-logo.png","width":864,"height":246,"caption":"Exelixis Media P.C."},"image":{"@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/javacodegeeks","https:\/\/x.com\/javacodegeeks"]},{"@type":"Person","@id":"https:\/\/www.javacodegeeks.com\/#\/schema\/person\/5eee031b356c7682e1fd24c8297561c6","name":"Emmanouil Gkatziouras","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/5c6d031d211ab786ec335687ad6f3f076f93f47e24c92d78041d2f805ee6c291?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/5c6d031d211ab786ec335687ad6f3f076f93f47e24c92d78041d2f805ee6c291?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5c6d031d211ab786ec335687ad6f3f076f93f47e24c92d78041d2f805ee6c291?s=96&d=mm&r=g","caption":"Emmanouil Gkatziouras"},"description":"He is a versatile software engineer with experience in a wide variety of applications\/services.He is enthusiastic about new projects, embracing new technologies, and getting to know people in the field of software.","sameAs":["http:\/\/egkatzioura.wordpress.com\/","https:\/\/gr.linkedin.com\/in\/gkatziourasemmanouil"],"url":"https:\/\/www.javacodegeeks.com\/author\/emmanouil-gkatziouras"}]}},"_links":{"self":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts\/110628","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/users\/936"}],"replies":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/comments?post=110628"}],"version-history":[{"count":0,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/posts\/110628\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/media\/112"}],"wp:attachment":[{"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/media?parent=110628"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/categories?post=110628"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.javacodegeeks.com\/wp-json\/wp\/v2\/tags?post=110628"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}