\n$ mkdir \/opt\/gradle\n$ unzip -d \/opt\/gradle gradle-8.10.2-bin.zip\n$ ls \/opt\/gradle\/gradle-8.10.2\n\n<\/pre>\nAfter setting up, you can run the gradle command to check the version. Gradle command is shown below:<\/p>\n
\ngradle -v\n<\/pre>\nThe output of the executed command is shown below:<\/p>\n
\nbhagvanarch@Bhagvans-MacBook-Air avro_oct_20 % gradle -v \n\n------------------------------------------------------------\nGradle 8.10.2\n------------------------------------------------------------\n\nBuild time: 2024-09-23 21:28:39 UTC\nRevision: 415adb9e06a516c44b391edff552fd42139443f7\n\nKotlin: 1.9.24\nGroovy: 3.0.22\nAnt: Apache Ant(TM) version 1.10.14 compiled on August 16 2023\nLauncher JVM: 23 (Homebrew 23)\nDaemon JVM: \/opt\/homebrew\/Cellar\/openjdk\/23\/libexec\/openjdk.jdk\/Contents\/Home (no JDK specified, using current Java home)\nOS: Mac OS X 14.6.1 aarch64\n\nbhagvanarch@Bhagvans-MacBook-Air avro_oct_20 %\n<\/pre>\n3. Java Classes Generation<\/h2>\n
Apache Avro is a widely used data serialization framework that efficiently stores and exchanges large datasets across diverse programming languages. It’s trendy in big data ecosystems, often seen in conjunction with Hadoop and Kafka. One of the key features of Avro is its ability to define schemas in JSON format, which makes it both human-readable and machine-processable. Avro schemas ensure that the data structure is clearly defined, facilitating smooth data communication between services.
<\/div> <\/div><\/p>\n
At its core, Avro provides a compact and fast binary data format that reduces storage overhead and accelerates data processing. It uses JSON to define the data structure, known as a schema, and supports schema evolution, allowing systems to handle data encoded with older versions of the schema. This feature is crucial for maintaining compatibility in distributed systems where updating all components simultaneously is impractical.<\/p>\n
Generating Java classes from Avro schemas automates the serialization and deserialization process, providing a type-safe way to handle data. This not only speeds up development but also reduces errors associated with manual serialization. Avro’s binary format ensures that the data is both compact and easy to process, making it ideal for high-performance applications.<\/p>\n
3.1 Steps to Generate Java Classes Using Apache Avro<\/h3>\n3.1.1 Define the Avro Schema<\/h4>\n
The first step is to define your Avro schema in a ‘.avsc’ file. This file is written in JSON and specifies the structure of the data, including the fields and their data types. Here’s an example of an Avro schema for a simple user record. Let us look at the JSON file:<\/p>\n
\n{\n \"namespace\": \"org.javacodegeeks\",\n\u00a0 \"type\": \"record\",\n\n\u00a0 \"name\": \"User\",\n\n\u00a0 \"namespace\": \"com.example\",\n\n\u00a0 \"fields\": [\n\n\u00a0\u00a0\u00a0 {\"name\": \"id\", \"type\": \"int\"},\n\n\u00a0\u00a0\u00a0 {\"name\": \"name\", \"type\": \"string\"},\n\n\u00a0\u00a0\u00a0 {\"name\": \"email\", \"type\": [\"null\", \"string\"], \"default\": null}\n\n\u00a0 ]\n\n}\n\n\n<\/pre>\nIn this schema, we define a ‘User’ record with three fields: ‘id’, ‘name’, and ’email’. The ’email’ field is nullable, indicated by the union type ‘[“null”, “string”]’.<\/p>\n
3.1.2 \u00a0Install Apache Avro Tools<\/h4>\n
Next, you need to install the Apache Avro tools. These tools include a compiler that generates Java classes from Avro schemas. You can download the Avro tools JAR file from the official Apache Avro website or add it to your build configuration if you’re using a build tool like Gradle or Maven.<\/p>\n
For Gradle, add the following dependency to your ‘build.gradle’ file: <\/p>\n
\nbuildscript {\n dependencies {\n classpath libs.avro.tools\n }\n}\n\nplugins {\n id 'java'\n alias libs.plugins.avro\n}\n\nrepositories {\n mavenCentral()\n}\n\ndependencies {\n implementation libs.avro\n \/\/ Use JUnit Jupiter for testing.\n testImplementation libs.junit.jupiter\n\n testRuntimeOnly 'org.junit.platform:junit-platform-launcher'\n\n}\n\n<\/pre>\n <\/strong>Update your libs.toml file which is found under gradle folder.<\/p>\n \n# This file was generated by the Gradle 'init' task.\n# https:\/\/docs.gradle.org\/current\/userguide\/platforms.html#sub::toml-dependencies-format\n\n[versions]\njunit-jupiter = \"5.10.0\"\navro = \"1.11.0\"\n\n[libraries]\njunit-jupiter = { module = \"org.junit.jupiter:junit-jupiter\", version.ref = \"junit-jupiter\" }\navro = {module = \"org.apache.avro:avro\", version.ref = \"avro\"}\navro-tools = {module = \"org.apache.avro:avro-tools\", version.ref = \"avro\"}\n\n[plugins]\navro = { id = \"com.github.davidmc24.gradle.plugin.avro\", version = \"1.9.1\" }\n\n\n<\/pre>\n3.1.3 Generate Java Classes<\/h4>\n
Avro Tools<\/strong><\/p>\nWith the schema defined and the Avro tools installed, you can now generate the Java classes. This can be done using the Avro tools command-line interface or through your build tool.<\/p>\n
For command-line usage, run the following command:<\/p>\n
\njava -jar avro-tools-1.10.2.jar compile schema user.avsc .\n<\/pre>\nThis command tells the Avro compiler to generate Java classes from the ‘user.avsc’ file and place them in the current directory.<\/p>\n
Gradle<\/strong><\/p>\nIf you’re using Gradle, you can run the build task<\/p>\n
\n.\/gradlew build\n<\/pre>\nRunning ‘.\/gradlew build’ will then generate the Java classes. You can find them under build\/generated-main-avro-java<\/em><\/p>\n3.1.4 \u00a0Implement Serialization and Deserialization<\/h4>\n
Once the Java classes are generated, you can use them to serialize and deserialize data. Avro provides specific classes to handle these operations, such as ‘DatumWriter’ and ‘DatumReader’ because some times we need to read and write data.<\/p>\n
Here’s an example of how to serialize a ‘User’ object to a file:<\/p>\n
\nimport org.apache.avro.file.DataFileWriter;\n\nimport org.apache.avro.generic.GenericDatumWriter;\n\nimport org.apache.avro.specific.SpecificDatumWriter;\n\nimport org.javacodegeeks.User;\n\npublic class AvroWriterExample {\n\n\u00a0\u00a0\u00a0 public static void main(String[] args) {\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 User user = new User();\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 user.setId(1);\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 user.setName(\"John Doe\");\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 user.setEmail(\"john.doe@example.com\");\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 DatumWriter userDatumWriter = new SpecificDatumWriter(User.class);\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 try (DataFileWriter dataFileWriter = new DataFileWriter(userDatumWriter)) {\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 dataFileWriter.create(user.getSchema(), new File(\"users.avro\"));\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 dataFileWriter.append(user);\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 } catch (IOException e) {\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 e.printStackTrace();\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 }\n\n\u00a0\u00a0\u00a0 }\n\n}\n<\/pre>\nPlease ensure you have the jars (avro and avro tools) downloaded.This example can be compiled and executed as below:<\/p>\n
\n javac -cp \".\/*\" AvroWriterExample.java\n java -classpath '.:.\/*' AvroWriterExample\n\n<\/pre>\nIn the above writer example, we create a ‘User’ object and serialize it to a file named ‘users.avro’.<\/p>\n
To deserialize the data, you can use the ‘DatumReader’ class because some times you need to read data:<\/p>\n
\nimport org.apache.avro.file.DataFileReader;\n\nimport org.apache.avro.generic.GenericDatumReader;\n\nimport org.apache.avro.io.DatumReader;\n\nimport org.apache.avro.specific.SpecificDatumReader;\n\nimport org.javacodegeeks.avro.User;\n\nimport java.io.*;\n\npublic class AvroReaderExample {\n\n public static void main(String[] args) {\n\n File file = new File(\"users.avro\");\n\n DatumReader userDatumReader = new SpecificDatumReader(User.class);\n\n try (DataFileReader dataFileReader = new DataFileReader(file, userDatumReader)) {\n\n User user = null;\n\n while (dataFileReader.hasNext()) {\n\n user = dataFileReader.next(user);\n\n System.out.println(user);\n\n }\n\n } catch (IOException e) {\n\n e.printStackTrace();\n\n }\n\n }\n}\n<\/pre>\nThis code reads the ‘users.avro’ file and prints each ‘User’ object to the console when the code is executed using the commands below.<\/p>\n
\n javac -cp \".\/*\" AvroReaderExample.java\n java -classpath '.:.\/*' AvroReaderExample\n\n<\/pre>\nThe output of the executed code above is shown below:<\/p>\n
\nbhagvanarch@Bhagvans-MacBook-Air avro_example_oct_20 % java -classpath '.:.\/*' AvroReaderExample\n{\"id\": 1, \"name\": \"John Doe\", \"email\": \"john.doe@example.com\"}\nbhagvanarch@Bhagvans-MacBook-Air avro_example_oct_20 % \n\n<\/pre>\n3.1.5 \u00a0Schema Evolution and Compatibility<\/h4>\n
One of the powerful features of Avro is schema evolution, which allows for backward and forward compatibility when schemas change over time. This means you can add new fields, remove fields, or change field types without breaking existing data.<\/p>\n
For example, if you add a new field to the ‘User’ schema then you can modify the object definition:<\/p>\n
\n{\n\n\u00a0 \"type\": \"record\",\n\n\u00a0 \"name\": \"User\",\n\n\u00a0 \"namespace\": \"com.example\",\n\n\u00a0 \"fields\": [\n\n\u00a0\u00a0\u00a0 {\"name\": \"id\", \"type\": \"int\"},\n\n\u00a0\u00a0\u00a0 {\"name\": \"name\", \"type\": \"string\"},\n\n\u00a0\u00a0\u00a0 {\"name\": \"email\", \"type\": [\"null\", \"string\"], \"default\": null},\n\n\u00a0\u00a0\u00a0 {\"name\": \"phone\", \"type\": [\"null\", \"string\"], \"default\": null}\n\n\u00a0 ]\n\n}\n\n<\/pre>\nThis new schema is still compatible with data written using the old schema because Avro supports default values for new fields.<\/p>\n
Generating Java classes from Avro schemas is a straightforward process that enhances the efficiency and reliability of data serialization in Java applications. By defining your data structure in a schema, using the Avro tools to generate classes, and implementing serialization and deserialization logic, you can seamlessly integrate Avro into your projects. The ability to evolve schemas without breaking compatibility makes Avro a robust choice for applications that require flexible and efficient data handling without the overhead of roundtrip engineering.<\/p>\n
5. Conclusion<\/h2>\n
In conclusion, generating Java classes from Avro schemas using Gradle provides a seamless and efficient workflow for integrating Avro serialization into your Java projects. By defining your data structures in Avro schema files and leveraging Gradle\u2019s build automation capabilities, you streamline the process of generating the necessary Java classes.<\/p>\n
This approach enhances development speed and ensures type safety and consistency across your application. Furthermore, the ease of updating schemas and regenerating classes means you can adapt to evolving data requirements without significant overhead. Embracing this methodology equips you with a robust solution for handling data serialization in a scalable and maintainable manner, ultimately contributing to more efficient and reliable software development.<\/p>\n
![\"\"](\"https:\/\/www.javacodegeeks.com\/wp-content\/uploads\/2024\/10\/avro.png\")
<\/a>