Apache Avro serialization and deserialization (Java implementation)

Like two exchanges, we need to find a language that can understand each other, Mandarin at home and English abroad, and the communication between the two processes also needs to find a data format that everyone can understand. Simple such as JSON, XML, which is a self-descriptive format, XML has a Schema definition, but there is no formal JSON Schema specification. In the case of efficiency, the plain text data exchange format can not meet the requirements, so there are binary Google Protobuf and Apache Avro. In the ecology of Apache like Hadoop, Kafka naturally chooses Avro.

Avro supports multiple languages, such as C, client, Java, PHP, Python and Ruby. It uses JSON to define Schema. Through tools, Schema can generate data objects in the corresponding language, such as Java's avro-tools.jar. In this way, objects can be exchanged transparently across processes and languages.

This article experiences serialization and deserialization of Avro data format in Java environment.

Avro Schema file is the communication protocol between data production and consumption; we can generate corresponding Java objects by Schema and exchange them with specific Java objects, or we can exchange them with specific Java objects, or we can interact with GenericRecord without generating Java objects. In order to manipulate data simply, we usually use the former method, that is, to generate specific data transfer objects. Read the full text > >