Shulou(Shulou.com)05/31 Report--

This article will explain in detail what is the use of serialization in Hadoop. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Serialization concept:

Serialization: refers to the conversion of structured objects into byte streams.

Deserialization: the reverse process of serialization, that is, transferring the byte stream back to a structured object.

Java serialization: (java.io.Serializable)

Hadoop serialization:

Serialization format features:

1. Compact: efficient use of storage space.

2. Fast: the extra cost of reading and writing data is small.

3. Extensible: data in the old format can be read transparently.

4. Interoperability: supports multi-language interaction.

If Hadoop wants to implement serialization, it must implement the Writable interface.

Writable interface:

The Writable interface is a simple and efficient serialization object implemented according to DataInput and DataOutput.

Any Key of MapReduce must implement the WritableComparable interface.

Any Key and Value of MapReduce must implement the Writable interface.

Common Writable implementation classes:

The purpose of Hadoop serialization:

There are two main functions of serialization in a distributed environment: interprocess communication and permanent storage.

Communication between Hadoop nodes:

This is the end of this article on "what is the use of serialization in Hadoop". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.