Shulou(Shulou.com)06/03 Report--

Lu Chunli's work notes, who said that programmers should not have literary style?

Serialization and deserialization are mainly used in process building communication and permanent storage in distributed data processing.

Serialization is the process of converting structured data into byte streams that can be transferred over the network or written to disk for permanent storage; deserialization is the inverse process of converting byte streams back to structured objects.

The inter-process communication of Hadoop system nodes is realized by RPC. Hadoop does not adopt the serialization mechanism of Java, but defines two serialization-related interfaces: Writable and Comparable, which abstracts a WritableComparable interface.

In Hadoop, the Writable interface defines two methods:

Write data to a DataOutput stream in binary format

DataInput stream that reads data from binary format

Package org.apache.hadoop.io;public interface Writable {/ * * Serialize the fields of this object to out. * / void write (DataOutput out) throws IOException; / * * Deserialize the fields of this object from in. * / void readFields (DataInput in) throws IOException;}

The structure of the Writable interface in Hadoop is

Writable WritableComparable IntWritable int (fixed length) VintWritable int (variable length) BooleanWritable boolean ByteWritable byte (single byte) ShortWritable short FloatWritable float DoubleWritable double LongWritable long ( Fixed length) VlongWirtable long (variable length) Text is a Writable class for UTF-8 sequences It is generally considered to be equivalent to java.lang.String BytesWritable byte (byte sequence) ArrayWritable array TwoDArrayWritable two-dimensional array MapWritable implements Map SortedMapWritable implements SortedMap

The WritableComparable interface provides the function of type comparison, which is critical to MapReduce.

The package org.apache.hadoop.io;public interface WritableComparable extends Writable, Comparable {} # WritableComparator class is a general implementation. 1. Provides a default implementation of the original compare () method; 2. Act as the factory (registered Writable implementation) package org.apache.hadoop.io;public class WritableComparator implements RawComparator, Configurable {/ * * For backwards compatibility for RawComparator instances. * * / public static WritableComparator get (Class

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.