Shulou(Shulou.com)06/03 Report--

This article mainly explains "what is serialization". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what serialization is.

What is serialization / deserialization serialization

Serialization is a mechanism for handling the flow of objects.

That is, the content of the object is streamed, and the data is converted into a byte stream so that it can be stored in a file or used for transmission in the network. Of course, the most commonly used is network transmission.

Deserialization

After the serialized object byte stream is obtained from the file or network, the object is reconstructed by deserialization according to the object state and description information saved in the byte stream.

The common way to serialize / deserialize: Serializable.

Who dares to say you haven't seen Serializable, give me a hit on the hanging tree that I pulled out.

The Serializable provided by JDK is slow, for example, due to the addition of serialized version number, class name and other information, resulting in larger code stream, slower speed, and so on.

So we're going to learn some fast serialization. It is not only fast, but also takes up a small amount of space.

It's a pity that the girl didn't learn such a good skill.

Serialization selection standard

Versatility: whether it can only be used for inter-java serialization / deserialization, cross-language and cross-platform

Performance: divided into space overhead and time overhead, serialized data is generally used for storage or network transmission, and its size is a very important parameter.

Of course, the time of parsing also affects the choice of serialization protocol.

Ease of use: whether the use of API is complex and affects development efficiency

Extensibility: will the property changes of the entity class lead to deserialization exceptions, which usually occur when the system is upgraded, which is not very referential?

Getting started with Kryo Serialization

Kryo is a fast serialization / deserialization tool that uses a bytecode generation mechanism (underlying relies on the ASM library), so it runs faster.

The result of Kryo serialization is its custom and unique format, which is no longer JSON or other existing general format.

Moreover, the serialized result is binary (that is, byte []; JSON is essentially the string String).

Binary data is obviously smaller and faster to serialize and deserialize.

Kryo is generally only used for serialization (then as a cache, or landed on a storage device) and deserialization, rather than for data exchange between multiple systems or even languages-currently kryo is only implemented by java.

Storage tools like Redis can store binary data securely, so Kryo can be used instead of JDK serialization in general projects.

Usage scenarios: (data exchange or data persistence) such as using kryo to serialize objects into byte arrays and send them to message queues or nosql scenarios such as redis.

Pom configuration

Com.esotericsoftware

Kryo

4.0.1

It should be noted that because kryo uses a higher version of asm, it may conflict with the asm on which the business currently depends, which is a common problem. Just change the dependency to:

Com.esotericsoftware

Kryo-shaded

4.0.2

Code implementation

Note: because kryo is not thread-safe, for the use of multithreading, it is necessary to make a simple encapsulation design for kryo, so that serialization and deserialization can be used safely in multithreading.

Interface / * *

* serialization tool (this interface is called by the program to serialize / deserialize between objbyte [])

, /

Public interface Serializer {

/ * *

* Serialization

, /

Public void serialize (Object tjinbyte [] bytes)

/ * *

* Serialization

, /

Public void serialize (Object obj, byte [] bytes, int offset, int count)

/ * *

* deserialization

, /

Public T deserialize (byte [] bytes)

/ * *

* deserialization

, /

Public T deserialize (byte [] bytes, int offset, int count)

}

Implementation class / * *

* serialization / deserialization tool based on kyro

, /

Public class kryoSerializer implements Serializer {

/ / because kryo is not thread-safe, each thread uses a separate kryo

Final ThreadLocal kryoLocal = new ThreadLocal () {

@ Override

Protected Kryo initialValue () {

Kryo kryo = new Kryo ()

Kryo.register (ct, new BeanSerializer (kryo, ct))

Return kryo

}

}

/ / serialize threadlocal

Final ThreadLocal outputLocal = new ThreadLocal ()

/ / deserialize threadlocal

Final ThreadLocal inputLocal = new ThreadLocal ()

Private Class ct = null

Public kryoSerializer (Class ct) {

This.ct = ct

}

/ * *

* Serialization

, /

@ Override

Public void serialize (Object obj, byte [] bytes) {

Kryo kryo = getKryo ()

Output output = getOutput (bytes)

Kryo.writeObjectOrNull (output, obj, obj.getClass ())

Output.flush ()

}

/ * *

* Serialization

, /

@ Override

Public void serialize (Object obj, byte [] bytes, int offset, int count) {

Kryo kryo = getKryo ()

Output output = getOutput (bytes, offset, count)

Kryo.writeObjectOrNull (output, obj, obj.getClass ())

Output.flush ()

}

/ * *

* deserialization

, /

@ SuppressWarnings ("unchecked")

@ Override

Public T deserialize (byte [] bytes, int offset, int count) {

Kryo kryo = getKryo ()

Input input = getInput (bytes, offset, count)

Return (T) kryo.readObjectOrNull (input, ct)

}

/ * *

* deserialization

, /

@ Override

Public T deserialize (byte [] bytes) {

Return deserialize (bytes, 0, bytes.length)

}

/ * *

* obtain kryo

, /

Private Kryo getKryo () {

Return kryoLocal.get ()

}

/ * *

* get Output and set the initial array

, /

Private Output getOutput (byte [] bytes) {

Output output = null

If ((output = outputLocal.get ()) = = null) {

Output = new Output ()

OutputLocal.set (output)

}

If (bytes! = null) {

Output.setBuffer (bytes)

}

Return output

}

/ * *

* obtain Output

, /

Private Output getOutput (byte [] bytes, int offset, int count) {

Output output = null

If ((output = outputLocal.get ()) = = null) {

Output = new Output ()

OutputLocal.set (output)

}

If (bytes! = null) {

Output.writeBytes (bytes, offset, count)

}

Return output

}

/ * *

* obtain Input

, /

Private Input getInput (byte [] bytes, int offset, int count) {

Input input = null

If ((input = inputLocal.get ()) = = null) {

Input = new Input ()

InputLocal.set (input)

}

If (bytes! = null) {

Input.setBuffer (bytes, offset, count)

}

Return input

}

Public Class getCt () {

Return ct

}

Public void setCt (Class ct) {

This.ct = ct

}

IO of Kryo

Kryo is committed to easy-to-use API, the main core of the serialization process are Kryo, Output, Input.

Output and Input are IO of Kryo, and they support serialized dest and deserialized source in the form of byte array or stream.

When writing out in stream form, you need to close these Output and Input.

When written out, when OutputDe buffer is full, it will be flush bytes to stream.

When writing, the bytes is fetched from the stream to the Input buffer and deserialized when it is full.

Registration of Kryo

Like many other serialization frameworks, Kryo provides a way to serialize object classes registered in order to provide performance and reduce the size of serialization results.

When registered, an int ID is generated for the serialized class, and later the type is uniquely identified by int ID when serialized.

The registration method is as follows:

Kryo.register (SomeClass.class)

Alternatively, you can explicitly specify the int ID of the registered class, but the ID must be greater than or equal to 0. If not, an orderly int ID generation will be maintained internally using int++.

Kryo.register (SomeClass.class, 1)

Object reference aspect

This is support for circular references and can effectively prevent stack memory overflows, which is turned on by default by kryo.

When you are sure that no circular reference will occur, you can improve some performance by turning off circular reference detection with the following code, but it is not highly recommended

Kryo kryo = new Kryo ()

Kryo.setReferences (false)

The way Kryo reads and writes

If the serialized object type is unknown and may be empty:

Kryo.writeClassAndObject (output, object)

/ /...

Object object = kryo.readClassAndObject (input)

If (object instanceof SomeClass) {

/ /...

}

If the object type is known and may be empty:

Kryo.writeObjectOrNull (output, someObject)

/ /...

SomeClass someObject = kryo.readObjectOrNull (input, SomeClass.class)

If the object type is known and cannot be empty:

Kryo.writeObject (output, someObject)

/ /...

SomeClass someObject = kryo.readObject (input, SomeClass.class)

Serialization thread safety

Kryo is thread-unsafe by default, and there are two ways to solve it:

One is to store an instance for a thread through Threadlocal:

Private static final ThreadLocal kryoThreadLocal = new ThreadLocal () {

Protected Kryo initialValue () {

Kryo kryo = new Kryo ()

/ / A series of configuration information can be added here

Return kryo

}

}

The other is through KryoPool, which is also better than ThreadLocal in performance:

Public KryoPool createPool () {

Return new KryoPool.Builder (()-> {

Kryo kryo = new Kryo ()

/ / A series of configurations can also be made here. You can implement the KryoFactory interface to satisfy the dynamic registration and abstract the class.

Return kryo

}). SoftReferences (). Build ()

}

Kryo supports serialized type booleanBooleanbyteBytecharCharactershortShortintIntegerlongLongfloatFloatdoubleDoublebyte [] StringBigIntegerBigDecimalCollectionDateCollections.emptyListCollections.singletonMapStringBuilderTreeMapCollections.emptyMapCollections.emptySetKryoSerializableStringBufferClassCollections.singletonListCollections.singletonMapCurrencyCalendarTimeZoneEnumEnumSet

The advantages and disadvantages of kryo are summarized. The performance of serialization is very high. The small size of serialization results in the disadvantage of easy-to-use API.

Cross-language support is more complex

Addition / deletion / modification of object fields is not supported

If you change the fields of an object and then deserialize from the bytes serialized before the change, an error will occur.

Of course, if you want support for Add, Remove, and so on, you can use other extensions to FieldSerializer, such as TaggedFieldSerializer, VersionFieldSerializer, and so on.

Performance comparison between kryo and JDK Simple class import java.io.Serializable

Import java.util.Map

Public class Simple implements Serializable {

Private static final long serialVersionUID =-4914434736682797743L

Private String name

Private int age

Private Map map

Public Simple () {

}

Public Simple (String name,int age,Map map) {

This.name = name

This.age = age

This.map = map

}

Public String getName () {

Return name

}

Public void setName (String name) {

This.name = name

}

Public int getAge () {

Return age

}

Public void setAge (int age) {

This.age = age

}

Public Map getMap () {

Return map

}

Public void setMap (Map map) {

This.map = map

}

}

JDK performance Test import java.io.FileInputStream

Import java.io.FileNotFoundException

Import java.io.FileOutputStream

Import java.io.IOException

Import java.io.ObjectInputStream

Import java.io.ObjectOutputStream

Import java.util.HashMap

Import java.util.Map

Public class OriginalSerializable {

Public static void main (String [] args) throws IOException, ClassNotFoundException {

Long start = System.currentTimeMillis ()

SetSerializableObject ()

System.out.println ("java native serialization time:" + (System.currentTimeMillis ()-start) + "ms")

Start = System.currentTimeMillis ()

GetSerializableObject ()

System.out.println ("java native deserialization time:" + (System.currentTimeMillis ()-start) + "ms")

}

Public static void setSerializableObject () throws IOException {

FileOutputStream fo = new FileOutputStream ("D:/file2.bin")

ObjectOutputStream so = new ObjectOutputStream (fo)

For (int I = 0; I < 1000000; iTunes +) {

Map map = new HashMap (2)

Map.put ("zhang0", I)

Map.put ("zhang1", I)

So.writeObject (new Simple ("zhang" + I, (iTun1), map))

}

So.flush ()

So.close ()

}

Public static void getSerializableObject () {

FileInputStream fi

Try {

Fi = new FileInputStream ("D:/file2.bin")

ObjectInputStream si = new ObjectInputStream (fi)

Simple simple = null

While ((simple= (Simple) si.readObject ())! = null) {

/ / System.out.println (simple.getAge () + "+ simple.getName ())

}

Fi.close ()

Si.close ()

} catch (FileNotFoundException e) {

E.printStackTrace ()

} catch (IOException e) {

/ / e.printStackTrace ()

} catch (ClassNotFoundException e) {

E.printStackTrace ()

}

}

}

Kryo performance Test import java.io.FileInputStream

Import java.io.FileNotFoundException

Import java.io.FileOutputStream

Import java.io.IOException

Import java.util.HashMap

Import java.util.Map

Import org.objenesis.strategy.StdInstantiatorStrategy

Import com.esotericsoftware.kryo.Kryo

Import com.esotericsoftware.kryo.KryoException

Import com.esotericsoftware.kryo.io.Input

Import com.esotericsoftware.kryo.io.Output

Public class KyroSerializable {

Public static void main (String [] args) throws IOException {

Long start = System.currentTimeMillis ()

SetSerializableObject ()

System.out.println ("Kryo serialization time:" + (System.currentTimeMillis ()-start) + "ms")

Start = System.currentTimeMillis ()

GetSerializableObject ()

System.out.println ("Kryo deserialization time:" + (System.currentTimeMillis ()-start) + "ms")

}

Public static void setSerializableObject () throws FileNotFoundException {

Kryo kryo = new Kryo ()

Kryo.setReferences (false)

Kryo.setRegistrationRequired (false)

Kryo.setInstantiatorStrategy (new StdInstantiatorStrategy ())

Kryo.register (Simple.class)

Output output = new Output (new FileOutputStream ("D:/file1.bin"))

For (int I = 0; I < 1000000; iTunes +) {

Map map = new HashMap (2)

Map.put ("zhang0", I)

Map.put ("zhang1", I)

Kryo.writeObject (output, new Simple ("zhang" + I, (iTun1), map))

}

Output.flush ()

Output.close ()

}

Public static void getSerializableObject () {

Kryo kryo = new Kryo ()

Kryo.setReferences (false)

Kryo.setRegistrationRequired (false)

Kryo.setInstantiatorStrategy (new StdInstantiatorStrategy ())

Input input

Try {

Input = new Input (new FileInputStream ("D:/file1.bin"))

Simple simple = null

While ((simple=kryo.readObject (input, Simple.class))! = null) {

/ / System.out.println (simple.getAge () + "+ simple.getName () +" + simple.getMap () .toString ())

}

Input.close ()

} catch (FileNotFoundException e) {

E.printStackTrace ()

} catch (KryoException e) {

}

}

}

Test results JDKjava native serialization time: 6614 msjava native deserialization time: 8609 mskryoKryo serialization time: 8609 msKryo deserialization time: 307 ms thank you for reading, the above is the content of "what is serialization", after the study of this article, I believe you have a deeper understanding of what is serialization, the specific use of the need for practice to verify. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.