Shulou(Shulou.com)06/02 Report--

This article mainly introduces "how to solve the FastJson long overflow problem". In the daily operation, I believe that many people have doubts about how to solve the FastJson long overflow problem. The editor consulted all kinds of data and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubt of "how to solve the FastJson long overflow problem"! Next, please follow the editor to study!

Background

FastJson (version 1.1.48.android) was connected to the strict selection project earlier (at the end of 2015). With the development of business, the values of individual request fields are beyond the range of int, exposing this overflow problem in the current version of FastJson.

Question 1. Error converting object to json string

In the network request response body data parsing, in order to map the json data to the object, the JSON.toJSONString () method is called, and there is an long data overflow and an error occurs in the data processing here.

Object result = isArray? JSON.parseArray (jsonObj.getJSONArray ("data"). ToJSONString (), modelCls): jsonObj.getObject ("data", modelCls); parseResult.setResult (result)

Array object mapping code looks a little strange, performance will be a bit wasteful, because there are not many interfaces involved and did not expect a better mapping, did not change, light spray.

Question 2. Object petabyte array error

The network requests the request body transbyte array procedure, calls the JSON.toJSONBytes interface, and an overflow occurs when there is a long field in the mBodyMap.

@ Overridepublic byte [] getContenteAsBytes () {/ / prevent repeated conversion of if (mBody = = null & & mBodyMap.size ()! = 0) {mBody = JSON.toJSONBytes (mBodyMap);} return mBody;} / / mBodyMap data content Map mBodyMap = new HashMap (); mBodyMap.put ("shipAddressId", 117645003002L);... InvoiceSubmitVO submit = new InvoiceSubmitVO (); submit.shipAddressId = 117645003002LbomachmBodyMap.put ("invoiceSubmite", submit) / / the backend receives data content {"invoiceSubmite": {"shipAddressId": 117645003002,...}, "shipAddressId": 1680886010,...}

The same two long fields shipAddressId, one parsed normally and the other overflowed.

1 problem analysis

Write test code:

Public static void test () {JSONObject jsonObj = new JSONObject (); jsonObj.put ("_ int", 100); jsonObj.put ("_ long", 1234567890120L); jsonObj.put ("_ string", "string"); String json0 = JSON.toJSONString (jsonObj); Log.i ("TEST0", "json0 =" + json0); TestModel model = new TestModel (); String json1 = JSON.toJSONString (model) Log.i ("TEST1", "json1 =" + json1);} private static class TestModel {public int _ int = 100; public long _ long = 1234567890120L; public String _ string = "string";}

Content output

I/TEST0: json0 = {"_ int": 100, "_ long": 1912276168, "_ string": "string"}

I/TEST1: json1 = {"_ int": 100," _ long ": 1234567890120," _ string ":" string "}

You can find that an overflow occurs during long value parsing in regular map, while the long field in the class object parses normally.

View the source code:

/ / JSON.javapublic String toJSONString () {SerializeWriter out = new SerializeWriter ((Writer) null, DEFAULT_GENERATE_FEATURE, SerializerFeature.EMPTY); String var2; try {(new JSONSerializer (out, SerializeConfig.globalInstance)) .write (this); var2 = out.toString ();} finally {out.close ();} return var2;} public static final String toJSONString (Object object, SerializerFeature...) Features) {SerializeWriter out = new SerializeWriter ((Writer) null, DEFAULT_GENERATE_FEATURE, features); String var4; try {JSONSerializer serializer = new JSONSerializer (out, SerializeConfig.globalInstance); serializer.write (object); var4 = out.toString ();} finally {out.close ();} return var4;}

As you can see, the final calls are all JSONSerializer.write methods.

/ / JSONSerializer.javapublic final void write (Object object) {... ObjectSerializer writer = this.getObjectWriter (clazz);...} public ObjectSerializer getObjectWriter (Class clazz) {ObjectSerializer writer = (ObjectSerializer) this.config.get (clazz); if (writer = = null) {if (Map.class.isAssignableFrom (clazz)) {this.config.put (clazz, MapCodec.instance);}. Else {Class superClass; if (! clazz.isEnum () & & (superClass = clazz.getSuperclass ()) = = null | | superClass = = Object.class | |! superClass.isEnum ()) {if (clazz.isArray ()) {...}. Else {... This.config.put (clazz, this.config.createJavaBeanSerializer (clazz));} else {...}} writer = (ObjectSerializer) this.config.get (clazz);} return writer;}

You can see that Map objects are processed using MapCodec, and ordinary Class objects are processed using JavaBeanSerializer.

MapCodec handles serialized write logic:

Class clazz = value.getClass (); if (clazz = = preClazz) {preWriter.write (serializer, value, entryKey, (Type) null);} else {preClazz = clazz; preWriter = serializer.getObjectWriter (clazz); preWriter.write (serializer, value, entryKey, (Type) null);}

The serialization class for the long field can be found to be the IntegerCodec class.

/ / SerializeConfig.javapublic SerializeConfig (int tableSize) {super (tableSize);... This.put (Byte.class, IntegerCodec.instance); this.put (Short.class, IntegerCodec.instance); this.put (Integer.class, IntegerCodec.instance); this.put (Long.class, IntegerCodec.instance);...}

Looking at the IntegerCodec source code, you can see the cause of the problem: due to the previous fieldType write dead null input, resulting in the last write is out.writeInt (value.intValue ()); there is an overflow.

\ IntegerCodec.javapublic void write (JSONSerializer serializer, Object object, Object fieldName, Type fieldType) throws IOException {SerializeWriter out = serializer.out; Number value = (Number) object; if (value = = null) {...} else {if (fieldType! = Long.TYPE & & fieldType! = Long.class) {out.writeInt (value.intValue ());} else {out.writeLong (value.longValue ()) }}}

When the long value is a class field, look at the JavaBeanSerializer.write method and it is indeed written correctly.

/ / JavaBeanSerializer.javapublic void write (JSONSerializer serializer, Object object, Object fieldName, Type fieldType) throws IOException {... If (valueGot & &! propertyValueGot) {if (fieldClass! = Integer.TYPE) {if (fieldClass = = Long.TYPE) {serializer.out.writeLong (propertyValueLong) } else if (fieldClass = = Boolean.TYPE) {...}} else if (propertyValueInt = =-2147483648) {...}...} 2 problem handling 2.1 use ValueFilter processing

For JSON.toJSONString, you can call the following method and set ValueFilter,FastJson to call the ValueFilter.process method before writing a string, and modify the data type of value in this method, thus bypassing the IntegerCodec writing logic with bug

Public static final String toJSONString (Object object, SerializeFilter filter, SerializerFeature... Features) public interface ValueFilter extends SerializeFilter {Object process (Object object, String name, Object value);} String json1 = JSON.toJSONString (map, new ValueFilter () {@ Override public Object process (Object object, String name, Object value) {if (value instanceof Long) {return new BigInteger (String.valueOf (value));} return value;}})

Here, the long type is changed to the BigInteger class, and the value remains the same. Finally, the write operation is given to BigDecimalCodec.

2.2 replace the problematic IntegerCodec

Looking at the SerializeConfig source code, you can find that all ObjectSerializer subclasses are integrated in SerializeConfig and use globalInstance internally.

Public class SerializeConfig extends IdentityHashMap {public static final SerializeConfig globalInstance = new SerializeConfig (); public ObjectSerializer createJavaBeanSerializer (Class clazz) {return new JavaBeanSerializer (clazz);} public static final SerializeConfig getGlobalInstance () {return globalInstance;} public SerializeConfig () {this (1024);}.}

To do this, you can replace IntegerCodec when Application initializes.

/ / MyApplication.java@Overridepublic void onCreate () {super.onCreate (); SerializeConfig.getGlobalInstance () .put (Byte.class, NewIntegerCodec.instance); SerializeConfig.getGlobalInstance () .put (Short.class, NewIntegerCodec.instance); SerializeConfig.getGlobalInstance () .put (Integer.class, NewIntegerCodec.instance); SerializeConfig.getGlobalInstance () .put (Long.class, NewIntegerCodec.instance);}

Since the SerializeWriter.features field used by NewIntegerCodec is protected, you need to place this class under the com.alibaba.fastjson.serializer package name.

2.3 upgrade FastJson

The latest version is 1.1.68.android (July 16, 2018.08). If you look at the IntegerCodec class, you can find that bug has been fixed

/ / IntegerCodec.javapublic void write (JSONSerializer serializer, Object object, Object fieldName, Type fieldType) throws IOException {... If (object instanceof Long) {out.writeLong (value.longValue ());} else {out.writeInt (value.intValue ());}.}

Taken together, it seems that the best solution is to upgrade FastJson, but other holes are triggered during the upgrade process.

Because of the fields defined on nei, part of the numeric variable definition type is Number, and the same basic type, the backend field part uses the boxing type, which is inconsistent with the client definition type (such as server definition Integer, client definition int).

Public static void test () {String json = "{\" code\ ": 200,\" msg\ ":\"\ ",\" data\ ": {\" _ long\ ": 1234567890120,\" _ string\ ":\" string\ ",\" _ int\ ": null}}"; JSONObject jsonObj = JSONObject.parseObject (json); AndroidModel AndroidModel = jsonObj.getObject ("data", AndroidModel.class);} private static class AndroidModel {public int _ int = 100; public long _ long = 1234567890L Public String _ string = "string";}

As for the test code above, there is no problem with this definition in earlier versions, and even if the _ int field is null, the client can resolve to the initial value of 100. After upgrading FastJson, json string parsing will crash.

/ / JavaBeanDeserializer.javapublic Object createInstance (Map map, ParserConfig config) / / throws IllegalAccessException, IllegalArgumentException, InvocationTargetException {Object object = null; if (beanInfo.creatorConstructor = = null) {object = createInstance (null, clazz); for (Map.Entry entry: map.entrySet ()) {. If (method! = null) {Type paramType = method.getGenericParameterTypes () [0]; value = TypeUtils.cast (value, paramType, config); method.invoke (object, new Object [] {value});} else {Field field = fieldDeser.fieldInfo.field; Type paramType = fieldDeser.fieldInfo.fieldType Value = TypeUtils.cast (value, paramType, config); field.set (object, value);} return object;}...} TypeUtils.java@SuppressWarnings ("unchecked") public static final T cast (Object obj, Type type, ParserConfig mapping) {if (obj = = null) {return null;}...}

If you look at the source code, you can find that when the value in the json string is null, TypeUtils.cast also returns null directly, while when field.set (object, value); is executed, the null is forcibly set to the int field, and an IllegalArgumentException exception occurs. Because of this exception, the client cannot upgrade FastJson.

At this point, the study on "how to solve the FastJson long overflow problem" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.