Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces in detail "how to solve the OOM problems caused by the use of okhttp in android". The content is detailed, the steps are clear, and the details are handled properly. I hope that this article "how to solve the OOM problems caused by the use of okhttp in android" can help you solve your doubts.

There is a problem: all requests need to be signed and verified to prevent brushing; incoming requests url and body generate a text string to be transmitted to the server as a header; there is a ready-made signature verification method String doSignature (String url, byte [] body); the current network library is based on com.squareup.okhttp3:okhttp:3.14.2.

This is very simple, of course, just write an interceptor and pass in the url and body of the request object. So there are:

Public class SignInterceptor implements Interceptor {@ NonNull @ Override public Response intercept (@ NonNull Chain chain) throws IOException {Request request = chain.request (); RequestBody body = request.body (); byte [] bodyBytes = null; if (body! = null) {final Buffer buffer = new Buffer (); body.writeTo (buffer); bodyBytes = buffer.readByteArray ();} Request.Builder builder = request.newBuilder (); HttpUrl oldUrl = request.url () Final String url = oldUrl.toString (); final String signed = doSignature (url, bodyBytes); if (! TextUtils.isEmpty (signed)) {builder.addHeader (SIGN_KEY_NAME, signed);} return chain.proceed (builder.build ());}}

Okhttp's ReqeustBody is an abstract class, content output only writeTo methods, write the content to a BufferedSink interface implementation, and then convert the data to byte [], that is, the memory array. The only class that can achieve this goal is Buffer, which implements the BufferedSink interface and provides a method readByteArray that converts to an array of memory. There seems to be no problem with this, can it cause OOM?

Yes, it depends on the type of request. What if it is an interface for uploading files? What if this file is larger? The upload interface may use the public static RequestBody create (final @ Nullable MediaType contentType, final File file) method. If it is for the implementation of the file, its writeTo method is sink.writeAll (source). The Buffer.readByteArray we use when passing to the signature method converts all the contents in the buffer into a memory array, which means that all the contents in the file are converted into memory arrays, which is easy to cause OOM at this time! The RequestBody.create source code is as follows:

Public static RequestBody create (final @ Nullable MediaType contentType, final File file) {if (file = = null) throw new NullPointerException ("file = = null"); return new RequestBody () {@ Override public @ Nullable MediaType contentType () {return contentType;} @ Override public long contentLength () {return file.length ();} @ Override public void writeTo (BufferedSink sink) throws IOException {try (Source source = Okio.source (file)) {sink.writeAll (source) }

You can see that the implementation holds the file, Content-Length returns the size of the file, and all the contents are transferred to the Source object.

This is indeed a point that used to be very easy to overlook, there are very few additional operations on the request body, and once this operation becomes an one-time large memory allocation, it is very easy to cause OOM. So how to solve it? How is the signature method handled? It turns out that this signature method is lazy here-it only reads the first 4K content of the incoming body, and then encrypts only this part of the content, regardless of the size of the incoming memory array itself, completely leaving the risk and trouble to the outside (excellent SDK!).

Of course, the quick way is to make a whitelist, and there is no endorsement verification for the upload interface server, but this is easy to miss, and it increases the maintenance cost. Asking the people who sign the method sdk to write another appropriate interface is killing them, so it still has to be solved fundamentally. Since the signature method only reads the first 4K content, why don't we just read the first 4K part of the content and convert it into the memory array needed by the method? So our goal is to expect RequestBody to read some but not all of the content. Can you inherit RequestBody and rewrite its writeTo? Yes, but not realistic, it is impossible to replace all the existing RequestBody implementation classes, and it is also possible for the ok framework to create private implementation classes. Therefore, we can only write about the parameter BufferedSink of writeTo, and we must first understand how BufferedSink is called by the okhttp framework.

BufferedSink-related classes include Buffer, Source, all belong to the okio framework, okhttp is just a piece based on okio, okio does not directly use the io operation of java, but writes a set of io operations, specifically data buffering operations. Following the description above, how to create Source and how to operate BufferedSink at the same time? In Okio.java:

Public static Source source (File file) throws FileNotFoundException {if (file = = null) throw new IllegalArgumentException ("file = = null"); return source (new FileInputStream (file));} public static Source source (InputStream in) {return source (in, new Timeout ());} private static Source source (final InputStream in, final Timeout timeout) {return new Source () {@ Override public long read (Buffer sink, long byteCount) throws IOException {try {timeout.throwIfReached (); Segment tail = sink.writableSegment (1) Int maxToCopy = (int) Math.min (byteCount, Segment.SIZE-tail.limit); int bytesRead = in.read (tail.data, tail.limit, maxToCopy); if (bytesRead =-1) return-1; tail.limit + = bytesRead; sink.size + = bytesRead; return bytesRead;} catch (AssertionError e) {if (isAndroidGetsocknameError (e) throw new IOException (e); throw e) } @ Override public void close () throws IOException {in.close ();} @ Override public Timeout timeout () {return timeout;}};}

Source reads the file as the input stream inputstream, but its read method parameter is a Buffer instance. Where does it come from and how does it relate to BufferedSink? We have to continue to look at the implementation of BufferedSink.writeAll.

The implementation class of BufferedSink is Buffer, and then its writeAll method:

Override public long writeAll (Source source) throws IOException {if (source = = null) throw new IllegalArgumentException ("source = = null"); long totalBytesRead = 0; for (long readCount; (readCount = source.read (this, Segment.SIZE))! =-1;) {totalBytesRead + = readCount;} return totalBytesRead;}

It turns out that the Source.read (Buffer,long) method is explicitly called, so it is strung together, and the Buffer parameter turns out to be itself.

It is basically certain that as long as the BufferedSink interface class is implemented, and then it is judged that the content read exceeds the specified size, it can be returned after stopping writing, which can be called FixedSizeSink.

However, the trouble is that BufferedSink has so many interfaces, nearly 30 methods, I do not know when the framework will call which methods, only all of them can be implemented! Secondly, there are many okio classes in the parameters of the interface method, and the usage of these classes needs to be understood, otherwise the effect will be counterproductive if you use them incorrectly. So the understanding of one class becomes the understanding of multiple classes, there is no way but to write.

The first interface is a bit sore: Buffer buffer (); BufferedSink returns an instance of Buffer for external calls, and the implementation of BufferedSink is Buffer, and then returns a buffer! I've been speculating for a long time that BufferedSink is to provide a writable buffer object, but the framework authors are too lazy to do interface decoupling (alas, everyone is so simple). So FixedSizeSink needs to hold at least one Buffer object, which is used as the actual data cache and can be passed as a parameter where Source.read (Buffer, long) is needed.

At the same time, you can see an implementation class FormBody of RequestBody, which writes some data directly with this Buffer object:

Private long writeOrCountBytes (@ Nullable BufferedSink sink, boolean countBytes) {long byteCount = 0L; Buffer buffer; if (countBytes) {buffer = new Buffer ();} else {buffer = sink.buffer ();} for (int I = 0, size = encodedNames.size (); I

0) buffer.writeByte ('&'); buffer.writeUtf8 (encodedNames.get (I)); buffer.writeByte ('='); buffer.writeUtf8 (encodedValues.get (I));} if (countBytes) {byteCount = buffer.size (); buffer.clear ();} return byteCount;}

With such an operation, you may not be able to limit the change in buffer size! However, the amount of data should be relatively small and this usage scenario is relatively small, and the size we specify should be able to cover this situation.

Then there is an interface BufferedSink write (ByteString byteString), and it's exhausting to know how to use ByteString.

@ Override public Buffer write (ByteString byteString) {byteString.write (this); return this;}

You can call ByteString.write (Buffer) directly in the Buffer implementation because it is accessed by the package name, and the FixedSizeSink declaration implemented by yourself can also be used with the same package name package okio;. If other package names can only be converted to byte [], ByteString should not be a big deal and cannot do so (no method has been found for ByteString to read a segment of data):

@ Override public BufferedSink write (@ NotNull ByteString byteString) throws IOException {byte [] bytes = byteString.toByteArray (); this.write (bytes); return this;}

In short, these objects are converted to memory arrays or held by parameters acceptable to Buffer!

WriteAll, which is the key concern, is relatively easy to achieve. We continuously read the content of a specified length until the content length reaches our threshold.

Another sore point is the direction of read/write data streams for various objects:

Caller.read (Callee) / Caller.write (Callee), some from Caller to Callee, some, on the contrary, are a bit of a headache by a small class.

Finally, complete the code, if you find any potential problems, you can also communicate ~:

Public class FixedSizeSink implements BufferedSink {private static final int SEGMENT_SIZE = 4096; private final Buffer mBuffer = new Buffer (); private final int mLimitSize; private FixedSizeSink (int size) {this.mLimitSize = size;} @ Override public Buffer buffer () {return mBuffer;} @ Override public BufferedSink write (@ NotNull ByteString byteString) throws IOException {byte [] bytes = byteString.toByteArray (); this.write (bytes); return this } @ Override public BufferedSink write (@ NotNull byte [] source) throws IOException {this.write (source, 0, source.length); return this;} @ Override public BufferedSink write (@ NotNull byte [] source, int offset, int byteCount) throws IOException {long available = mLimitSize-mBuffer.size (); int count = Math.min (byteCount, (int) available) Android.util.Log.d (TAG, String.format ("FixedSizeSink.offset=%d,"count=%d,limit=%d,size=%d", offset, byteCount, mLimitSize, mBuffer.size ()); if (count > 0) {mBuffer.write (source, offset, count);} return this;} @ Override public long writeAll (@ NotNull Source source) throws IOException {this.write (source, mLimitSize); return mBuffer.size () } @ Override public BufferedSink write (@ NotNull Source source, long byteCount) throws IOException {final long count= Math.min (byteCount, mLimitSize-mBuffer.size ()); final long BUFFER_SIZE = Math.min (count, SEGMENT_SIZE); android.util.Log.d (TAG, String.format ("FixedSizeSink.count=%d,limit=%d", size=%d,segment=%d ", byteCount, mLimitSize, mBuffer.size (), BUFFER_SIZE)); long totalBytesRead = 0 Long readCount; while (totalBytesRead < count & & (readCount = source.read (mBuffer, BUFFER_SIZE))! =-1) {totalBytesRead = readCount;} return this;} @ Override public int write (ByteBuffer src) throws IOException {final int available = mLimitSize-(int) mBuffer.size (); if (available < src.remaining ()) {byte [] bytes = new byte [available]; src.get (bytes) This.write (bytes); return bytes.length;} else {return mBuffer.write (src);} @ Override public void write (@ NotNull Buffer source, long byteCount) throws IOException {mBuffer.write (source, Math.min (byteCount, mLimitSize-mBuffer.size ();} @ Override public BufferedSink writeUtf8 (@ NotNull String string) throws IOException {mBuffer.writeUtf8 (string); return this } @ Override public BufferedSink writeUtf8 (@ NotNull String string, int beginIndex, int endIndex) throws IOException {mBuffer.writeUtf8 (string, beginIndex, endIndex); return this;} @ Override public BufferedSink writeUtf8CodePoint (int codePoint) throws IOException {mBuffer.writeUtf8CodePoint (codePoint); return this;} @ Override public BufferedSink writeString (@ NotNull String string, @ NotNull Charset charset) throws IOException {mBuffer.writeString (string, charset); return this } @ Override public BufferedSink writeString (@ NotNull String string, int beginIndex, int endIndex, @ NotNull Charset charset) throws IOException {mBuffer.writeString (string, beginIndex, endIndex, charset); return this;} @ Override public BufferedSink writeByte (int b) throws IOException {mBuffer.writeByte (b); return this;} @ Override public BufferedSink writeShort (int s) throws IOException {mBuffer.writeShort (s); return this } @ Override public BufferedSink writeShortLe (int s) throws IOException {mBuffer.writeShortLe (s); return this;} @ Override public BufferedSink writeInt (int I) throws IOException {mBuffer.writeInt (I); return this;} @ Override public BufferedSink writeIntLe (int I) throws IOException {mBuffer.writeIntLe (I); return this;} @ Override public BufferedSink writeLong (long v) throws IOException {mBuffer.writeLong (v); return this } @ Override public BufferedSink writeLongLe (long v) throws IOException {mBuffer.writeLongLe (v); return this;} @ Override public BufferedSink writeDecimalLong (long v) throws IOException {mBuffer.writeDecimalLong (v); return this;} @ Override public BufferedSink writeHexadecimalUnsignedLong (long v) throws IOException {mBuffer.writeHexadecimalUnsignedLong (v); return this;} @ Override public void flush () throws IOException {mBuffer.flush ();} @ Override public BufferedSink emit () throws IOException {mBuffer.emit () Return this;} @ Override public BufferedSink emitCompleteSegments () throws IOException {mBuffer.emitCompleteSegments (); return this;} @ Override public OutputStream outputStream () {return mBuffer.outputStream ();} @ Override public boolean isOpen () {return mBuffer.isOpen ();} @ Override public Timeout timeout () {return mBuffer.timeout ();} @ Override public void close () throws IOException {mBuffer.close () }} after reading this, the article "how to solve the OOM problems caused by the use of okhttp in android" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.