Shulou(Shulou.com)06/01 Report--

This article mainly explains "Content-Length instance Analysis of Http Protocol". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "Http protocol Content-Length case analysis" bar!

Detailed explanation of Http Protocol Content-Length

Some of the resources that our mobile App downloads from the server when making updates are usually small files. The updated code is similar to the following:

Static void update () throws IOException {URL url = new URL ("http://172.16.59.129:8000/update/test.so"); HttpURLConnection conn = (HttpURLConnection) url.openConnection (); if (conn.getResponseCode () = 200) {int totalLength = conn.getContentLength (); BufferedInputStream in = new BufferedInputStream (conn.getInputStream ()); byte [] buffer = new byte [512]; int readLength = 0; int length = 0 While ((length=in.read (buffer))! =-1) {readLength + = length; / / progress bar System.out.println (float) readLength) / ((float) (totalLength);}

For example, the above code updates a so file, first obtains the total size of the file through content-length, and then reads Stream, each reading paragraph, calculates the total size of the current read, divided by content-length, used to display the progress bar.

As a result, after weblogic was upgraded from 10 to 12, content-length kept returning-1, so that the progress bar could not be displayed, but the file stream could be read normally. Restart weblogic, at first it can return to content-length, and then it will be-1 again.

Cause analysis

The request message and reply message of Http protocol have header and body,body, which are the resources you want to get, such as a html page and a jpeg image, while header is used to make certain conventions. For example, the client and the server agree on some transmission formats, and the client first gets the header, gets some format information, and then starts to read the body.

Client: Accept-Encoding:gzip (compress it for me, I use traffic, download it first and then decompress it slowly)

Server 1:Content-Encoding:null (no Content-Encoding header. I don't give compression, CPU is not available, do you like it or not?

Server 2:Content-Encoding:gzip (save you traffic, compress it)

Client: Connection: keep-alive (Brother, we finally set up a TCP connection, and we'll use it next time)

Server 1: Connection: keep-alive (none of which is easy, continue to use)

Server 2: Connection: close (who will continue to use it with you? our TCP is disposable. I will have to reconnect with you next time.)

There is no three-way handshake in http protocol. When the client requests resources from the server, the server shall prevail. There are also some header does not negotiate the process, but the server directly tells the client what to press. For example, in the Content-Length above, the server tells the client how big the body is. But! The server may not be able to tell you exactly how big the body is. The server should write header first, then body. If you want to write the size of body in header, you need to know the size of body in advance. If the body is generated dynamically, the server finishes, and then starts writing the header, which requires a lot of extra overhead, so there may not be a content-length in the header.

How does the client know the size of the body? The server can tell you in three ways.

1. The server already knows the size of the resource and tells you through the header of content-length.

Content-Length:1076 (the size of body is 1076B, you can finish the task by reading 1076B) Transfer-Encoding: null

two。 If the server is unable to know the size of the resource in advance, or is unwilling to spend the resource to calculate the size in advance, it will add a header to the http reply message called Transfer-Encoding:chunked, which means multipart transmission. Each block uses a fixed format, with the size of the block at the front, the data behind it, and the last block with a size of 0. In this way, the client needs to pay attention to removing some useless fields when parsing.

Content-Length:nullTransfer-Encoding:chunked (I want to pass the next body piece by piece. Each piece starts with the size of this piece, and when I reach the block with the size of 0, it will be gone.)

3. The server does not know the size of the resource and does not support the transmission mode of chunked, so there is neither content-length header nor transfer-encoding header. In this case, a short connection must be used to indicate the end of the data transfer, and the size can be known at the end of the transmission. At this time, Connection must be close in the header returned by the server.

Content-Length:nullTransfer-Encoding:nullConnection:close (I don't know the size, I can't use chunked, when I close the tcp connection, it means the transmission is over) experiment

I do experiments in a virtual machine through nginx. By default, nginx supports chunked mode and can be turned off.

The code used is as follows, and the parameters may be adjusted.

Static void update () throws IOException {URL url = new URL ("http://172.16.59.129:8000/update/test.so"); HttpURLConnection conn = (HttpURLConnection) url.openConnection (); / / conn.setRequestProperty (" Accept-Encoding "," gzip "); / / conn.setRequestProperty (" Connection "," keep-alive "); conn.connect () If (conn.getResponseCode ()) {System.out.println (conn.getHeaderFields (). KeySet ()); System.out.println (conn.getHeaderField ("transfer-encoding")); System.out.println (conn.getHeaderField ("Content-Length")); System.out.println (conn.getHeaderField ("Content-Encoding")); System.out.println (conn.getHeaderField ("Connection")) }} when 1.nginx opens chunked_transfer_encoding

(1) do not use gzip in reqeust header, that is, do not add accept-encoding:gzip

Test.so file size result 100B can return content-length normally, without transfer-encoding header 69m can return content-length normally, without transfer-encoding header 3072m can return content-length normally, without transfer-encoding header

You can find that nginx no matter how large the resources, if the client does not accept the compressed format of gzip, it will not use chunked mode, and it has nothing to do with the use of short connections.

(2) add gzip,accepting-encoding:gzip to request header

Test.so file size result 100B No content-length,transfer-encoding=trunked69M, no content-length,transfer-encoding=trunked3072M, no content-length,transfer-encoding=trunked

You can see that when nginx enables chunked_transfer_encoding and the client accepts gzip, it will use chunked mode. When nginx enables gzip, it will not calculate the size of resources, but will directly use chunked mode.

2.nginx shuts down chunked_transfer_encoding

(1) do not use gzip in reqeust header, that is, do not add accept-encoding:gzip

Test.so file size result 100B can return content-length normally, without transfer-encoding header 69m can return content-length normally, without transfer-encoding header 3072m can return content-length normally, without transfer-encoding header

Because it is easy to know the file size, nginx can still return content-length.

(2) add gzip,accepting-encoding:gzip to request header

Test.so file size result 100B has no content-length and transfer-encoding headers. No matter whether the client connection is keep-alive or close, the connection header returned by the server is close69M without content-length and transfer-encoding headers. Whether the client connection is keep-alive or close, the connection header returned by the server is close3072M without content-length and transfer-encoding headers. Whether the client connection is keep-alive or close, the connection header returned by the server is close.

This is the third case mentioned above. If you do not know the size and do not support trunked, you must use a short connection to mark the end.

Solution to the problem

After consulting my colleagues in the middleware group, I encountered a similar problem before, because upgrading Weblogic caused an error in parsing XML on the client side, because the chunked pattern was used and there were some formatted characters in the middle, while the code parsed on the client side did not take into account the parsing of the chunked pattern, resulting in an error in parsing.

Because our client has to use content-length to show progress, we can't use chunked mode. Weblogic can turn off chunked mode. Use the following methods:

#! java weblogic.WLST connect ('username','password',' t3edit edit () startEdit () cd ("Servers/AdminServer/WebServer/AdminServer") cmo.setChunkedTransferDisabled (true) save () activate () exit ()

After the change, it is true that chunked is not returned, but there is no content-length, because Weblogic does not get the file size in advance, but forces the addition of connection:close, that is, the third one mentioned above, to identify the end of the data through the end of the connection. In the end, these resources can only be put into apache.

At this point, I believe you have a deeper understanding of "Content-Length instance Analysis of Http Protocol". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.