Shulou(Shulou.com)11/24 Report--

Usually we open web pages, such as a shopping website. Click on the list of goods, jump to the web page to the details of the goods.

From the perspective of HTTP protocol, it is to click a button on the web page, the front end sends an HTTP request, and the website returns an HTTP response.

This kind of active request by the client and response by the server also meets the functional scenarios of most web pages.

But have not found, in this case, the server will never take the initiative to send a message to the client.

Just like the girl you like never comes to you.

But now, if you suddenly pop up a small advertisement in the lower right corner when you swipe the web page, you will be prompted to play secretly at home.

Knowledge, learning, hard work, these things engraved in your DNA are moving.

You click on it and you see.

The ordinary-looking ancient one reminds you that "Taoist priest has 9 dogs, all of whom walk horizontally."

A certain teacher of the film emperor said to you,"if you are a brother, come and cut me."

Come to all come, you choose a character to enter the game interface.

Create a character page at this time, up is a small monster, from a distance, and then crazy with a stick to hit you.

You didn't click a mouse the whole time. The server will automatically send you the monster's mobile data and attack data.

This... It's so warm.

Moved, the problem comes,

How does it look like the server is actively sending messages to the client?

Before we answer this question, let's talk about some background knowledge.

The real pain point of HTTP polling is how to get the page to receive the message and change it without the user doing anything.

The most common solution is that the front-end code of the web page constantly sends HTTP requests to the server, and the server receives the request and sends a response message to the client.

This is a form of pseudo-server push in real time.

In fact, it is not the server actively sending messages to the client, but the client itself constantly secretly requests the server, but the user does not perceive it.

There are also many scenarios in this way, the most common is scanning code login.

For example, after the two-dimensional code appears on the login page of a certain letter public number platform, the front-end webpage does not know whether the user scans it at all, so it keeps asking the back-end server to see if anyone has scanned this code. Moreover, the request is continuously sent at intervals of about 1 to 2 seconds, which can ensure that the user can get timely feedback within 1 to 2 seconds after scanning the code, so as not to wait too long.

But there are two obvious problems with HTTP polling.

When you open the F12 page, you will find a full screen of HTTP requests. Although small, it also consumes bandwidth and increases the burden on downstream servers.

In the worst case, after scanning the code, the user needs to wait for 1~2s, just to trigger the next http request, and then jump to the page, the user will feel obvious stuck.

The experience of using it is that after the QR code appears, the mobile phone scans, and then click on the mobile phone to confirm, at this time, the card waits for 1~2s before the page jumps.

Keep polling to see if there is a scan code. Then the problem comes again. Is there a better solution?

Yes, and the cost of implementation is very low.

Long polling We know that after the HTTP request is issued, it will generally leave a certain amount of time for the server to respond, such as 3s. If it does not return within the specified time, it is considered to be a timeout.

If our HTTP request timeout is set to a very large value, such as 30 seconds, the server will immediately return to the client page as soon as it receives the scan request within 30 seconds. If the timeout occurs, the next request is made immediately.

This reduces the number of HTTP requests, and because most of the time, users will scan code within a certain 30s interval, so the response is also timely.

Long polling, for example, a certain degree of cloud network disk is done like this. So you will find a scan code, click on the mobile phone to confirm, the computer side web page will jump in seconds, the experience is very good.

Long polling instead of really killing two birds with one stone.

A mechanism like this that initiates a request and waits for a server response for a long time is called a long polling mechanism. RocketMQ, our popular message queue, also uses this approach when consumers fetch data.

RocketMQ consumers get data via long polling

Like this, in the case of users do not perceive, the server will push data to the browser technology, is the so-called server push technology, it also has an unrelated English name, comet technology, we have heard of good.

The two solutions mentioned above, in essence, are still the client actively fetching data.

It can also be used for simple scenarios such as scanning code login.

However, if it is a web game, the game generally has a large amount of data that needs to be actively pushed from the server to the client.

I have to say websocket.

We know that both ends of a TCP connection can actively send data to each other at the same time. This is called full duplex.

The most widely used HTTP 1.1 is also based on the TCP protocol. At the same time, only one party can actively send data between the client and the server. This is the so-called half-duplex.

In other words, a good full-duplex TCP is used as half-duplex by HTTP.

Why not?

This is because at the beginning of HTTP protocol design, the consideration is to look at the scene of web page text, it is enough to make the client initiate the request and then the server responds, and it does not consider the scene of web game, where the client and the server have to actively send a large amount of data to each other.

So in order to better support this scenario, we need another new protocol based on TCP.

A new application-layer protocol, websocket, was designed.

Don't be fooled by the name. Although the name carries a socket, in fact, the relationship between socket and websocket is the same as Lei Feng and Lei Feng Pagoda. The two are close to nothing.

Websocket in the four-layer network protocol location How to establish a websocket connection We usually brush the web page, generally brush on the browser, brush the text for a while, this time using the HTTP protocol, open the web game for a while, this time we have to switch to our new websocket protocol.

To accommodate these usage scenarios. After the TCP three-way handshake is established, the browser uses the HTTP protocol to communicate once.

If this is a normal HTTP request, then the subsequent two parties will continue to interact with the normal HTTP protocol as usual, there is no doubt about this.

If you want to establish a websocket connection at this time, you will add some special headers to the HTTP request.

Connection: UpgradeUpgrade: websocketSec-WebSocket-Key: T2a6wZlAwhgQNqruZ2YUyg==\r\n These headers mean that the browser wants to upgrade to the Connection: Upgrade protocol and wants to upgrade to the websocket protocol.

At the same time, it carries a randomly generated base64 code (Sec-WebSocket-Key) and sends it to the server.

If the server supports upgrading to websocket protocol. It will go through the websocket handshake process, and at the same time, according to the base64 code generated by the client, use a public algorithm to turn it into another string, put it in the Sec-WebSocket-Accept header of the HTTP response, and send it back to the browser with the 101 status code.

HTTP/1.1 101 Switching Protocols\r\nSec-WebSocket-Accept: iBJKv/ALIW2DobfoA4dmr3JHBCY=\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nhttp status code = 200 (normal response) is a common occurrence. 101 is really uncommon, it actually refers to protocol switching.

After base64 is converted into a new string, the browser uses the same public algorithm to convert base64 into another string. If this string matches the string sent back by the server, the verification passes.

Compared with the string generated by the client and the server, the websocket is established after two HTTP handshakes, and the subsequent two parties can communicate using the webscoket data format.

We can use Wireshark to grab a packet, actually look at the situation of the next packet.

The client request is upgraded to websocket. Note that line 2445 of the message with red box is the first handshake of websocket, which means that an HTTP request with special Header is initiated.

The server agrees to upgrade to the websocket protocol. The message in line 4714 with the red box in the above figure is the second handshake that the server responds to after getting the first handshake. You can see that this is also an HTTP type message, and the status code returned is 101. At the same time, you can see that the returned message header also carries various websocket-related information, such as Sec-WebSocket-Accept.

After two HTTP requests, websocket communication is officially used. The above figure is the whole picture. As can be seen from the comments on the screenshot, websocket and HTTP are both TCP based protocols. After three TCP handshakes, the HTTP protocol is upgraded to the websocket protocol.

You may see a statement on the Internet: "websocket is a new protocol based on HTTP", which is actually not true, because websocket only uses HTTP when establishing a connection, and has nothing to do with HTTP after the upgrade is completed.

It's like the girl you like gets your college roommate's WeChat through you, and then they start chatting on their own. Can you say that this girl communicated with your roommate through you? No. Like HTTP, you are just a tool man.

This is a bit of a "shell to lay eggs" that means.

The relationship between HTTP and websocket The message format of websocket mentioned above After the completion of the protocol upgrade, the two ends will communicate in the data format of webscoket.

Packets are called frames in websockets.

Let's see what its data format looks like.

Websocket Message Format There are a lot of fields here, but we only need to pay attention to the following.

Opcode field: This is used to indicate what type of data frame this is. For example.

1 refers to packets of type text (string)

2 is a binary data type ([] byte) packet

Equal to 8 is the signal to close the connection.

Payload field: stores the length of the data we really want to transfer, in bytes. For example, if the data you want to send is the string "111," its length is 3.

In addition, we can see that there are several fields for storing the payload length, we can use the first 7 bits, or we can use the 7+16 bits or 7+64 bits.

Then the question arises.

We know that at the data level, everyone is a binary stream of 01. How do I know when to read 7 bits and when to read 7+16 bits?

Websocket uses the first 7 bits as a flag. No matter how big the next data is, read the first 7 bits first, and decide whether to read another 16 bits or 64 bits according to its value.

If the first 7 bits have values between 0 and 125, then it represents the full payload length and reads only the first 7 bits.

Payload length is between 0 and 125 if 126 (0x7 E). It means that the payload length ranges from 126 to 65535, and then you need to read 16 bits. These 16 bits contain the true payload length.

Payload length is between 126 and 65535 if 127 (0x7F). Then it indicates the payload length range>=65536, and then you need to read 64 bits. The 64 bits contain the payload length. This can put 2 to the 64th byte of data, convert a lot of TB, it must be enough.

If the payload length is greater than or equal to 65536, the payload data field: The data stored here is the data to be transmitted. After knowing the payload length above, you can intercept the corresponding data according to this value.

Have you found a small detail, websocket data format is also the data header (including payload length)+ payload data form.

As mentioned in "Since there is HTTP protocol, why should there be RPC" written before, TCP protocol itself is full duplex, but directly using pure bare TCP to transmit data, there will be "problem" of sticky packets. In order to solve this problem, the upper layer protocol generally uses the format of message header + message body to repackage the data to be sent.

The message header generally contains the length of the message body, through which the real message body can be intercepted.

The HTTP protocol and most RPC protocols, as well as the websocket protocol we're introducing today, are designed this way.

Websocket perfectly inherits the full-duplex capability of TCP protocol, and also provides a solution to sticky packets. It is suitable for most scenarios that require frequent interaction between the server and client (browser). For example, web/Mini programs games, web chat rooms, and web collaboration software such as flying books.

Back to the question at the beginning of the article, in web games using websocket protocol, monster movement and attacking player behavior are generated by server logic, and data such as damage to players need to be actively sent to the client by the server, and the client displays the corresponding effect after obtaining the data.

TCP protocol itself is full-duplex, but our most commonly used HTTP 1.1, although it is based on TCP protocol, but it is half-duplex, for most of the need for the server to actively push data to the client scenarios, are not very friendly, so we need to use full-duplex support websocket protocol.

In HTTP 1.1. As long as the client does not ask, the server does not answer. Based on such characteristics, for simple scenarios such as login pages, the effect of server push (comet) can be achieved by using timed polling or long polling.

For complex scenarios that require frequent interaction between the client and the server, such as web games, consider using the websocket protocol.

Websocket and socket have almost nothing to do with each other, just similar names.

Because all browsers support HTTP protocol, websocket will first use HTTP protocol plus some special header header to handshake upgrade operation, after the upgrade is successful, it has nothing to do with HTTP, and then use websocket data format to send and receive data.

This article comes from Weixin Official Accounts: Xiaobai debug (ID: xiaobaidebug), author: Xiaobai

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.