Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to analyze the network protocols in big data. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

I. what is an agreement?

Agreement is equivalent to a kind of norm that must be mastered in normal communication and a kind of constraint rule that both hair and hair understand. For example, to send and receive telegrams, you need to have a password to decrypt the message, which is the agreement between the two parties. Why is there an agreement? The Internet world can be understood as an interaction between machines. If you need two machines to accomplish something, then both sides of the "conversation" of the two machines must understand. We cannot ask the two machines to be exactly the same, just as we cannot ask two people. So in order to facilitate the interaction, an agreement has been made, as long as everyone can understand the agreement in their own way.

II. Three elements of the agreement

1. Grammar means that the content of a paragraph should conform to certain rules and formats.

2. Semantics, that is, this paragraph should represent a certain meaning. For example, it makes sense to subtract a number from a number, while a number minus the text is generally meaningless.

3. Order, that is, what you do first and what you do later. An order that must be strictly observed.

Take the HTTP protocol as an example to see what the protocol looks like.

1. When we open the browser and enter the URL for online shopping, the URL is actually an agreement format. For example, the NetEase koala format is like this:

HTTP/1.1 200 OKDate: Tue, 27 Mar 2018 16:50:26 GMTContent-Type: text/html;charset=UTF-8Content-Language: zh-CN the main venue for the 3rd anniversary of the ekoala

2. Let's see if this meets the three elements of the agreement.

First of all, it conforms to the syntax, and the browser will recognize it only if it follows the above format. That is, status, radical and content

Second, it conforms to the semantic meaning, that is, in accordance with the agreed meaning. For example, status 200 means that the web page returned successfully, or 404 if it is not successful

Third, in order, a bit of browser is to send a HTTP request, and then there is a string of things returned by HTTP above.

The browser obviously did what the agreement agreed on, and the last colorful page appeared in front of you.

Third, commonly used network protocols

Use a shopping order process to see which network protocols are used in the operation of the Internet world.

1. Type https://www.kaola.com in the browser first, which is a URL. The browser only knows the name "www.kaola.com", but it doesn't know the exact location, so it doesn't know how to access it. So it opened the address book to look for it. You can use the general address book protocol DNS to find it, and you can also use another more accurate address book lookup protocol, HTTPDNS.

2. No matter which method you use to find it, you will eventually get this address: 106.114.138.24. This is the IP address, the "house number" of the Internet world.

3. Once the destination address is known, the browser begins to package its request. For ordinary browsing requests, the HTTP protocol is often used, but for shopping requests, encrypted transmission is often required, so the HTTPS protocol is used. No matter what the agreement is, it will say "what and how much you want to buy".

4. The layer where DNS, HTTP and HTTPS are located is called the application layer. After being encapsulated in the application layer, the browser will hand over the package of the application layer to the next layer to complete, which is realized by socket programming. The next layer is the transport layer. There are two protocols in the transport layer, one is the connectionless protocol UDP, and the other is the connection-oriented protocol TCP. For payment, the TCP protocol is often used. The so-called connection-oriented means that TCP ensures that the packet can reach its destination. If it cannot be reached, it will be resend until it arrives.

5. There will be two ports in the TCP protocol, one is the port that the browser listens to, and the other is the port that the e-commerce server listens to. The operating system often uses the port to determine which process it should give the packet it gets.

6. After the transport layer is encapsulated, the browser will hand over the package to the network layer of the operating system. The network layer protocol is the IP protocol. In the IP protocol, there will be the source IP address, that is, the IP address of the machine where the browser is located and the destination IP address, that is, the IP address of the server where the e-commerce website is located.

7. Now that the operating system knows the target IP address, it begins to think about how to find the target machine according to this house number. The operating system often determines whether the target IP address is a local or an outsider. If you are a local, you can see from the house number that the e-commerce website is not local, but in a distant place.

8. The operating system knows to leave the local place and go far away. Although we don't know where the distance is, we can make an analogy like this: if you want to go to the customs when you go abroad, you have to go to the gateway if you go out of town. When the operating system starts, the IP address will be configured by the DHCP protocol, as well as the default gateway IP address 192.168.1.1.

9. How does the operating system send the IP address to the gateway? In the local communication basically depends on the roar, so the operating system roars, who is 192.168.1.1 ah? The gateway will answer it, I am, my local address is in the east of the village. This local address is the MAC address, and the roar is the ARP protocol.

10. So the operating system hands the IP package to the next layer, the MAC layer. The network card will send out the package again. Because the packet contains an MAC address, it can reach the gateway.

11. After receiving the packet, the gateway will judge what to do next according to its own knowledge. A gateway is often a router. How to get to a certain IP address is called a routing table.

12. The router is a bit like the city passes of one country after another as Xuanzang traveled westward. Each city pass is connected to two countries, each country is equivalent to a local area network, in each country, you can use the local address MAC to communicate.

13. Once you cross the city pass, you need to take out the IP head, which says that the poor monk is from Datang (that is, the source IP address) and wants to worship the Buddha and seek scriptures in the west (refers to the target IP address). Pass by the treasure land, stay for one night, and leave tomorrow. How can I get there next?

14. Chengguan is often aware of this "knowledge", because Chengguan and its neighbors often communicate with each other. The protocols for this kind of communication are called routing protocols, and the common ones are OSPF and BGP.

15. There is a country between chengguan and chengguan. When the network packet knows which chengguan to go next, it still needs to use the MAC address within the country, through the MAC address of the next chengguan, to find the next chengguan, and then ask the next way to get out of the last chengguan.

16. The last chengguan knows where this network packet is going. So, yell at this country, who is the target IP? The target server will reply with an MAC address. After the network packet passes, the target server can be found through this MAC address.

17. The target server finds that the MAC address is matched, takes down the MAC header and sends it to the network layer of the operating system. When you find that the IP is also matched, take off the IP head. The IP header will write a layer encapsulating the TCP protocol, and then give it to the transport layer, that is, the TCP layer.

In this layer, for each packet received, there will be a reply indicating that it has been received. This reply package is by no means the result of this order request, such as whether the purchase was successful, how much money was deducted, etc., but only a description of the TCP layer, that is, the reply after receipt. Of course, this reply will walk back in the direction I just came and let me know that I am safe. Because once out of the country, the westbound road is extremely difficult and dangerous, what if the network packet gets lost in the process? So it's time to report peace.

19. If it does not arrive after a period of time, the TCP layer of the sender will resend the packet, which is the same as the above process, until one day it receives a reply that it has arrived safely. This retry is by no means that your browser will rerequest the action of placing an order again. As far as the browser is concerned, a request to place an order is sent, and the TCP layer keeps trying again. Unless there is something wrong with the TCP layer, such as a broken connection, it is the browser's application layer's turn to resend the order request.

20. When the network packet arrives safely at the TCP layer, there is a destination port number in the TCP header. Through this port number, you can find that the process of the e-commerce website is listening to this port number. Suppose a Tomcat, and send the package to the e-commerce website.

21. The process of the e-commerce website gets the content requested by HTTP and knows what to buy and how much to buy. Often, the Tomcat who initially receives the request on an e-commerce site is just a receptionist who is responsible for handling the request as a whole, rather than doing everything himself. For example, the receptionist should tell the specially managed order process, register an item to be bought, how much to buy, tell the inventory management process, how much the inventory should be reduced, tell the payment process, how much should be paid, and so on.

22. What if you tell the relevant process? It is often realized through RPC calls, that is, remote procedure calls. Remote procedure call means that when telling the management order process, the receptionist does not have to care about the network interconnection in the middle, but will be handled by the RPC framework. There are many kinds of RPC frameworks, some are placed in HTTP messages based on HTTP protocol, and some are directly encapsulated in TCP messages.

23. When the receptionist finds that the corresponding departments have been disposed of, he replies with a HTTPS package, informing him that the order has been placed successfully. This HTTPS bag, as it did when it came, will go through thousands of difficulties to your personal computer and finally enter the browser to show the success of the payment.

IV. Summary

A simple order issuing process that involves so many protocols. And managing a large area of machines is a very technical thing. Technologies such as cloud computing, containers, services and other technologies also need the help of various agreements to achieve cooperation between large-scale machines. The common protocols in the five-tier model are as follows:

On how to analyze big data in the network protocol to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.