Shulou(Shulou.com)06/03 Report--

OSI (Open System Interconnect) Open system Interconnection Model

OSI reference model: application layer, presentation layer, session layer, transport layer, network layer, data link layer, physical layer

Application layer: the application layer provides an interface for applications to access network services and directly provides a variety of network services for users. Common application layer network service protocols are: HTTP,HTTPS,FTP,POP3, SMTP and so on. Presentation layer: provides a variety of coding and conversion functions for application layer data to ensure that the data sent by the application layer of one system can be recognized by the application layer of another system: responsible for establishing, managing, and terminating communication sessions between presentation layer entities. Transport layer: provides end-to-end reliable or unreliable data transmission services for upper layer protocols. Transmission units for data segment segment; protocols are: TCP, UDP network layer: through IP addressing to establish a connection between two nodes (to solve network routing and addressing problems). The protocols are: IP, ARP, ICMP, and the transmission unit is the packet packet data link layer: it accepts the data in the form of bit stream from the physical layer, encapsulates it into a frame, and transmits it to the upper layer; similarly, it also forwards the data frame from the upper layer, disassembled and disassembled into the form of bit stream, to the physical layer; and is also responsible for processing the information of the confirmation frame sent back by the receiver in order to provide reliable data transmission. (transmission unit is frame frame) MAC belongs to the physical layer of this layer: it uses transmission media to provide physical connection for the data link layer, realizes the transparent transmission of bit stream between adjacent computer nodes, and shields the differences between specific transmission media and physical devices as much as possible. Network card and hub in physical layer (data transmission unit bit) protocol stack

Protocol stack (Protocol Stack) refers to the sum of the protocols of each layer in the network.

Router

It is a device that connects each local area network and wide area network in the Internet. it will automatically select and set routes according to the situation of the channel, and send signals in the order of the best path.

The router consists of two parts. WAN and LAN,WAN are used for dialing, which is a part of routing itself to access the Internet. LAN is used to exchange data in the local area network. Just like the function of the switch, our computer is plugged into the LAN port to access the Internet.

Routing built-in DHCP server, which can automatically assign IP to computers that use routing

The router switches at the network layer

It is a network device for transmitting electrical (optical) signals. The function can be simply understood as connecting some machines to form a local area network. It can provide an exclusive electrical signal path for any two network nodes of the access switch.

He only has LAN, not WAN.

Plug the network cable into any interface of the switch, computer 1 into any interface of the switch, and computer 2 into any interface of the switch, so that both computers need to dial up to access the Internet, so that the switch is used for data exchange.

The switch is at the data link layer (there are also multilayer switches: data link layer + partial network layer)

The cat converts the optical fiber into a TCP/IP model of the network cable that can be plugged directly into the computer.

Application layer (the session layer and presentation layer of the OSI model are merged into the application layer)

Transport layer

The Internet layer (corresponding to the network layer of the OSI reference model) is also often the IP layer.

Network access layer or host-to-network layer (physical layer and data link layer in the OSI reference model correspond)

The IP layer includes Internet Control message Protocol (ICMP:Internet Control Message Protocol) and address Resolution Protocol (ARP). In fact, they are not part of the IP layer, but work directly with the IP layer. ICMP is used to transmit error information, time, echo, network information and other message control data. ARP, which is between the IP and the data link layer, is a protocol that performs translation between 32-bit IP addresses and 48-bit LAN addresses

Ethernet data format:

Ethernet uses 48bit (6 bytes) to represent the original address and destination address. The source and destination addresses here refer to the hardware address, such as the MAC address of the network card.

After the address is a two-byte field that represents the type, for example, 0800 indicates that the data of the frame is IP data, and 0806 indicates that the frame is an ARP request

The data after the type field, for Ethernet, specifies that the size of the data segment ranges from 46 bytes to 1500 bytes, and the insufficient data should be filled with empty characters. For example, the data format of the ARP protocol is 28 bytes. In order to comply with the specification, a placeholder of 18 bytes is used to meet the requirement of at least 46 characters.

The length of the data segment has a maximum value, Ethernet is 1500, this characteristic is MTU, that is, the maximum transmission unit. If there is a data to be transmitted in the IP layer that is longer than the MTU, the data in the IP layer should be sliced so that each slice is smaller than the MTU.

The CRC field is used to check the intra-frame data to ensure the correctness of data transmission, which is usually implemented by hardware, such as the CRC check of network data in the network card device.

The 14-byte header of Ethernet will cause efficiency problems in the implementation of some platforms, such as 4-byte aligned platforms, which are usually copied again when IP data is obtained.

ARP address Resolution Protocol

ARP (Address Resolution Protocol, address Resolution Protocol) is a network layer located in the TCP/IP protocol stack, which is responsible for resolving an IP address into a corresponding MAC physical address.

In an Ethernet-based local area network, each network interface has a hardware address, which is a 48bit value that identifies different Ethernet devices. In the local area network, the hardware address of the network device must be known before sending data to the destination host, while in the internetwork, the destination address of data transmission is the IP address. If the data can be transmitted normally, the mapping record between the IP address and the hardware address must be established.

ARP Protocol for 32-bit IP address to 48-bit hardware address Mapping

IP (Internet Protocol)

The main purpose of the IP layer is to provide the interconnection of subnets to form a larger network so that data can be transmitted between different subnets.

The main roles of the IP layer are:

Data transmission: data transmission from one host to another host addressing: according to subnetting and IP address, finding the correct destination host address routing: selecting the transmission path of data on the Internet, data packet segmentation: when the transmitted data is greater than MTU, the data is sent and received in segments and assembled.

IPV4 IP address 32-bit, consisting of four sets of decimal numbers, each group of values range from 0 to 255, in the middle. Separation

An IP address consists of IP address type, network ID, and host ID.

The network type identifier identifies the type to which this IP address belongs

Network ID identifies the network on which the IP identifies the device or host

The host ID identifies a workstation, server, or router on the network.

The address of the host ID with all zeros indicates the network address of a network

The address where the host ID is all 1 indicates the broadcast address

The address with all zero IP represents the host itself, and the data packets sent to this IP address are received by the local machine.

An address with all 1 IP indicates a limited broadcast address

IP address 127.0.0.1 is a special loopback address, which is generally used for local testing

The IP protocol is used to connect multiple packet-switched networks. It transmits something called a packet between the source address and the destination address. It also provides the function of reassembling the data size to meet the packet size requirements of different networks.

The network layer IP provides an unreliable service. It only sends packets from the source node to the destination node as soon as possible, but does not provide any guarantee of reliability.

Network Control message Protocol (ICMP)

It is used to transmit error information, time, echo, network information and other message control data. It is often used to detect whether the network is not accessible, whether the host is reachable, and whether the router is available.

ICMP protocol can be divided into two categories, one is query message, the other is error message.

UDP protocol

UDP:user data protocol user data message protocol-an unreliable, connectionless protocol; suitable for scenarios where there is no fear of data loss, no need to sort messages, and no flow control

UDP protocol does not guarantee the order of data message transmission and the accurate arrival of data.

The execution speed of UDP protocol is much faster than that of TCP protocol, because UDP protocol is much simpler and causes less load on the system.

Application scenarios: streaming media transmission, domain name server, embedded set-top box system

TCP protocol

TCP:transmission control protocol transmission control protocol-provides a reliable, connection-oriented and flow-controlled transport layer protocol on the unreliable ip layer. In order to provide this reliable service, TCP adopts mechanisms such as timeout retransmission, sliding window, sending and receiving end-to-end acknowledgement packets to ensure that the receiver can receive all packets from the sender, and the order is consistent with the sending order.

Sliding window: the receiver controls the sending speed of the sender by notifying the sender of its own window size, so as to prevent the sender from inundating itself by sending too fast.

TCP features:

Connection-oriented service: before the data is transmitted, a connection needs to be established, and then the TCP message is transmitted reliably on the basis of this connection: the reliable buffer transmission is guaranteed based on the check and reply retransmission mechanism: the transmission of application layer data can be delayed Allows applications to accumulate to a certain amount of data to be transmitted before centralized full-duplex transmission: full-duplex data flow switching flow control transmission: through the sliding window mechanism, support end-to-end flow control byte flow services between hosts: byte flow-based services do not have byte order problems plaguing three-way handshakes

Both client An and server B are in the CLOSED state. Server B creates socket, binds ip, listens to the designated port, is in LISTEN state, waits for client A to request client A to create socket, sends connection request (SYN = 1pm seq = x) message, enters SYN-SENT state, waits for server to confirm that server B receives connection request, and sends acknowledgement (ACK=1, confirmation number ack=x+1) message to client A. At the same time, it also sends a SYN packet (SYN=1,seq = y). After client A receives the confirmation from server B, it sends an acknowledgement (ACK=1,ack=y+1,seq=x+1) to server B. after An enters ESTABLISHED status, server B enters ESTABLISHED state after receiving confirmation from client A.

Unconnected queue

In the three-way handshake protocol, the server maintains an unconnected queue, which is listed as the SYN packet (syn=j) of each client and opens an entry that indicates that the server has received the SYN packet and sends an acknowledgement to the customer, waiting for the customer's confirmation packet. The connection identified by these entries is in the SYN_RECV state of the server. When the server receives the acknowledgement packet from the customer, the entry is deleted and the server enters the ESTABLISHED state.

Waving four times

Both client An and server B are in ESTABLISHED state. Client A sends connection release message (FIN=1, serial number seq=u) to the server, stops sending data, actively closes the TCP connection, enters FIN-WAIT-1 (termination waiting 1) state, waits for server B to confirm that server B receives the connection release message segment, and then sends out the acknowledgement message segment. (ACK=1 The confirmation number ack=u+1, serial number seq=v), the server side enters the CLOSE-WAIT (waiting to be closed). At this time, the TCP is in the semi-closed state. After receiving the confirmation from the server side B, the client A releases the connection from the server side B. after receiving the confirmation from the server side B, it enters the FIN_WAIT-2 (termination waiting 2) state and waits for the server side B to issue the connection release message segment. Server B has no data to be sent to client A. Server B sends out a connection release message segment (FIN=1,ACK=1, serial number seq=w, confirmation number ack=u+1), the server enters the LAST-ACK (final acknowledgement) state, waits for client A's confirmation client A to receive the connection release message of server B, sends out acknowledgement message segment (ACK=1,seq=u+1,ack=w+1), and client An enters TIME-WAIT (time waiting) state. At this time, the TCP is not released, and it is necessary to wait for the time 2MSL set by the timer before An enters the CLOSED state.

Https://www.cnblogs.com/Andya/p/7272462.html

Why does the TIME_WAIT state need to go through 2MSL (maximum segment lifetime) to return to the CLOSE state?

A: although according to reason, after all four messages have been sent, we can directly enter the CLOSE state, but we must pretend that the network is unreliable and that the last ACK can be lost. So TIME_WAIT status is used to resend ACK messages that may be lost.

Server side is vulnerable to SYN***.

The resource allocation on the server side is allocated during the second handshake, while the resource on the client side is allocated when the three-way handshake is completed, so the server is vulnerable to SYN flooding. SYN*** means that Client forges a large number of non-existent IP addresses in a short period of time and constantly sends SYN packets to Server. Server replies to the confirmation packet and waits for Client confirmation. Because the source address does not exist, Server needs to be retransmitted until timeout. These fake SYN packets will occupy the unconnected queue for a long time, resulting in normal SYN requests being discarded because the queue is full, resulting in network congestion and even system paralysis.

Measures to prevent SYN***: reduce the waiting time of the host so that the host can release the occupation of semi-connection as soon as possible, and the repeated SYN of a certain IP in a short time will discard subsequent requests.

The TCB transmission control block Transmission Control Block stores important information in each connection, such as the TCP connection table, the pointer to the send and receive cache, the pointer to the retransmission queue, and the current send and receive sequence number.

The message segment of SYN=1 cannot carry data

Why is it a three-way handshake when connecting and a four-way handshake when it is closed?

Because when the Server side receives the SYN connection request message from the client side, it can send the SYN+ACK message directly. The ACK message is used to reply, and the SYN message is used to synchronize. However, when the connection is closed, when the Server side receives the FIN message, it is likely that the SOCKET will not be closed immediately, so it can only reply an ACK message first, telling the client, "I received the FIN message you sent." Only when all the messages on my Server side have been sent, can I send FIN messages, so I can't send them together. So you need a four-step handshake.

What is HTTP?

HTTP protocol is the abbreviation of Hyper Text Transger Protocal (Hypertext transfer Protocol).

A transport protocol for transferring hypertext from a www:World Wide Web server to a client

HTTP is a communication protocol based on TCP/IP to transfer data. The default port number of HTTP is 80.

HTTP is an application layer protocol

Composed of requests and responses, it is a standard client-server model.

HTTP allows the transfer of any type of data object. The type being transmitted is marked by Content-Type

HTTP is a stateless protocol, which means that the protocol has no memory ability for transactions.

Risks faced by HTTP Transmission

Eavesdropping risk: * can obtain communication content.

Risk of tampering: * the content of the communication can be modified.

Impersonate risk: * can participate in communication by pretending to be someone else.

What is SSL/TLS?

SSL is the abbreviation of Secure Sockets Layer, which is called "secure socket layer" in Chinese.

TLS is the abbreviation of "Transport Layer Security" and is called "Transport layer Security Protocol" in Chinese.

SSL refers to the secure socket layer protocol (and the transport layer protocol TLS), which is located between the TCP/IP protocol and various application layer protocols to provide security support for data communication.

What is HTTPS?

The HTTPS protocol (HyperText Transfer Protocol over Secure Socket Layer) is still transmitted based on TCP (the so-called "HTTP over SSL" is actually a layer of SSL encapsulation on top of the original HTTP data. The original GET, POST and other mechanisms of the HTTP protocol are basically intact)

The HTTPS protocol is carried on top of the TLS or SSL protocol layer, and the port number of HTTPS is 443,

Characteristics of HTTPS

Confidentiality (anti-disclosure-all information is encrypted and cannot be eavesdropped), integrity (tamper-proof-there is a verification mechanism, once tampered with, both sides of the communication will immediately find out), authenticity (anti-counterfeiting-equipped with × × books, prevent identity from being impersonated)

Prevent traffic hijacking

Https is a solution to eradicate the hijacking of traffic among operators and intermediaries, which can not only put an end to small advertisements inserted in web pages, but also protect the privacy and security of users. Improve search rankings

The use of Https can help improve the search rankings and improve the credibility and brand image of the site. Put an end to fishing websites

The green × × mark of the Https address bar can help users identify phishing websites, protect the interests of users and enterprises from being harmed, and enhance user trust. The relationship between HTTP and TCP

HTTP protocol is usually carried over TCP protocol; HTTP persistent connection and short connection are essentially TCP long connection and short connection. HTTP belongs to the application layer protocol, which uses TCP protocol at the transport layer and IP protocol at the network layer. IP protocol mainly solves the problem of network routing and addressing, while TCP protocol mainly solves how to reliably transmit data packets above the IP layer, so that the receiver on the network receives all the packets sent by the sender, and the order is consistent with the sending order. TCP protocol is reliable and connection-oriented.

Network layer: IP protocol / ARP protocol

Transport layer: TCP/UDP protocol

Application layer: HTTP protocol

Socket is the middle software abstraction layer for the communication between the application layer and the TCP/IP protocol family, and it is a group of interfaces. In the design pattern, Socket is actually a facade pattern, which hides the complex TCP/IP protocol family behind the Socket interface. For users, a simple set of interfaces is all, allowing Socket to organize the data to comply with the specified protocol.

If the application of host A can communicate with the application of host B, it must establish a connection through Socket, and the establishment of Socket connection must require the underlying TCP/IP protocol to establish TCP connection. Establishing a TCP connection requires the underlying IP protocol to address hosts in the network. We know that the IP protocol used in the network layer can help us find the target host based on the IP address, but there may be multiple applications running on one host, and how to communicate with the specified application is specified by the address of TCP or UPD, that is, the port number.

Long connection and short connection

HTTP uses TCP connections in two ways: commonly known as "short connections" and "long connections" ("Keep-Alive" or "Persistent Connection")

Persistent connection: when a web page is opened, the TCP connection used to transmit HTTP data between the client and the server will not be closed, and the client will continue to use this established connection when the client accesses the server again. Keep-Alive does not stay connected permanently, it has a hold time, which can be set in different server software (such as Apache). The realization of persistent connection requires that both client and server support persistent connection.

Using the long-connected HTTP protocol, this line of code is added to the response header: Connection:keep-alive

If the HTTP1.1 version of the HTTP request message does not want to use persistent connections, add Connection: close to the header of the HTTP request message.

TCP's keep-alive function is mainly provided for server applications. It attempts to detect a semi-open connection on the server side and decide whether to close the connection according to the response.

Short connection: every time the client and server perform a HTTP operation, a connection is established, and the connection is interrupted at the end of the task. Either of the two parties can initiate the close operation, but usually client initiates the close operation first.

The advantage of short connection is that it is easy to manage, the existing connections are all useful connections, and no additional control means are needed.

Operation procedure of short connection

Establish a connection-data transfer-close the connection. Establish connection-data transfer-close connection

The procedure for a long connection is:

Establish a connection-data transmission. (stay connected). Data transfer-close the connection

Advantages and disadvantages of long connection and short connection

Long connections can save more TCP establishment and shutdown operations, reduce waste and save time. For customers who frequently request resources, it is more suitable for long connections. However, there is a problem here. The detection cycle of the survival feature is too long, and it is a relatively gentle way to detect the survival of TCP connections. When you encounter malicious connections, the survival feature is not enough. In the application scenario of persistent connections, the client usually does not actively close the connection between them. If the connection between Client and server is not closed all the time, there will be a problem. As there are more and more client connections, server will be unable to handle it sooner or later. At this time, the server side needs to adopt some strategies, such as closing some connections that have not been read or written for a long time. This can prevent some malicious connections from causing damage to server-side services. If the conditions permit, you can take the client machine as the granularity and limit the maximum number of long connections per client, which can completely prevent a painful client from affecting the back-end service.

Short connections are relatively simple to manage for the server, and existing connections are useful connections that do not require additional control. However, if the customer requests frequently, it will waste time and bandwidth on the establishment and shutdown of the TCP.

The generation of long connection and short connection lies in the shutdown strategy adopted by client and server, the specific application scenario adopts specific strategy, there is no perfect choice, only the appropriate choice.

Long-connected data transmission completes identification

Determine whether the transmitted data has reached the size indicated by Content-Length

The dynamically generated file does not have Content-Length, it is a block transfer (chunked). At this time, it should be judged according to the chunked coding. The data encoded by chunked has an empty chunked block at the end, indicating the end of this transfer.

Expiration time of persistent connection

Keepalive_timeout 20;-- persistent connection timeout

Keepalive_requests 8192;-- maximum number of requests per connection

When to use long connection, short connection

Long connections are mostly used for frequent operations, point-to-point communication, and the number of connections should not be too many. Each TCP connection requires a three-step handshake, which takes time. If each operation is connected first, then the processing speed will be much lower, so it will continue to open after each operation, and the OK will be sent directly during the second processing. There is no need to establish a TCP connection. For example, database connections use long connections, frequent communication with short connections will cause socket errors, and frequent socket creation is also a waste of resources.

Http services like WEB sites generally use short links, because long links will consume certain resources for servers, while thousands or even hundreds of millions of clients with short connections like WEB sites will save some resources. If long links are used, and there are thousands of users at the same time, it can be imagined if each user occupies a connection. Therefore, the amount of concurrency is large, but each user needs to use short connection when there is no need for frequent operation.

Set HTTP persistent connection, which has expiration time.

Set request header domain

Set Connection:keep-alive and Keep-Alive: timeout=60 in the header field to indicate that after the connection is established, after the idle time exceeds 60 seconds, it will expire. If you use this connection again when you are idle for 58 seconds, the connection is still valid, and after you finish using it, re-count it and expire after 60 seconds of idle.

Set the HTTP persistent connection with no expiration time; only set Connection:keep-alive in the header field, indicating that the connection is permanent. The connection field is valid only for server settings.

The connection field is valid only for server settings.

The client sets Connection: Keep-Alive and Keep-Alive: timeout=60, and the server sets Connection: Keep-Alive and Keep-Alive: timeout=5.

How to understand that HTTP protocol is stateless

The HTTP protocol is stateless, which means that the protocol has no memory ability for transactions, and the server does not know what the client state is. That is, there is no connection between opening a web page on a server and the last time you opened a web page on that server.

Comparison of MTU-related issues between GET and POST

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.