In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-19 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Network Security >
Share
Shulou(Shulou.com)06/01 Report--
Detailed explanation of network traffic monitoring
The behavior characteristics of the network can be reflected by the dynamic characteristics of the traffic it carries, so we can monitor various parameters of the traffic in the network (such as receiving and sending Datagram size, packet loss rate, Datagram delay, etc.). The running state of the network can be analyzed from these parameters. By analyzing and studying the characteristics of the traffic carried on the network, it is possible to provide an effective way to explore the internal operation mechanism of the network.
In addition, the network traffic reflects the running state of the network, which is the key to judge whether the network is running normally or not. If the traffic received by the network exceeds its actual carrying capacity, it will cause a decline in network performance. Traffic measurement can not only reflect whether the network equipment (such as routers, switches, etc.) is working properly, but also reflect the resource bottleneck of the whole network operation. Therefore, the health of network traffic in the enterprise network is as important as the blood in the human body.
1. Key technologies of network monitoring
1. Network monitoring
Network monitoring is a management tool for monitoring network status, data flow, and information transmission on the network. the workflow of network monitoring is as follows: the listener collects the data flow of the target network segment through a single or distributed probe, aggregates it to the remote / local data center through a predetermined tunnel, and uses the network traffic / protocol analysis system to complete the preliminary analysis and preprocessing of the massive data, finally according to the task requirements. Complete the identification, location and evaluation of the key data, so as to provide a basis for further action. Network monitoring includes two core technologies, namely, data flow collection technology and network traffic / protocol analysis technology. Data flow collection refers to the collection of data streams from the monitored objects (including stand-alone or intranet network segments) through the deployment of network monitoring probes in a specific location; protocol analysis, it usually refers to the cooperative processing of computer artificial intelligence and intelligence analysis experts to find the key information needed for the task from massive data, and try to achieve the best balance in terms of efficiency and accuracy.
Network traffic / protocol analysis technology can help network operation and maintenance personnel to fully understand and grasp all network behaviors such as network traffic occupation, application distribution, communication connection, original content of data packets, as well as the operation of the whole network. When there is a problem in the network, it can quickly and accurately analyze the cause of the problem, locate the key points, fault points and threat points and deal with them accordingly, so as to ensure that the network runs as expected. It can help us figure out the "details of the internal operation of the network."
2. Deficiency of SNMP protocol
SNMP is the predecessor of the RMON model. At present, SNMP is a network management protocol based on TCP/IP and widely used in Internet. Network administrators can use it to monitor and analyze the operation of the network, but SNMP also has some obvious shortcomings. SNMP uses polling to collect data, and polling in large networks will generate huge network management messages, which will lead to network congestion. SNMP only provides general verification and does not provide a reliable security guarantee. In addition, SNMP does not support distributed management, but uses centralized management. Because only the network management workstation is responsible for collecting and analyzing data, the processing capacity of the network management workstation may become a bottleneck. In order to improve the effectiveness of transmission management messages, reduce the load of network management workstations, and meet the needs of network administrators to monitor the performance of network segments, IETF developed RMON to solve the limitations of SNMP in the expanding distributed interconnection.
3. Key technologies of monitoring
The network monitoring system includes two core technologies: data flow collection technology and network traffic / protocol analysis technology. At the same time, there is another division method in the industry, which summarizes the key technologies of network monitoring into the following three aspects:
Data flow acquisition technology solves the problem of "how to get the network data flow we need from different locations of the network". From the point of view of the location of data acquisition, it can be divided into three types: network-based, host-based and mixed acquisition.
(1) flow monitoring technology.
Traffic monitoring technology mainly includes SNMP-based traffic monitoring and Netflow-based traffic monitoring. Traffic information collection based on SNMP. Collect some specific devices and variables related to traffic information by extracting the MIB provided by the network device agent. The network traffic information collected based on SNMP includes the number of bytes, the number of broadcast packets, the number of packets lost and the length of output queue length, etc.
(2) based on Netflow traffic information collection.
Based on the Netflow mechanism provided by network equipment, the efficiency and effect of traffic information collection can meet the needs of network traffic anomaly monitoring. Based on the above traffic detection technology, there are many traffic monitoring and management software. this kind of software is an effective tool to judge the flow direction of abnormal traffic. through the monitoring of the change of traffic, it can help network administrators to find abnormal traffic. especially the flow direction of abnormal traffic, so as to further find the source and destination addresses of abnormal traffic.
(3) Protocol analysis technology.
Protocol analysis technology is used to understand what protocols and applications are used by users, including protocol and application identification, packet decoding analysis and so on.
4 the difference between NetFlow and sFlow
At present, traffic-based solutions are mainly divided into sFlow and NetFlow. SFlow is jointly developed by Hewlett-Packard and FoundryNetworks. It uses random data flow collection technology, which can adapt to super-large network traffic, such as in the environment of 10 gigabytes of traffic, to implement and analyze network transmission, but there are not many hardware devices that support sFlow. At present, it is supported by Hewlett-Packard, FoundryNetworks and Extreme Networks manufacturers. NetFlow is a Cisco technology currently widely supported in a variety of high-end devices, but the current support for 10 Gigabit traffic is not ideal, it uses timing sampling to collect data. Support for sFlow and Netflow traffic collection is provided in the plug-in of the Ntop tool.
5. Protocol and application identification
According to the contents of the collected Datagram header, the traffic identification technology based on protocol automata is adopted to comprehensively analyze a variety of characteristics, including IP address, port number, keyword, message format, transport layer protocol, etc., and classify the traffic and accurately identify various application layer protocols. Such as database protocols, P2P using dynamic port allocation, encrypted or non-encrypted instant messaging, virtual tunnel applications and so on.
Analysis based on packet decoding. First of all, the collected Datagram is decoded into readable data segments according to the definition of message format, and then the massive data segments are matched with intelligent state pattern. The principle of this technique is to decode in the same way as the client or server in the session. After identifying the types of each part of the communication data, each protocol component searches for information patterns according to the rules defined by RFC. In some cases, it can be done by pattern matching in a specific protocol domain, while others require the use of more advanced technologies or the introduction of human intervention. For example, it is detected according to some specific variables, such as the length of a domain or the number of independent variables.
6. Network data flow acquisition Technology
The best way to control the network communication is to collect the network data flow comprehensively. At present, there are mainly two types, namely hardware probe and software agent. The network probe (Sensor) usually relies on devices such as Hub/ switches / TAP, such as the common switch port analyzer (SPAN) function, which is used in the monitoring parts involved in this book; it can also be used to connect TAP devices in series in the network segment. The network that uses the hub (Hub) as the central switching device of the network is a shared network, the hub works in the way of shared bandwidth, and all the devices connected to the hub are in a conflict domain. Therefore, if the central switching device of the user network is a hub, all data communications in the entire subnet can be captured simply by connecting the listening device to the hub.
Switch port analyzer (commonly known as SPAN) is usually relatively common and acts on the network data flow collection port on the switch. The network administrator configures a port on the switch as a SPAN port, and then the switch copies and sends traffic from its designated port / VLAN to the SPAN port to listen for network traffic. Of course, using SPAN method also has its shortcomings, it works at the expense of switch performance (normally, after SPAN is enabled, the CPU utilization rate of the switch is less than 10%, if more than half of it is not possible to use SPAN scheme). In order to solve this problem, hardware acceleration technology should be used to try traffic collection and analysis in networks with gigabit rates and above. At present, the better one is the GAG series test card developed by Endace Company. Interested readers can inquire deeply on the Internet.
7. Limitations of span
SPAN technology is used in all the cases in the book "Open Source Security Operation and maintenance platform-OSSIM Best practices", but it should be pointed out that Cisco, Huawei and other manufacturers have some restrictions on SPAN:
There can be only one destination port in a SPAN session
There can be only one destination port for different SPAN sessions.
Generally, mid-range Cisco devices usually support only one session.
In situations where the security level and requirements are relatively high (for example, when multiple IDS systems and multiple traffic analysis systems are used in parallel), more than 2 security devices or traffic analysis devices will be required. At this time, due to the limit on the number of SPAN ports of the switch, it is impossible to meet the requirements, so users will usually consider using dedicated traffic analysis access equipment-TAP (Test Access Point), while the traditional SPAN can be used as a supplement. TAP-based traffic replication / aggregator, it is a hardware device, the role is to support multi-port traffic aggregation, and can achieve a real full-line speed, that is, it can be completely copied to multiple monitoring ports for multiple analysis systems to use. Why is it so powerful? because the TAP device uses hardware ASIC to replicate the switching engine, it can guarantee gigabit full-line-speed replication monitoring. Usually, the deployment method is to connect TAP devices in series between the firewall and the core switch, and then connect multiple security devices such as IDS/IPS to the designated port of TAP to achieve the purpose of multiple security devices working at the same time. Below, through Table 1, readers can have a clear understanding of the advantages and disadvantages of the three.
Table 1 comparison of HUB/SPAN/TAP snooping methods
In some large enterprises with highly developed network applications, setting up users to use IBM WebSphere applications in the background, when the problem is that the operation and maintenance personnel will create SPAN ports on multiple switches, we know that Cisco6500 series switches can only set 2 SPAN ports, so if there are multiple monitoring systems, they cannot be used at the same time. Moreover, when the load is heavy, SPAN can not be used, so using the matrix switch can ensure the normal operation of the monitoring tool. And can connect more network sniffing tools to the above for analysis. The matrix switch uses the built-in filtering function more than TAP, which allows operators to select specific data streams through specified tools. Imagine that in a TAP interface that cannot be filtered, it is suddenly washed out by data from 10 Gigabit channels. Using the filtering function of the matrix switch will not overload the sniffer tool.
Second, use Netflow to analyze abnormal network traffic.
With the rapid increase of all kinds of network applications, the network traffic increases rapidly. What is the online behavior of network users in these traffic? How are various types of traffic distributed? In this case, NetFlow, an effective tool, can be used to meet the needs of network traffic management, which is NetFlow. NetFlow was originally developed by Cisco. Because of its wide use, many manufacturers can achieve functions similar to NetFlow, such as Juniper, Extreme, Foundry, H3C. For Cisco, there are many versions of NetFlow, such as V5, V7, V8, V9. Currently, NetFlow V5 is mainstream. Therefore, this article mainly focuses on NetFlowV5. What are the basic elements in this version of the packet? let's start with Flow. For more information, see "Open Source Security Operation and maintenance platform-OSSIM Best practices". The book not only describes how to deploy the Netflow system and how to use it to analyze abnormal traffic, but also uses another open source tool to analyze application layer traffic in detail, and finally presents ways to prevent sniffing technology to fully satisfy your appetite.
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 223
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.