Shulou(Shulou.com)06/01 Report--

With the rapid development and wide application of information technology, the emergence of new technologies such as the Internet of things, big data, cloud computing and so on, new products and new models of the Internet continue to emerge. Take the financial industry as an example, the emergence of new technologies such as online banking, online trading and Internet finance has not only brought great convenience to people, but also brought many security problems. At present, the field of computer network and information security is facing a new challenge. On the one hand, with the arrival of big data and the era of cloud computing, the security issue is becoming a big data problem. The networks and information systems of enterprises and organizations are generating a large amount of security data every day, and the speed is getting faster and faster. On the other hand, the cyberspace security situation faced by countries, enterprises and organizations is grim, and the threats that need to be dealt with are becoming more and more complex. These threats have the characteristics of strong concealment, long incubation period and strong persistence.

In the face of these new challenges, the limitations of the traditional enterprise security management platform are exposed, mainly in the following aspects:

The processing of massive data

Enterprise security management platform management involves a variety of security devices, network devices, application systems and so on in the enterprise network. A large number of security events and operation logs and other security data are generated every day, and the amount of data may be very huge. In the face of massive security data, it is difficult for security managers to find valuable information; on the other hand, in the face of massive data, the traditional technical architecture of enterprise security management platform also encountered different bottlenecks in data collection, storage, analysis, processing and presentation. Multi-source heterogeneous data acquisition

All kinds of security devices, network devices and application systems in the enterprise network may involve different kinds and different manufacturers. due to the differences in the products of each equipment, the security data faced by the enterprise security management platform is not unified in structure and format, which brings difficulties to data analysis. This problem reduces the data collection efficiency of the enterprise security management platform, which leads to a bottleneck in performance. Security data fragmentation and isolation

All kinds of security devices, network devices and application systems in the enterprise network will be scattered in different locations of the network. if there is a lack of effective correlation between each data, it will lead to the isolation of security information and the formation of an isolated island of information. it is impossible to analyze a large amount of data as a whole. Currently, the * behavior in the network is generally segmented *. Each step may be detected by different security devices and exist in different logs. If you only analyze the security logs of individual devices, it is difficult to find complete * * behavior. In order to improve the accuracy of security data analysis, it is necessary to find out the correlation between multiple alarms and find potential threat behavior or * behavior through event correlation analysis based on big data. Lack of in-depth mining means

In the current network environment, new methods emerge one after another. Different from the traditional methods, the new methods are more hidden, and it is more difficult to find them with traditional detection methods, such as APT***. In the face of the long-term, concealment and advanced nature of the new means, the traditional monitoring technology based on real-time analysis is no longer adaptable. in order to prevent the harm caused by the new means, it is necessary to deeply mine the historical security data and find the clues of the new behavior from a large number of historical data.

The above problems can be summarized in one sentence, that is, massive, multi-source, heterogeneous, decentralized and independent security data, which brings many problems in analysis, storage and retrieval to the traditional enterprise security management platform. From this point of view, the new generation of enterprise security management platform should be supported by the architecture of big data platform, supporting the collection, fusion, storage, retrieval, analysis, situational awareness and visualization of a large amount of data. the past scattered security information is integrated and associated, and independent analysis methods and tools are integrated to form interaction, so as to realize intelligent security analysis and decision-making. Machine learning, data mining and other technologies are applied to security analysis, and security decisions should be made faster and better. The development of big data has brought new challenges to the enterprise safety management platform, but its big data technology has also brought opportunities and new vitality to the enterprise safety management platform.

What is big data?

Big data's popular definition is "a collection of large amounts of data that are difficult to manage with existing general technologies" and in a broad sense as "a comprehensive concept, which includes data that is difficult to manage because of its 4V (massive / diverse / fast / value, Volume/Variety/Velocity/Value) characteristics, and techniques for storing, processing and analyzing these data. And people and organizations who can gain practical meaning and opinions by analyzing these data. "

Big data has four important characteristics (i.e. 4V characteristics): Volume (magnanimity), Variety (diversity), Velocity (fast) and Value (value).

Volume refers to the fact that the amount of data is too large to be effectively processed and analyzed by the current mainstream software tools, so it is necessary to change the traditional data processing and analysis methods.

Variety refers to a wide range of data sources and various forms, including structured data and unstructured data. The growth rate of unstructured data is faster than that of structured data, and it has considerable use value. The analysis of unstructured data can reveal important information that is difficult or impossible to determine in the past.

Velocity means that compared with the traditional data processing system, big data analysis system has higher real-time requirements and needs to complete the calculation in a very short time, otherwise the results will be outdated and invalid.

Value means that big data is valuable, but in the huge amount of data, only a small part of it is really valuable and meaningful.

The Application of big data in Information Security

Big data's application in information security is mainly manifested in that the explosive growth of data has brought challenges to the current information security technology, and the traditional information security technology is no longer suitable in the face of a large amount of data. it is necessary to develop a new generation of security technology based on the characteristics of big data environment. At present, the popular security practice mainly depends on border defense and on static security control measures that require predetermined network threat knowledge. But this security practice is no longer appropriate for today's extremely extended, cloud-based, highly mobile business world. Based on this background, the industry began to shift the research focus of information security to the intelligent-driven information security model, which is a risk-aware, context-based, flexible model that can help enterprises resist unknown advanced network threats. This intelligent-driven information security method supported by big data analysis tools can integrate dynamic risk assessment, analysis of huge amounts of security data, adaptive control measures and information sharing about network threats and technologies. Secondly, big data's concept can be used in information security technology, for example, through big data analysis, we can quickly and effectively analyze the massive network security data and find out the information related to network security. It can be predicted that the integration of big data into security practice will greatly enhance the visibility of the IT environment and improve the ability to distinguish between normal and suspicious activities, thus helping to ensure the credibility of the IT system and greatly improve the security incident response capability.

Safety Analysis of big data

Big data security analysis, as the name implies, refers to the use of big data technology for security analysis. With the help of big data security analysis technology, we can better solve the problem of collecting and storing massive security data, and with the help of machine learning and data mining algorithm based on big data analysis technology, we can have a more intelligent insight into the situation of information and network security, and deal with new and complex threats and unknown and changeable risks more actively and flexibly.

In the field of network security, big data security analysis is the core technology of enterprise security management platform security event analysis, while big data security analysis mainly depends on analysis methods for security data processing. However, when applied to the field of network security, we must also take into account the characteristics of security data and the goal of security analysis, so that the application of big data security analysis is more valuable.

Application of big data Analysis in Enterprise Safety Management platform

At present, the mainstream technical framework applied to big data's analysis is Hadoop, and the industry pays more and more attention to its role in big data's analysis. Hadoop's HDFS technology and HBase technology match big data's super-capacity storage needs, and Hadoop's MapReduce technology can also meet big data's fast real-time analysis needs.

Based on the challenges and limitations of the traditional enterprise security management platform introduced above, Hadoop technology can be applied to the enterprise security management platform and developed into a new generation of enterprise security management platform to support the collection, fusion, storage, retrieval, analysis, situational awareness and visualization of a large amount of data.

The new generation of enterprise security management platform using Hadoop architecture has the following characteristics:

Scalability: supports dynamic addition and deletion of system nodes, and cluster construction is flexible and controllable.

High efficiency: the distributed file system is used to store data, supporting fast read / write and query operations of massive data; distributed computing is used for data analysis and business operations, and each business node is independent of each other. The more nodes, the faster the operation.

Reliability: system automatic disaster recovery (HA); use master-slave mechanism (Master-Slave) to build clusters, real-time backup data between nodes in the system, switch directly to backup nodes when nodes are down, and directly switch to backup nodes when computing units are down.

Low cost: the hardware requirements of the nodes in the system are not high, and the Java technology can be developed across platforms, and the related technology is open source.

In a word, compared with the enterprise security management platform with traditional architecture, the next generation enterprise security management platform using Hadoop can greatly improve the operation speed of data analysis, reduce the operation cost, improve data security, and provide users with various analysis engines and analysis means flexibly.

To sum up, we can see that with the help of big data analysis framework and big data security analysis technology, we can solve the problems of security data collection, analysis, storage and retrieval of the traditional enterprise security management platform. In the long run, the future enterprise security management platform should also improve the function of the enterprise security management platform through the research of new technologies such as machine learning, data mining algorithm, visual analysis and intelligent analysis based on big data analysis technology. so that it can analyze the network security situation more intelligently, so as to deal with new and complex threats and unknown and changeable risks more actively and flexibly. However, no matter how the technology of the enterprise security management platform develops and how it combines with big data, the fundamental customer problems to be solved by the enterprise security management platform and the trend of business integration with customers remain unchanged. The application of big data should still serve the fundamental goal of solving the actual safety management problems of customers.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.