Shulou(Shulou.com)05/31 Report--

This article will explain in detail what is the solution to big data's security. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have some understanding of the relevant knowledge after reading this article.

Construction background

At present, the global big data industry is in a period of active development, and technological evolution and application innovation are accelerating in parallel. New key technologies for data storage, computing and analysis, such as non-relational database, distributed parallel computing, machine learning and deep mining, emerge as the times require and evolve rapidly. Big data excavates and analyzes to create commercial value and application value in telecommunications, Internet, finance, transportation, medical and other industries. Beginning to transmit and infiltrate to the traditional primary and secondary industries, big data has gradually become the national basic strategic resources and social basic factors of production.

At the same time, big data's safety problems are gradually exposed. Big data has become the key target of network attacks because of its great value and centralized storage management mode. the problems of blackmail attacks and data leakage against big data are becoming more and more serious, and big data security incidents occur frequently all over the world. Accordingly, big data's safety demand has spawned the R & D and production of related safety technologies, solutions and products, but it lags behind compared with the development of the industry.

During the second collective study of the political Bureau of the CPC Central Committee on the implementation of the National big data Strategy, the Chairman pointed out that it is necessary to build a digital economy with data as the key element, promote the integrated development of the real economy and the digital economy, and promote the deep integration of the Internet, big data and artificial intelligence with the real economy. At the same time, it is necessary to effectively ensure the security of national data. This requires us to adhere to the overall concept of national security, establish a correct concept of network security, and adhere to "ensuring development through security and promoting security through development". While giving full play to big data's important role in promoting industrial transformation and upgrading and raising the modernization level of national governance, we should deeply understand the importance and urgency of big data's security, recognize big data's security challenges, and actively deal with complex and severe security risks. We will attach equal importance to security and development, speed up the construction of big data's security system, and ensure the smooth implementation of the national big data development strategy.

Based on the current big data security situation and environment, our company is committed to creating a set of products that can meet the current big data security needs. Strengthen big data's security and reduce the security threat to big data. To solve the increasingly serious security concerns of users in the complex and diverse environment of big data.

Demand analysis

I. requirements of policies and regulations

According to the requirements of national and industry policies, the construction of data security platform should ensure the integrity, confidentiality and availability of data. The requirements for data security construction in the National Network Security Law and the basic requirements for Network Security level Protection are as follows:

1. Network security law

1) take technical measures to monitor and record network operation status and network security events, and keep relevant network logs for not less than six months in accordance with the regulations.

2) protect the network from interference, destruction or unauthorized access, and prevent network data from leaking or being stolen or tampered with

3) take measures such as data classification, important data backup and encryption, etc.

4) the network operator shall not divulge, tamper with or destroy the personal information collected by the network operator, and shall not provide personal information to others without the consent of the collector. However, the exception is that a specific individual cannot be identified and cannot be recovered after processing.

two。 Basic requirements of network security level protection

1) the security audit function covering each user should be provided, the important security events of the application system should be audited, and the functions of statistics, query, analysis and generation of audit report should be provided.

2) the access control function should be provided, the user's access to files, database tables and other objects should be controlled according to the security policy, and the user's operation to the sensitive marked important information resources should be strictly controlled according to the security policy.

3) the system above the third level should adopt encryption or other effective measures to realize the storage confidentiality of system management data, authentication information and important business data.

3. Big data Appendix of Network Security level Protection

1) big data platform should implement identity authentication for the use of data collection terminal, data import service component, data export terminal and data export service component.

2) big data platform should be able to identify and identify big data applications of different customers.

3) big data platform should provide big data applications with the ability to centrally control the use of their computing and storage resources.

4) big data platform should shield the failures of computing, memory and storage resources to ensure the normal operation of business.

5) big data platform should provide static desensitization and desensitization tools or service component technology.

6) big data platform, platform or third party that provides services can access, use and manage the data resources of big data applications only under the authorization of big data applications.

7) big data platform should provide data classification and hierarchical security management function for big data application to take different security protection measures for different types of data.

8) big data platform should provide the function of setting data security tags, authorization and access control measures based on security tags to meet the requirements of fine-grained authorization access control management.

9) the big data platform should support the classified and hierarchical disposal of the data in all aspects of data collection, storage, processing and analysis, and ensure that the security protection strategies are consistent.

10) access control shall be implemented for calls involving important data interfaces and important service interfaces, including but not limited to data processing, use, analysis, export, sharing, exchange and other related operations

11) the processes of data collection, processing, analysis and mining should be tracked and recorded to ensure that the traceability data can reproduce the corresponding process and meet the requirements of compliance audit.

12) the big data platform should ensure the isolated storage of audit data applied by different customers big data, and provide the ability of collection, collection and centralized analysis of audit data from different customers.

4. Information Security Technology big data Service Security capability requirements

Data service is a kind of network information service that covers the data activities related to the life cycle of data through the scalable big data platform at the bottom and a variety of big data applications at the upper layer, aiming at the datasets with the characteristics of huge quantity, variety of types, fast flow speed and changeable characteristics. Big data service providers should ensure that big data platform and applications run safely and reliably, and meet big data service security goals such as confidentiality, integrity, availability and so on. The standard specifies the basic security requirements of the organization and the data service security requirements related to the data life cycle that big data service providers should have.

The standard divides big data's service security capability into two levels: general requirements and enhanced requirements. The general requirement is that the big data service provider should have the security capability when carrying out the big data service, be able to resist or deal with common threats, and control the loss after the destruction of big data's service to a limited scope and extent. have basic event traceability. The enhancement requirement is that when big data's service involves national security, or has a great impact on economic development and social and public interests, big data's service provider should have the security capability, that is, the ability to actively identify and prevent potential attacks. be able to deal with security incidents efficiently and control their losses in a small range. It can ensure the effectiveness of security incident traceability, the reliability, scalability and scalability of big data service. According to the importance of the data carried and the scope and severity of the impact that may be caused when big data's service is unable to provide services normally or is damaged, the security capabilities that big data service providers should have are also different.

The standard specifies the basic security capabilities of the organization and the data service security capabilities related to the data life cycle that big data service providers should have. It can provide a reference for the big data service security capacity building of government departments, enterprises and institutions, and it is also applicable to third-party organizations to review and evaluate the big data service security capability of big data service providers.

II. Risk Analysis of big data platform

Fig. 1 risk analysis chart of big data platform

Big data platform is composed of a number of open source and semi-open source components. Most of the open source components have weak control in data access and authority division. In the process of data exchange between components through interfaces, there is a lack of necessary access control measures for data. At the same time, most of the data in HBASE, HIVE, HDFS and other components are stored in plaintext. These factors lead to the lack of data security protection capability of big data platform itself. At the same time, in big data platform, there are various risks of data leakage during the flow of data in various scenarios such as user access, data sharing, and OPS, development, testing, etc. The specific analysis is as follows:

1. User access

When users access data, there may be some risks, such as attacks such as SQL injection, data not desensitized properly, data precipitation, not accessing data through security interface, data plaintext storage, lack of detailed audit, and so on.

1) attacks such as SQL injection

External users use hacker technology to carry out SQL injection, buffer overflow and other hacker attacks on big data platform.

2) the data is not desensitized properly.

When the data is provided to the user, the data should be provided differently according to the user's access rights. If the sensitive data is not desensitized properly, it will lead to the disclosure of sensitive information.

3) slow data precipitation attack

Business personnel have access to sensitive data, and over time, sensitive data accumulate and become towers, which will become a risk point for data leakage.

4) data is not accessed through the security interface

Users do not access the data through a secure interface, which may result in data disclosure.

5) data plaintext storage

Sensitive data is stored in clear text, and attacks such as bumping into the database and dragging the database are carried out. Once the protection is broken, all the data in HBASE, HIVE, HDFS and other components will be exposed.

6) lack of detailed audit

When users access data, there is a lack of detailed audit of data access behavior, in the event of data leakage, it is impossible to trace the source in time.

two。 Interface data sharing

In the process of sharing data through interfaces, there may be risks such as lack of access control and detailed audit of data.

1) lack of access control

When each component shares data through the interface, it lacks access control to the data and cannot block illegal access.

2) lack of detailed audit

Lack of detailed audit of data in the sharing process, in the event of data leakage, it is impossible to trace the source in time.

3. Operation and maintenance, development, testing

In the process of operation and maintenance, there may be risks such as misoperation or malicious operation by operation and maintenance personnel, lack of proper desensitization of data, lack of detailed audit and so on.

1) misoperation or malicious operation by operation and maintenance personnel

When the operation and maintenance personnel are carrying out data maintenance, it is inevitable that misoperation or malicious operation will occur. This kind of behavior brings great risk to the database.

2) the data is not desensitized properly.

Operation and maintenance personnel, developers and testers have access to real data when working. They are not prevented from being exposed to sensitive data and need to be desensitized properly.

3) lack of detailed audit

Lack of detailed audit of data in the process of data operation and maintenance, in the event of data leakage, it is impossible to trace the source in time.

Construction goal principle

I. Construction objectives

Based on the current security situation and environment of big data, protecting data security has become the core idea of the overall security construction plan. Design and plan the security technology from the aspects of data generation, use, transmission, interaction, storage and destruction, and realize the overall goal of data life cycle security through the construction of "big data security platform". Through data access control, data encryption, data desensitization, data security audit and data situation analysis, combined with leading technologies at home and abroad to carry out multi-dimensional security protection of data, strengthen the technical guarantee of big data's security protection, reduce security threats from multiple dimensions, and solve the increasingly serious security concerns of users in the complex and diverse big data environment.

II. Principles of construction

1. Standardized design and unified standards

The construction of big data security situation management and control platform is a system project, which involves network, host, user, data, application, environment and so on. Therefore, the design and construction of the system strictly follow the requirements of relevant national policies and regulations, and have a certain technical foresight in the field of data security. The system construction adopts unified technical standards, makes full use of existing equipment and resources, and strictly implements various technical indicators. After the completion of the system, each data node and the data itself can achieve the goal of greatly reducing the hidden danger of leakage.

2. Quick deployment and easy to use

The deployment is convenient and quick, and the use is simple and convenient. The deployment of the platform relies on the existing network environment, does not change the original network architecture of the unit, and the system has convenient management capabilities to achieve network management and maintenance, unified audit, and establish an efficient, unified and rapid response data security protection system. At the same time, the front-end business users use the required business systems and underlying numbers without feeling.

3. Ensure safe and stable operation

The whole platform can run stably, normally and continuously, which is the key guarantee to support the business system and the security of front-end users at the data level. The system fully considers the actual work requirements and makes it meet the stable and reliable requirements of the information system without affecting the original office efficiency and the use efficiency of the business system.

4. Flexible design and support for expansion

The whole platform configuration scheme can not only improve the current computing power and overall efficiency, but also take into account the convenient extended storage and application service support according to the business needs and technological development characteristics in the future. The design of the platform takes full account of the current situation of the pilot environment, and can be deployed based on big data's cloud environment or by using highly scalable hardware equipment. For the support of application services, the platform can provide software compatible interface, so that the whole platform can fully meet the requirements of later expansion.

5. Economical and practical, cost-effective

On the one hand, the construction of the platform meets the relevant requirements and closely follows the future trend of information security development, on the other hand, under the premise of meeting the requirements, the system cost should be reduced as far as possible, and the original network environment, equipment and resources should be fully utilized. In line with the practical, adequate, moderately advanced principles of system construction, to avoid repeated construction and waste of funds.

6. Strengthen management and highlight actual results

The platform provides technical support for strictly preventing the leakage of sensitive data and effectively maintaining the security interests of various businesses and data nodes. By using the platform, a data security protection system with prevention, control and management can be formed to enhance the unit's ability to control the data in an all-round way. improve the ability to perceive the risk of leakage, and improve the operation quality of the system through professional maintenance and institutionalized management.

Safety protection scheme

1. Data security protection scheme of big data platform

For the data security protection of big data platform, we can ensure the secure application of big data platform by building big data security platform and uniformly realizing the security of data management components and data flow monitoring. Protect the data content stored and applied in big data platform by protecting the data storage and management components of the platform such as authority control, data desensitization, data encryption and data audit; by monitoring the flow of data in the system, achieve a variety of security objectives, such as data security situation awareness, data leak detection, data behavior audit and traceability.

Fig. 2 schematic diagram of data security protection on big data platform

1. User access

Data security measures are as follows: big data security platform can be protected through data access control, data desensitization, data encryption, data audit and data situation awareness.

1) access control

Effectively block hacker attacks such as SQL injection from external users of the data center.

2) data desensitization

Through data desensitization, the sensitive information is blurred to ensure the safe use of the data.

3) data situational Awareness

Through the situational awareness of data security, analyze and judge whether the sensitive information of the data center has been accessed continuously by a specific user for many times, alarm in time, and prevent data precipitation. At the same time, the situation awareness system can trace the source after the data is leaked.

4) data audit

Data audit can audit the user's data access interface and alarm the data access of non-secure interface. At the same time, it can record data access behavior.

5) encryption protection

Encrypt sensitive data in the database through password computing services.

two。 Interface data sharing

Data security measures are as follows: big data security platform can be protected through data access control, data audit and situational awareness.

1) access control

Real-time access control of data sharing behavior among various components, and block illegal data exchange behavior.

2) data audit and situational awareness

Big data security platform can record data access behavior, perceive the overall situation of data security, visualize risk points and quantify risks through data audit and data situation awareness, so as to form a global vision of data security. The situation awareness system can also realize data traceability at the same time.

3. Operation and maintenance, development, testing

Data security protection measures are as follows: through big data security platform, data access control, data desensitization, data audit and situation awareness can be protected.

1) access control

For high-risk operations such as database deletion and table deletion, it can be blocked in time.

2) data desensitization

Through data desensitization, the sensitive information is blurred to ensure the safe use of the data.

3) data audit and situational awareness

Big data security platform can record data access behavior, perceive the overall situation of data security, visualize risk points and quantify risks through data audit and data situation awareness, so as to form a global vision of data security. The situation awareness system can also realize data traceability at the same time.

Advantages and value of the scheme

Scheme advantage

Association analysis based on multimodal data: establish a complex model system based on multiple dimensions of time and space, run through the multi-layer relationship among users, business and data, and make the model suitable for decision analysis of business scenarios; retrieval methods of ciphertext index and fuzzy index for very large-scale data encryption Provide ES-based log storage technology to achieve big data-level storage as well as efficient retrieval and statistical analysis, graphical display of risks and sensitive data flow status.

Accurate calculation based on intelligent analysis: based on artificial intelligence analysis technology, intelligent analysis of all kinds of alarm data, built-in automatic learning rules, accurate matching of user behavior trajectory whitelist, according to user access model, generate data access baseline, deeply mine the essential meaning of structured data content, achieve accurate identification of alarm data and reduce the false alarm rate of alarm data. Improve the work efficiency of business management personnel.

Custom model design for business requirements: business-oriented requirements provide a security policy warehouse, providing visual and interactive operation windows for creation, modification, replication, deletion, result display and retrieval, so as to achieve centralized security control throughout the network. Through the business-oriented visual custom model, it can effectively improve the efficiency of manual analysis and expand the scope of data security control.

Security protection for the whole life cycle of data: based on the fully independent development technology system, transparent encryption and decryption, data desensitization, data access control combined with multi-threading and multi-cache technology are used to carry out security protection in all aspects of data access, data use, data transmission, data storage and data destruction, so as to achieve efficient, stable and flexible life cycle security protection for data.

Scheme value

This scheme mainly provides a set of overall security solution for data life cycle security protection.

Simplify business governance, improve data security management capabilities, and help customers reduce the risk of data disclosure.

Help the security administrator to open the "black box" of the database system, combined with the system built-in security policy model and behavior self-learning baseline, comprehensively discover all kinds of behaviors and system configuration risks in the use of the database, and give reasonable suggestions for modification. The data security platform comprehensively monitors data access through various means, provides rich statistical reports, graphically displays data access and risks, and provides access control capability. it greatly simplifies business governance and improves the ability of data security management.

Improve the defense system in depth and enhance the overall security protection capability.

The establishment of in-depth defense system has been the consensus of data security construction, from the application system to the database, is the last line of defense of data security, involving the most direct sensitive data security management. The data security platform solution closely revolves around the core data and provides complete protection means, which is conducive to data leveling and improving the defense system in depth, repairing vulnerabilities, reducing the risk of database system being attacked, and improving the overall security protection capability.

Reduce the infringement of core data assets and ensure business continuity.

Data is the most valuable asset and the ultimate goal that attackers will peep, tamper with, and even delete. If the core data is violated, it will lead to business interruption, serious will lead to information disclosure and tampering, a serious threat to information security. The data security platform solution can achieve the visibility and control of data security, ultimately reduce the possibility of core data assets being violated, and ensure normal business continuity.

Upgrade the security level of big data's Hadoop platform

Based on the existing security protection measures of big data platform itself, combined with big data security platform technology, the security protection of the whole life cycle is realized from the bottom of the data to the application layer. Customers have ownership and control over the data. Users without customer authorization or formal authorization can not touch the data, cannot view the audit log and content, and achieve fine-grained control, enhanced security audit and situation early warning analysis. To achieve early warning, monitoring in the event, and tracing back to the source afterwards.

This is the end of what is the solution for big data's security. I hope the above content can be helpful to everyone and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.