Shulou(Shulou.com)05/31 Report--

This article introduces why you need a map database, the content is very detailed, interested friends can refer to, hope to be helpful to you.

At present, Internet data is growing exponentially, but what is increasing at a faster rate is the relationship between the data. The CIO and CTO of an enterprise not only manage large amounts of data, but also mine business value from existing data, in which case it is more important to deal with the relationship between data than to deal with individual data.

Traditional relational databases perform poorly in dealing with complex data relational operations. with the increase of the amount and depth of data, relational databases can not calculate the results in an effective time. Therefore, in order to make better use of the connection between data, enterprises need a database technology that stores relational information as entities and flexibly expands the data model, which is called graph database (Graph Database).

The graph database is naturally interpretable.

Graph database is a technology that stores, manipulates and accesses graph data based on graph model, which can be easily understood even if there is no professional knowledge reserve of graph theory. It can accept more complex analysis requirements than real-time query to tap the potential value of graph data. In terms of classification, graph database belongs to a kind of NoSQL.

Graph model is an important concept in graph database. The graph model consists of two elements: nodes and edges. Each node represents an entity (a person, place, thing, or other data), and each edge represents a connection between two nodes. this general structure can model a variety of scenarios, such as social networks and anything else defined by relationships.

For example, the following graph model contains three nodes: China, Sichuan, and giant panda. Among them, their two sides are: giant pandas are characteristic of Sichuan, and Sichuan belongs to China.

The basic elements of a graph model: nodes and edges

As can be seen from the graph model above, the goal of the graph database is to simulate these relationships in an intuitive way based on the graph model. Because it is a model representation based on the relationship between things, the graph also has natural interpretability.

The advantages of graph database in dealing with associated data

Compared with relational databases, graph databases have three outstanding technical advantages in dealing with associated data:

High performance: with the increase of the amount of data and the increase of correlation depth, the traditional relational database is restricted by the join operation between multiple tables when retrieving, and foreign key constraints also need to be considered when data writing, which leads to large additional overhead and serious performance problems. The inherent data index structure of the graph model makes its data query and analysis faster.

Flexibility: the graph database has a very flexible data model, and users can adjust the data model at any time according to business changes, such as adding or deleting vertices, edges, expanding or shrinking the graph model, which can be easily realized. Such frequent Schema changes can not be well supported in relational databases.

Agile: the graph model of the graph database is very intuitive and supports the test-driven development model, and functional testing and performance testing can be carried out each time, which meets the popular agile development requirements and helps to improve the efficiency of production and delivery.

We can continue to extend the diagram model use case described earlier to demonstrate the advantages of the diagram database. Beijing also belongs to China, the Great Wall is located in Beijing, Tom has been to the Great Wall, the hot pot restaurant Master Zhang was born in Sichuan, Tom was born in China and likes giant pandas, Master Zhang opened a shop in Beijing, and Tom is a customer of Master Zhang.

Extended graph model

If you are a business / product worker, you must want your product or business to expand to all aspects of the user. If you are a developer, you must want to be able to describe this complicated world simply and efficiently.

In traditional relational databases, how many tables do we need to build in order to make relational queries? Country, province / city, human, animal, landmark, relationship between animal and province / city, relationship between country and province / city, relationship between person and province / city, person to person. Do a rough calculation of at least a dozen tables.

It's okay to build these tables. But if, now we need to ask: which cities do people like giant pandas best?

First of all, you need to associate the animal table, the personnel table and the animal table that people like. If you associate these three tables, you can find that Tom likes giant pandas. But then you need to associate two more tables to find out which landmarks they work in, and then join two tables to find out which city these landmarks are in. Wait, it's not over yet. You still have to group by and sort it out.

You will find this query too difficult! But this is precisely the most basic work of data analysts, and it is also a microcosm of massive information processing in big data's era. Using the graph database, we can easily describe and query the relationship shown in the figure above. In dealing with complex data relational operations, the query efficiency of graph database is much higher than that of relational database.

Application scenario of Graph Database

Graph database technology has been applied to all aspects of real life, such as Google, Facebook and other technology giants have begun to use the power of Graph database to flourish business. According to Gartner's "Ten big data Analytical Technology Trends", the application of global map processing and map database will grow rapidly at an annual rate of 100% from 2012 to 2022.

If we say that knowledge graph is the underlying application scenario of graph database, it makes full use of the advantages of graph model in storage and query to provide knowledge services for multi-industries. Then financial risk control is a high-level application scenario with industry characteristics.

Knowledge graph

As the underlying application of graph database, knowledge graph has served a variety of industries, including intelligent question and answer, search, personalized recommendation and so on. Take intelligent question answering as an example, the products are mainly divided into chat robot and industry intelligent question answering system. The knowledge graph of open domain can provide a wide range of knowledge for chatbots, and machines can not only chat with users but also provide daily knowledge. The industry intelligent question answering system uses the industry knowledge graph, which can provide users with professional knowledge, and has been used in the legal and medical industries.

In the application of knowledge graph, there are two main factors that affect the quality and implementation of knowledge graph-NLP natural language processing engine and algorithm library. NLP natural language processing engine determines the quality and quantity of data obtained by NLP crawler platform, and these raw data as the knowledge raw materials of knowledge graph determine the level of knowledge graph. The graph algorithm in the algorithm library determines the ability of graph construction, graph storage and graph operation. The knowledge raw material is rich and the graph algorithm is backward, so it is still unable to build a powerful knowledge graph.

Financial anti-fraud

By using multi-dimensional cross-related information to depict application and transaction behavior, graph database can effectively identify large-scale and hidden fraud network and money laundering network; combined with machine learning, clustering analysis, risk propagation and other related algorithms, users' risk score can be calculated in real time and identified in advance before the occurrence of risk behavior, which can effectively help financial institutions to improve efficiency and reduce risk. There are many financial risk control scenarios in the application graph database, such as personal credit, money laundering path tracking, personal / enterprise credit information, etc.

Based on the excellent performance of graph database in financial risk control, many enterprises say they are optimistic about this technology, and some forward-looking enterprises have taken the lead in using this technology and gained competitive advantage. Map technology has been developed for many years, but this technology is still not used by many enterprises. what is the reason that hinders the promotion of technology?

First of all, there is the problem of data storage. In the scene of anti-money laundering, users' debit card and credit card data need to be analyzed. When storing, it is found that only 10 months debit card data + 1 month credit card data scale has 5 T, this amount of data can not be supported by the past chart database.

The second point is multi-step analysis of the problem. In the application scenario of anti-money laundering, we need to do more than 3-10 steps of analysis, while the current graph database in the enterprise scenario, there will be timeout or memory overflow when querying from 2 to 3 degrees. Such performance is of little help to fraud detection.

In order to solve these problems, graph database manufacturers are actively building mature solutions to meet these two requirements, and there are more and more high-performance graph databases on the market. At present, the alternative adopted by some enterprises is to achieve the effect of a large amount of data through the map database + big data platform, but such a solution can not be easily mastered because of the high technical threshold.

Industrial field

The graph model has a strong expressive force and strong adaptability to fast-updating things, and is used to manage rapidly changing inventory and supply chain relationships in the industrial field. At present, Volvo and other automobile manufacturers rely on the graph database to optimize the production process and supply chain management.

In the manufacturing industry, supply chain management involves multi-person cooperation and real-time inventory information feedback, including the query of aggregated information and detailed data, which involves many entities and complex relationships. At this time, when facing such deeply related scenarios, the advantages of the graph database are revealed, because the associated data can be found only through the query of the edges, without the need to scan a vertex globally. The graph database can update the inflow data in real time and traverse the data depth.

The Architecture of Graph Database Technology

The technical architecture of the figure database is shown in the following figure, the overall use of hierarchical architecture model, from top to bottom: interface layer, computing layer, storage layer.

The system architecture of graph database

(1) Interface layer: the interface layer provides services in the following ways:

Query language interface: provides language query in addition to the original query language of the graph database, such as Cypher, Gremlin and other mainstream graph query language interfaces.

API: provides ODBC, JDBC, RPC, RESTful and other interfaces to interact with the application side.

SDK: the interface of calling graph database through library functions in Python, Java, C++ and other programming languages.

Visual components: display and realize user interaction in the form of a graphical interface.

(2) Computing layer: provides the processing and calculation of operations, including syntax parsing, query engine, optimizer, transaction management, task scheduling and graph algorithm implementation. Among them, the graph algorithm may be provided by the graph database itself, or it may provide an interface with the graph processing engine.

(3) Storage layer: there are two storage modes of graph database: native storage and non-native storage. Graph storage engine provides the management of graph data structure and index logic.

The unified standard of graph query language represents the improvement of market recognition.

Different from relational database, there is no unified query language in the field of graph database, and most query languages are closely related to products. When enterprises need to use new graph data, they need to relearn syntax, which brings unnecessary learning costs. Whether there is a unified query language standard or not also marks the maturity of the graph database market.

On September 17, 2019, the International Committee on SQL Standards voted to use GQL as a new standard language for querying graph data. It is not possible to determine the first implementable version of GQL, but it is likely that a complete draft of the GQL graph query language will be released in the second half of 2020.

Benefits of uniform query language:

Reduce the cost of enterprise learning-early learning results can be accumulated to play a role in the future. The new query language is not only a simple syntax, but also a new way of thinking about the use of the language. After the unified language, the use of different graph databases will only mean different tools, but the language foundation is the same.

Improve technology maturity-companies worry not only about the cost of learning, but also about the maturity of the technology as a whole. If the industry has a unified query language, that is, when the enterprise thinks that this way of analysis is stable and mature, it will recognize it.

Cloud makes data query and analysis easy to use

At present, there are not many manufacturers who will map the cloud on the database, and a few of them provide cloud database deployment for data scientists, developers, business analysts, students and other enthusiasts. Developers can turn on graph-based solution configuration with simple steps in a short time.

The business growth in the era of big data has brought about a sharp increase in the amount of data and the complexity of data association. at the same time, enterprises have higher and higher expectations for the value of data. According to the popular trend of DB Engines database in the past seven years, graph database is far more popular than other mainstream databases. at present, more and more domestic manufacturers enter the field of graph database and begin to build their own graph database. The construction of graph database requires not only comprehensive big data technology, but also the continuous cooperation of graph database engineers and business experts, which is a long-term and continuous work in the future. Graph database technology will become one of the hottest technologies.

About why you need the map database to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.