Shulou(Shulou.com)05/31 Report--

This article mainly shows you "what the NoSQL ecosystem refers to". The content is simple and clear. I hope it can help you solve your doubts. Let the editor lead you to study and learn this article "what does the NoSQL ecosystem mean?"

Name of NoSQL

Before we define NoSQL, let's try to interpret it from its name. As the name implies, the data manipulation interface of the NoSQL system should be non-SQL type. But in the NoSQL community, NoSQL is given a more inclusive meaning, which means that Not Only SQL, that is, NoSQL, provides a different storage model from traditional relational databases, which provides developers with an alternative to relational databases.

The inspiration of NoSQL

The NoSQL movement has been inspired by many related research papers. Among all the materials, there are two core ones: Google's BigTable paper and Amazon's Dynamo paper.

Overview of Properti

The NoSQL system abandons some functions in the SQL standard and is replaced by some simple and flexible functions. The construction idea of NoSQL is to simplify data operations as much as possible and make the execution efficiency of operations predictable as far as possible. When you check out a NoSQL system, the following points are worth noting.

① data model and operation model: is your application layer data model row, object, or document? Can this system support you to do some statistical work?

② reliability: when you update data, is the new data immediately written to persistent storage? Is the new data synchronized to multiple machines?

③ scalability: how much data do you have and can you accommodate it on a stand-alone machine? Can you support your reading and writing requirements on a stand-alone machine?

④ partitioning strategy: do you need a piece of data to be stored on multiple machines, considering the requirements for scalability, availability, or persistence? Do you need to know or can you know which machine the data is on?

⑤ consistency: is your data replicated to multiple machines? How can the data of these different nodes be consistent?

⑥ transaction mechanism: does the business need an ACID transaction mechanism?

⑦ stand-alone performance: if you plan to persist data on disk, which data structure will meet your needs (do you need to read more or write more)? Will write operations become disk bottlenecks?

⑧ load is assessable: for an application that reads more and writes less, such as a web application that responds to user requests, we always pay a lot of attention to the load. You may need to monitor the data scale and summarize the data of multiple users. Does your application scenario need such a feature?

NoSQL data model and operation model

The data model of the database refers to the organization of the data in the database, and the operation model of the database refers to the way to access the data. Usually, data models include relational models, key-value models and various graph structure models. Operation languages may include SQL, key-value query, MapReduce and so on. NoSQL usually combines a variety of data models and operational models to provide different architectures.

NoSQL data Model based on Key value Storage

In key-based systems, complex joint queries and data query operations that meet multiple conditions are not so easy to achieve, and a different way of thinking is needed to establish and use key names. For example, to get the information of all employees with department number 20, the application layer can first get the list whose Key is employee_departments:20, and then cycle through the ID in this list to get the information of all employees by getting employee:ID.

Key-Value storage

Key-Value storage can be said to be the simplest NoSQL storage, with each Key value corresponding to an arbitrary data value. For the NoSQL system, it doesn't care what this arbitrary data value is. For example, in the employee belief database, the Key of employee:30 may correspond to a piece of binary data that contains all the information of the employee. It doesn't matter if the binary format may be Protocol Buffer, Thrift, or Avro.

Key- structured data storage

A typical example of a Key- structured data store is that Redis,Redis turns the Value stored by Key-Value into a structured data type. The types of Value include numbers, strings, lists, collections, and ordered collections. In addition to set/get/delete operations, Redis also provides many special operations for the above data types, such as adding and subtracting operations for numbers and push/pop operations for list. By providing this specific type of operation for a single Value, Redis can be said to achieve a balance between function and performance.

What is the NoSQL ecosystem?

Key- document storage

Representatives of Key- document storage are CouchDB, MongoDB, and Riak. Under this storage structure, the Value of Key-Value is a structured document, which is usually converted to JSON or a structure similar to JSON for storage. Documents can store lists, key-value pairs, and documents with complex hierarchies.

Column Cluster Storage of BigTable

The data models of HBase and Cassandra are borrowed from BigTable of Google. This data model is characterized by column storage, where items of each row of data are stored in different columns (the set of these columns is called column clusters). Each data in each column contains a timestamp property so that multiple versions of the same data item in the column can be saved.

Column storage can be understood as follows: row ID, column cluster number, column number, and timestamp are combined to form a Key, and then the Value is stored in Key order. The structure of Key value enables this data structure to achieve some special functions, the most commonly used is to save multiple versions of a data into several values with different timestamps, so that historical data can be easily saved. This structure can also naturally store loose column data (there is no data for a column in many rows). Of course, for those columns that rarely have a NULL value in a row, this will result in a waste of space because each data must contain a column identity.

Graph structure storage

Graph structure storage is another storage implementation of NoSQL. The guiding idea is that the data is not equal, relational storage or key-value pair storage may not be the best way to store. Graph structure is one of the basic structures of computer science. Neo4j and HyperGraphDB are the most popular graph structure databases at present.

Complex query

In NoSQL storage systems, there are many more complex operations than key-value lookups. For example, MongoDB can build indexes on any row of data, and you can use Javascript syntax to set complex query conditions. BigTable systems usually support traversing data in a single row, allowing data in a single column to be filtered according to specific criteria. CouchDB allows you to create multiple views of the same data and implement more complex queries or updates by running MapReduce tasks. Many NoSQL systems support large-scale data analysis in combination with Hadoop or other MapReduce frameworks.

Transaction mechanism

Unlike relational databases, NoSQL systems usually focus on performance and scalability rather than transaction mechanisms. Transactions in traditional SQL databases are usually strong transaction mechanisms that support ACID. The support of ACID enables applications to be clear about their current data status. For many NoSQL systems, the performance consideration is far above the guarantee of ACID. Usually the NoSQL system only provides the guarantee of atomicity at the row level, that is to say, two operations on the data under the same Key will be carried out serially in the actual execution, ensuring that each Key-Value pair will not be destroyed.

Storage of Schema-free

Another thing that many NoSQL have in common is that it usually does not have mandatory data structure constraints. Even on document or column storage, it is not required that a data column must exist on every row of data.

Data reliability

Ideally, the database will immediately write all writes to the persistent storage device and copy multiple copies to different nodes in different geographic locations to prevent data loss. However, this requirement of data security has an impact on performance, so different NoSQL systems adopt different strategies in data security under the consideration of their own performance.

Single machine reliability

Stand-alone reliability is very simple to understand, and it is defined that write operations will not be lost due to machine restart or power outage. Usually, the guarantee of stand-alone reliability is accomplished by writing data to disk, which usually causes disk Imando O to become the bottleneck of the whole system. Let's talk about some ways to improve performance under the guarantee of stand-alone reliability.

Control the frequency of fsync calls

Redis provides several ways to control the frequency of fsync calls. Application developers can configure Redis to execute fsync once after each update operation, which is safer and slower. Redis can also be set to call fsync once in N seconds for better performance. But the consequence is that in the event of a failure, it may lead to data loss within N seconds at most. For some occasions where reliability is not too high (for example, when using Redis only as Cache), application developers can even turn off the call to fsync directly: let the operating system decide when to flush the data to disk. Redis can close aof logs, and Redis itself supports the mechanism of dump data in memory into rdb files, which is not the same as above.

Use a log-based data structure

Cassandra, HBase, Redis, and Riak all write to a log file sequentially. The log files mentioned above can be fsync frequently relative to other data structures in the storage system, thus turning random writes to the disk into sequential writes.

Improve throughput by merging write operations

Cassandra has a mechanism that puts several concurrent writes together for a short period of time to make a fsync call, which is called group commit.

Multi-machine reliability

As the hardware level sometimes causes irreparable damage, the guarantee of stand-alone reliability is out of reach at this time. For some important data, cross-machine backup and preservation is a necessary security measure. Some NoSQL systems provide support for multi-computer reliability.

Redis adopts the traditional way of master-slave data synchronization.

MongoDB provides a highly available architecture called Replica Sets.

Riak, Cassandra, and Voldemort provide some more flexible configurable policies and a configurable parameter N, which represents the number of copies of each data that will be backed up. In order to deal with the failure of the whole data center, it is necessary to realize the multi-machine backup function across the data center.

Scale-out brings performance improvement

The goal of scale-out is to achieve a linear effect, that is, if you double the number of machines, you should be able to double the load capacity accordingly. The main problem to be solved is how to distribute data among multiple machines, which involves slicing technology.

These are all the contents of the article "what does the NoSQL ecosystem mean?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.