Shulou(Shulou.com)06/01 Report--

The big data craze in recent years has activated a large number of hadoop learning enthusiasts. Some self-study hadoop, some sign up for training courses to study. Anyone who has been in contact with hadoop knows that building each build in a separate hadoop requires a process such as running the environment, modifying the configuration file testing, and so on. For us beginners, every one is a trap. There are so many domestic distributions of hadoop, it seems that they have not come to fill such a hole? I don't know if I can't solve it, or I didn't think of it?

Install the operating environment of such a pit, those who do the bottom development of domestic big data, if they can not solve this problem, I do not think it is a qualified big data bottom development organization. Fortunately, however, I applied to get a three-node distribution of DKHadoop, the fast open source distribution hadoop, in March. This domestic distribution integrates a variety of commonly used components, such as HDFS,Hbase,Storm,Flume,Kafka,Mahout,Es, and finally does not need to rack their brains to build and configure the underlying platform, and simply complete the installation. This is good news for hadoop beginners.

I've pulled a little too much. I'm sharing the installation and use of DKHadoop with my family. What I want to share with you today is the database in big data's basic content: SQL and NOSQL. To understand these two kinds of data, you only need to understand the concepts of the two and how they are different.

Two concepts:

1. SQL database, which refers to relational database. Main representatives: SQL Server,Oracle,MySQL (open source), PostgreSQL (open source).

2. NoSQL generally refers to non-relational database. Main representative: MongoDB,Redis,CouchDB.

The difference between the two:

In fact, there is a big difference between SQL data and NOSQL data. To sum up, it can be compared and analyzed from the following aspects:

(1) usage scenarios: SQL is a number, which is most suitable for well-defined, precisely standardized independent projects. Typical use cases are online shopping malls and banking systems; NoSQL is a simulation, which is best suited for organizational data without fixed requirements. Typical use cases are social networking, customer management, and network analysis systems.

(2) Storage mode: SQL data is stored in tables with a specific structure, and SQL usually stores data in the form of database tables. For example, save data on student borrowing books:

NoSQL storage is flexible and can be stored in JSON documents, hash tables, or other ways. For example, use a JSON-like file to store the large loan data in the above table:

(3) if external correlation data needs to be added in SQL, the normalization practice is to add a foreign key to the original table to associate the external data table. For example, if you need to add auditor information to the loan table, first create an auditor list:

Then add the auditor foreign key to the original borrower table, so that if we need to update the auditor's personal information, we only need to update the auditor table instead of the borrower table.

In addition to this standardized external data table practice in NoSQL, we can also use the following non-standardized way to put external data directly into the original data set to improve query efficiency. The disadvantage is also obvious, it will be more troublesome to update the auditor data.

(4) data coupling: deletion of external data that has been used is not allowed in SQL. For example, if the "bear three" in the auditor table has been assigned to the borrower bear, then the data bear three will not be deleted in the auditor table to ensure data integrity; while there is no such strong coupling concept in NoSQL, any data can be deleted at any time.

(5) query performance: under the premise of the same level of system design, the performance of NoSQL is theoretically better than that of SQL because the consumption of JOIN query is omitted in SQL.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.