Shulou(Shulou.com)06/01 Report--

Today, with the rapid development of big data, the amount of data is constantly increasing, especially the rise of Internet web2.0 applications and the development of mass storage and massive computing needed by cloud computing. The traditional relational database has exposed many insurmountable problems and has been unable to meet the needs of this aspect. On the other hand, the non-relational database (NoSQL=Not Only SQL, that is, "not just SQL") has developed rapidly because of its own characteristics, because it is produced to solve the problem of big data application in the storage and calculation of large-scale collection and multiple data.

According to the international data company IDC testing statistics, in 2011, the total amount of global data has reached 1.8ZB (1ZB equals 1 trillion GB), and the total data growth doubles every two years. It is estimated that the global total will reach 40 ZB by 2020, and the per capita data on the earth is expected to reach 5247GB. According to statistics, Google's search engine now uses more than 2 million users per second. Twitter generates more than 10TB data every day, Facebook data is dozens of TB, and some special enterprises generate TB-level data every hour.

However, how to use these data to generate the greatest business value is a common challenge for all companies and institutions. This problem is more complex, although technicians have become more and more aware of the data with the help of skills and professional tools, but as the data is growing faster and faster and accumulating more and more, the proportion of data available to companies and institutions is declining rapidly.

We know that sql database can store and process data, but one of the biggest advantages of NOSQL is that in dealing with big data, it can deeply deal with big data, analyze data, and get useful resources from a large amount of data.

However, the traditional relational database appears to be inadequate in big data's processing, exposing many insurmountable problems, such as:

1. The demand for high concurrent read and write of database.

Web2.0 applications need to generate dynamic pages and provide dynamic information in real time according to users' personalized information, so it is basically impossible to use the static technology of dynamic pages, so the concurrent load of the database is very high, often reaching tens of thousands of read and write requests per second. Relational database can barely cope with tens of thousands of SQL queries, but the hard disk IO can no longer handle tens of thousands of SQL write data requests. In fact, for ordinary BBS sites, there is often a demand for high concurrent write requests, such as real-time statistics of online user status, recording the number of clicks on popular posts, voting count and so on, so this is a very common demand.

2. The need for efficient storage and access of massive data

SNS sites like Facebook,twitter,Friendfeed generate a large amount of user dynamics every day. Take Friendfeed as an example, there are 250 million user dynamics in a month. For relational databases, it is extremely inefficient and even unbearable to query SQL in a table of 250 million records. For example, the user login systems of large web sites, such as Baidu, Ali and Tencent, often have hundreds of millions of accounts, and relational databases are also difficult to cope with.

3. Requirements for high scalability and high availability of the database

In the web-based architecture, the database is the most difficult to scale out. When the number of users and visitors of an application system is increasing day by day, your database has no way to expand the performance and load capacity as easily as web server and app server by adding more hardware and service nodes. For many websites that need to provide 24-hour uninterrupted service, it is very painful to upgrade and expand the database system, which often requires downtime maintenance and data migration. why can't the database be expanded by constantly adding server nodes?

Therefore, the digital media technology department uses MongoDB in the non-relational database to store data. The advantages of MongoDB are beyond the reach of traditional relational databases. We use MongoDB to solve the difficult problems of big data application in the storage and calculation of large-scale collection and multiple data.

Digital gathering Media has an accurate crowd database and a powerful crowd cookie model, which can directly locate the target audience; nearly 6, 000 user tags allow advertisements to find users intelligently; multiple optimization mechanisms, rendering to release the best results, to build a full scene data marketing model, and finally through intelligent algorithms and bidding principles to maximize the effect of advertising. The reasonable bidding mode makes the advertising of advertisers accurate and cost-effective.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.