Shulou(Shulou.com)06/03 Report--

Hadoop is an open source framework that allows big data to be stored and processed in a distributed environment of computers with a simple programming model throughout the cluster. Its purpose is to extend from a single server to thousands of machines, each of which can provide local computing and storage.

Ninety percent of the world's data have been generated in the past few years.

With the advent of new technologies, devices and similar communication devices on social networking sites, the amount of data generated by humans is growing rapidly every year. The amount of data generated in the United States from its inception to 2003 was 50 billion gigabytes. If in the form of stacked data disks, it can fill the entire football field. It takes only two days to create the same amount of data in 2011, and the rate is still growing dramatically every 10 minutes in 2013. Although all this information produced is meaningful and useful to deal with, it is ignored. Recommend the big data Learning Exchange Qun: 710219868, which I created by myself. Enter the Qun chat invitation code and fill in the sharing open course with learning route. After listening, you will know how to learn big data.

Headline user qq (630892562)

What is big data?

Big data is a collection of large data sets that cannot be processed by traditional computing techniques. It is not a single technology or tool, but involves many areas of business and technology.

What will happen in big data?

Big data includes data generated through different devices and applications. Here are some areas within the framework of data.

Black box data: this is an integral part of helicopters, airplanes, and jets. It captures the sound of the flight crew, the recording of microphones and headphones, and the performance information of the aircraft.

Social media data: social media such as Facebook and Twitter maintain the opinions of millions of people around the world.

Stock exchange data: exchange data keeps information about "buy" and "sell" and that customers are determined by the share of different companies.

Grid data: grid data maintains information relative to specific nodes consumed by the base station.

Transportation data: traffic data include vehicle model, capacity, distance and availability.

Search engine data: search engines get a lot of data from different databases.

The advantages of big data

By retaining information on social networks such as Facebook, marketing organizations understand the response of advertising media such as their campaigns and promotions.

Use information planning to produce products that like and make consumers aware of products in social media, product companies and retail enterprises.

Using previous medical records of patients, hospitals provide better and faster services.

Big data's technology

Big data's technology is to provide more accurate analysis, which may affect more specific decisions leading to greater operational efficiency, lower costs, and reduced risks to the business.

In order to take advantage of the power of big data, we need to manage and process real-time structured and unstructured massive data, which can protect data privacy and security infrastructure.

Currently in the market a variety of technologies, from different suppliers, including Amazon, IBM, Microsoft and so on to deal with big data. Although we have found techniques to deal with big data, we have studied the following two types of technologies:

Operate big data

These include systems like MongoDB, which provide real-time business capabilities, mainly data capture and storage interaction.

The design of the NoSQL big data system takes full advantage of the advantages that have emerged in the past decade while allowing a large amount of computing to run the new cloud computing architecture cheaply and efficiently. This makes operating big data workloads easier to manage, cheaper and faster to implement.

Some NoSQL systems can provide an in-depth understanding of real-time data patterns based on least-used coding without the need for data scientists and additional infrastructure.

Analysis of big data

These include systems such as massively parallel processing (MPP) database systems and MapReduce systems that provide retrospective and complex analysis that may touch most or all of the data.

Based on the analysis data provided by MapReduce, MapReduce can proportionally increase the functionality provided from a single server to thousands of complementary SQL of high-end and low-end computers, which is a new approach to the system.

These two class technologies are complementary and are often deployed together.

Big data's Challenge

The main challenges related to big data are as follows:

Collect data

Curatorial exhibition

Storage

Search

Share

Transmission

Analysis.

Show

To meet the above challenges, enterprises often need the help of enterprise-class servers.

Traditional enterprise method

In this method, an enterprise will have a computer to store and process big data. For storage, programmers will do it with the help of database vendors of their own choice, such as Oracle,IBM, and users interact with applications to obtain and process data storage and analysis.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.