Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "what are the similarities and differences between big data and the block chain". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

First, the similarities between big data and the block chain

1. Distributed storage: HDFSvs. block

Big data needs to deal with sea-based and fast-growing storage, which requires that the underlying hardware architecture and file system have a much higher performance-to-price ratio than traditional technology, and can flexibly expand storage capacity. Google's GFS and Hadoop's HDFS laid the foundation for big data's storage technology. In addition, another challenge posed by big data to storage technology is the adaptability of multiple data formats, so now big data's underlying storage layer is not only HDFS, but also storage architectures such as HBase and Kudu.

Blockchain, which is the underlying technical architecture of Bitcoin, is essentially a decentralized distributed ledger. As a continuously growing chained data structure sorted into blocks in sequence, blockchain technology participates in the calculation and recording of data through multiple nodes in the network, and verifies the validity of its information. From this point of view, blockchain technology is also a specific database technology. Because of the characteristics of security and convenience of decentralized database, many people in the industry are optimistic about its development and think that it is an upgrade and supplement to the existing Internet technology.

2. Distributed computing: MapReduce vs. Consensus mechanism

Big data's analytical mining is data-intensive computing, which requires huge distributed computing power. Node management, task scheduling, fault tolerance and high reliability are the key technologies. The MapReduce of Google and Hadoop is the representative of this distributed computing technology, which can linearly expand the total processing power (Scale Out) of the system by adding server nodes, which has great advantages in cost and scalability. Now, in addition to batch computing, big data also includes flow computing, graph computing, real-time computing, interactive query and other computing frameworks.

The consensus mechanism of block chain is how to reach a consensus among all distributed sections, to generate and update data through algorithms, to determine the validity of a record, which is not only a means of identification, but also a means to prevent tampering. Blockchain mainly includes four different consensus mechanisms, which are suitable for different application scenarios and strike a balance between efficiency and security. Bitcoin, for example, uses "Proof Of Work" (POW), which makes it possible to forge a record that does not exist only if it controls more than 51 per cent of the accounting nodes in the network.

3. The separation and combination of the development of IT technology.

Like human society, the development of IT technology also shows a "long-term division, division must be United", that is, the spiral of concentration and distribution.

In the early days of the birth of computers, only one-to-one use could be achieved, and it was centralized. In order to enable a mainframe to provide services for multiple customers at the same time, IBM introduced the design idea of virtualization, so that when multiple customers use the same mainframe at the same time, it is as if they have divided it into multiple miniaturized virtual hosts, which is the centralized computing of time division multiplexing.

Entering the era of minicomputers and PC, it has returned to one-on-one use, but the devices have been distributed to thousands of households. In the Internet era, the client and server of Cmax S model are distributed computing, but the servers are still distributed.

In the era of cloud computing, computing power is under unified control. Based on the distributed computing of clients and servers, distributed collaborative work begins between servers. Because of collaboration, they can also be considered as a centralized computing service as a whole.

Entering the era of big data, cloud computing has become the infrastructure of big data, which also makes big data's core idea and cloud computing come down in a continuous line. MapReduce decomposes the tasks into distributed computing, and then merges the results to realize the integrated analysis of information.

Blockchain is a distributed system in a pure sense.

Second, the difference between big data and the block chain.

1. The two technologies are in different life cycles.

In 2011, "big data" made the list for the first time, in the climbing stage of the budding stage of technology, when it was collectively referred to as "'Big Data' and Extreme Information Processing and Management" ("big data" and extreme information processing and management). It went further in 2012 and almost reached the peak of the overheating period in 2013. After the decline in 2014, "big data" suddenly disappeared from the curve in 2015, which can be interpreted as the Gartner's position of big data has changed from "emerging" to "mainstream". At present, the significance of big data to the enterprise has risen from the element of ability to the core of strategy.

Relatively speaking, the "blockchain" did not appear in the "technology maturity curve" for the first time until 2016 and entered the "overheating period" directly. Generally speaking, the life cycle stages of "big data" and "blockchain" are very different, and there is a gap of about 5 years between them.

2. What are the main differences?

Big data is usually used to describe data sets that are large and complex enough that it is difficult to deal with them in the traditional way. However, the information and data that the block chain can carry is limited, which is far from the "big data" standard. There are several significant differences between Block chain and big data:

Structured vs unstructured: blockchain is a well-defined block, a chain of pointers, typical structured data, while big data needs to deal with more unstructured data

Independent vs integration: in order to ensure the security of the blockchain system, the information is relatively independent, while big data focuses on the integration and analysis of information.

Direct vs indirect: the blockchain system itself is a database, while big data refers to the in-depth analysis and mining of data, which is a kind of indirect data.

Mathematical vs data: blockchain tries to speak with mathematics, blockchain advocates "code is law", while big data tries to speak with data.

Anonymous vs personality: the blockchain is anonymous (public account book, anonymous owner, as opposed to the public account of traditional financial institutions, account confidentiality), while big data intends to personalize

Only by understanding the similarities and differences between big data and the block chain, can we combine big data with the block chain, make the data in the block chain more valuable, and put big data's predictive analysis into action, which will be the cornerstone of the digital economy era.

This is the end of the content of "what are the similarities and differences between big data and the block chain". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.