Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "sample Analysis of Gitee Block chain Open Source Project". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Md_blockchain

Java block chain platform, based on Springboot development block chain platform. Blockchain qq exchange group 737858576, together to learn blockchain platform development, of course, also exchange Springboot, springcloud, machine learning and other knowledge.

cause

In order to develop the block chain, the company originally wanted to use Ether Square to develop a contract or use a third-party platform to do it, but later found that it did not meet the business requirements. The reason is very simple, Taifang, super account books and other platforms are to do shared accounts, there are tokens and mining and other modules. What we need is for several companies to form an alliance to witness and record some untampered interactive information, such as Company A sends a xxx request to Company B and Company B responds to what. In fact, all you want is a distributed database with good performance, and you can't generate a block in 10 minutes like bitcoin. What we want is more about the performance of the database and some of the features of the blockchain.

Pass through

The research and development of the project began in early March 18, and the first edition was released in January. It mainly does the storage module, encryption module, network communication, PBFT consensus algorithm, public key and private key, block content analysis, storage and so on. It has initially possessed the basic characteristics of blockchain, but it is not in place in terms of merkle tree, intelligent contract and other details.

It is hoped that the experts will not hesitate to give advice, brainstorm, and put forward opinions or proposals to do a blockchain platform project, which is suitable for more blockchain scenarios, not just account books and various deceiving tokens.

Project description

There are mainly storage module, network module, PBFT consensus algorithm, encryption module, block parsing storage and so on.

The project belongs to "chain", not "currency". No virtual coins and mining are involved.

Storage module

Stored in Block are Sql-like statements. The database table structure that meets the needs of the business scenario is set in advance between the alliances, and then the operation authority (ADD,UPDATE,DELETE) of each node on the table is set. In the future, each node can write Sql statements according to their allowed permissions, package them to Block, and broadcast them across the network, waiting for the validity of signatures, permissions and other information to be verified across the network. If Block is legal, then enter the PBFT consensus algorithm mechanism, and each node starts to execute according to the states of PrePrepare, Prepare, Commit and so on, until after 2f+1 commit, it starts to generate new blocks locally. After the new block is generated, each node parses the content of the block and stores it on the ground.

The scenario is more extensive, you can set different table structures, or multiple tables, and then you can complete the storage of each type of information. For example, tracing the source of goods, from manufacturers, transportation, distributors, consumers and so on, each link can carry on the operation of ADD information on a commodity.

The storage uses the key-value database rocksDB, and those who know about Bitcoin know that Bitcoin uses levelDB, and it's all similar. You can dynamically switch which database to use by changing db.levelDB to true,db.RocksDB to false in yml.

Statements with a structure similar to sql, such as ADD (add, delete, modify) tableName (table name) ID (primary key) JSON (json of the record). The logic for rollback is set here, that is, when you do an ADD operation, a Delete statement is stored for possible future rollback operations.

Network module

In the network layer, each node is long connected to each other, disconnected and reconnected, and then the heartbeat packet is maintained. The network framework uses t-io, which is also a well-known open source project of oschina. T-io adopts the way of AIO, which has excellent performance in the case of a large number of long connections, takes up very little resources, and has the function of group, so it is especially suitable for the SaaS platform with multiple alliance chains. And it includes excellent functions such as heartbeat, disconnected reconnection, retry and so on.

In the project, each node is not only server, but also client. As server, it is connected by other NMUE 1 node, and as client, it connects the server of other NMUE 1 node. For the same federation, set a Group. Every time you send a message, you can call the sendGroup method directly.

However, it should still be noted that because the pbft consensus algorithm is adopted in the project, the third power of N network communication will be generated in the process of reaching consensus. When the number of nodes is large, such as 100, each consensus will bring heavy burden to the network. This is the limitation of the algorithm itself.

Consensus module PBFT

Distributed consensus algorithm is the core of distributed system, such as Paxos, pbft, bft, raft, pow and so on. The common ones in blockchain are POW, POS, DPOS, pbft and so on.

Bitcoin uses POW workload proof that it takes a lot of resources to do hash operations (mining), and miners have the right to generate Block. Others mostly use election voting to decide who will generate the Block. The common feature is that only specific nodes can generate blocks and then broadcast them to others.

Block chains are divided into three categories:

Private chain: this refers to the blockchain application deployed within the enterprise. All nodes can be trusted and there are no malicious nodes.

Alliance chain: a semi-closed ecological trading network with unequal trust nodes and possibly malicious nodes

Public chain: an open ecological trading network that provides a global trading network for alliance chains and private chains.

Because the private chain is a closed ecological storage system, the best performance can be achieved by using Paxos class consensus algorithm (more than half consent). The federation chain is semi-open and semi-open, so Byzantine fault tolerance is one of the suitable choices, such as the IBM super ledger project. For the public chain, the requirements of this consensus algorithm have gone beyond the scope of ordinary distributed system construction, coupled with the characteristics of transactions, so it is necessary to introduce more security considerations. So Bitcoin's POW is a very good choice.

What we can choose here is raft and pbft, which are private chain and alliance chain respectively. I used the modified pbft consensus algorithm in the project.

Let's start with a brief look at pbft:

(1) A master node (Leader) is elected from the nodes of the whole network, and the new block is generated by the master node.

(2) each node broadcasts the transactions sent by the client to the whole network, and the main node will collect from the network to sort a number of transactions to be placed in the new block, and then store the list into a list, and broadcast the list to the whole network.

(3) after receiving the transaction list, each node executes these transactions according to the sort simulation. After all transactions are executed, the hash summary of the new block is calculated based on the transaction results and broadcast to the whole network.

(4) if a node receives a summary from 2f (f is the tolerable number of Byzantine nodes), it broadcasts a commit message to the whole network.

(5) if a node receives a 2f+1 (including its own) commit message, it can submit the new block to the local block chain and status database.

(6) if the client receives the return of f + 1 successes (even if there are f failures and then f maliciously returned error messages, f + 1 correct ones are also a majority), the write request can be considered successful.

As you can see, the traditional pbft needs to elect the leader first, and then the leader collects the transactions, packages them, and then broadcasts them. Then each node begins to check, vote, accumulate the number of commit for the new Block, and finally land.

And I have made changes to pbft here, this is an alliance, each node is equal, and the performance is high. So I don't want each node to generate an instruction, send it to other nodes, and then elect a node to collect instructions on the network and regenerate it into Block. It's too complicated, and there are hidden dangers of leader nodes.

My modification to pbft is that there is no need to select leader, any node can build Block, and then broadcast all over the network. When other nodes receive the Block request, they enter the Pre-Prepare state, verify the permissions of the format, hash, signature, and table, and enter the Prepare state after the verification is passed, and broadcast across the network. When the number of accumulated Prepare of each node is greater than 2f+1, enter the commit state, and broadcast the state throughout the network. When the number of accumulated Commit of each node is greater than 2f+1, it is considered that a consensus has been reached, Block is added to the chunk chain, and then the sql statement in Block is executed.

Obviously, there is a lack of the concept of order compared with when you have leader. When there is leader, the order of Block can be guaranteed, and when there is a need for Block, leader can broadcast sequentially. For example, everyone has already reached the block of number=5, and then two more need to be generated. When there is leader, it will be generated in the order of 6 and 7. Without leader, it is possible for multiple nodes to generate 6 at the same time. In order to avoid bifurcations, I have done some processing, specifically you can see the implementation logic in the code.

Block information query

Each node implements a synchronous sqlite database (or other relational database such as mysql) by executing the same sql. In the future, the query to the data will be a direct query sqlite, which is better than the traditional block chain project.

Because each node can generate Block, there will be block inconsistency under high concurrency. If the chain forks for some reason, a rollback mechanism is also provided, and sql can roll back. The principle is also very simple, when you ADD a data, I will record two instructions in the block at the same time, one is ADD, the other is DELETE for rollback. In the same way, old data is saved in UPDATE. The sql in the block hits the ground, such as executing 1-10 instructions sequentially, and the rollback instruction is executed from 10-1 when rolling back.

Each node records the value of the block it has synchronized so that the sql can be stored on the ground at any time.

It is easy to query the block chain information, just do the database query directly. Compared with Bitcoin, which needs to retrieve the index tree of the entire block chain, speed and convenience are very different.

Simple instructions for use

How to use: first download the md_blockchain_manager project, then import the sql database file in the project, modify the application.yml database configuration, and finally start the manager project.

Then modify a value corresponding to name, appid and manager project database in application.yml in md_blockchain as a node. If there are multiple nodes, then each node corresponds to the database. Fill in the ip of each node. ManagerUrl is the url of the manager project, giving the project access to the manager project.

When the md_blockchian project starts, it is visible in the ClientStarter class, and at startup, all nodes' data is pulled from the manager project and connected. If your ip, appId, and so on are not in the manager database, you cannot start.

You can generate a chunk by accessing localhost:8080/block?content=1. In normal use, at least 4 nodes must be started, otherwise no consensus can be reached. PBFT requires the consent of 2f+1 nodes before Block can be generated. To facilitate testing, you can directly change the return value of pbftSize to 0 so that you can play on your own node. If you have more than one node, you will find that other nodes will automatically synchronize their newly generated Block after generating the Block. At present, a table message is set by default in the code, and there is only one field content, which is equivalent to a simple block chain notepad. When there are four nodes, you can generate Block at the same time to test by accessing several of them concurrently to see if it will fork. You can also shut down one of them to see if the other three can reach a consensus (Byzantium allows a maximum of f node failures, 4 nodes allow 1 failure), restore the failed one, and see if you can synchronize the Block of other normal nodes. All kinds of tests can be carried out, welcome to mention bug.

You can view the data stored in sqlite through localhost:8080/block/sqlite, which is the result of execution according to the sql statement in Block.

I deployed the project to docker and started a total of 4 nodes, as shown in the figure:

The four ip nodes are all written dead, and after they are all started, they will all connect to each other, maintain long connections and heartbeats, and exchange the latest Block information with each other.

Other nodes will be like this, receive a request from the block project to generate a block, start the verification, and then enter the voting state of pbft

In addition, in the case of high concurrency, each node generates Block at the same time, and the system handles consensus and ensures that the block chain does not bifurcate.

The API for generating blocks is written for testing. The normal process is to call the instuction API, first produce instructions that meet your needs, and then combine multiple instructions to call the generated block API in BlockController.

This is the end of the "sample Analysis of Gitee Block chain Open Source Project". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.