Shulou(Shulou.com)06/03 Report--

HyperLeger Fabric Development (5)-- HyperLeger Fabric account Book Storage I. introduction to HyperLeger Fabric account Book

The data in Fabric is stored as a distributed ledger. The ledger consists of a series of sequential and tamper-proof records that contain all state changes of the data. The data items in the account book are stored in the form of key-value pairs, and all the key-value pairs in the account book constitute the state of the account book, also known as World State.

There is a unique ledger in each channel, which is maintained by all the members of the channel, and a copy of the ledger of the channel is kept on each accounting node, so it is a distributed ledger. The access to the account book needs to add, delete, update and query the key-value pairs of the account book through the chain code.

The account book consists of two parts: block chain and state database.

A blockchain is an immutable set of ordered blocks (data blocks) that keep a log of all transactions. Each block contains data for several transactions, and the number of transactions contained in different blocks can be different. Chunks are associated with a Hashed-link: each block contains the hash values of all transactions in that block and the hash values of the previous block head. The chained structure ensures that the data of each block cannot be changed and the sequential relationship between each block cannot be changed. Therefore, blocks of a block chain can only be added at the end of the chain.

The status database records the current values of all key value pairs in the account book, which is equivalent to indexing the transaction log of the current account book. When the chain code executes the transaction, the current status of the account book needs to be read, and the latest status of the key value can be quickly obtained from the status database.

If there is no state database, in order to obtain a key value, we need to traverse the transactions related to the key value in the whole block chain, which is very inefficient. Therefore, reading the state database can be regarded as a way to quickly locate and access a key value. In addition, when the state database fails, it can be regenerated by traversing the ledger.

When a block is attached to the end of the block chain, if the valid transaction in the block modifies the key-value pair, the corresponding update will be made in the state database to ensure that the block chain and the state database are always consistent.

The data blocks of the blockchain are saved in the form of files in each node. The state database can be a variety of key-value databases, and Fabric uses LevelDB by default, and also supports the option of CouchDB. CouchDB not only supports key-value data, but also supports document model in JSON format, which can do complex queries.

2. HyperLeger Fabric transaction process 1. The client sends a transaction proposal request to the endorsement node.

The client first builds the transaction proposal, which is used to call the chain code in the channel to read or write the data in the account book. The client uses Fabric SDK to create the transaction proposal and signs the transaction proposal using the user's private key.

Fabric SDK has two functions, one is to encapsulate the transaction proposal into a message in Protobuf format that conforms to the gRPC protocol, and the other is to sign the created deal proposal.

After the client has packaged the transaction proposal, it submits the transaction proposal to the endorsement node in the channel. The endorsement strategy of the endorsement node defines which endorsement nodes can sign and endorse the transaction effectively. The client selects the corresponding endorsement node according to the endorsement strategy and submits the transaction proposal to the corresponding endorsement node.

According to the endorsement strategy, the three endorsement nodes need to complete signature endorsement.

2. Endorsement node signs and endorses the transaction proposal.

After receiving the transaction proposal, the endorsement node uses the MSP module to verify the signature and determine whether the requester is reasonably authorized to operate the transaction proposal (using the ACL corresponding to each channel for verification). The endorsement node takes the transaction proposal voucher as input and generates the transaction result based on the database execution of the current state, and the output includes feedback value, read set and write set. At this time, the account book has not been updated. The endorsement node will send the output of the endorsement node, the signature of the endorsement node, and the endorsement declaration to the SDK of the client as feedback on the transaction proposal.

First of all, verify whether the signature of the transaction is legal, and then confirm whether it has the authority to conduct the relevant transaction according to the identity of the signer. In addition, the endorsement node also needs to check whether the transaction proposal is in the correct format and whether it has been submitted (to prevent replay).

After all the validity checks are passed, the endorsement node invokes the chain code to simulate the execution of the transaction according to the transaction proposal. When the chain code is executed, the read data (key-value pair) is the local state database in the endorsement node, and the data read by the chain code is summed up to the read set (Read Set); the write operation of the chain code to the state database will not change the account book, and all write operations will be recorded in a Write Set. The read and write sets will be used in the confirmation node to determine whether the transaction is eventually written to the account.

After the chain code execution is completed, the endorsement node signs the read-write set (Read-Write Set) and other information obtained from the chain code simulation execution and sends it back to the client. At this time, the transaction information only reaches a consensus between the client and a single endorsement node, and does not complete the network-wide consensus, and the transaction order of each client is not determined, so there may be a double flower problem, so it is not an effective transaction. At the same time, the client needs to receive the verification reply from most endorsement nodes before the verification is successful. The specific endorsement strategy is controlled by the intelligent contract code and can be freely configured by the developer.

The endorsement node E0memeE1recoverE2 executes signature endorsement.

3. The endorsement node returns the signature endorsement result to the client

After the endorsement node completes the signature endorsement, the feedback value, the read set and the write set, the endorsement node signature, and the endorsement declaration are sent back to the client as the transaction proposal feedback.

4. The client submits the transaction to the sorting service

After receiving the signature endorsement result of the endorsement node, the client checks the signature of the endorsement node and compares whether the results of different endorsement nodes are consistent. If the transaction proposal is a request to query the ledger, the client does not need to submit the transaction to the sorting node. If the transaction proposal is a request to update the account book, the client collects the signature endorsement results of enough endorsement nodes that meet the endorsement policy, and sends the read / write set returned by the endorsement node, the signatures and channel numbers of all endorsement nodes to the sorting service node.

5. Sorting service nodes generate blocks

After receiving the transaction from each node, the sorting service node does not check all the contents of the transaction, but sorts the transaction according to the channel number in the transaction, and then packages the transactions of the same channel into blocks. The consensus algorithm of the sorting service is inserted into the Fabric network in the form of components, that is, developers are free to choose the appropriate consensus algorithm.

The sorting service node broadcasts the chunks generated for a particular channel to the leading nodes of all organizations in the channel. There are two trigger conditions for block broadcasting, one is when the number of transactions in the channel reaches a preset threshold, and the other is when the number of transactions does not exceed the threshold but the time from the last broadcast exceeds a certain threshold, broadcast data blocks can also be triggered. The two methods are combined to make the sorted transactions generate blocks in time and broadcast to the Peer node (accounting node) of the channel.

The sorting service node only determines the order of transaction processing, does not verify the legitimacy of the transaction, and is not responsible for maintaining the account book information. Only the accounting node has the right to write the account book.

6. The accounting node verifies the transaction

After receiving the block sent by the sorting service node, the accounting node checks the transactions in the block one by one. First check the legality of the transaction and whether it has ever occurred. Then the verification system chain code (VSCC,Validation System Chaincode) is called to verify whether the signature endorsement of the transaction is legal and whether the number of endorsements meets the requirements of the endorsement strategy.

The accounting node performs a multi-version concurrency control (MVCC) check on the transaction, that is, to verify that the read set (Read Set) of the transaction is consistent with the version in the current account (that is, there is no change). If there is no change, the modification of the data in the transaction write set (Write Set) is valid, the transaction is marked as valid, and the write set of the transaction is updated to the status database. If the data of the current account is inconsistent with the version of the read set, the transaction is marked as invalid and the status database is not updated. After being marked as valid or invalid, the transaction data in the block is encapsulated as a block and written into the block chain of the account book.

In the transaction process, the optimistic lock model of MVCC is adopted to improve the concurrency ability of the system. But MVCC also brings some limitations. For example, if two transactions in the same block update a data item successively, the later transaction will fail because the read set version of the subsequent transaction is no longer consistent with the current data item version.

7. The accounting node notifies the client

The client receives a notification from the accounting node of each connection.

8. Summary of transaction process

The account book of the blockchain is maintained by the Peer node, not by the sorting service cluster, so only the Peer node (endorsement node and accounting node) contains complete blockchain information, while the sorting service cluster is only responsible for sorting transactions and retaining only part of the blockchain information in the process.

The node in the Hyperledger Fabric network is a logical concept, which is not necessarily a physical device, but for the production environment, in order to decouple the system architecture, improve scalability and improve security through host isolation, the Peer node cannot be deployed on the same machine as the sorting service node, while the endorsement node and the accounting node can be deployed on the same machine. The endorsement node verifies the client's signature and then executes the intelligent contract code to simulate the transaction. After the transaction processing is completed, the transaction information is signed and returned to the client. After receiving the signed transaction information, the client sends it to the sorting service node for sorting. After the sorting service node packages the transaction information into blocks, it broadcasts it to the accounting node and writes it into the block chain.

Third, HyperLeger Fabric account book storage principle 1. Fabric account book storage principle

In the Fabric block chain network, each channel has its own account book, and each Peer node keeps the account book of the channel it joins. The account book of the Peer node contains the following data:

A, account book number, which is used to quickly query which account books exist.

B, ledger data, for block data storage

C, block index for quick query of blocks / transactions

D, status data, for the latest world state data

E, historical data: track the history of keys

There are four kinds of databases in Fabric's Peer node ledger: idStore (ledgerID database), blkstorage (block file storage), statedb (state database) and historydb (historical database).

Based on the file system, the account database stores the block in the file block, and then stores the file block and its offset corresponding to the block transaction in the block index LevelDB, that is, the block index LevelDB is used as the index of the account database. Currently, the supported block indexes are: block number, block hash, transaction ID, block transaction number.

The state database stores the latest values of all key-value pairs that have appeared in the transaction. Calling the chain code to execute the transaction can change the state data, and in order to execute the chain code call efficiently, the latest values of all data are stored in the state database; the state database is a snapshot of the ordered transaction log, which can be regenerated at any time according to the transaction log; the state database will be automatically restored or reconstructed when the Peer node is started, and the Peer node will not accept new transactions until it is complete. State databases can use LevelDB or CouchDB,CouchDB to store arbitrary binary data and support rich text queries.

The historical status database is used to query the historical modification records of a certain key. The historical status database does not store the specific value of the key, but only records a change of a key in a transaction in a block. When you need to query later, query the actual change value according to the change history.

The ledgerID database stores chainID, which is used to quickly query which account books exist in the node.

For single-chain Peer node account books (excluding ledgerID database), the structure is as follows:

2. Peer book structure

LedgersData is the root directory of the Peer node's account book. The Peer node's account book is stored in the / var/hyperledger/production/ledgersData directory of the Peer node container. You can enter the Peer node container to view it through the command line, as follows:

Docker exec-it peer0.org1.example.com bash

The local account structure of the Peer node is as follows:

The name of the mychannel directory channel under the chains/chains directory. Fabric supports a multi-channel mechanism, while the account books between channels are isolated, and each channel has its own account space.

The chains/index directory contains levelDB database files, stores the block index database, and is implemented using leveldb.

The historyLeveldb directory stores the index address of the history of key written in the smart contract, which is implemented using leveldb.

The ledgerProvider directory stores information about the channel contained in the current node (the channel id that has been created and the channel id that is being created), which is implemented using leveldb.

The stateLeveldb directory stores data written by smart contracts, optionally using leveldb or couchDB

3. Ledger data

Account data is stored in the form of binary files, and each account data is stored in a different directory. All operations of the ledger data are implemented through the block file manager. The file created by the block file manager is prefixed with blockfile_ and suffixed with 6 digits. The suffix must be a continuous number from small to large, and there can be no deletions in the middle, so the account book can hold up to 1000000 file blocks. The default upper limit for block file block size is 64m (currently hard-coded in code). The structure of the account data is as follows:

B represents blockchain ledger data. Among them, B0 is the creation block, which contains block head H0, block data D0 (the creation block does not contain transaction data), and block metadata M0. Block B1 block head H1, which contains the encrypted hash of the previous block B0 and the encrypted hash of its own block.

4. Block structure

(1) Block head

The Block Header consists of three parts:

A, number of chunks (Block Number): an integer starting at 0 (Genesis chunk) and increasing by 1 for each new block attached to the chunk chain.

C, current block hash (Current Block Hash): the hash of all transactions contained in the current block.

C. Previous block hash (Previous Block Hash): hash of the previous block.

(2) Block data

Chunk data (Block Data) contains a set of transactions that are written when the chunk is created.

(3) Block metadata

Chunk metadata (Block Matadata) contains the chunk creation time, the certificate, public key, and signature written to the client.

5. Transaction structure

(1) Trading head

The transaction header contains some important transaction metadata, such as the name and version of the chain code.

Signature

(2) signature

The signature contains an encrypted signature created by the client and generated by the client's private key to check whether the transaction has been tampered with.

(3) transaction proposal

The deal proposal contains the transaction request parameters generated by the client.

(4) transaction proposal is returned

The transaction proposal returns the simulation execution result (read / write set RWset) returned by the endorsement node.

(5) endorsement

Endorsement contains the endorsement of the transaction, and one return corresponds to multiple endorsements.

6. Block index

The block index is used to quickly locate blocks.

The index key can be a block height, a block hash, a transaction hash.

The index value is the block file number + intra-file offset + block data length.

Hyperledger Fabric provides a variety of ways to index blocks so that they can be found quickly. The content of the index is the file location pointer (File Location Pointer). The file location pointer consists of three parts: the number of the file (fileSuffixNum), the offset within the file (offset), and the number of bytes occupied by the chunk (bytesLength).

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.