Shulou(Shulou.com)06/02 Report--

This article focuses on "the use of storing and retrieving transactions in Filecoin". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let the editor take you to learn "the use of storing and retrieving transactions in Filecoin".

The Filecoin network consists of hundreds of storage providers distributed around the world. The content addressing and encrypted storage certificate verifies that the data is stored correctly and securely on the miner's hardware for a long time, thus creating a powerful and reliable service.

This paper describes the two types of transactions in Filecoin, the various stages of storing transactions and retrieving transactions, and explains their life cycle in detail. It also explains how cryptographic proof is used to verify whether the participants in the system perform their duties as promised.

Data on Filecoin

To store a file on Filecoin, users must first import the file in their local Filecoin node. This step results in a data CID, the content identifier, that describes the unique ID of the content. After that, the data were passed on to the miners.

Importing data into a local Filecoin node can be done through the lotus client import command. Keep in mind that the resulting data CID is also available on the local node later, as it will be used later when retrieving data from the miners.

After importing the data into the local node, the user needs to initiate a transaction. This is done through the lotus client deal command. This command takes a data CID as input, generates a Filecoin Piece, and interactively guides the user through the storage transaction process.

Filecoin Piece is the main negotiation unit for users to store data on the Filecoin network. Filecoin Piece does not have a specific size, but is subject to the upper limit of sector size and is controlled by network parameters. If a Filecoin Piece is larger than the sector size supported by the miners, it must be divided into more fragments so that each fragment fits a sector.

Filecoin Piece

Each Filecoin Piece is a CAR file containing an IPLD DAG with corresponding data CID and piece CID.

CAR is a content addressable archive, and each CAR file is a serialized representation of IPLD DAG, that is, stringing together its data blocks, plus header information (and root CID) that describes the DAG graph.

When a user wants to store a file in a Filecoin network, the first thing to do is to use UnixFS to make the IPLD DAG of the file (that's what the lotus client import command does). The hash that represents the DAG root node is an IPFS-style CID called the data CID.

UnixFS is an protobuf-based format for describing files, directories, and soft links in IPFS. In Filecoin, UnixFS is the standard file format in which files are submitted to the Filecoin network.

The resulting CAR file is completed with additional zeros so that the file is written as a binary merkle tree.

Store transaction proc

Users access data through transactions in the Filecoin network. Network participants, including miners (supply side) and users (demand side), interact with each other by storing and retrieving transactions.

The lifecycle of the storage transaction is as follows:

1. find

The user first determines the miner and its pricing, that is, the price per epoch (30 seconds) per GiB that the miner wants to receive in order to accept the transaction, in attoFIL. Currently, the minimum duration of a transaction in Filecoin is 180 days.

You can query the synchronized nodes through JSON RPC API, listing all currently active miners, using the Filecoin.StateListMiners method. You can choose according to the reputation and ability of the miners in the network. The reputation index of the miners has not yet entered the Filecoin agreement.

After you have selected the miner, you can use methods such as Filecoin.StateMinerInfo to obtain the miner's PeerID, which is used to establish a secure connection with the other party in the libp2p protocol.

Next, you can use the Filecoin.ClientQueryAsk method to get a signed StorageAsk.

The results include the details of the transaction that the miner is willing to accept, such as the range of Filecoin Piece accepted and the price per GiB per epoch. It is important to note that the proposal of a storage transaction that matches the miner's storage requirements is only a prerequisite, but not sufficient to ensure that the transaction is accepted-the storage provider may run its own decision logic later.

two。 Bargaining and data transmission

At this stage, the two sides reach an agreement on the terms of the transaction, such as transaction cost, transaction duration, transaction start time and so on.

The data is then sent from the user to the miner.

3. Publish

Publish the transaction on the chain through PublishStorageDeals messages, making the storage provider publicly responsible for the transaction.

4. Complete

Once the transaction is published on the chain, it is handed over to the mining subsystem, packaged into sectors, then encapsulated, and then continuously proven to be available.

Storage mining subsystem

The storage and mining subsystem ensures that the data of the Filecoin network is effectively saved by the miners, and:

Participate in Filecoin storage market, undertake user data, participate in storage transactions.

Participate in Filecoin storage computing consensus, validate and produce blocks, let Filecoin block chain grow, and get block awards.

The system oversees the following processes:

Commitment to new storage and registration of new sectors

In order to register a sector in Filecoin, the miner must encapsulate the sector. The _ encapsulation _ process requires a lot of computation to produce a unique representation of the data in the form of proof, that is, a copy certificate or PoRep. Once it is proved to be generated, the miners will compress it and submit the results to the block chain. This proves that the miners did make copies of the data they agreed to store.

To prove that the storage is continuously available, all storage miners need to continuously submit certificates on the chain to verify that the sectors are fully stored.

Declare a storage failure and recover from the failure. If the above proof required by the sector is not successfully submitted, it will lead to a failure and the miner will be punished.

Storage miners and users' considerations, as mentioned above, storage transactions are activated and encapsulated only after they are published on the chain. This is important because releasing a deal locks the user's money on the chain. Only in this way, after the sealed data into the sector, the miners' income can be guaranteed.

You can think of publishing a deal on the chain as signing a contract and encapsulating and activating the deal as starting to make a commitment.

From the user's point of view, if you want to store data in Filecoin, the transaction goes through the following stages:

The transaction is put into money, and the user locks the money into escrow.

Make a deal proposal to the miners

Check the intention to accept the transaction

The data is transmitted to the miners for trading, which is done through the GraphSync protocol. GraphSync is a protocol for synchronizing IPLD graphs between nodes. The protocol allows the local node to make a request to the remote node to get the results of the selector search on the IPLD graph of the remote node. Lotus uses ipfs/go-graphsync, an implementation of the GraphSync protocol.

Check whether it is accepted-make sure the miner has accepted the deal and posted it on the chain.

Encapsulation-the transaction is already on the chain and the miners are encapsulating the sector containing the transaction.

Activate-the deal is sealed and active. From here on, storage providers / miners should periodically certify that they continue to store data.

From the miner's point of view, the transaction goes through the following stages to provide services by storing users' data:

Validate the deal-receive the deal offer and check its parameters (size, price, etc.).

Check if there are locked funds-make sure that users have locked funds and can pay for transactions.

Wait for data-receive transaction data provided by the customer.

Provide collateral for transactions to be used for chain transactions.

Post the transaction on the chain.

Encapsulating sector activation transactions, storage providers (miners) submit WindowPoSt periodically to prove that they are continuously storing data.

Retrieve the transaction process

Unlike storing transactions, retrieving transactions uses payment channels, which are mainly completed under the chain. Data transmission is priced by volume, and users gradually pay fees to miners in the process of data transmission. In the whole process, only the creation of payment channels and exchange vouchers involves interaction with the Filecoin block chain.

The overall process is as follows:

Discovery-the user finds the miner who has the data he needs and asks him for details of the search quotation-price per byte, unsealed price, payment interval.

Set up a payment channel-the user needs to set up a payment channel with the miner (if it does not already exist).

Data transfer and payment-miners send data to users until they need to pay. When a certain threshold is reached, payment processing is required, and then data transmission continues. Depending on whether the miners have data in their block storage, they may need to unencapsulate the data first-this is an unconventional and non-instantaneous operation, which is the reverse of the encapsulation described in the Storage transactions section.

At this time, the user has not yet obtained the complete data.

Space-time proof

The above chapter quickly enumerates many of the details that make Filecoin unique, providing a probability guarantee for user data. This section describes the two proofs used by Filecoin and explains how they become part of the protocol and the problems they solve.

Space-time proof (PoSt) is the proof submitted by miners to the Filecoin network that it is continuing to be the only copy of data stored on the network.

At present, spatio-temporal proof exists in two types in Filecoin:

WindowPoSt

WinningPoSt

WinningPoSt

WinningPoSt is a mechanism that rewards storage miners for their contributions to the Filecoin network. At the beginning of each epoch, a small number of storage miners are selected, and each miner digs out a new block. The specific requirement is that these miners submit compressed storage certificates for the specified sectors. Each elected miner who successfully creates a block is given a FIL (Block Award), as well as an opportunity to charge other Filecoin participants who want to include information in the block.

If the storage miner fails to do so within the necessary time window, he will lose the opportunity to produce the block, but will not be punished for not producing the block.

WindowPoSt

WindowPoSt is the mechanism by which the Filecoin blockchain reviews the commitments made by storage miners.

Every storage miner should maintain their commitment sector. These sectors contain transactions with the user, or may be empty. The latter is called committed capacity, that is, miners can make capacity commitments to fill a sector with arbitrary data rather than user data. Maintaining these sectors allows storage miners to prove that they are reserving space on a generation network.

Each day is divided into time windows, currently 48 time windows, each lasting 30 minutes (60 epoch, because 1 epoch equals 30 seconds).

Each miner's commitment sector is divided into several groups, each corresponding to a time window.

Within a time window (30 minutes), each storage miner must submit a space-time certificate for each sector in the time window. This requires access to each sector of the time window at any time and the generation of zk-SNARK proof that the join block is published to the Filecoin block chain. In this way, each committed storage sector is audited at least once every 24 hours, and a permanent, verifiable, public record is maintained to prove each storage miner's conservative commitment.

In the example above, you can see that a miner should submit window space-time proof in deadline 0 (> 16TB), deadline 1 (< 8TB), and deadline 2 (< 8TB), most of which are in deadline 0. The deadline for each miner is random, starting with epoch 1635, epoch 1695 and epoch 1755 for this particular miner. You can check these deadline and more details about miners on the SpaceGap tool.

The Filecoin network expects the stored data to be continuously available. Failure to commit WindowPoSt for a sector will result in a failure, and the storage miners who supply the sector will be punished. This encourages storage miners to operate healthily.

Malfunction

A failure occurs when it is proved that it has not been included in the Filecoin blockchain within the time limit due to loss of network connection, storage failure, or malicious behavior.

When a sector is registered as a failure, the Filecoin network will punish the storage miners who should have stored the sector; that is, the penalty for the miner's failure to keep the storage will be assessed (paid from the miner's prepaid collateral).

Sector failure fees are divided into three categories:

Sector failure fee: it needs to be paid by each sector in the failure state every day. The size of the fee is slightly higher than the amount of block reward expected to be received by the sector every day. If a sector is in a failure state for more than 2 weeks, the sector will pay a termination fee and be removed from the block chain state.

Sector fault detection fee: this is an one-time payment if the fault is detected by the chain mechanism rather than when the miner reports honestly. Considering the probability of the spatio-temporal proof check, the charge is set as the block reward of the corresponding sector in several days.

Sector termination fee: a sector may be terminated before the expiration date due to failure or miner initiative. The termination fee charged is in principle equivalent to the current revenue generated by a sector and does not exceed a limit so as not to hinder the long-term sector.

Conclusion

This paper describes some concepts about storing and retrieving data on Filecoin, the protocols used by users and miners to access data, and the various proofs and guarantees involved in these processes.

From the perspective of users and miners, this paper introduces in detail the process of storing and retrieving transactions, as well as the punitive measures imposed by Filecoin protocol on a party when there is malicious behavior.

At this point, I believe you have a deeper understanding of "the use of storing and retrieving transactions in Filecoin". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.