Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how to use the rust version of peerstore". In the daily operation, I believe many people have doubts about how to use the rust version of peerstore. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "how to use the rust version of peerstore"! Next, please follow the editor to study!

Realize the idea

First of all, since our startup core is swarm, peerstore will be one of the attributes. Secondly, in go-libp2p-peerstore, peerstore mainly stores three pieces of data: the AddrBook of address information, the KeyBook of public and private key information, and the ProtoBook; of protocol information. We can combine the three to form a new struct, named PeerRecord, and put it in the Hashmap with peer_id as the key.

Garbage collection mechanism

The purpose of GC is to prevent hashmap from overexpanding. As the network state is changing all the time, the connection between peer may also change, and an important role of peerstore is to store the address information of peer. If the invalid peer information is not cleaned up, the efficiency of peerstore will be affected.

The current effect is to use task::spawn in swarm's start method to start a task, create a channel of mpsc, use select syntax to wait for a message from the pipeline or task to wait for 10 minutes of logical completion, for some addresses, if the limit of ttl has been exceeded, clean up the current address; at the same time, if there is no address information in the address set of the current peer_id, it is removed from the Hashmap.

Pinned

Although the existence of GC mechanism, so that Hashmap will not be unlimited expansion, good help to the operation of the system. But there are still some inadequacies. Consider the following situation:

For KAD protocol, peer needs to iteratively query known nodes and populate its own KBucket step by step, which is a time-consuming process. If the node address information that was queried earlier is removed from the peerstore during gc, then iterative query needs to be re-enabled to obtain the address, so we added a bool value pinned to the PeerRecord. GC determines the category of the record, and if pinned is true, the cleanup step is skipped.

Serialization and persistence

For every peer, the node information stored in the peerstore should not be discarded when it needs to be offline or downtime for some reason. For the data that needs to be persisted, it also needs to be serialized to facilitate storage.

In libp2p-rs, the call to the main loop is also initiated through task::spawn. When swarm receives a message from close, it exits the event processing loop and sends a close () event to the GC thread running peerstore, ending the gc process. Next, call the save_data () method of peerstore, serialize the data into json format using serde, and use std::io to store the serialized data in the txt file in the root directory.

Method analysis

Parse with the GC method:

The swarm main cycle, spawn, runs task, triggering select every ten minutes.

Hashmap is wrapped by Arc and can be obtained through lock () to ensure concurrency security.

If the information of the peer is not obtained through kad, call retain to filter addresses that do not exceed the ttl time limit.

If the address data for the current peer has been cleared, remove the peer from the hashmap.

/ / swarm/lib.rs / / The GC task is to remove all expired addresses from the peer store task::spawn (async move {log::info! ("starting Peerstore GC..."); loop {let either = future::select (rx.next (), task::sleep (PEERSTORE_GC_PURGE_INTERVAL). Boxed ()). Await Match either {Either::Left ((_, _)) = > break, Either::Right ((_)) = > peer_store.remove_expired_addrs (),}} log::info! ("quitting Peerstore GC...");}) / / core/peerstore.rs / Removes all expired address. Pub fn remove_expired_addrs (& self) {let mut to_remove = vec! []; let mut guard = self.inner.lock () .unwrap (); for (peer, pr) in guard.iter_mut () {if! pr.pinned {log::debug! ("GC attempt for {:?}", peer) Pr.addrs.retain (| record | record.expiry.elapsed () < record.ttl); / / delete this peer if no addr at all if pr.addrs.is_empty () {log::debug! ("remove {:?} from peerstore", peer); to_remove.push (peer.clone ()) } for peer in to_remove {guard.remove (& peer);}}

Netwarps is composed of a domestic experienced cloud computing and distributed technology development team, which has rich experience in finance, power, communications and Internet industries. Netwarps has set up R & D centers in Shenzhen and Beijing with a team size of 30 years. Most of them are technicians with more than 10 years of development experience, from Internet, finance, cloud computing, blockchain, scientific research institutions and other professional fields. Netwarps focuses on the R & D and application of secure storage technology products, including decentralized file system (DFS) and decentralized computing platform (DCP). It is committed to providing distributed storage and distributed computing platform based on decentralized network technology, with the technical characteristics of high availability, low power consumption and low network, which is suitable for scenarios such as Internet of things and industrial Internet.

At this point, the study on "how to use the rust version of peerstore" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.