Shulou(Shulou.com)06/03 Report--

5. Code structure. (3) I hand O writing process.

In the last article, we introduced the space management of dm dedup.

In this article, we introduce the core process, IWeiO writing process.

To see this section very clearly, you need to learn it together with the "device-mapper Block-level dm dedup Design" I wrote earlier. Please add a link description.

This series of processes have been described in the re-delete design article at the block level.

The previous code structure has analyzed the lookup and insert of kvs_hash and kvs_lbn.

Next let's take a look at the use of lookup and insert in the writing process.

First of all, let's take a look at the function that alloc_pbn_block gives to lbn, which will be used later.

[/ * because there is no pbn to place the lbn yet So first apply for a pbn_new*/ static int alloc_pbnblk_and_insert_lbn_pbn (struct dedup_config * dc, U64 * pbn_new, struct bio * bio) Uint64_t lbn) {int r = 0 Struct lbn_pbn_value lbnpbn_value; r = allocate_block (dc, pbn_new); / * find a new pbn*/ lbnpbn_value.pbn = * pbn_new; do_io (dc, bio, * pbn_new) / * place the data of bio in the location of pbn_new * / r = dc- > kvs_lbn_pbn- > kvs_insert (dc- > kvs_lbn_pbn, (void *) & lbn, sizeof (lbn), (void *) & lbnpbn_value, sizeof (lbnpbn_value)) / * record the relationship of the new lbn_pbn to kvs_lbn_pbn * / return r;}]

1. No hash & & no lbn

That is, ① compute_hash_bio (dc- > desc_table, bio, hash); / / A pair of data of bio is hash to get the hash_pbn

-> ② dc- > kvs_hash_pbn- > kvs_lookup; / / find the corresponding hash_pbn_value, that is, pbn, through the hash value.

Did not find pbn- > ③ handle_write_no_ hash.[ DC-> kvs_lbn_pbn- > kvs_lookup] / / find out whether the lbn of bio exists

Did not find lbn-> ④ handle_write_no_ hash.[ _ _ handle_no_lbn_pbn] / / here comes the final handler function

In order to reduce the chapter, resource application error handling and resource access error handling are removed.

Static int _ _ handle_no_lbn_pbn (struct dedup_config * dc, struct bio * bio, uint64_t lbn, U8 * hash) {int r, ret; U64 pbn_new; struct hash_pbn_value hashpbn_value; / * Create a new lbn-pbn mapping for given lbn, note here is pbn_new * / r = alloc_pbnblk_and_insert_lbn_pbn (dc, & pbn_new, bio, lbn) / * Inserts new hash-pbn mapping for given hash. Because dm_io has placed bio data in pbn, then hash_pbn records * / hashpbn_value.pbn = pbn_new; r = dc- > kvs_hash_pbn- > kvs_insert (dc- > kvs_hash_pbn, (void *) hash, dc- > crypto_key_size, (void *) & hashpbn_value, sizeof (hashpbn_value)) / * Increments refcount for new pbn entry created. Add a reference * / r = dc- > mdops- > inc_refcount (dc- > bmd, pbn_new); / * On all successful steps increment new write count. * / dc- > newwrites++;/* records the increase of newwrites in dc for better statistics and analysis of behavior * / goto out;/* Error handling code path * / inc_refcount_err: / * Undo actions taken in hash-pbn kvs insert. * / ret = dc- > kvs_hash_pbn- > kvs_delete (dc- > kvs_hash_pbn, (void *) hash, dc- > crypto_key_size); kvs_insert_err: / * Undo actions taken in alloc_pbnblk_and_insert_lbn_pbn. * / ret = dc- > kvs_lbn_pbn- > kvs_delete (dc- > kvs_lbn_pbn, (void *) & lbn, sizeof (lbn)); ret = dc- > mdops- > dec_refcount (dc- > bmd, pbn_new); out: return r;}

2. No hash & & has lbn

That is, ① compute_hash_bio (dc- > desc_table, bio, hash); / / A pair of data of bio is hash to get the hash_pbn

-> ② dc- > kvs_hash_pbn- > kvs_lookup; / / find the corresponding hash_pbn_value, that is, pbn, through the hash value.

Did not find pbn- > ③ handle_write_no_ hash.[ DC-> kvs_lbn_pbn- > kvs_lookup] / / find out whether the lbn of bio exists

Find lbn-> ④ handle_write_no_ hash.[ _ _ handle_has_lbn_pbn] / / here comes the final handler

Static int _ _ handle_has_lbn_pbn (struct dedup_config * dc, struct bio * bio, uint64_t lbn, U8 * hash, U64 pbn_old) {int r, ret; U64 pbn_new; struct hash_pbn_value hashpbn_value; / * Allocates a new block for new pbn and inserts lbn-pbn lapping. Note that this overrides * / r = alloc_pbnblk_and_insert_lbn_pbn (dc, & pbn_new, bio, lbn); / * Inserts new hash-pbn entry for given hash. * / hashpbn_value.pbn = pbn_new; r = dc- > kvs_hash_pbn- > kvs_insert (dc- > kvs_hash_pbn, (void *) hash, dc- > crypto_key_size, (void *) & hashpbn_value, sizeof (hashpbn_value)); / * Increments refcount of new pbn. * / r = dc- > mdops- > inc_refcount (dc- > bmd, pbn_new); / * Note that pbn_new+,pbn_old-, needs to be reduced to recyclable pbn_old * / / * Decrements refcount for old pbn and decrement logical block cnt. * / r = dc- > mdops- > dec_refcount (dc- > bmd, pbn_old); dc- > logical_block_counter--; / * On all successful steps increment overwrite count. * / dc- > overwrites++; / * the overwrites++, here is different from the newwrites above * / goto out;/* Error handling code path. * / dec_refcount_err: / * Undo actions taken while incrementing refcount of new pbn. * / ret = dc- > mdops- > dec_refcount (dc- > bmd, pbn_new); inc_refcount_err: ret = dc- > kvs_hash_pbn- > kvs_delete (dc- > kvs_hash_pbn, (void *) hash, dc- > crypto_key_size); kvs_insert_err: / * Undo actions taken in alloc_pbnblk_and_insert_lbn_pbn. * / ret = dc- > kvs_lbn_pbn- > kvs_delete (dc- > kvs_lbn_pbn, (void *) & lbn, sizeof (lbn)); ret = dc- > mdops- > dec_refcount (dc- > bmd, pbn_new); out: return r;}

3. Hash & & no lbn

That is, ① compute_hash_bio (dc- > desc_table, bio, hash); / / A pair of data of bio is hash to get the hash_pbn

-> ② dc- > kvs_hash_pbn- > kvs_lookup; / / find the corresponding hash_pbn_value, that is, pbn, through the hash value.

Find pbn- > ③ handle_write_with_ hash.[ DC-> kvs_lbn_pbn- > kvs_lookup] / / find whether the lbn of bio exists

Did not find lbn-> ④ handle_write_no_ hash.[ _ _ handle_no_lbn_pbn_with_hash] / / here comes the final handler function

Now that hash_pbn has been found, pbn can be reused. Just associate lbn with pbn directly.

Static int _ handle_no_lbn_pbn_with_hash (struct dedup_config * dc, struct bio * bio, uint64_t lbn, U64 pbn_this, struct lbn_pbn_value lbnpbn_value) {int r = 0, ret; / * Increments refcount of this passed pbn * / r = dc- > mdops- > inc_refcount (dc- > bmd, pbn_this); if (r

< 0) goto out; lbnpbn_value.pbn = pbn_this; /* Insert lbn->

Pbn_this entry * / r = dc- > kvs_lbn_pbn- > kvs_insert (dc- > kvs_lbn_pbn, (void *) & lbn, sizeof (lbn), (void *) & lbnpbn_value, sizeof (lbnpbn_value)); if (r

< 0) goto kvs_insert_error; dc->

Logical_block_counter++; bio- > bi_status = BLK_STS_OK; bio_endio (bio); / * bio completes * / dc- > newwrites++; goto out;kvs_insert_error: / * Undo actions taken while incrementing refcount of this pbn. * / ret = dc- > mdops- > dec_refcount (dc- > bmd, pbn_this); out: return r;}

4. Hash & & lbn

That is, ① compute_hash_bio (dc- > desc_table, bio, hash); / / A pair of data of bio is hash to get the hash_pbn

-> ② dc- > kvs_hash_pbn- > kvs_lookup; / / find the corresponding hash_pbn_value, that is, pbn, through the hash value.

Find pbn- > ③ handle_write_with_ hash.[ DC-> kvs_lbn_pbn- > kvs_lookup] / / find whether the lbn of bio exists

Find lbn-> ④ handle_write_no_ hash.[ _ _ handle_has_lbn_pbn_with_hash] / / here comes the final handler

Now that hash_pbn and lbn_pbn are found, there are two situations:

First, overwrite, that is, hash_pbn 's pbn and lbn_pbn are one.

2. No relationship, that is, writing the contents of a pbn to a new lbn location

Static int _ _ handle_has_lbn_pbn_with_hash (struct dedup_config * dc, struct bio * bio, uint64_t lbn, U64 pbn_this, struct lbn_pbn_value lbnpbn_value) {int r = 0, ret; struct lbn_pbn_value this_lbnpbn_value; U64 pbn_old; pbn_old = lbnpbn_value.pbn / * special case, overwrite same LBN/PBN with same data * / if (pbn_this = = pbn_old) goto out; / * if hash_pbn and lbn_pbn are not equal, 1. Add pbn reference and new lbn and pbn association * / / * Increments refcount of this passed pbn * / r = dc- > mdops- > inc_refcount (dc- > bmd, pbn_this); if (r

< 0) goto out; this_lbnpbn_value.pbn = pbn_this; /* Insert lbn->

Pbn_this entry * / r = dc- > kvs_lbn_pbn- > kvs_insert (dc- > kvs_lbn_pbn, (void *) & lbn, sizeof (lbn), (void *) & this_lbnpbn_value, sizeof (this_lbnpbn_value)); if (r

< 0) goto kvs_insert_err; /*减小lbn之前记住的pbn_old的引用，这个pbn_old将可能被回收*/ /* Decrement refcount of old pbn */ r = dc->

Mdops- > dec_refcount (dc- > bmd, pbn_old); if (r

< 0) goto dec_refcount_err; goto out; /* all OK */dec_refcount_err: /* Undo actions taken while decrementing refcount of old pbn */ /* Overwrite lbn->

Pbn_this entry with lbn- > pbn_old entry * / ret = dc- > kvs_lbn_pbn- > kvs_insert (dc- > kvs_lbn_pbn, (void *) & lbn, sizeof (lbn), (void *) & lbnpbn_value, sizeof (lbnpbn_value)); if (ret

< 0) DMERR("Error in overwriting lbn->

Pbn_this [% llu] with "" lbn-pbn_old entry [% llu]. ", this_lbnpbn_value.pbn, lbnpbn_value.pbn); kvs_insert_err: ret = dc- > mdops- > dec_refcount (dc- > bmd, pbn_this); if (ret

< 0) DMERR("Error in decrementing previously incremented refcount.");out: if (r == 0) { bio->

Bi_status = BLK_STS_OK; bio_endio (bio); dc- > overwrites++;} return r;}

This article introduces the four situations of writing the process, and explains the flow chart of dm dedup design more clearly.

I hope that after reading it, readers will be attracted to dm dedup, which is a simple logical way, and thus like the technology of block re-deletion.

-to be continued-

[this article is only posted by 51cto blogger "underlying Storage Technology" https://blog.51cto.com/12580077, official account release: storage Valley], if you need to reprint, please contact me, thank you.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.