Shulou(Shulou.com)06/03 Report--

How to use Node.js+COW technology for process creation and file replication, I believe that many inexperienced people do not know what to do, so this paper summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

COW is not a cow, it's an acronym for Copy-On-Write, a technique that replicates but not exactly.

Generally speaking, copying is to create two identical copies, and the two copies are independent:

However, sometimes it is not necessary to copy this, and you can reuse the previous one. At this time, you can just quote the previous copy and copy the corresponding part of the content when you write the content. In this way, if the content is used for reading, there is no need to copy, and if you need to write, you will actually copy part of the content to make changes.

This is called "copy-on-write", or Copy-On-Write.

The principle is simple, but it is common in operating system memory management and file systems, and Node.js has become "lazy" because of this technology.

In this article, we will explore the application of Copy-On-Write in process creation and file replication in Node.js. [recommended: "nodejs tutorial"]

File copy

The most common way to copy a file is to write exactly the same file to another location, but there are two problems:

Write exactly the same content, if the same file is copied hundreds of times, then create the same content hundreds of times? It's a waste of hard disk space.

What if the power is cut off in the middle of writing? How can the overwritten content be restored?

What should I do? This is when operating system designers think of COW technology.

After realizing file copy with COW technology, the above two problems are solved perfectly:

Copy only adds a reference to the previous content, if it is not modified, it will not really copy, and the corresponding data block will not be really copied until the content is modified for the first time, so as to avoid a lot of waste of hard disk space.

When you write a file, you will first make changes in another free disk block, and then copy it to the target location after the modification, so that there will be no problem of power outage and rollback.

The api of fs.copyFile in Node.js can use the Copy-On-Write mode:

By default, copyFile writes to the target file, overwriting the original content

Const fsPromises = require ('fs'). Promises; (async function () {try {await fsPromises.copyFile (' source.txt', 'destination.txt');} catch (e) {console.log (e.message);}}) ()

However, you can specify the replication policy through the third parameter:

Const fs = require ('fs'); const fsPromises = fs.promises;const {COPYFILE_EXCL, COPYFILE_FICLONE, COPYFILE_FICLONE_FORCE} = fs.constants; (async function () {try {await fsPromises.copyFile (' source.txt', 'destination.txt', COPYFILE_FICLONE);} catch (e) {console.log (e.message);}}) ()

There are 3 flag supported:

COPYFILE_EXCL: an error will be reported if the target file already exists (overwrite by default)

COPYFILE_FICLONE: replicate in copy-on-write mode, and switch to real replication if the operating system does not support it (direct replication by default)

COPYFILE_FICLONE_FORCE: replicate in copy-on-write mode and report an error if the operating system does not support it

These three constants are 1, 2, 4, respectively, and can be passed in either bitwise or by combining them:

Const flags = COPYFILE_FICLONE | COPYFILE_EXCL;fsPromises.copyFile ('source.txt',' destination.txt', flags)

Node.js supports the copy-on-write technology of the operating system, which can improve performance in some scenarios. It is recommended to use COPYFILE_FICLONE, which is better than the default.

Process creation

Fork is a common way to create processes, and its implementation is a copy-on-write technology.

We know that the process is divided into three parts in memory: code segment, data segment, and stack segment:

Code snippet: stores the code to be executed

Data segment: store some global data

Stack segment: stores the state of execution

If you create a new process based on this process, copy these three parts of memory. If these three parts of memory are the same, then memory space is wasted.

So fork doesn't actually copy memory, but creates a new process that references the memory of the parent process, and when the data is modified, it actually copies that part of the memory.

This is why process creation is called fork, that is, bifurcation, because it is not completely independent, but some part is forked into two, but most of them are the same.

But what if the code to be executed is different? it's time to use exec, which creates new code segments, data segments, stack segments, and executes new code.

The api of fork and exec can also be used in Node.js:

Fork:

Const cluster = require ('cluster'); if (cluster.isMaster) {console.log (' I am master'); cluster.fork (); cluster.fork ();} else if (cluster.isWorker) {console.log (`I am worker # ${cluster.worker.id} `);}

Exec:

Const {exec} = require ('child_process'); exec (' my.bat', (err, stdout, stderr) = > {if (err) {console.error (err); return;} console.log (stdout);})

Fork is the foundation of linux process creation, which shows how important copy-on-write technology is.

Copying multiple copies of the same content is undoubtedly a waste of space, so the operating system uses Copy-On-Write technology when copying files and creating processes. Only when it is really modified will it be copied.

Node.js supports the flags setting of fs.copyFile. You can specify COPYFILE_FICLONE to use Copy-On-Write to copy files. It is also recommended that you use this way to save hard disk space and improve the performance of file replication.

The fork of the process is also the implementation of Copy-On-Write, and it does not directly copy the code segment, data segment, stack segment of the process to the new content, but before the reference, and the real memory copy will be done only when it is modified.

In addition, Copy-On-Write has many applications in Immutable, distributed read-write separation and other fields.

COW makes Node.js "lazy", but its performance is higher.

After reading the above, have you mastered how to use Node.js+COW technology to create processes and copy files? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.