Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "what is the difference between pnpm and npm/yarn". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

The following is the mind map of this article:

What is pnpm?

The official documentation of pnpm says:

Fast, disk space efficient package manager

Therefore, pnpm is essentially a package manager, which is no different from npm/yarn, but its two advantages as a killer are:

Package installation is extremely fast

Disk space utilization is very efficient.

It is also very easy to install. How simple can it be?

Npm I-g pnpm

II. Overview of features

1. High speed

How fast does pnpm install the package? Take the React package as an example to compare:

As you can see, in most scenarios, the package installation speed of pnpm, as a yellow part, is obviously faster than that of npm/yarn, which is 2-3 times faster than that of npm/yarn.

Students who are familiar with yarn may say, doesn't yarn have PnP installation mode? Directly remove the node_modules and write the contents of the dependency package to disk, saving the overhead of the node file Iploo, which can also improve the installation speed.

Next, let's take such a warehouse as an example. Let's take a look at benchmark data, mainly comparing pnpm and yarn PnP:

As you can see, overall, the package installation speed of pnpm is significantly faster than that of yarn PnP.

two。 Efficient use of disk space

Pnpm internally uses a content-addressable file system to store all files on disk, and the advantages of this file system are:

The same package will not be installed repeatedly. When using npm/yarn, if all 100 projects rely on lodash, lodash will probably be installed 100 times, and this part of the code will be written in 100 places on disk. But in the use of pnpm will only be installed once, there is only one place on the disk to write, and later use will directly use hardlink (hard link, unclear students see this article (https://www.cnblogs.com/itech/archive/2009/04/10/1433052.html))).

Even if there are different versions of a package, pnpm reuses previous versions of the code to a great extent. For example, if lodash has 100 files and one more file is added after the newer version, the 101files will not be rewritten to the disk, but the hardlink of the original 100files will be retained, only the new file will be written.

3. Support for monorepo

With the increasing complexity of front-end engineering, more and more projects begin to use monorepo. In the past, for the management of multiple projects, we usually use multiple git repositories, but the purpose of monorepo is to use a git repository to manage multiple subprojects, and all subprojects are stored in the packages directory of the root directory, then one subproject represents a package. If you have not been exposed to the concept of monorepo before, it is recommended to take a closer look at this article and the open source monorepo management tool lerna, the project directory structure can refer to the babel repository.

Another big difference between pnpm and npm/yarn is that it supports monorepo, which is reflected in the functions of various subcommands, such as pnpm add A-r in the root directory, then all package will be added with A dependency, and of course, the-filter field is also supported to filter package.

4. High security

In the past, when using npm/yarn, because of the flat structure of node_module, if A depends on B and B depends on C, then C can be used directly in A, but the problem is that C is not declared in A. As a result, there will be such illegal visits. However, pnpm is particularly clever and has created a set of dependency management, which solves this problem well and ensures security. How to embody security and avoid the risk of illegal access dependence will be discussed in detail later.

III. Dependence management

Npm/yarn install principle

It is mainly divided into two parts. First, how the package arrives in the project node_modules after the npm/yarn install is executed. Second, how dependencies are managed within node_modules.

After executing the command, the dependency tree is first built, and then for the packages under each node, you go through the following four steps:

-1. Parse the version range of the dependent package to a specific version number

-2. Download the tar package that the corresponding version depends on to the local offline image

-3. Extract the dependency from the offline mirror to the local cache

-4. Copy the dependency from the cache to the node_modules directory of the current directory

The corresponding package will then arrive in the node_modules of the project.

So what is the directory structure of these dependencies within node_modules, in other words, what is the dependency tree of the project?

Nested structures are presented in npm1 and npm2, such as the following:

Node_modules └─ foo ├─ index.js ├─ package.json └─ node_modules └─ bar ├─ index.js └─ package.json

If there are dependencies in the bar, then they will continue to be nested. Imagine what's wrong with such a design:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

The dependency level is too deep, which can lead to the problem of long file paths, especially on window systems.

A large number of duplicate packages are installed and the file size is super large. For example, if there is a baz in the same directory as foo, and both rely on the same version of lodash, then lodash will be installed in the node_modules of both, that is, repeated installation.

Module instances cannot be shared. For example, React has some internal variables, and the React introduced in two different packages is not the same module instance, so the internal variables cannot be shared, resulting in some unpredictable bug.

Then, starting with npm3, including yarn, start to solve this problem by flattening dependencies. I believe everyone has this experience, I obviously installed an express, why there are so many things in node_modules?

Yes, this is the result of flattening dependency management. Compared to the previous nested structure, the current directory structure looks like this:

Node_modules ├─ foo | ├─ index.js | └─ package.json └─ bar ├─ index.js └─ package.json

All dependencies are flattened to the node_modules directory, no longer have deep nesting relationships. In this way, when installing a new package, according to the node require mechanism, you will keep looking for it in the node_modules of your superior. If you find the same version of the package, you will not reinstall it. This solves the problem of repeated installation of a large number of packages, and the dependency level is not too deep.

The previous problem has been solved, but if you think about this flattening approach, is it really unassailable? It's not. It still has a lot of problems. Sort it out:

Depends on the uncertainty of the structure.

The flattening algorithm itself is very complex and time-consuming.

Packages that do not declare dependencies can still be illegally accessed in the project

The latter two are easy to understand, so what does the uncertainty in the first point mean? Here, let's explain in detail.

If the project now relies on two packages, foo and bar, the dependencies of these two packages are as follows:

So in npm/yarn install, after flattening, what is it like this?

Or like this?

The answer is: both are possible. Depending on the position of foo and bar in the package.json, if the foo declaration is in front of it, it is the front structure, otherwise it is the latter structure.

This is why there is uncertainty about the dependency structure, and it is also the reason for the birth of lock files, whether it is package-lock.json (npm 5.x only appeared) or yarn.lock, to ensure that a definite node_modules structure is generated after install.

In spite of this, npm/yarn itself still has the problems of complex flattening algorithm and illegal access to package, which affects performance and security.

Pnpm dependency management

Zoltan Kochan, author of pnpm, found that yarn did not intend to solve these problems, so he started a new package manager and created a new dependency management mechanism. let's find out now.

Or take installing express as an example, let's create a new directory and execute:

Pnpm init-y

Then execute:

Pnpm install express

Let's take a look at node_modules:

.pnpm .modules.yaml express

We directly saw the express, but it is worth noting that here is only a soft link, do not believe you open to see, there is no node_modules directory, if it is the real file location, then according to the node package loading mechanism, it can not find the dependency. So where is its real location?

We continue to look for .pnpm:

▾ node_modules ▾ .pnpm ▸ accepts@1.3.7 ▸ array-flatten@1.1.1... ▾ express@4.17.1 ▾ node_modules ▸ accepts ▸ array-flatten ▸ body-parser ▸ content-disposition... ▸ etag ▾ express ▸ lib History.md Index.js LICENSE package.json Readme.md

Oh, boy! Found it under .pnpm / express@4.17.1/node_modules/express!

Open any other bag at random:

It seems to be the same rule, with the directory structure of @ version/node_modules/. And express's dependencies are all under .pnpm / express@4.17.1/node_modules, and these dependencies are all soft links.

Let's take a look at the .pnpm, .pnpm directory, although it presents a flat directory structure, but think about it, slowly unfold along the soft link, in fact, it is a nested structure!

▾ node_modules ▾ .pnpm ▸ accepts@1.3.7 ▸ array-flatten@1.1.1... ▾ express@4.17.1 ▾ node_modules ▸ accepts->.. / accepts@1.3.7/node_modules/accepts ▸ array-flatten->.. / array-flatten@1.1.1/node_modules/array-flatten. ▾ express ▸ lib History.md index.js LICENSE package.json Readme.md

The package itself and dependencies are placed under the same node_module, fully compatible with native Node, and well organized with package and related dependencies.

In retrospect, the node_modules in the root directory is no longer a dazzling dependency, but basically consistent with the dependency declared by package.json. Even though there are some packages inside pnpm that will set dependency promotion and will be promoted to the root directory node_modules, on the whole, the root node_modules is still much clearer and more standardized than before.

IV. Talk about safety again

I don't know if you have noticed, but pnpm's dependency management approach also cleverly avoids the problem of illegal access dependencies, that is, as long as a package does not declare dependencies in the package.json, it is inaccessible in the project.

But it cannot be done in npm/yarn, then you may ask, if A depends on B and B depends on C, then even if A does not declare the dependence of C, due to the existence of dependency promotion, C is installed in A's node_modules, then I use C in A, there is no problem with running, after I am online, it can work normally. Isn't it safe?

Not really.

First, you should know that the version of B may change at any time, if you relied on the new version of npm/yarn install 1.0.1, and now the new version of B depends on 2.0.1, then after npm/yarn install in project A, the version of C is installed, and the old version of API in C is still used in A, which may be misreported directly.

Second, if C may not be needed after B update, C will not be installed in node_modules when installing dependencies, and the code referencing C in A will report an error directly.

In another case, in the monorepo project, if A depends on X, B depends on X, and there is a C, it doesn't depend on X, but it uses X in its code. Due to dependency promotion, npm/yarn will put X in the root node_modules, so that C can run locally, because according to node's package loading mechanism, it can be loaded into X in node_modules under the root directory of the monorepo project. But imagine that once C sends out a separate package and the user installs C separately, then the X will not be found and the error will be reported directly when the code referencing X is executed.

These are all dependent on the promotion of potential bug. If it is your own business code, imagine if it is a toolkit for many developers, then the harm is very serious.

Npm also wants to solve this problem in the past, specifying the-global-style parameter can prohibit variable promotion, but this is tantamount to going back to the era of nested dependencies before liberation overnight, the shortcomings of the above-mentioned nested dependencies are still exposed.

It seems difficult for npm/yarn itself to solve the problem of relying on promotion, but the community already has a specific solution to this problem: dependency-check, address: https://github.com/dependency-check-team/dependency-check

However, it is undeniable that pnpm has done more thoroughly, and its original set of dependency management not only solves the security problem of dependency promotion, but also greatly optimizes the performance in time and space.

5. Daily use

Having said so much, I guess you will think that pnpm is quite complicated. Is it very expensive to use?

On the contrary, pnpm is very easy to use, and if you have previous experience with npm/yarn, you can even seamlessly migrate to pnpm. If you don't believe me, let's give some examples of daily use.

Pnpm install

Similar to npm install, all dependencies under the installation project are installed. For monorepo projects, however, all dependencies for all packages under workspace are installed. However, package can be specified with the-- filter parameter, and only dependent installations of package that meet the criteria are performed.

Of course, you can also use this to install a single package:

/ / install axios pnpm install axios / / install axios and add axios to devDependencies pnpm install axios-D / / install axios and add axios to dependencies pnpm install axios-S

Of course, you can also specify package through-- filter.

Pnpm update

Updates the package to the latest version according to the specified scope, and the package can be specified in the monorepo project with-- filter.

Pnpm uninstall

Removes the specified dependency in node_modules and package.json. Project monorepo is the same as above. Examples are as follows:

/ / remove axios pnpm uninstall axios-- filter package-a

Pnpm link

Connect a local project to another project. Note that hard links are used instead of soft links. Such as:

Pnpm link.. /.. / axios

In addition, for the fact that we often use npm run/start/test/publish, it is the same to replace these directly with pnpm, so I won't repeat them.

As you can see, although there are a lot of complex designs inside pnpm, it is actually imperceptible to users and very user-friendly. And now the author is still in maintenance, npm last week has been downloaded 10w +, has experienced the test of large-scale users, stability can also be guaranteed.

Therefore, on the whole, pnpm is a better solution than npm/yarn, and it is expected that pnpm will have more landings in the future.

This is the end of the content of "what's the difference between pnpm and npm/yarn". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.