Shulou(Shulou.com)06/03 Report--

This article mainly explains "how to start interactive mode with Puppeteer". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to start interactive mode with Puppeteer.

When we use Selenium to develop crawlers, writing code in Jupyter is more convenient than writing code in PyCharm. If you use PyCharm to write code and run it directly, when a line reports an error, the whole program will fail, and you have to start completely from scratch after changing the code, which will be a waste of time. But if we use Jupyter to write code, then we only need to retry the line of code in question, and we don't need to restart the whole program again.

But if we use JavaScript to manipulate Puppeteer, how do we write and run a single line of code?

We know that the command line of Node.js does make it possible to write and run one line of code, as shown in the following figure:

But the Puppeteer code is a little different from the above code. Let's take a look at how the code is written in the official Puppeteer documentation:

Note that there is a long section of functions that are declared using async, which is asynchronous. In the interactive environment of Node.js, the whole function must be written in and run at once, not segment by segment. If you try to run paragraph by paragraph, the code will report an error. Let's take a look at this:

This is because the await keyword must be used in an asynchronous function. It can't appear on the outermost layer alone. This requires us to write all the code into an asynchronous function and then run the asynchronous function.

But what's the difference between doing this and writing a .js file to run it? Every time I want to test whether a XPath statement works properly, I have to run the whole code back to one side. Isn't this a waste of the advantages of an interactive environment?

But in fact, Node.js and Chrome themselves have a very useful interaction environment, but many people may not know it.

It is actually very simple to start this interaction mode. Let's start with 0 to create this environment.

First, we create a folder test_puppeteer, and then install puppeteer-core using npm or yarn. Puppeteer-core is used instead of puppeteer because the former can directly use the system's Chrome, while the latter needs to download a hundreds of MB Chromium, which is a waste of time.

Mkdir test_puppeteer cd test_puppeteer yarn add puppeteer-core

After the above command is executed, we can start the interactive environment.

Execute the command:

Node-inspect

The running effect is shown in the following figure:

Now, open any Chrome window and open the developer tools, as shown in the following figure:

Notice that in the upper-left corner of the developer tools menu bar, the green Logo of Node.js appears where the arrow points. Let's click on it.

At this point, a separate developer tools window pops up, as shown in the following figure:

This window is automatically associated with the Node.js we just started.

Now, let's try writing some code directly on the Console tab of the developer tool:

As you can see, what is printed on the Console tab also appears in the terminal window. It seems that it just changed from writing code in a black window to writing code in a white window. What is there to show off?

Now, let's first see what the version of Node.js on your computer is. If it is less than 14, upgrade it quickly so that it is greater than or equal to 14. Then execute our command again, as shown in the following figure.

Next, in the same step as just now, we start the developer tool and then try to call Puppeteer directly, and an incredible scene occurs:

Now we can use await directly! Now, on the Console tab, we can write JavaScript code like in Jupyter, write one line, run one line, see how it works, write another line, and run another line.

Let's try to open my blog here:

As you can see, I deliberately wrote a wrong part of the code. I should have used await browser.NewPage (), but I wrote await browser.page, which caused the program to report an error. But it doesn't matter. I'll just change it back to the right one. The whole process does not require a browser restart. You run a line of code, it executes a line of code, the code is wrong, just change this line and re-execute it. This is called an interactive environment.

Thank you for your reading, the above is the content of "how to start the interactive mode of Puppeteer". After the study of this article, I believe you have a deeper understanding of how to start the interactive mode of Puppeteer, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.