Shulou(Shulou.com)05/31 Report--

Most people do not understand the knowledge points of this article "Buffer and event cycle case Analysis in Node.js", so the editor summarizes the following content, detailed content, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this "Buffer and event cycle case Analysis in Node.js" article.

The use of Buffer

Binary of data

Everything in the computer: text, numbers, pictures, audio, video will eventually be represented in binary.

JS can directly deal with very intuitive data: such as strings, which is what we usually show users.

But you might think that JS can also handle pictures.

In fact, on the web side, pictures are always handled by browsers.

JS or HTML is just responsible for telling the browser the address of the image.

The browser is responsible for sending a request to get the picture and finally rendering the picture.

But it's different for the server.

The server has a relatively large number of local file types to deal with

For example, if a file that saves text is not encoded in utf-8, but in GBK, we must read their binary data and convert it to the corresponding text through GKB.

For example, what we need to read is a picture data (binary), and then carry on the secondary processing to the picture data by some means (cropping, format conversion, rotation, adding filter). There is a library called sharp in Node, which is responsible for reading pictures or passing in pictures. Buffer is responsible for re-processing it.

For example, when a long connection is established through TCP in Node, TCP transmits a byte stream. We need to convert the data into bytes and then pass it in, and we need to know the size of the transmission bytes (the client needs to judge how much content is read according to the size).

Buffer and binary

We will find that for front-end developers, we usually rarely deal with binaries, but for the server side, in order to achieve many functions, we have to manipulate its binary data directly.

So in order to make it easier for developers to complete more functions, Node provides us with a class called Buffer, which is global.

As we said earlier, binary data is stored in Buffer, so how exactly is it stored?

We can think of Buffer as an array that stores binaries

Each item in this array can hold an 8-bit binary: 00000000, which is exactly one byte.

Why eight?

In a computer, there are very few cases in which we directly manipulate one-bit binary because the data stored in one-bit binary is very limited.

So we usually combine 8 bits together as a unit, which is called a byte.

In other words, 1 byte = 8 bit,1kb = 1024 byte,1M = 1024kb, 1 G = 1024 M

For example, in many programming languages, the int type is 4 bytes and the long type is 8 bytes.

For example, TCP transmits a byte stream and needs to specify the number of bytes when writing and reading.

For example, the values of RGB are all 255, so they are essentially stored in one byte in the computer.

Buffer and string

Buffer is equivalent to an one-byte array, and each item in the array is the size of one byte.

What if we want to put a string into Buffer?

Pass the string directly into the Buffer class, and then create a buffer instance

An English string has a characteristic that each character corresponds to a byte of binary coding.

Const message = 'Hello'// uses the new keyword to create a buffer instance, but this creation method has expired const buffer = new Buffer (message) console.log (buffer); / / console.log (buffer.toString ()); / / Hello

Coding and Decoding of Chinese string

The default encoding for buffer is utf-8, so in the following code, the Buffer class uses utf-8 encoding to encode our string, and utf-8 to decode our string.

There is a characteristic of Chinese string. In utf-8 coding, a text corresponds to a 3-byte binary code.

Const message = 'Hello' / / use Buffer.from to decode our string const buffer = Buffer.from (message) console.log (buffer); / there is a toString method in the buffer instance that can decode the encoding console.log (buffer.toString ()); / / 'Hello'

What happens if different forms of encoding are used for encoding and decoding?

There is no doubt that what is decoded is not the string we originally encoded.

Const message = 'Hello' const buffer = Buffer.from (message, 'utf16le') console.log (buffer); / / console.log (buffer.toString ()); / / `O} YJU

Other ways to create Buffer

There are many ways to create a buffer. Here we can create a Buffer through alloc.

We can directly modify each bit of the buffer instance in the form of an array

If you modify a decimal number, it will automatically help us convert it to a hexadecimal number

If you modify a hexadecimal number, it will be written directly to

/ / it can specify the number of digits of our buffer. For example, if 8 is passed in here, the created buffer has 8 elements, and the binary number corresponding to each element is 0const buffer = Buffer.alloc (8) console.log (buffer). / if a decimal number is assigned, buffer will convert it to a hexadecimal number and then write it to the corresponding position buffer [0] = 88 / / in js, the number buffer [1] = 0x88console.log (buffer) that begins with 0x is represented as hexadecimal; / /

Buffer and file operation

1. Text file

If no character encoding is specified, the original buffer, which is the binary number encoded by the utf-8 of the file content, is returned instead of decoding.

Const fs = require ('fs') fs.readFile ('. / a.txtshipping, (err, data) = > {console.log (data); / /})

If you use utf-8 to encode and decode, you can get the correct content in the file.

Const fs = require ('fs') / / encoding indicates the character encoding used for decoding. The default is utf-8fs.readFile ('. / a.txtencoding, {encoding: 'utf-8'}, (err, data) = > {console.log (data); / / })

If the character encoding used for encoding and decoding is different, the final read will be garbled.

Const fs = require ('fs') / / uses utf16le character encoding and decoding uses utf-8 format, which must be fs.readFile ('. / a.txtencoding, {encoding: 'utf16le'}, (err, data) = > {console.log (data)) / / ""}) / / the above code is similar to the following code const msg = '' const buffer = Buffer.from (msg, 'utf-8') console.log (buffer.toString (' utf16le')); / / ""

2. Picture file

Copy the picture code to achieve the purpose of copying the picture.

Do not specify the encoding attribute when reading pictures, because character encoding is useful only when reading text files

Const fs = require ('fs') fs.readFile ('. / logo.png', (err, data) = > {console.log (data); / / the binary code corresponding to the image file is printed / / We can also write the image code to another file, which is equivalent to we copied a fs.writeFile of the picture ('. / bar.png', data, err = > {console.log (err);})})

To flip and crop pictures, you can use sharp as a library

Const sharp = require ('sharp') / / cut the picture of logo.png to 200x300 and copy it to the file bax.png sharp ('. / logo.png') .resize (200,300) .toFile ('. / bax.png', (err, info) = > {console.log (err)) }) / / you can also convert the picture file to buffer first, and then you can copy the picture to sharp ('. / logo.png') .resize (300,300) .toBuffer () .then (data = > {fs.writeFile ('. / baa.png', data, err = > {console.log (err);})})

The process of creating Buffer

In fact, when we create a Buffer, we do not frequently apply for memory from the operating system. By default, it will apply for a memory of 8 * 1024 bytes, that is, 8kb.

Wait until there is not enough memory or run out of memory before you apply for new memory.

Event loops and asynchronous IO

What is an event loop?

What is the event loop?

In fact, I think of the event loop as a bridge between the JS we wrote and the browser or Node.

The browser event loop is a bridge between the JS code we wrote and browser API calls (setTimeout, AJAX, listening events, etc.). The bridge communicates through callback functions.

The event loop of Node is a bridge between the JS code we wrote and the system calls (file system, networ, etc.), and the bridge is also communicated through callback functions.

Processes and threads

Processes and threads are two concepts in the operating system:

Process (process): a program that the computer is already running

Thread: the smallest unit of operation scheduling that the operating system can run, so CPU can directly manipulate threads.

It sounds abstract, but let's explain it intuitively:

Processes: we can assume that by starting an application, one process (or multiple processes) will be started by default.

Thread: in each process, a thread is started to execute the code in the program, which is called the main thread.

So we can also say that a process is a container for threads.

Explain it with a visual example.

The operating system is similar to a factory

There are many workshops in the factory, and this workshop is the process

There may be more than one worker in each workshop, and this worker is the thread.

Multi-process and multi-thread development

How does the operating system make multiple processes (listening to music, writing code, and consulting data) work at the same time?

This is because the computing speed of CPU is very fast, and it can quickly switch between multiple processes.

When the threads in our process get the time slice, they can quickly execute the code we wrote.

It is impossible for users to feel this kind of fast switching.

Browser and JavaScript

We often say that JavaScript is single-threaded, but JS threads should have their own container processes: browsers or Node

Is the browser a process? is there only one thread in it?

At present, most browsers are actually multi-process. When we open a tab page, we will start a new process. This is to prevent one page from being stuck and all the pages can't respond. The whole browser needs to be forced to exit.

There are many threads in each process, including threads that execute JavaScript code

But the code execution of JavaScript is executed in a separate thread

This means that JS code can only do one thing at a time.

If this is very time-consuming, it is assumed that the current thread will be blocked

The execution process of JavaScript

The function will not be executed until it is pressed into the function call stack. Let's analyze the execution process of the code.

Const message = 'Hello World'console.log (message); function sum (num1, num2) {return num1 + num2} function foo () {const result = sum (20,30) console.log (result);} foo ()

Our JS code, like other programming languages, can also be seen as executed in main functions.

So first we need to push the main function into the function call stack.

Define the variable message

When the log function is executed, the log function will be put into the function call stack, and the stack will be released after execution.

When the foo function is called, the foo function is pushed into the function call stack, but the sum function needs to be called during execution.

So the sum function will be pushed into the function call stack, and the sum function will be off the stack after execution.

At this time, the foo function also gets the value returned by the sum function, and performs the assignment operation, but encounters the log function.

Therefore, it is necessary to push the log function into the call stack, the log function is executed, the foo function is also executed after the stack is off the stack, and the foo function is off the stack.

After the foo function is executed, the whole js code is executed, and the main function is out of stack.

Event loop of the browser

What if there is an asynchronous operation during the execution of JS code?

For example, we insert a function call of setTimeout in the middle.

Then the function setTimeout is put into the call stack, and the execution ends immediately and does not block the execution of subsequent code.

So when will the function passed into the setTimeout function (which we call the timer function) be executed?

In fact, setTimeout calls web api, and the browser will store the callback function in advance and add the timer function to an event queue at the right time.

The functions in the event queue are put into the function call stack and executed in the call stack

Why doesn't setTimeout block code execution? Because there is a very important thing maintained in the browser-- the event loop.

The browser will help us save the callback function in setTimeout in some way. The more common method is to save it in a red-black tree.

When the setTimeout timer time arrives, it will take our timer callback function out of the saved place and put it in the event queue.

Once the event loop discovers that there is something in our queue, and the current function call stack is empty and other synchronization code has been executed, the callback functions in our queue will be dequeued and executed in turn on the function call stack (the next function will not enter the stack until the previous function in the queue is out of the stack)

Of course, there may not be only one event in the event queue. For example, if the user clicks a button in the browser during a certain process, we may monitor the click of this button and correspond to a callback function. That callback function will also be added to our queue, and the execution order will be in the order in which they are in the event queue. And the callback of our ajax request, which is also added to the event queue.

Conclusion: in fact, the event loop is a very simple thing, that is, in a special case, when you need to perform a callback, it will stuff the saved callback into the event queue, and then take it out and put it into the function call stack.

Macro task and micro task

But there is not only one queue maintained in the event loop, in fact, there are two queues, and the tasks in the queue must wait until all the script is executed.

Macro task queue (macrotask queue): ajax, setTimeout, setInterval, DOM snooping, UI Rendering, etc.

Micro task queue (microtask queue): Promise then callback, Mutation Observer API, queueMicrotask (), etc.

So what is the priority of the event loop for the two queues?

Priority execution of code in main script (top-level script code written)

Before executing any macro task (not a queue, but a macro task), check to see if there are any tasks in the micro task queue that need to be executed

That is, before the macro task is executed, it is necessary to ensure that the micro task queue is empty.

If it is not empty, then priority is given to the tasks in the micro-task queue (callback)

Interview questions

Test sites: main stcipt, setTimeout, Promise, then, queueMicrotask

SetTimeout (() = > {console.log ('set1'); 4 new Promise (resolve = > {resolve ()}). Then (resolve = > {new Promise (resolve = > {resolve ()}). Then (() = > {console.log (' then4');}) console.log ('then2');}) new Promise (resolve = > {console.log (' pr1')) Resolve ()}) .then (() = > {console.log ('then1');}) setTimeout (() = > {console.log (' set2');}) console.log (2); queueMicrotask () = > {console.log ('queueMicrotask');}) new Promise (resolve = > {resolve ()}). Then (() = > {console.log (' then3');}) / / pr1// 2lump / then1// queueMicrotask// then3// set1// then2// then4// set2

SetTimeout immediately pushes into the function call stack, and immediately leaves the stack after execution, and its timer function is put into the macro task queue.

The function passed into the Promise class is executed immediately, it is not a callback function, so the pr1 is printed, and because the resolve method is executed, the state of the Promise immediately changes to fulfilled, so that the corresponding callback function is put into the micro-task queue when the then function is executed.

Another setTimeout function is encountered, which pushes the stack off the stack, and its timer function will be put into the macro task queue.

When you encounter the console.log statement, the function presses the stack, prints out 2, and then goes out of the stack.

Here we bind a function through queueMicrotask, which is put into the micro-task queue.

The new Promise statement is encountered again, but it immediately changes the state of promise to fulfilled, so the callback corresponding to the then function is also put into the micro-task queue.

Now that the synchronization script code has been executed, the event loop begins to put the tasks confronted by the micro task queue and the macro task into the function call stack in order of priority. Note: the priority of the micro task is higher than that of the macro task. Check whether the micro task queue is empty every time you want to execute the macro task. If it is not empty, you need to execute the tasks of the micro task queue first.

The first micro task is to print then1, the second micro task is to print queueMicrotask, and the third micro task is to print then3. After execution, you start to execute macro tasks.

The first macro task is relatively complex. First, the set1 is printed, and then a new promise statement with an immediate state transition is executed, and its then callback is put into the micro-task queue. Note that the micro-task queue is not empty now, so you need to execute the micro-task queue with higher priority, which is equivalent to the then callback being executed immediately. It is the same new Promise statement, and the corresponding then counter-call is put into the micro-task queue. Notice that there is a console function after the new Promise statement, which is executed immediately after the new Promise statement is executed, that is, printing the then2. Now there is still a task for the micro task, so the next step is to print the then4. So far, the micro task queue is empty, and you can continue to execute the macro task queue.

So the next macro task set2 will be printed, and the macro task will be completed.

The printed result of the whole code is: pr1-> 2-> then1-> queueMicrotask-> then3-> set1-> then2-> then4-> set2.

Interview questions

Test sites: main script, setTimeout, Promise, then, queueMicrotask, await, async

Knowledge supplement: async, await is a syntax sugar of Promise, when dealing with the problem of event loop

We can think of the code executed after the await keyword as code wrapped in new Promise ((resolve,rejcet) = > {function execution})

The code after the await statement can be thought of as the code from then (res = > {function execution}) in the previous Promise

Async function async1 () {console.log ('async1 start'); await async2 () console.log (' async1 end');} async function async2 () {console.log ('async2');} console.log (' script start'); setTimeout () = > {console.log ('setTimeout');}, 0) async1 () new Promise (resolve = > {console.log (' promise1'); resolve ()}). Then () = > {console.log ('promise2') }) console.log ('script end'); / / script start// async1 start// async2// promise1// script end// async1 end// promise2// setTimeout

At the beginning, it is the definition of the function, and it does not need to be pressed into the function call stack until the first console statement is encountered. After the stack is pressed, the script start is printed and then out of the stack.

When the first setTimeout function is encountered, its corresponding timer will be put into the macro task queue.

The async1 function is executed, first printing out async1 start, and then executing the async2 function after the await statement, because as mentioned earlier, the function after the await keyword is regarded as a statement in new Promise, and this function will be executed immediately, so the async2 will be printed, but the code behind the await statement is equivalent to putting it into the then callback. In other words, the line of console.log ('async1 end') is put into the micro-task queue.

The code continued to execute and encountered another new Promise statement, so it was immediately printed out that the functions in the promise1,then callback were put into the micro-task queue.

The last console function executes the print script end, and the synchronization code is finished, and the event loop will go to the macro task and micro task queue to execute the task.

The first is to go to the micro task queue, and the print statement corresponding to the first micro task will be executed, that is, the async1 end will be printed, and then the promise2 will be printed. At this time, the micro task queue is empty, and the tasks in the macro task queue will be executed.

The setTimeout corresponding to the timer function is printed, and the macro task is executed. The final printing order is: script start-> async1 start-> async2-> promise1-> script end-> async1 end-> promise2-> setTimeout.

The above is about the content of this article on "Buffer and event cycle example Analysis in Node.js". I believe we all have some understanding. I hope the content shared by the editor will be helpful to you. If you want to know more about the relevant knowledge, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.