Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to use Node.js to achieve JavaScript full-stack development, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let Xiaobian take you to understand.

What are the characteristics of js? 1. Js belongs to an explanatory scripting language. 2. With the support of most browsers, js can run on multiple platforms and has cross-platform characteristics. 3. Js is a weakly typed scripting language that does not make strict requirements on the data types used. It is able to type conversion, simple and easy to use. 4. Js language is highly secure, and it can only browse information or interact dynamically through the browser, so as to effectively prevent the loss of data. 5, object-based scripting language, js can not only create objects, but also use existing objects.

Background of front and rear separation

"Front-end separation" is obviously not a new topic. Zakas published a blog "Node.js and the new web front-end" in October 2013 to discuss the front end of a new era in the context of Node. There is no doubt that the emergence of Node has brought new vitality to the JavaScript language, but also makes the front-end developers have more possibilities.

On the surface, front-end separation seems to be a "enclosure movement", but in fact, front-end separation is to solve some criticisms and pain points of previous development models, and it is also a wise move to cater to large industry trends. My Meituan Hotel Division was established in July last year, with new business and new development team, which makes our front and rear separation very thorough. So far, all the business and online services hosted by the front end are based on Node, and there are nearly 20 servers in the production environment. This brings a new way of front-end collaboration that allows professionals to do professional things, both front-end and back-end can focus more on what they are good at than before.

Development mode, technology stack

The traditional development mode only needs to focus on multi-terminal presentation (browser, WebView). Now, the browser is only one part of the front end, and there are also the architecture of the Node side, the operation and maintenance capabilities of the service, and so on. The figure above shows our current service architecture: Nginx is located in front of Node service and is used for load balancing, service scheduling, Gzip compression, and so on. Then there is the Node service. We deploy and load balance the Node service through PM2.5 (taking full advantage of multi-cores). At the same time, as a lightweight middle layer, we are responsible for routing, Controllers, Views, and view rendering. Data acquisition uses JSON format through RESTful's API interface. The back end is only responsible for business logic, data storage, Models, and provides JSON data for the front end.

After this change, the Node side can optimize the page loading such as the first screen rendering. After the page is rendered, the subsequent interaction and rendering will be completed by the JavaScript code on the browser side. In most cases, the template on the Node side and the template on the browser side are the same, so we need to consider the problem of template reuse. We have replaced the default template engine of the Express framework with Juicer. Juicer is an efficient and lightweight front-end (Javascript) template engine with the goal of efficiency and ease of use. In addition, it can also run in a Node.js environment. Through Juicer, we can solve the problem of template and Helper reuse on the node side and the browser side. And based on the front-end separation of the engineering architecture, the front-end code warehouse and the back-end isolation, the front-end is independently responsible for the front-end static resource files, template files, Controller maintenance and release.

After the division of labor between the front and rear ends is redefined in this way, there are more things the frontend can do than before, such as the access of Wechat SDK. Because the use of Wechat JS SDK requires signature on the server side, now we do not need the intervention of the backend. The frontend can complete the access of Wechat SDK independently. In addition, access to our internal and commercial SSO login logic is completely completed independently by the front end.

Thoughts on the selection of Technology

For the front-end technology selection, we always maintain a rational and embracing attitude. We will not introduce new technologies for blind innovation, technology selection is aimed at our current large team scenarios, in order to solve some of the pain points and deficiencies found in the past collaboration process. For example, the introduction of Node is to improve the workflow and efficiency of the front and back end, and enhance the development experience of the front and back end. For example, Angular and React used in our project are also aimed at specific business scenarios, in order to improve development efficiency and enhance the maintainability of the code. In our business applications, Angular can significantly improve the development efficiency of some add, delete, change and search systems for merchants and backstage, while React we are only doing some attempts and practices in user-oriented PC projects.

The challenges brought by

This division of labor and architecture mode not only brings more possibilities and convenience to the front-end, but also brings a lot of challenges. Compared with the traditional front-end roles, we need to pay more attention to the status of online services, the detailed status of process memory and CPU usage, and the monitoring of online exceptions. While we embrace Node, the capability requirements of the front end are higher. A piece of JS code that looks normal may expose some previously unnoticed problems on the browser side and on the Node side, such as memory leaks: a closure or an object used to cache data. Unlike browsers, Node is very sensitive to memory leaks because online applications have thousands or even millions of traffic. So even an one-byte memory leak can cause memory accumulation, resulting in increased garbage collection time, slow application response, knowing process memory overflow, application restart or crash.

Location of memory leak problem

The following is a case we encountered in a production environment: it was recently found that the memory footprint of online services increases linearly after the service is restarted. 18 hours after the process starts, the memory footprint is close to 1.6 gigabytes. Soon after that, the memory limit of V8 will be exceeded to cause the service to restart. As can be seen from the figure, memory usage fluctuates periodically over time before the repair, which is caused by the continuous restart of the online Node process.

As we all know, under the garbage collection mechanism of V8, general code rarely has memory leaks, but once memory leaks occur, it is often difficult to troubleshoot. However, there is only one essential reason for memory leakage, that is, the objects that should be recycled are not normally recycled and become resident objects in the old generation. Fortunately, some common troubleshooting tools can help us locate the specific cause of memory leaks:

-v8-profiler

-node-heapdump

-node-mtrace

-dtrace

-node-memwatch

Here we use node-heapdump to generate snapshot of heap memory under the condition of simulated access, and analyze the generated snapshot file through Chrome's developer debugging tool. By comparing the heapdump information when the service is started and after using AB to simulate concurrent access for a period of time, you can easily locate the problem of memory leak: because Juicer enables cache by default, the compiled template is cached by default, so with the growth of access and concurrent requests, cache objects will continue to grow and will not be recycled, so close cache and redeploy it online and return to normal online.

Because of the short running time in the browser scenario and running on the user's machine, even if the memory is used too much or the memory is leaked, it will only affect the user's terminal. And the running time is short, as the process exits, memory will be released, and there is little need for memory management. But the same code on the Node side can expose problems.

Operation and peacekeeping monitoring of online services

Front-end separation means not only the separation of code repositories and development collaboration, but also the independent release and deployment of online services. What comes with it is, of course, how the front end can better monitor the operation and maintenance of online services with finer granularity. Our SA will pay more attention to the overall metrics and availability of online services, while the frontend wants to have a fine-grained understanding of the process status and exceptions of online Node.

PM2 is an excellent and open source Node process management tool. We have made some modifications on the basis of PM2, and deployed data collection and real-time data acquisition services in the cloud, thus forming the Node deployment monitoring platform PM2.5, which we have applied online. It can aggregate the fine-grained information at the process level of online Node services in the cloud for processing and visualization, and PM2.5 can monitor the status of various indicators of Node Server and processes. And the alarm can be configured and displayed in each terminal (Web, iPhone, Apple Watch).

Service Architecture of PM2.5

Briefly introduce the service architecture of PM2.5: Node services in production environment are deployed through PM2.5 CLI, and PM2.5 CLI will continuously report the data of Node process to the cloud of PM2.5. After receiving the reported data, the cloud will process the original data and store it to MongoDB. On the other hand, both the web side and the iOS application will obtain the real-time data flow from the server side through the WebSocket service, and then display the visual information through the front end.

Internal implementation of PM2.5

When the Node process is started through PM2.5, PM2.5 CLI will shake hands with the cloud service, and the data will not be reported until the handshake is successful. When reporting, the data will first be encrypted by AES256, and then the data will be reported to the server using TCP communication. Open source Axon is used here. After receiving the data, the cloud server will store the data in MongoDB and scan the monitoring alarm. If the current data meets the monitoring alarm conditions subscribed by the user, the alarm message will be pushed to the iOS client through the cloud Push service. The cloud runs WebSocket service at the same time to provide real-time data push for multiple terminals (Web platform, iOS application).

It is worth mentioning here that the client of PM2.5 is developed based on React-Native and has been submitted to Apple Store for review. After the verification, it can be downloaded from Apple Store. The client provides the view of the basic indicators of services and processes, and can cooperate with the monitoring alarm settings of Web platform to realize the hourly monitoring of services by 7x24.

Access to other monitoring facilities

In order to ensure the reliability and stability of online services, we have also connected some other monitoring facilities and logging platforms to facilitate the tracking, analysis and positioning of online errors and access logs.

Zabbix

Zabbix is an enterprise-level open source middleware with distributed system monitoring and network monitoring functions, which is mainly used by operation and maintenance. Zabbix is mainly used to detect the heartbeat of the service and monitor the indicators of the service. When some indicators are abnormal or exceed the set threshold, the alarm of SMS, elephant and email is carried out.

Sentry error log collection

Sentry is an error log server that centrally captures the details of program errors. It also provides SDK in various common languages for service access. However, Sentry will have sampling on the server side, which generally can not replace the monitoring of real-time error log alarm.

Log monitoring platform

Log monitoring platform is a log collection system within Meituan. At present, Meituan uniformly uses flume to collect logs. Flume has the ability to receive logs in scribe format, and log monitoring platform also collects logs in scibe format. Logs exist in two forms throughout the collection process, namely the original log and the parsed log. At present, we use the log monitoring platform to report the formatted data of the access log, and then we can query the access data through Hive/Presto.

Performance monitoring platform

The performance monitoring platform provides easy-to-use, end-to-end performance data services for Meituan's platforms and product lines. At the same time, it also provides SDK in various common languages for service access. It is mainly used to analyze the interface response of the Node side and the page loading performance of the browser side.

Thank you for reading this article carefully. I hope the article "how to use Node.js to achieve full-stack development of JavaScript" shared by the editor will be helpful to everyone. At the same time, I also hope that you will support and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.