Shulou(Shulou.com)06/01 Report--

Business from the initial stage of development to gradually mature, the server architecture is also from a relatively single to cluster, and then to distributed services.

A service that can support high concurrency needs a good server architecture, load balancing, master-slave cluster for database, master-slave cluster for NoSQL cache, and upload CDN for static files. All these are powerful backing to make business programs run smoothly, and the server needs operation and maintenance personnel to build.

The general server architecture that needs to be used is as follows:

Server

Load balancing (such as nginx, Aliyun SLB)

Resource monitoring

Distributed system

Database

Master-slave separation, cluster

DBA table optimization, index optimization, etc.

Distributed system

NoSQL

Master-slave separation, cluster

Master-slave separation, cluster

Master-slave separation, cluster

Redis

MongoDB

MemCache

CDN

HTML

CSS

JS

Image

Concurrent testing

High concurrency-related services need to be tested for concurrency, and the amount of concurrency that the whole architecture can support is evaluated through a large number of data analysis.

Testing high concurrency can use a third-party server or your own test server, use testing tools to test concurrent requests, and analyze test data to get an evaluation that can support the number of concurrency, which can be used as an early warning reference. as the saying goes, you can fight a hundred battles without defeat.

Third-party services:

Aliyun performance test

Concurrent testing tools:

Apache JMeter

Visual Studio performance load test

Microsoft Web Application Stress Tool

Actual combat plan

Suitable for: daily user traffic is large, but it is relatively scattered, and occasionally there is a high concentration of users.

Scenario: user check-in, user center, user order, etc.

Service architecture diagram

Description:

These services in the scenario are basically operated by users after entering APP. Except for the activity days (618, double 11, etc.), the number of users of these services will not be highly aggregated. At the same time, the tables related to these services are big data tables, and most of the businesses are query operations, so we need to reduce the number of queries that users directly hit DB; priority query cache, if the cache does not exist, then conduct DB queries to cache the query results.

Updating user-related caches requires distributed storage, such as using user ID for hash grouping to distribute users to different caches. The total amount of such a cache set will not be very large and will not affect query efficiency.

Options such as:

Users check in to get points

Find the user's check-in information in the key,redis hash that calculates the user distribution

If the check-in information is found, the check-in information is returned

If not, DB queries whether you have checked in today, and if so, synchronizes the check-in information to the Redis cache.

If today's check-in record is not found in DB, check-in logic is performed, DB is used to add today's check-in record and check-in points are added (the whole DB operation is a transaction)

Cache check-in information to redis and return check-in information

Note that there will be logic problems in the case of concurrency, such as checking in multiple times a day and issuing multiple points to users.

User order

Here, we only cache the order information on the first page of the user, with 40 pieces of data on one page, and users usually only look at the order data on the first page.

The user accesses the order list, if it is the first page read cache, if not the read DB

Calculate the key,redis hash of user distribution to find user order information.

If the user order information is queried, the order information is returned.

If it does not exist, query the order data on the first page by DB, then cache redis and return the order information.

User center

Calculate the key,redis hash of user distribution to find user order information.

If the user information is found, the user information is returned

If there is no user DB query, then cache the redis and return the user information

Other business

Most of the above examples are for user storage cache. If it is a common cache data, you need to pay attention to some problems, as follows

Note that common cached data needs to be considered that concurrency can lead to a large number of hits to DB queries. You can use the administrative background to update the cache, or lock operations on DB queries.

My blog post "lie Redis Advanced" (http://blog.thankbabe.com/2016/08/05/redis-up/) shares update caching issues and recommended solutions.

The above example is a relatively simple high concurrency architecture, which can be well supported when the concurrency is not very high. However, with the growth of the business and the increase in the number of user concurrency, our architecture will be continuously optimized and evolved, such as service-oriented, each service has its own concurrency architecture, its own balanced server, distributed database, nosql master-slave cluster, such as user service and order service.

Message queue

Applicable activities: second kill, second grab and other active services, users pour in in an instant to produce high concurrent requests.

Scene: collect red packets regularly, etc.

Server architecture diagram

Description:

Timed collection in the scenario is a highly concurrent service. For example, flash sale activity users will pour in at the arrival time, DB will receive a critical blow instantly, hold will go down continuously, and then the whole business will be affected.

For businesses like this, which are not only query operations but also have high concurrency to insert or update data, the general solution mentioned above cannot be supported. When concurrency occurs, it will directly hit DB.

Message queues are used when designing this business. You can add the information of participating users to the message queue, and then write a multithreaded program to consume the queue and give red packets to the users in the queue.

Options such as:

Collect red packets regularly

It is generally customary to use list of redis

When the user participates in the activity, push the user participation information to the queue

Then write a multithreaded program to pop data to carry out the business of handing out red packets.

In this way, users with high concurrency can participate in activities normally and avoid the risk of database server downtime.

Attach:

Many services can be done through message queuing.

For example: scheduled short message sending service, using sset (sorted set), sending timestamp as the sorting basis, short message data queue according to time ascending order, and then writing a program to read the first item in the sset queue in a regular cycle, whether the current time exceeds the sending time, and if it exceeds, send the short message.

First-level cache

The high concurrent request connection cache server exceeds the number of request connections that the server can receive, and some users have the problem that they cannot read the data when establishing a connection. Therefore, there needs to be a solution that can reduce hits to the cache server when there is high concurrency.

At this time, there is a first-level cache scheme, the first-level cache is to use the site server cache to store data, pay attention to store only part of the data with a large number of requests, and the amount of data cached should be controlled. Do not excessively use the memory of the site server and affect the normal operation of the site application. The first-level cache needs to set the expiration time in seconds, which is set according to the business scenario. The purpose is to let the data acquisition hit the first-level cache when there are high concurrent requests, instead of connecting to the cache NoSQL data server, reducing the pressure on the NoSQL data server.

For example, the APP home screen commodity data interface, these data are public and will not be customized for users, and these data will not be updated frequently. If the number of requests for this API is relatively large, you can join the first-level cache.

Server architecture diagram

Reasonable specification and use of NoSQL cache database, split the cache database cluster according to the business, which can basically support the business. After all, the first-level cache uses the site server cache, so you should make good use of it.

Static data

Under the condition that the high concurrency request data does not change, if you can not request your own server to obtain data, you can reduce the resource pressure on the server.

If the update frequency is not high, and the data allows a short delay, you can upload CDN by statically turning the data into JSON, XML, HTML and other data files. When pulling the data, you will first pull it to CDN. If you do not get the data, you will get it from the cache and the database. When the manager operates the background to edit the data and then re-generate the static file and upload it to CDN. In this way, in the case of high concurrency, the data acquisition can be hit on the CDN server.

CDN node synchronization has a certain delay, so it is also important to find a reliable CDN server provider.

Other options

For data that is not updated frequently, APP and PC browsers can cache the data locally, and then upload the version number of the current cached data each time the API is requested. The server receives the version number to determine whether the version number is consistent with the latest data version number. If not, query the latest data and return the latest data and the latest version number. If the same, the status code is returned to tell the data is up-to-date. Reduce server pressure: resources, bandwidth.

I specially sorted out the above technology. There are many technologies that can not be explained clearly by a few words, so I simply asked my friends to record some videos. The answers to many questions are actually very simple, but the thinking and logic behind them are not simple. To know it, you need to know why. If you want to learn Java engineering, high-performance and distributed, profound and simple.

Hierarchical, segmented, distributed

Large websites need to support high concurrency well, which requires long-term planning and design.

In the initial stage, the system needs to be layered, the core business is divided into module units in the process of development, distributed deployment according to the requirements, and independent team maintenance and development can be carried out.

Stratification

The system is divided into several parts in the horizontal dimension, each department is responsible for a relatively simple and single responsibility, and then a complete system is formed through the dependence and scheduling of the upper layer on the lower layer.

For example, the e-commerce system is divided into application layer, service layer and data layer. (how many levels are divided according to your own business scenario)

Application layer: website home page, user center, commodity center, shopping cart, red packet business, activity center, etc., responsible for specific business and view display

Service layer: order service, user management service, red packet service, commodity service, etc., to provide service support for the application layer

Data layer: relational database, NoSQL database, etc., providing data storage and query services

The hierarchical architecture is logical and can be deployed on the same physical machine in physical deployment, but with the development of website business, it is necessary to separate the layered modules and deploy them on different servers, so that the website can support more users to visit.

Split up

Divide the business vertically, dividing a relatively complex business into different module units.

Packaging as a module with high cohesion and low coupling is not only conducive to the development and maintenance of the software, but also facilitate the distributed deployment of different modules, and improve the concurrent processing ability and function expansion of the website.

For example, the user center can be divided into account information module, order module, recharge module, withdrawal module, coupon module and so on.

Distributed system

Distributed applications and services, distributed deployment of hierarchical or segmented business, independent application server, database, cache server

When the business reaches a certain number of users, the server will balance the load, database, and cache the master-slave cluster.

Distributed static resources, such as static resource upload CDN

Distributed computing, such as using hadoop for big data's distributed computing

Distributed data and storage, for example, distributed nodes store data separately according to hashing algorithms or other algorithms

The website is layered, and the pictures come from the network.

Cluster

For users to access centralized business independent deployment servers, application servers, databases, NoSQL databases. The core business basically needs to set up a cluster, that is, multiple servers deploy the same applications to form a cluster, and the server cluster can provide more concurrent support for the same service through load balancing devices. Therefore, when more users visit, you only need to add new machines to the cluster. In addition, when one of the servers fails, Requests can be transferred to other servers in the cluster through the failure transfer mechanism of load balancing, so the availability of the system can be improved.

Application server cluster

Nginx reverse proxy

Slb

... ...

(relational / NoSQL) database cluster

Master-slave separation, slave library cluster

Reverse proxy load balancing, picture from the network

Async

If database operations are involved in high concurrent business, the main pressure is on the database server. although master-slave separation is used, database operations are all operated on the master database. The maximum number of connections allowed by a single database server connection pool is limited.

When the number of connections reaches the maximum, other requests that require connection data operations need to wait for free connections, so many requests will have the case of connection time out when there is high concurrency.

So how can we design a development solution like this high concurrency business to reduce the pressure on the database server?

Such as:

Automatic pop-up window check-in, when double 11 crosses 0 o'clock, concurrent requests for check-in interface

Double 11 snatching red packets

Double 11 orders in storage, etc.

Design considerations:

Reverse thinking, the pressure is on the database, then the business interface will not operate the database, there will be no pressure.

Does data persistence allow delay?

How to make the business interface not directly manipulate DB, but also make the data persistent?

Scheme design:

For a highly concurrent business like this, which involves database operations, we should consider using asynchrony.

The client initiates the interface request, the server responds quickly, the client shows the result to the user, and the database operation is synchronized asynchronously.

How to achieve asynchronous synchronization?

Use the message queue to enqueue the stored content to the message queue, and the business interface responds quickly to the user's result (you can warmly remind you that the peak period can be delayed to the account)

Then write an independent program to input the dequeue data from the message queue, refresh the user-related cache after the entry is successful, and log if the entry fails to facilitate feedback query and re-persistence

In this way, there is only one program (multi-thread) to complete the database operation, which will not put pressure on the data.

Add:

Message queuing can be used not only in highly concurrent services, but also in other services with the same needs, such as SMS sending middleware, etc.

Asynchronously persisting data under high concurrency may affect the user's experience. You can switch between time and time or use async in a configurable way or automatically monitor resource consumption. In this way, in the case of normal traffic, you can use constant operation of the database to improve the user experience.

Asynchronism can also refer to programming asynchronous functions, asynchronous threads, which can sometimes use asynchronous operations to put operations that do not need to wait for results into async, and then continue with subsequent operations, saving time for waiting for this part of the operation.

Caching

Most of the high concurrent business interfaces query business data, such as commodity list, commodity information, user information, red packet information and so on. These data do not change frequently and are persisted in the database.

In the case of high concurrency, if you directly connect to the database to do query operations, multiple slave servers cannot withstand such a large number of connection requests (as mentioned earlier, the maximum number of connections allowed by a single database server is limited)

So how do we design this highly concurrent business interface?

Design considerations:

Or reverse thinking, the pressure is on the database, then we will not query the database.

The data doesn't change very often, so why do we keep querying DB?

The data does not change. Why should the client request the same data from the server?

Scheme design:

Data does not often change, we can cache the data, there are many ways, general: application server directly Cache memory, mainstream: stored in MemCache, Redis memory database

Cache is directly stored in the application server, the reading speed is fast, the number of connections allowed by the in-memory database server can support up to a large number, and the data is stored in memory, the reading speed is fast, coupled with the master-slave cluster, it can support a large number of concurrent queries.

According to the business scenario, using local storage with the client, if the content of our data does not change often, why do we keep asking the server to get the same data? by matching the data version number, if the version number is different, the API re-queries the cache to return the data and the version number, if it is the same, you will respond directly without querying the data.

This can not only improve the response speed of the interface, but also save the server bandwidth. although some server bandwidths are charged by traffic, they are not absolutely infinite. In the case of high concurrency, the server bandwidth may also lead to slow response to requests.

Add:

Caching also refers to static resource client cache

CDN cache, static resources cache our static resources by uploading cdn,CDN nodes to reduce server pressure

Service oriented

SOA Service oriented Architecture Design

Micro-service is more fine-grained service, and a series of independent services form a system together.

Use service-oriented thinking to extract the core business or general business functions into services deployed independently and provide functions by providing interfaces to the outside world.

The most ideal design is that a complex system can be extracted into multiple services to form the business of the system, with the advantages of loose coupling, high availability, high scalability and easy maintenance.

Through service-oriented design, independent server deployment, load balancing, database clustering, services can support higher concurrency.

Examples of services:

User behavior tracking record statistics

Description:

Record the user's operation behavior by reporting application module, operation event, event object, etc.

For example, record a user in a product module, click on a product, or browse an item

Background:

Because the service needs to record a variety of user actions, and can be repeatedly reported, and the business preparing to access the service is the user behavior tracking of the core business, so the number of requests is very large, and a large number of concurrent requests will be generated during the peak period.

Architecture:

Nodejs WEB application server load balancing

Redis master-slave cluster

MySQL master

Nodejs+express+ejs+redis+mysql

The server uses nodejs,nodejs as a single process (PM2 starts multiple working processes according to the number of cpu cores), and adopts event-driven mechanism, which is suitable for Imax O-intensive business and has strong ability to handle high concurrency.

Business Design:

Large amount of concurrency, so it can not be directly stored in the library, using: asynchronous synchronous data, message queue

Request the API to report the data. The API will push the reported data to the list queue of Redis.

Nodejs writes scripts to the library, circulates pop redis list data, stores the data in the library, and makes related statistics, Update, sleep for a few seconds when there is no data.

Because the amount of data will be relatively large, the reported data tables will be stored by day.

Interface:

Escalation data interface

Statistical query interface

Go online to follow up:

Service business is basically normal.

There are tens of millions of data in the daily report.

Redundancy, automation

When the server where the high concurrency business is located is down, a backup server is needed for quick replacement, and machines can be quickly added to the cluster when the application server is under great pressure, so we need a backup machine that can be on standby at any time. The most ideal way is to automatically monitor the resource consumption of the server to alarm, automatically switch the downgrade scheme, automatically replace and add the server, and so on, through automation can reduce the cost of manual operation, and can operate quickly to avoid human errors.

redundancy

Database backup

Standby server

Automation

Automatic monitoring and control

Automatic alarm

Automatic degradation

Through the GitLab event, we should reflect that backing up the data does not mean it is foolproof. We need to ensure high availability, first of all, whether the backup is normal, whether the backup data is available, we need to check regularly, or automatic monitoring, and how to avoid human errors.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.