Shulou(Shulou.com)06/02 Report--

For the understanding of large-scale website technology, we can take the architecture elements as the dimension from the organization of the principle of architecture technology, and explain the technical key points of website architecture from the perspectives of system performance, availability, scalability, expansibility and security. There is another more intuitive way of organization, which is described from the dimension of website architecture technology used at different architectural levels.

The system architecture of the website is shown in the following figure:

I. Front-end architecture

The front end refers to the link experienced by the user before the request arrives at the website application server, which usually does not include the website business logic and does not deal with dynamic content.

Browser optimization technology

It is not to optimize the browser, but to speed up the loading and display of the browser page by optimizing the response page, such as page caching, merging HTTP to reduce the number of requests, using page compression and so on.

CDN

The content distribution network, deployed in the computer room of the network operator, distributes the static page content to the nearest CDN server, so that the user can obtain the content through the shortest path.

Static and static separation, independent deployment of static resources

Static resources, such as JS and CSS files, are deployed on dedicated server clusters, separated from Web application dynamic content services, and use dedicated (second-level) domain names.

Picture service

Pictures do not refer to website Logo, button icons, etc., these files belong to the static resources mentioned above and should be deployed with JS and CSS. The images here refer to the pictures uploaded by users, such as product images, user avatars, etc. The image service also applies to independently deployed image server clusters and uses independent (second-level) domain names.

Reverse proxy

Deployed in the website computer room, page caching service is provided before application server, static resource server and picture server.

DNS

Domain name service, which resolves domain names to IP addresses. You can use DNS to achieve DNS load balancing. You also need to modify DNS to configure CDN to point to the CDN server after domain name resolution.

II. Application layer architecture

The application layer is the place where the main business logic of a website is handled.

Development framework

The website business is changeable, most of the website software engineers are working overtime to develop the website business, a good development framework is very important. A development framework should be able to separate concerns, so that artists, development engineers can do their own things, easy to cooperate. At the same time, some security policies should be built in to protect against Web attacks.

Page rendering

The dynamic content developed and maintained separately and the static page template are integrated to form a complete page that is finally displayed to the user.

Load balancing

Multiple application servers are formed into a cluster, and user requests are distributed to different servers through load balancing technology to cope with the high concurrent load pressure caused by a large number of users accessing at the same time.

Session management

In order to realize the highly available application server cluster, the application server is usually designed as stateless and does not store the user request context information, but the website business usually needs to maintain the user session information and needs a special mechanism to manage the Session, so that the application servers within the cluster or even across the cluster can share Session.

Static dynamic page

For dynamic pages that have a large number of visits and are not updated very frequently, we can make them static, that is, generate a static page and use the optimization means of static pages to accelerate user access, such as reverse proxy, CDN, browser cache and so on.

Business split

Split the complex and huge business to form a number of smaller products, independent development, deployment and maintenance, not only reduce the system coupling, but also facilitate the database business sub-database. To split the relational database according to business, the technical difficulty is relatively small, and the effect is relatively good.

Virtualized server

Virtualizing a physical server into a polymorphic virtual server makes it easier to build a highly available application server cluster with fewer resources for services with lower concurrent access.

III. Service layer architecture

Provide basic services, provide layer calls, and complete the website business.

Distributed message

The message queue mechanism is used to realize the asynchronous message transmission and low-coupling business relationship between business and service, service and service.

Distributed service

Provide distributed services with high performance, low coupling, easy reuse and easy management, and implement service-oriented architecture (SOA) on the website.

Distributed cache

Caching large-scale hot data through scalable server clusters is an important means of website performance optimization.

Distributed configuration

Many parameters need to be configured to run the system. If these parameters need to be modified, such as adding a new cache server to the distributed cache cluster, you need to modify the cache server list configuration of the application client and restart the application server. The distributed configuration provides the configuration dynamic push service during the system run time, and the configuration modification is pushed to the application system in real time without restarting the server.

IV. Storage layer architecture

Provide persistent storage access and management services for data and files.

Distributed file

Most of the files that the website online business needs to store are relatively small files such as pictures, web pages, videos and so on, but the number of these files is very large and usually continues to increase, which requires a distributed file system with good scalability and design.

Relation database

Most of the main business is based on relational database, but the supporting table of relational database for cluster scalability is poor. By adding the routing function of database access to the data access layer of the application, the database access can be routed to different physical databases according to the business configuration, and the distributed access of relational database can be realized.

NoSQL database

At present, all kinds of NoSQL databases emerge one after another, and each has its own advantages in memory management, data model, cluster distributed management and so on. But from the perspective of community activity, HBase is undoubtedly the best at present.

Data synchronization

Before the maturity of distributed database technology that supports global data sharing, websites with multiple data centers must synchronize data among multiple data centers to ensure that each data center has complete data. In practice, in order to reduce the pressure on the database, the transaction log of the database (or the write operation Log of NoSQL) is synchronized to other data centers, and the data is replayed according to Log to achieve data synchronization.

Fifth, background architecture

In the website application, in addition to dealing with users' real-time access requests, there are also some background non-real-time data analysis to deal with.

Search engine

Even the search engine inside the website needs to update the data incrementally and fully, build the index and so on. These operations are performed regularly by the background system.

Data warehouse

Provide data analysis and data mining services according to offline data.

Recommendation system

Social networking sites and shopping sites provide personalized recommendation services for users by mining the relationship between people and goods, and developing potential interpersonal relationships and shopping interests.

VI. Data acquisition and monitoring

Monitor the visit of the website and the operation of the system to provide support for website operation decision and operation and maintenance management.

Browser data acquisition

The user's behavior is analyzed by embedding JS script in the website page to collect the user's browser environment and operation record.

Server business data acquisition

There are two kinds of server business data, one is to collect the user request operation log recorded on the server side, and the other is to collect application run-time business data, such as the number of messages to be processed.

Server performance data acquisition

Collect server performance data, such as system load, memory utilization, network card traffic and so on.

System monitor and control

The data collected above will be displayed in the form of a chart, so that the operation and maintenance personnel can monitor the operation of the website, so that this step is only system monitoring. A more advanced approach is to carry out automatic operation and maintenance according to the collected data, automatically deal with abnormal conditions of the system, and absorb automatic control.

System alarm

If the collected data exceeds the preset normal threshold, for example, the system load is too high, an alarm signal will be sent out by e-mail, text message, voice call and so on, waiting for the engineer to intervene.

VII. Security architecture

Protect the website from attacks and disclosure of sensitive information.

Web attack

The most harmful attacks launched in the form of HTTP requests are XSS and SQL injection attacks. However, as long as proper measures are taken, both attacks are relatively easy to prevent.

Data protection

Encrypted transmission and storage of sensitive information to protect websites and user assets.

VIII. Data center computer room architecture

Large websites require hundreds of thousands of servers, and the physical architecture of the computer room also needs attention.

Computer room architecture

For a large website with 100, 000 servers, the power consumption of each server (including the server itself and air conditioning) costs about 2000 yuan per year, so the annual electricity cost of the computer room of the website needs 200 million yuan. The problem of data center energy consumption is becoming more and more serious. Google and Facebook tend to choose places with good heat dissipation and abundant power supply when choosing the geographical location of the data center.

Cabinet architecture

Including cabinet size, network cable layout, indicator specification, uninterruptible power supply, voltage specification (48V DC or 220V civil AC) and so on.

Server architecture

Due to the large scale of server procurement, large-scale websites mostly use customized servers instead of purchasing the whole server. According to the website application requirements, customize the hard disk, memory, and even CPU, while removing unnecessary peripheral interfaces (display output interface, mouse, keyboard input interface), and make the spatial structure conducive to heat dissipation.

Source of this article: http://www.hollischuang.com/archives/1132

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.