Shulou(Shulou.com)06/01 Report--

This article mainly introduces "the evolution process of a mature large-scale website system architecture". In daily operation, I believe many people have doubts about the evolution process of a mature large-scale website system architecture. The editor consulted all kinds of materials and sorted out simple and useful operation methods. I hope it will be helpful for you to answer the doubts about "a mature large-scale website system architecture evolution process". Next, please follow the editor to study!

I. the initial structure of the website

The initial architecture, applications, databases, and files are all deployed on a server, as shown in the figure:

II. Separation of applications, data and files

With the expansion of business, a server can no longer meet the performance requirements, so applications, databases and files are deployed on separate servers, and different hardware is configured according to the use of the server to achieve the best performance.

Third, improve the performance of the website by using cache

While hardware optimizes performance, it also optimizes performance through software. In most website systems, caching technology is used to improve system performance. The use of caching is mainly due to the existence of hot data. Most website visits follow the principle of 28 (that is, 80% of access requests end up on 20% of the data), so we can cache hot data.

The common ways of cache implementation are local cache and distributed cache. Of course, there are CDN, reverse proxy and so on. We'll talk about this later. Local cache, as the name implies, caches data locally on the application server, either in memory or in files. OSCache is a commonly used local cache component. The local cache is characterized by high speed, but the amount of cached data is also limited because of the limited local space. The characteristic of distributed cache is that it can cache a large amount of data, and it is very easy to expand. It is often used in portal websites, and its speed is not as fast as local cache. The commonly used distributed cache is Memcached and Redis.

Fourth, using clusters to improve the performance of application servers

As the entrance to the website, the application server will bear a large number of requests, and we often share the number of requests through the application server cluster. The load balancing server deployed in front of the application server dispatches user requests and distributes the requests to multiple application server nodes according to the distribution policy.

The commonly used load balancing technology hardware is F5, the price is more expensive, and the software is LVS, Nginx, HAProxy. LVS is a layer-4 cloud load balancer. Internal servers are selected according to the target address and port. Nginx and HAProxy are layer-7 load balancers, and you can select internal servers according to the content of the message. Therefore, LVS distribution path is better than Nginx and HAProxy, with higher performance, while Nginx and HAProxy are more configurable. For example, it can be used to separate static and static servers (select static resource server or application server according to the characteristics of the request message).

With the increase of the number of users, the database has become the biggest bottleneck. The common means to improve the performance of the database is the separation of read and write and the separation of databases and tables. As the name implies, the database is divided into read and write databases, and data synchronization is realized through the main and standby functions. The sub-database sub-table is divided into horizontal split and vertical split, while the horizontal split is to split a large table in a database, such as a user table. Vertical segmentation is divided according to different services, such as user business, commodity business-related tables are placed in different databases.

Use CDN and reverse proxy to improve the performance of the website

If all our servers are deployed in the computer room in Chengdu, the access is faster for users in Sichuan and slower for users in Beijing, because Sichuan and Beijing belong to different developed areas of Telecom and Unicom respectively. Beijing users need to go through interconnected routers to access the servers in Chengdu, and the return path is the same, so the data transmission time is relatively long. In this case, CDN is often used to solve the problem. CDN caches the data content to the operator's computer room, and the user first obtains data from the nearest operator when accessing, which greatly reduces the path of network access.

The reverse proxy, on the other hand, is deployed in the computer room of the website. when the user's request arrives, the reverse proxy server first accesses the reverse proxy server, and the reverse proxy server returns the cached data to the user. If there is no cached data, it will continue to access the application server to obtain, which reduces the cost of obtaining data. Reverse proxy has Squid,Nginx.

Using a distributed file system

With the increase of users day by day, the volume of business is getting larger and larger, and more and more files are generated, a single file server can no longer meet the demand, so it needs the support of distributed file system. The commonly used distributed file systems are GFS, HDFS, TFS.

Use NoSql and search engines

For the query and analysis of massive data, we can achieve better performance by using nosql database and search engine. Not all data should be placed in relational data. The commonly used NOSQL is mongodb, hbase, redis, and the search engines are lucene, solr and elasticsearch.

Split the business of the application server

With the further expansion of the business, the application becomes very bloated. At this time, we need to split the application into news, web pages, pictures and other businesses. Each business application is responsible for relatively independent business operations. Businesses communicate with each other through messages or share databases.

Build distributed services

At this time, we find that various business applications will use some basic business services, such as user services, order services, payment services, security services, these services are the basic elements to support each business application. We extract these services and use the partial service framework to build distributed services. Ali's Dubbo is a good choice.

At this point, the study on "the evolution process of a mature large-scale website system architecture" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.