Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what are the technical characteristics of the website architecture of eBay". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

The change of eaby Technology Architecture

The change of ebay system architecture mainly goes through four stages. The following picture shows the timetable of ebay system architecture change.

In the V1 version of ebay, ebay uses FREEBSD + APACHE + PERL + DGBM, which is a relatively primitive model and relatively simple. The operating system, application server, web server and database server are all on the same machine, and the network structure has only one layer physically. The whole website has four domain names, each domain name corresponds to a different application, and each group of applications corresponds to a server.

Figure 1 ebayV1 system architecture

With the increasing volume of business and visits, ebay began to upgrade the architecture in 1999, and the technical architecture has undergone great changes, mainly from 1999 to 2004, while the version number of the architecture is from V2.0 to v2.5. Let's take a look at the Ebay V2.0 technical architecture.

V2.0

The ORACLE server, database server and web server are separated, and the database is independently deployed to a new machine.

The program logic has begun to be layered, which is what we often call the mvc3 layer structure: display layer, business logic layer, data access layer, while physically it is still a two-tier structure web server and database server.

At a time when java was just emerging, there were probably no other good languages to choose.

V2.1

Each group of applications corresponds to multiple servers, and multiple servers form a servler pool (service pool), which forwards requests to different servers through a load balancing server.

Deploy the database to a better-performing server

V2.2

A database server is added as a backup server to prevent failure

V2.3

This version just adds more servers to each application and keeps server pool.

V2.4

The biggest and most important change in this version is to split the database vertically, that is, to divide the database according to different functional modules, such as transaction library, membership library, account library.

V2.5

This version separates part of the database from reading and writing, and splits the Item database horizontally, assigning Items to different Categoty repositories according to different Categoty, which greatly expands the access performance of Items database.

Figure 2 ebayV2 system architecture

From the above, we can see the architecture change of ebay V2, mainly through the addition of the server, the vertical and horizontal split of the database, and the read-write separation of the database to improve the performance of the whole website. In the web layer, the server is added to scale horizontally, and the application service function is divided vertically into different systems according to different business functions. At the database level, an attempt is made to separate read and write, split the database vertically, and split the Items library horizontally according to Category, which disperses the centralized access to the product library items, but needs to provide a transparent access mechanism in the DAL layer. Ebays does not seem to have such a mature framework here, and we do not know how the distributed transaction ebay is implemented at this stage.

V3

The whole application development platform is replaced by j2ee platform, and the whole website is rewritten with java. It seems to be a big job. The purpose is to decouple and reuse modules, from which we can see the advantages of java in developing complex enterprise applications.

V3 version makes a more optimized design at the database level, and ebay continues to optimize the database.

Split the database vertically and divide it into more sublibraries according to functional modules

Split the database horizontally, and assign the same kind of data to different databases according to the different key values (there are many ways to divide the database at the specific level, which will not be introduced here. When splitting the database horizontally, ebay must also establish a set of transparent DAL access methods, provide transparent database access mechanism and transparent database routing functions, and the changes in the physical structure of the database will not affect the logical changes of the code.

Here, ebay also gives best practices at the database level:

Minimize the consumption of database CPU, such as using only a small number of triggers without using stored procedures

Reduce the logical functions at the database level, such as data transformation and combination, which are placed in the logical layer.

Reducing dynamic SQL is mainly the function of dynamic generation of parameters in SQL, which is also emphasized by the company's DBA.

Shorten the transaction time of the database as much as possible and end things as early as possible

Update the database asynchronously as much as possible to disperse the pressure of the database, such as the operation that consumes the time of the database should be processed at night.

Do not use distributed transactions, it seems that distributed transactions do not use systems with high concurrency

In the application logic level, ebay divides the system into many different modules according to function, each module as a subsystem, and improves the scalability of the whole system by horizontally expanding the number of subsystem servers.

Let's take a look at the best practices given by ebay at the application level

Keeping the application layer subsystem completely stateless, it can be expanded horizontally to improve scalability, and is equally distributed to the instance pools of each subsystem through the load balancing server.

Use caching as much as possible. Caching can reduce the pressure on the database and use space for time.

Strictly divide all levels of the system, presentation layer, business logic layer, service integration layer, DAO layer, infrastructure layer.

In the design of the application layer, ebay divides many domain by different functions. Each domain is only responsible for the business logic of its own function, and there is no dependence between domain and domain. At the same time, common domain provides interaction and dependency between various domain, as shown in the following figure:

Since the ebay database is logically divided into many different font libraries, ebay must provide transparent access to the database. For example, ebay divides Items into many sub items libraries according to categoray. If you need to query all the Items purchased by a user, then you must query all the sub items libraries and combine the databases, so the DAL layer must mask the physical structure of the database. Query all the corresponding data in the sub items library at one time. And this access is transparent to the application. Applications don't need to care about how many sublibraries there are in items.

The architectural features of ebay:

Partition Everything

At the beginning of a website, there may be only dozens of people visiting it, or hundreds a day, or an ordinary server may be enough. Db and applications can all be put together, but with the increase of users and business, one server is far from enough, so you naturally want to add servers, and the system should follow the changes. One more server reduces the pressure. In this way, there is a split business and split the data.

In fact, it is not easy to achieve just right. Ebay divides applications and databases according to the level of business functions. This is done in many domestic websites, it is not surprising, but after the horizontal division of functions, the segmentation of individual functional applications also has a lot of articles to do. How to divide it? ebay's architecture document talked about this a long time ago.

After dividing the database according to the business, we can divide the number of tables according to certain rules, in which there are many rules, which can be divided according to the guidance of the main business producers, and all the data can be followed by the producers, as to what rules can express their own opinions.

Asynchrony Everywhere

Synchronous applications will bring strong coupling and poor usability, especially in terms of user experience. Imagine that a website home page will lose a large number of users if it is synchronized. If the network is slow, when mosquitoes are asleep, where do people have time to look? in fact, distributed systems should try to use asynchronous processing.

The coping strategies of EBay are: event-driven and pipeline, multicast messages, and the technologies involved are message middleware (disorder, at least one arrival) and reliable multicast based on SRM technology.

Automate Everything

Dynamic configuration information, involving technologies: implementation of configuration publish / subscribe mechanism, machine learning. This super cow, I don't know how many websites in China have done it. I heard that Taobao did it (hehe).

Remember Everything Fails

Fault detection and rollback

Many websites do this now, but ebay is doing a good job. Ebay has a log of 2TB almost every day, and Taobao has done a good job by monitoring events to make effective judgment and early warning.

The coping strategies of eBay are as follows: sending messages after exception, receiving message alerts, downgrading according to functions, ensuring the availability of core functions, including message middleware and how to achieve functional degradation.

Embrace Inconsistency

In fact, this is a bit like what we call "embracing change" all day. If there are too many transactions in the system, it will greatly affect the performance, especially the distributed transactions. If you blindly pursue consistency, the performance will be serious. The practice of ebay is that the process is inconsistent and ultimately consistent. The technologies involved are: message middleware, CAP (Consistency consistency; Availability availability; Tolerance of network Partition partition tolerance (which can be understood as the system can still work properly in case of partial node failure or connection failure between nodes), etc.

Expect (R) evolution

Here eBay is mainly talking about how to better deal with changes, including functional evolution, architecture evolution, eBay response strategies are: flexible schema, pluggable processing flow and incremental system release, this technology is still quite complex, eBay uses: configuration process, system release process to support the coexistence of multiple versions.

Dependencies Matter

This will become more obvious with the increase of distributed and asynchronous applications, as well as the increasing number of functions, and so does eBay.

Their coping strategies: service topology management, design control (only dependencies are allowed. ), the client assumes responsibility.

Speaking of this, I have to say that it is really important for the client to assume responsibility. Nowadays, many architectures like to solve N-multiple problems on the server, but it is really necessary to put it on the client on many occasions. Of course, this will also bring some problems, such as upgrading and so on.

This is the end of the content of "what are the technical features of eBay's website architecture"? thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.