Shulou(Shulou.com)05/31 Report--

Use open source software to build an extensible SNS website mixi.jp is like, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can gain something.

Mixi is currently the third largest website in Japan, ranking 42nd in the world, mainly providing SNS services: diaries, groups, on-site messages, comments, photo albums, etc., and is the largest SNS website in Japan. Mixi was developed in December 2003 and has been soldered by its current CTO-Batara Kesuma for four months. It was put into operation in February 2004. Two months later, I signed up for 1w users with daily visits to 60wPV. In the following year, the number of users grew to 21w, and the next year, to 200w. It has grown to 370w registered users by April this year, and the number of registered users is still growing at 1.5w per day. 70% of these users are active users (active users: users who log in at least once in three days), and the average user is online for nearly three and a half hours a week. [@ more@] Let's take a look at its technical architecture. Mixi uses open source software as the basis of its architecture: Linux 2.6, Apache 2.0, MySQL Perl, 5.8, memcachedSquid, and so on. So far, there are more than 100 MySQL database servers, and the number is growing at a rate of more than 10 per month. Mixi's database connection method is to connect to each query instead of a persistent connection. Most databases run in InnoDB mode. Mixi to solve the extension problem mainly depends on the segmentation of the database.

First of all, the vertical segmentation is carried out, and different table divisions are divided into different databases according to the contents of the table. Then there is horizontal sharding, which divides the content of different users into different databases according to their ID, which is more common and useful. The key to partition lies in the implementation of the application, which needs to encapsulate the operation in the data layer without affecting the business layer as much as possible. Of course, it is impossible not to change the logic layer at all. at this time, it is best to test whether the previous design is in place. If the previous design is good, pass a table name when creating the connection, and the user ID will almost solve the problem. In the past, if the sql code flies everywhere, or if the data layer encapsulation is not very good, then you will be tired.

After doing this can not fundamentally solve the problem, especially for SNS sites like mixi, the page often needs to reference a large number of user information, friend information, pictures, article information, cross-table, cross-database operation quite a lot. At this time, it is necessary to play the role of memcached, caching all the immutable data with large memory, and notifying cache to expire when it is modified, so that the application layer can basically solve most of the problems, and only a small number of requests penetrate the application layer and use the database. Mixi's experience is that the average load time per page is around 0.02 seconds (of course, it varies according to the size of the page), which shows that this approach is effective. Mixi has cache servers on 32 machines, each Cache Server has 2G memory, and these Cache Server are installed with App Server. Because Cache Server does not consume much CPU, and with the support of Cache Server, App Server does not require too much memory, so it can coexist peacefully and use resources more efficiently.

The processing of the picture is relatively simple. For mixi, there are two main parts of images: one is often used, such as user avatars, group avatars, etc., there are probably more than 100 GB, they are cached by Squid and CDN, and the hit rate is relatively high The other part is a large number of photos uploaded by users, their individual visits are relatively small, and the hit rate is relatively low, so it is not cost-effective to use Cache, so the strategy for these photos is to distribute them directly to the image storage server when uploaded by the user, and access them directly when the user accesses them. Of course, the location of the picture needs to be recorded in the database. Otherwise, you will be depressed if you can't find which server to put it on.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.