Shulou(Shulou.com)06/03 Report--

1. Using an in-memory database

In fact, an in-memory database is a database that operates directly in memory. Compared with disk, the data read and write speed of memory is several orders of magnitude higher, and storing data in memory can greatly improve the performance of applications compared to accessing from disk. The in-memory database abandons the traditional way of disk data management, redesigns the architecture based on all the data in memory, and makes corresponding improvements in data cache, fast algorithm and parallel operation. therefore, the data processing speed is much faster than that of the traditional database. But the problem of security can be said to be the biggest injury to the in-memory database. Because the memory itself has the natural defect of power loss, when we use the in-memory database, we usually need to take some protection mechanisms for the data in the memory in advance, such as backup, logging, hot backup or clustering, synchronizing with disk database and so on. So 56 cloud editor reminds you that some of the data that is of low importance but want to respond quickly to user requests can be stored in an in-memory database and can be solidified to disk on a regular basis.

two。 Use RDD

In some applications related to big data's cloud computing, Spark can be used to speed up data processing. The core of Spark is the earliest source of RDD,RDD and a paper "Resilient Distributed Datasets: a Fault-Tolerant Abstraction for In-Memory Cluster Computing" by Berkeley laboratory. The existing data flow systems are not efficient in dealing with two applications: one is iterative algorithms, which are common in the field of graph applications and machine learning, and the other is interactive data mining tools. In both cases, keeping data in memory can greatly improve performance.

3. Increase cach

Many web applications have a lot of static content, which is mainly small files and will be read frequently, using Apache and nginx as web servers. When the web access volume is small, the two http servers can be said to be very fast and efficient. If the load is very large, we can build the cache server at the front end and cache the static resource files in the server to the operating system memory for direct read operation, because the speed of reading data directly from memory is much faster than that from the hard disk. In fact, this is to increase the cost of memory to reduce the time consumption caused by accessing the disk.

4. Use SSD

In addition to memory optimization, you can also optimize this side of the disk. Compared with the traditional mechanical hard disk, solid-state hard disk has the characteristics of fast reading and writing, light weight, low energy consumption and small size. However, the price of ssd is more expensive than that of traditional mechanical hard drives, and ssd can be used to replace mechanical hard drives under certain conditions.

5. Optimize the database

Most of the server requests eventually fall into the database, and with the increase of the amount of data, the access speed of the database will become slower and slower. If you want to improve the speed of request processing, you have to operate on the original single table. At present, the database used by mainstream Linux servers is mysql. If we use mysql to store data in a single table with tens of millions of records, the query speed will be very slow. Dividing the database into tables according to the appropriate business rules can effectively improve the access speed of the database and improve the overall performance of the server. In addition, for business query requests, the index can be set up according to the relevant requirements when building the table to improve the query speed.

6. Select the appropriate IO model

The IO model is divided into:

(1)。 Blocking Ipicuro model: Ipicuro blocks until the data arrives, and returns if the data arrives. Typical is recvfrom, which generally defaults to blocking. (2)。 Non-blocking Imax O model: in contrast to blocking, Imax O returns immediately as long as the result is not available. The current thread is not blocked. IO reuse model: that is, the part you want to learn. Multiplexing means merging multiplex signals into one channel for processing, similar to multiple pipes converging into one pipe, as opposed to demultiplexing. IO reuse model is mainly select,poll,epoll; to an IO port, two calls, two returns, there is no advantage over blocking IO; the key is to be able to listen to multiple IO ports at the same time; the function will also block the process, but different from blocking IIO O, these two functions can block multiple IO O operations at the same time. Moreover, multiple read operations and multiple write operations can be detected at the same time, and the Icano operation function is not really called until there is data to read or write. Signal driver: first, turn on the socket signal driver Icano function, and install a signal processing function through the system call sigaction. When the Datagram is ready to be read, a SIGIO signal is generated for the process. Recvfrom can then be called in the signal processor to read the Datagram, informing the main loop that the data is ready to be processed. You can also notify the main loop to read the Datagram. Asynchronous IO model: tells the kernel to start an operation and tells the kernel to notify us when the entire operation is complete (including copying data from the kernel to the user's own buffer). This is not to say that a certain model must be used, and epoll is not better than select in all cases, and it is necessary to take into account the business requirements when making the choice.

7. Use multicore processing strategy

Now the mainstream machine configuration of running server is multi-core CPU. When we design the server, we can take advantage of multi-core characteristics and adopt multi-process or multi-thread framework. The choice of multi-thread or multi-process can be made according to the actual needs, combined with their respective advantages and disadvantages. For the use of multithreading, especially when using thread pools, you can set the appropriate thread pool by testing the performance of different thread pool servers.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.