Shulou(Shulou.com)06/01 Report--

How to store logs and popular articles in Redis? To solve this problem, today the editor summarizes this article about storing logs and popular articles on Redis, hoping to help more students who want to learn Redis to find a more simple and easy way.

You can implement a variety of data structures using Redis's list data type, which can be thought of as an indexed array in php. It can implement a variety of data structures of stack, queue and message queue.

Keep a log

As we all know, the nginx log will not be automatically cut by default, it will always be stored in a file, always additional write, we need to do our own operation to cut the log. Logging is useful in many places except nginx. After something goes wrong, the log is one of the main ways for us to find clues.

We are now going to write system logs to redis, and daily logs are recorded in a list list to prevent a single log file from becoming too large.

The basic idea is that the daily log information is written into a separate list list, and then a scheduled task is done. The function of the scheduled task is to take out the log list from one month ago, persist it into a text file, and then delete the log list from one month ago in redis to prevent redis from taking up too much memory.

You can use the compression function to compress the log information to reduce memory footprint. In addition, maintain a list to store the key name of the log list, which is convenient to retrieve the key name of the log list. The pseudo code for storing the log is as follows:

$log =. / / Log information / / Log list key name $key = 'log:'.strtotime (date (' Ymurmmurd')); / / maintain a list of key names if (! $redis- > exists ($key)) {$listlogkey = 'log:key'; $redis- > rpush ($listlogkey, $key);} / / Log information is stored in redis $redis- > rpush ($key, $log)

The scheduled task code is as follows:

$lastMonth = strtotime ("- 30 day"); while ($logkey = $redis- > lpop ('log:key')) {$logTime = explode (':', $logkey) [1]; if ($logTime)

Lrange ($logkey, $start, $end); if (! $logs) break; / / decompress the log information and then append it to the text file. / / delete the log list $redis- > del ($logkey);}} else {/ / within one month, re-push to the left $redis- > lpush ('log:key', $logkey); exit;}}

There are a few points to note here. If you fail to persist the log, or if the log is within a month, you need to re-push the log list key name from the left. In addition, when fetching logs from the log list, do not pull them all at once, which can easily lead to redis blocking. Each time, take a certain number (such as 50) and take it out in a cycle.

Store hot news ID

Here, do not paste the code, mainly talk about ideas. I used to set up a system with a single function, which is the hottest today, the hottest week, and the hottest January. At that time, the traffic of our website was quite large, and within a few days, the website was dead. The reason is the slow query problem of mysql. Because, the sql of this piece has grouping, COUNT (), condition judgment and so on.

Let's talk about our solution: write a mysql stored procedure and call the stored procedure regularly. The function of this stored procedure is to filter out the hottest articles of today, week and January, take 100 articles id, and store their article id in the queue of redis. For the hottest articles, we only show the top 100 articles. In this way, there are no slow queries in our system.

The method of storing logs and popular articles on Redis is shared here, and the solution to the problem is not just the method analyzed in the article and everyone, but the accuracy of the analysis method in this article is beyond doubt. If you like this article, you might as well share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.