Shulou(Shulou.com)11/24 Report--

This article comes from Weixin Official Accounts: Low Concurrent Programming (ID: dipingfa), Author: Flash Guest

I'm a redis service. I'm hanging up.

I've been running it for years, and I have a lot of key-value pairs stored in my memory.

If I die, then all my data in memory will be gone.

I need to figure out a way to copy data to a hard drive every now and then and save it.

I call this great plan the persistence plan.

Stop what you're doing.

The easiest thing I thought of first was to reject the incoming command and start copying the data from memory to the hard drive.

Wait until the copy is complete before starting to accept new commands.

This ensures that when I copy, there is no new command to modify memory, which ensures the timing.

Simply put, I save the state of Redis memory at a certain point in time.

But this will block client commands every time persistence, and will definitely be scolded.

Don't stop what you're doing.

This is easy to handle, then I will not stop the work at hand, while accepting orders, while doing persistence, as follows.

This improves efficiency a lot, persistence no longer blocks client execution of commands.

But have you noticed that there are only three ways to store data at a given moment in memory:

Flash 18 Low Concurrent Programming

Low Brother 18 Low Concurrent Programming

Brother 18, ask for three times.

At this time, the persistent data in the hard disk is:

Flash 18 for three

It cannot represent memory data at any one time.

Then such snapshots lose their meaning, that is, there is no guarantee of timeliness.

This was obviously impossible.

Make a copy of memory first

What could he do?

Stopping the task at hand guarantees timeliness, but blocks the client.

Do not stop the work at hand, although not blocking the client, but can not guarantee the timing.

That's a headache.

After scratching my head for a while, I calmed down and began to analyze.

Timing must be guaranteed, otherwise snapshots are meaningless, and you can only try to shorten the time of blocking clients.

Before blocking client time, it was consumed in persistence, i.e. copying memory to hard disk.

Optimize by copying a copy from memory to another memory space and then persisting the new memory space.

In this way, the persistence process does not delay the client command, and is not affected by the client command, ensuring the timing.

The time to block the client is simply the time to copy a copy of data between memory and memory, which is negligible compared to the entire persistence process.

Perfect!

With this perfect plan, I went to my master to claim credit.

copy on write

Me: Master, I've made a lasting plan!

Master: Well, let me see... Gee, copying memory is a good idea, but it's a bit premature. You don't know enough about the operating system.

Me: Ah, why?

Host: Think about it, your goal now is to separate the memory space used for persistence and processing client commands, right?

Me: Yeah, yeah.

Master: Yes, in fact, you only need to create a new process to do the process of persistence, the memory between different processes is isolated, that is, a new process will completely copy the memory space of the original process.

Me: Ah, isn't this the same as copying a copy of my own memory? It takes about the same time, right?

Master: I just gave the user the impression that this is the case, in fact, linux uses copy-on-write technology, in the fork out of the child process and not immediately copy the memory, just copy a mapping relationship, so that they temporarily point to the same memory space.

Master: When the parent-child process writes to this memory space, it will actually copy memory, and it is in pages.

Me: So, that is to say, I can use the principle of copying memory when writing the process of the operating system to replace my own copy of all memory, because the persistence process, the memory write operation will not be particularly large, most values are unchanged, so this improves efficiency.

Master: Yes, exactly.

Me: Wonderful!

I quickly modified the scheme, to persist when I fork a child process to do it, by the operating system process memory isolation characteristics for me to ensure that the time, write copy principle for me to ensure efficiency, that is, to reduce client blocking time, pseudocode is about this.

void rdbSaveBackground() { //child process handling (using copy-on-write techniques of the operating system) if ((childpid = fork()) == 0) { //Main method of tray drop rdbSave(); Perfect!

We haven't decided on the structure yet.

Just patronize the process of persistence, not yet decided to write to disk data format it.

Then order one.

Suppose my Redis memory has only one piece of data, written with the following command:

set dibingfa niubi

The persistent rdb file that falls to disk will look like this.

Well, done, I don't have to worry about dying anymore, someone will help me recover my memory data from persistent files.

But if it doesn't last, I don't care.

When to make a persistence, I left a configuration for the owner

save m n

Indicates that a persistence is automatically triggered when there are n modifications to the dataset in m seconds.

The owner can also configure multiple such configuration items.

And I also kindly gave the owner a default configuration item and wrote a comment.

# Save the DB on disk:

# In the example below the behaviour will be to save:

# after 900 sec (15 min) if at least 1 key changed

# after 300 sec (5 min) if at least 10 keys changed

# after 60 sec if at least 10000 keys changed

save 900 1

save 300 10

save 60 10000

I think with the master's English level, he should be able to read it.

Well, it's really done this time!

I'll name this thing RDB.

It doesn't mean anything, it just starts with my name, Redis DB.

PostScript

The rdb persistence process can also be triggered manually, that is, directly enter bgsave, which is exactly the same as automatic triggering.

In redis source code, it is called bgsaveCommand method.

The whole source code is very simple and readable, but a lot of interference just stop.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.