Shulou(Shulou.com)06/01 Report--

This article introduces you how to understand the FastDFS file synchronization mechanism, the content is very detailed, interested friends can refer to, hope to be helpful to you.

FastDFS file synchronization adopts binlog asynchronous replication mode. Storage server uses binlog files to record file uploads, deletions and other operations, and synchronizes files according to binlog. Only the file ID and operations are recorded in binlog, not the contents of the file. Here are some examples of the contents of the binlog file:

1572660675 C M00/00/00/oYYBAF285cOIHiVCAACI-7zX1qUAAAAVgAACC8AAIkT490.txt

1572660827 c M00/00/00/oYYBAF285luIK8jCAAAJeheau6AAAAAVgABI-cAAAmS021.xml

1572660911 D M00/00/00/oYYBAF285cOIHiVCAACI-7zX1qUAAAAVgAACC8AAIkT490.txt

1572660967 d M00/00/00/oYYBAF285luIK8jCAAAJeheau6AAAAAVgABI-cAAAmS021.xml

As you can see from the above, the binlog file has three columns, followed by the timestamp, the operation type, and the file ID (without the group name).

The file operation type is encoded by a single letter, in which the source operation is represented by uppercase letters and the synchronized operation is the corresponding lowercase letter. The meaning of the file operation letter is as follows:

C: upload files (upload)

D: delete files (delete)

A: append files (append)

M: partial file update (modify)

U: entire file update (set metadata)

T: truncate the file (truncate)

L: create a symbolic link (file deduplication function, save only one copy of the same content)

The storage server in the same group is peer-to-peer. File upload, deletion and other operations can be performed on any storage server. File synchronization is only carried out between storage server in the same group, using push method, that is, the source server is synchronized to other storage servers in the group. For other storage server in the same group, each storage server starts a thread for file synchronization.

File synchronization is incremental, recording the synchronized locations to the mark file. The path to the mark file is $base_path/data/sync/. Example of mark file content:

Binlog_index=3

Binlog_offset=382

Need_sync_old=1

Sync_old_done=1

Until_timestamp=1571976211

Scan_row_count=2033425

Sync_row_count=2033417

Using the asynchronous replication mode of binlog, there must be the problem of synchronization delay, such as the master-slave data synchronization of mysql. The next article will show you how FastDFS solves the latency problem caused by asynchronous file replication.

On how to understand the FastDFS file synchronization mechanism to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.