Shulou(Shulou.com)05/31 Report--

How to quickly migrate massive local data to cloud object storage COS, many novices are not very clear about this. In order to help you solve this problem, the following editor will explain it in detail. People with this need can come and learn. I hope you can get something.

In the era of big data, faced with a large number of local files, with the popularity of cloud storage, more and more users need to migrate massive data from traditional local storage to new distributed cloud infrastructure, which requires fast, efficient and secure migration methods.

Operation scene

For users with local IDC, COS COS supports the following migration methods on different migration types to help users quickly migrate massive data from local IDC to COS COS.

Users can consider how to choose the migration method based on data migration volume, IDC egress bandwidth, IDC free space resources, acceptable migration completion time and other factors.

The following figure shows the estimated time consumption when using online migration. You can see that if the migration period exceeds 10 days or the data volume exceeds 50TB, we recommend that you choose offline migration. Otherwise, select online migration.

Note: the large number of small files under 1MB and the poor performance of disk IO will also affect the progress of data migration.

Description of operation method

1. COS Migration online migration

Migration steps:

1. Install the Java environment.

two。 Install the COS Migration tool.

3. Modify the configuration file.

4. Start the tool.

For more information on how to do this, see the COS Migration tool documentation.

(for details, please see: https://cloud.tencent.com/document/product/436/15392)

How to configure to maximize migration speed:

1. Adjust the threshold and migration concurrency of distinguishing large and small files according to their own network environment, so as to achieve the best migration mode of large files divided into blocks and small files transferred concurrently. Adjust the execution time of the tool and set bandwidth limits to ensure that the operation of your business will not be affected by the bandwidth consumption of the migrated data. You can common the above adjustments in the configuration file config.ini by modifying the following parameters:

two。 The adoption of distributed parallel transmission can further speed up the migration speed. Users can consider using multiple machines to install COS Migration and perform different source data migration tasks.

II. Cloud data migration CDM offline migration

Migration steps:

1. Go to the CDM console for cloud data migration and submit an application.

two。 After the application is approved, the user waits for the equipment to be signed.

3. After receiving the device, copy the data to the device according to the migration device manual.

4. After copying the data, submit a return application in the console and wait for Tencent Cloud to move the data to COS.

How to migrate data to COS efficiently and securely through offline migration:

1. In the network environment of configuring 10Gbps in IDC, in order to prevent the local data environment from becoming a transmission bottleneck, a high-performance mount point machine can be equipped to speed up the copy as much as possible.

two。 The fastest way for CDM to transfer data is to transfer data in parallel. Users can monitor the CPU and memory utilization of the device. If the current migration speed is lower than expected, you can choose the following parallel transfer method.

Multiple devices connect the same CDM device through different network interfaces.

Multiple devices connect multiple CDM devices through different network interfaces.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.