Shulou(Shulou.com)06/02 Report--

Optimization idea: large files are divided into several 100m files through tar, and then uploaded through multiple processes.

The script is as follows: #! / bin/bash

# #

# Author:Jianjun Zhang

# Create Data:2017-03-22

# Propose:For big file upload

# Modify Data:

# #

# use tar to split large files into multiple 100m small files

Tar-czf-ftp_test.dmp | split-b 100m-d-ftp_test.tar.gz

# calculate how many small files are split

Fc= `ls-l | grep "ftp_test.tar.gz*" | wc-l`

Echo $fc

# using for loops

For ((aqui0; a ftp "$I" .txt

Echo binary > > ftp "$I" .txt

Echo hash > > ftp "$I" .txt

Echo prompt > > ftp "$I" .txt

Echo cd result_bj > > ftp "$I" .txt

Echo passive > > ftp "$I" .txt

Echo put ftp_test.tar.gz "$I" > > ftp "$I" .txt

Echo bye > > ftp "$I" .txt

Echo start upload the expdp file.

Sleep 2

# execute FTP at backend

Ftp-v-n 192.168.1.100 < ftp "$I" .txt &

Rm-rf ftp "$I" .txt

Done

Conclusion: the upload time is about 1x5 of a single file upload, and the FTP upload process will not be deadlocked. The disadvantage is that this upload takes up a lot of bandwidth.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.