Shulou(Shulou.com)06/03 Report--

1. Introduction of Google Cloud Storge

Google Cloud Storage, a service of Google Cloud Platform, provides unified object storage for a wide range of data types, including active data and archived data. It can be divided into middle storage categories: regional, multi-area, near-line and far-line. All categories provide unlimited data, the same tools for data access, and API, OAuth and fine-grained access control, access to other Google Cloud Storage services, and pay-as-you-go capabilities.

1.1. Comparison of four storage categories

You may not be able to immediately determine which of our four categories is suitable for your project. However, you can choose one to start using, and you can easily switch schemes in the future. As your needs change, object lifecycle management capabilities can help you automatically use a variety of categories to optimize cost performance.

Storage category Multi-RegionalRegionalNearlineColdline price $0.026-$0.036 per month per GB $0.026-$0.035 per month $0.007-$0.014 per GB per month applicable scenarios suitable for frequently accessed data less than once a month for data accessed less than once a year data redundancy geolocation redundant regional level Achieve redundancy across availability regions region-level availability SLA promises 99.95% SLA promises 99.9% SLA promises 99% SLA promises 99% durability 99.99999999% 99.9999999% 99.99999999% 99.99999999% 99.99999999% 99.99999999% consistent API millisecond access speed, consistent API millisecond access speed Consistent API millisecond access speed, consistent API use scene content storage and distribution, business continuity storage in the region and data analysis storage infrequently accessed content archive storage, backup and recovery application video, multimedia, business continuity transcoding, data analysis, compute-intensive data processing backup long tail content, rarely accessed document archive source file backup, disaster recovery 1.2, requirements

We now need to back up our files on the Google virtual machine to GCS (Google Cloud Storage). We are using the Gsutil tool, which is a Python application provided through the Google Cloud Software Development Kit. Administrators can use this tool to perform bucket and object management tasks from the command line, such as deleting objects and buckets.

1.3. Add authorization authentication to the virtual machine

We have logged in to the virtual machine and the gsutil command has been added to us by default. If not, please install Google Cloud SDK.

Then we log in to the virtual machine and use the command gcloud auth login to authorize it.

[root@sspanel ~] # gcloud auth loginYou are running on a Google Compute Engine virtual machine.It is recommended that you use service accounts for authentication.You can run: $gcloud config set account `account to switch accounts if necessary.Your credentials may be visible to others with access to thisvirtual machine. Are you sure you want to authenticate withyour personal account?Do you want to continue (YBO)? YGo to the following link in your browser: https://accounts.google.com/o/oauth3/auth?redirect_uri=uxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faptype=offlineEnter verification code:

Then the random code is generated, we enter it into the terminal, and then the authorization is completed.

Enter verification code: 4/RQGq4RAUBrp0pA_R3SvuREXyLtczDuy-WLi1sqo23jJhAxEgchEMsosWARNING: `gcloud auth login` no longer writes application default credentials.If you need to use ADC, see: gcloud auth application-default-- helpYou are now logged in as [wzlinux18@gmail.com]. Your current project is [proxy1-179115]. You can change this setting by running: $gcloud config set project PROJECT_ID

Now the authorization has been completed.

Note: if you are using a Google Cloud Shell or Google Compute Engine instance, Python and gsutil are pre-installed and verified. If the Google Compute Engine instance you set up does not have a service account that is scoped to Cloud Storage, run gsutil config and follow the instructions.

Backup operation 2.1 create a storage partition

Storage partition is the basic container used to store data in Cloud Storage.

To create a storage partition, use the gsutil mb command to create a storage partition and give it a unique name:

[root@sspanel ~] # gsutil mb-l asia-east1 gs://myweb-backup/Creating gs://myweb-backup/...

We can also see the created storage partition in Google Cloud Platform Console, which can be set in detail.

For example, our data is rarely accessed, we can change the storage category to Coldline, or we can add object life cycle to save our space.

2.2 upload objects to the storage partition

The operation of the command is similar to our usual operation command in Linux, we upload my entire website directory, for some multitasking operations, we can add a-m parameter.

[root@sspanel wwwroot] # gsutil cp-r www.wzlinux.xyz gs://myweb-backup/Copying file://www.wzlinux.xyz/index.html [Content-Type=text/html]... Copying file://www.wzlinux.xyz/404.html [Content-Type=text/html]. Copying file://www.wzlinux.xyz/.htaccess [Content-Type=application/octet-stream]... Copying file://www.wzlinux.xyz/.gitattributes [Content-Type=application/octet-stream]...-[4 files] [1.6 KiB/ 1.6 KiB] = > NOTE: You are performing a sequence of gsutil operations that mayrun significantly faster if you instead use gsutil-m cp. Pleasesee the-m section under "gsutil help options" for further informationabout when gsutil-m can be advantageous.Copying file://www.wzlinux.xyz/.gitignore [Content-Type=application/octet-stream].

Then we go to Console to check the uploaded files. In order to increase the transfer speed, it is recommended that you package the files before transferring them.

2.3 download objects from storage partition

Download is also very simple, I will take the official case to write it down, lazy.

Gsutil cp gs://my-awesome-bucket/kitten.png Desktop/kitten2.png, if the operation is successful This command returns the following: Copying gs://my-awesome-bucket/kitten.png...Downloading file://Desktop/kitten2.png: 0 Bhand 164.3 KiBDownloading file://Desktop/kitten2.png: 164.3 KiB/164.3 KiB2.4 copies objects to the folder gsutil cp gs://my-awesome-bucket/kitten.png gs://my-awesome-bucket/ of the storage partition Just-a-folder/kitten3.png2.5 lists the contents of storage partitions or folders gsutil ls gs://my-awesome-bucket. Backup schedule

Now that we know the backup command, it's very simple, so let's make a script and add it to the scheduled task to back up the data to storage at the same time every day.

#! / bin/bashexport HOME= "/ home/wangzan" cd / root/tar-zcPf www.wzlinux.xyz$ (date +% Y%m%d). Tar.gz / www/wwwroot/www.wzlinux.xyz;/bin/gsutil cp www.wzlinux.xyz$ (date +% Y%m%d). Tar.gz gs://myweb-backup/;rm-f www.wzlinux.xyz$ (date +% Y%m%d) .tar.gz

Add a scheduled task to back up at 2 a.m. Every day.

0 2 * / home/data/scripts/www_backup.sh

We tested it to see the effect.

Previous upload has been unsuccessful, the lack of an environment variable HOME, plus, the certificate can be added to whatever path.

Reference document: https://cloud.google.com/storage/docs/quickstart-gsutil

Welcome to scan the code and follow us for more information.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.