Shulou(Shulou.com)06/03 Report--

Overview of 1.Cloudera upgrade

Upgrading consists of two main steps, upgrading Cloudera Manager and upgrading CDH cluster

Upgrade CM:

* you can use the rpm command when upgrading CM, and Cloudera Navigator will also upgrade when Cloudera Manager is upgraded.

Upgrade CDH:

* CM and CDH do not need to be upgraded at the same time, but CM and CDH versions must be compatible. Cloudera Manager 6.0 can manage clusters running from CDH 5.7to CDH 5.14, as long as the major + minor version of Cloudera Manager is equal to or higher than the major + minor version of CDH.

For example:

Support:

Cloudera Manager 6.0.0 and CDH 5.14.0

Cloudera Manager 5.14.0 and CDH 5.13.0

Cloudera Manager 5.13.1 and CDH 5.13.3

Not supported:

Cloudera Manager 5.14.0 and CDH 6.0.0

Cloudera Manager 5.12 and CDH 5.13

Cloudera Manager 6.0.0 and CDH 5.6

two。 Assess the impact of upgrad

Plan a maintenance window long enough (downtime) to perform the upgrade. Depending on the components you need to upgrade, the number of hosts in the cluster, and the type of hardware, it may take you a whole day.

Before you start the upgrade, you need to gather some information and prepare in advance.

Please refer to Cloudera Manager's:

Https://www.cloudera.com/documentation/enterprise/release-notes/topics/rg_release_notes.html

Learn about API changes, outdated features, new features, and incompatible changes.

And check it out.

Https://www.cloudera.com/documentation/enterprise/upgrade/topics/rg_requirements_supported_versions.html#c6_requirements

To ensure that the supported operating system JDK database and other components are being used

There are three types of upgrades: major, minor, and maintenance

Major upgrade

Upgrades typically have the following characteristics:

The functionality of Hadoop has changed a lot, with a lot of updates.

Incompatible data format

Major changes have taken place in Cloudera Manager interface

Schema changes in Cloudera Manager database

A lot of downtime is required to upgrade the cluster

Redeploy client configuration

Minor upgrade

Minor upgrade refers to upgrading the minor version based on the same major, such as upgrading from 5.7.x to 5.8.x, and usually includes the following:

New feature Bug fixes potential database Schema changes automatically handled by Cloudera Manager

Minor upgrades usually do not result in incompatible changes and changes in data format

Maintenance upgrade

Maintenance upgrades can fix serious bug or solve security problems without compatibility modifications and new features

Overview of 3.Cloudera Manager upgrade

There are the following steps to upgrade CM:

Back up the Cloudera Manager server database, working directory, and several other entities. If there are problems during the upgrade process, you can use these backups to restore the Cloudera Manager deployment

Use the package command to upgrade Cloudera Manager Server on the command line on the CM host. For example, Redhat's yum command cm automates most of the process.

The upgrade Cloudera Manager agent,Cloudera Manager upgrade wizard on all cluster hosts can help you upgrade agent (and, optionally, JDK) or install agent and JDK software manually. CDH did not upgrade during this process.

For the steps to upgrade Cloudera Manager, please refer to the official website documentation:

Https://www.cloudera.com/documentation/enterprise/latest/topics/cm_upgrade.html#concept_q1q_mbs_wx

If you are upgrading from 5.x to a later version of 5.x, you can also upgrade Cloudera Manager using tarballs for procedures related to upgrading Cloudera Manager 5 using Tarball.

Https://www.cloudera.com/documentation/enterprise/5/latest/topics/cm_ag_ug_cm5_tarballs.html

Overview of 4.CDH upgrade

CDH upgrade includes updates of Hadoop software and related components you can use Cloudera Manager to upgrade CDH depending on the version of Cloudera Manager you use and the version of CDH you are upgrading to, these procedures will vary depending on the version you use and the version of CDH you are upgrading to. You can use Cloudera Manager to upgrade CDH using parcels or software packages.

After completing the preparation steps, you can use the Cloudera Manager upgrade wizard to complete the upgrade if you use parcels (recommended) to enable HDFS high availability and have a Cloudera Enterprise license you can perform a rolling upgrade without taking the cluster offline during the upgrade.

For CDH upgrade steps, see upgrading CDH clusters

Https://www.cloudera.com/documentation/enterprise/upgrade/topics/ug_cdh_upgrade.html#cdh_upgrade5

It is recommended to use Parcels to upgrade CDH

Upgrading CDH using parcels is the preferred method because Parcels is managed by Cloudera Manager and automatically downloads, distributes, and activates the correct version of the software. For an easier upgrade process, consider switching from package to parcels so that Cloudera Manager can automate more procedures. You can also switch from package to parcels when upgrading CDH.

Upgrade CDH using Packages

This option is the most time-consuming and requires you to log in using ssh and execute a series of package commands on all hosts in the cluster. It is recommended that you upgrade the cluster using Parcels so that Cloudera Manager can distribute the upgraded software to all hosts in the cluster without logging in to each host. If you install the cluster using packages, you can upgrade using parcels, and the cluster will use parcels for subsequent upgrades.

Overview of 5.Cloudera Navigator upgrade

Cloudera Navigator Metadata and Audit servers upgrade automatically when you upgrade Cloudera Manager you can also choose to upgrade other Cloudera Navigator components such as Cloudera Navigator Key Trustee Server, Cloudera Navigator Key HSM and Cloudera Navigator Encrypt you do not need to upgrade these components with Cloudera Manager or CDH upgrade for compatibility information, see: Cloudera Navigator encryption Product compatibility Matrix (Cloudera Manager 5.x) and

Upgrade Cloudera Navigator data encryption.

Https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html#pcm_navigator_encryption

Https://www.cloudera.com/documentation/enterprise/upgrade/topics/ug_cn_upgrade.html#cn_upgrade

6.Cloudera Manager upgrade

Rpm package download address

Http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.14.4/RPMS/x86_64/

Local yum source settings

Put the downloaded package into a directory to set up a yum index

Cd / etc/yum.repos.d/ # cd to this directory

Vi cloudera.repo creates a yum source profile configured as follows

Check yum source availability

Yum list

Shut down the Cloudera Management Service service:

Backup database

Shut down Cloudera Manager Server and agent on all servers and stop services

Stop server and agent services on all servers

Upgrade cloudera related components

Yum upgrade 'cloudera-*'

Check that the installation was successful:

Start the Cloudera Manager Server service

Service cloudera-scm-server start

Prepare to upgrade the agent of other servers

Click to continue

Select a custom repository

Note: you can also choose the private key here.

Upgrade succeeded Click to continue

Click continue after waiting for inspection

OK

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.