Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about the new features of CDP Private Cloud basic version 7.1.6. This article is rich in content and analyzed and described from a professional point of view. I hope you can get something after reading this article.

According to IDG, when customers consider updating to the latest version of the product, they expect new features, enhanced security, and better performance, but increasingly want a simpler upgrade process. With each new version of the CDP private cloud, we are working to provide this content. With many new features, we are simplifying the upgrade process as much as possible. In this blog, we will introduce the new features in version 7.1.6 and new in-place upgrades from HDP, which completely eliminates the hassle of replacing infrastructure and data migration.

As shown below, the CDP private cloud base stack provides an upgrade path for customers of CDH 5.13-5.16 and HDP 2.6.5. In this release, we also support upgrades from HDP 3.1.5.

CDP Private Cloud Foundation 7.1.6 provides the following categories of benefits:

Better upgrade support

Support for in-place upgrades from HDP 3.1.5 to CDP Private Cloud Foundation, and enhanced automation tools that make it easy to transition from Ambari to Cloudera Manager.

Support for rollback of HDP2.6.5 and CDH5. [13-16] and corresponding documentation

The conversion from Fair Planner to capacity Planner has been improved using enhanced tools that deal with more complex placement rules.

Platform enhancement

YARN's new placement rules engine provides better conversion for fair scheduler to capacity scheduler migration and better placement rule management.

Automatic dynamic queue support for parent and child queues

Added weight mode to support easier transition for CDH customers.

Added support for stand-alone NiFi / Kafka clusters

We have added operating system support for RHEL / CentOS 7.9 and database support for MySQL8 and Postgres 12 to further assist with the migration.

Object storage

Ozone is a distributed key object storage that provides 20 times the scalability of traditional HDFS, reduces cluster sprawl, removes restrictions on the number of small files, and simplifies cluster management.

Ozone supports 350TB-intensive node configuration, which currently increases available storage capacity by 350% compared to HDFS and reduces storage costs by 50%.

SDX-Security and Governance

The Ranger audit filter provides better audit management. Ranger Audit filters help to control captured audit events using JSON-defined filters to simplify audit volume only by including related events.

Improvements to Ranger Audit UI, which provide enhancements such as adjustable columns and options for selecting visible columns through UI.

Data engineering

Doubled performance with Spark 3 and is now released as a separate component

Hive Warehouse Connector (HWC) makes data engineering easier and faster.

Better Hive-Spark interaction with HWC makes data engineering applications easier and more efficient to create.

Data warehouse

Compared with traditional MapReduce, using Hive on Tez directed acyclic graph (DAG) and data transfer primitive can improve query performance.

The improvement of Impala increases the performance by two to seven times.

Faster Hive queries with instantiated views and query caching

We have enabled role-related statements in Impala to allow the use of Ranger as the authorization provider rather than the Sentry of CDH users.

Operational database

Transaction support added to Phoenix 5.1,

Both SQL and No SQL are supported, and the throughput performance is improved by 15-20%.

Complex x-row / x-table distributed transactions that support running TPC-C benchmarks and support for ANSI SQL can be easily migrated from MySQL databases to Operational Database.

We have added "ApacheAccumulo powered OpDB" based on Accumulo 2.0 to enable HDP customers using Accumulo to upgrade to CDP Private Cloud Foundation using semantic versioning, bulk import, and simplified scripts.

Now, let's draw your attention to three of these features and extend their capabilities to the platform.

Platform-HDP 3 in-place upgrade enhancements

HDP 3.1.5 customers can now upgrade their HDP 3 clusters directly to CDP Private Cloud Foundation without having to build new clusters and migrate workloads or data.

The upgrade path includes the following steps:

A new version of the AM2CM tool (1.2.0) has been created to support the transition from Ambari managed clusters to Cloudera Manager managed clusters.

The AM2CM tool takes the Ambari 2.7.5 blueprint as input and converts it into a Cloudera Manager deployment template. Next, the customer can migrate the deployment template to Cloudera Manager, which allows the customer to start the CDP cluster through Cloudera Manager.

In addition, you can now upgrade from HDP2 and CDH5 clusters using the rollback process.

Platform-Fair Scheduler upgrade tool enhancements

The Fair Scheduler to capacity Scheduler (FS2CS) conversion tool provides an improved scheduler transition for customers upgrading from previous versions of CDH.

With this tool, customers can use the same or better SLA to run their jobs or applications without any interruptions or code changes. After upgrading the cluster to CDP, customers can now use YARN QueueManager to adjust the cluster resource management configuration in a more user-friendly manner.

We have introduced the following new features

Enhanced placement rule engine

Dynamic queue support

Weight model

Placement rules determine the queues to which applications and jobs are assigned. The new layout rule evaluation engine has been enhanced to provide customers with the following:

Static and dynamic queues from single parents are supported.

Other policy options with alternate action configuration, which can be defined as actions that should be taken when the target queue for placing the rule does not exist or cannot be created

Introducing a placement rule strategy provides a better solution than mapping rule creation and provides shortcuts for the most common use cases.

The placement rules engine now supports the create flag, which creates queues that do not exist when automatic dynamic queue creation is enabled.

Automatically converts the old placement rule (queue mapping) to the new JSON-based format

Create a new placement rule from a single page that allows you to configure all options:

In versions prior to 7.1.6, customers could assign resources to queues using either absolute mode (allocating resources per unit) or relative mode (allocating resources as a percentage of total available resources). In this release, we have added a new mode of allocating resources, called the weight mode. The features of the weight mode include:

Assign capacity to a numeric value with the suffix "w". Weight is a small part of the total resource. The queue priority is used as a weight to determine the proportion of total resources that each application should receive.

Switch between relative and weight modes with just a few clicks

One click to enable automatic dynamic child creation of the queue. This allows the parent queue to have both static and dynamic child queues. Static queues have rules and expressions as well as pre-created target queues and user mappings. Dynamic queues allow queues to be created automatically based on rules and expressions. This feature can be easily enabled through the YARN queue manager UI.

Operational database-Apache Phoenix 5.1

We have released Apache Phoenix 5.1 to the CDP private cloud as part of Operation Database to provide the following features:

Scale-out RDBMS based on Apache HBase

Star schema support and evolutionary schema support

View and secondary index support

Full support for Apache Omid

With Phoenix 5.1, we added complex x-row, x-table transaction support (TPC-C benchmarking is supported out of the box). Prior to this release, Phoenix only supported single-line atomic transactions. In this release, it becomes easier to migrate sharded MySQL and PostgreSQL deployments to Cloudera, Cloudera partition management is fully automated, and extension does not mean increased operational complexity.

This version also improves our secondary index to ensure that index updates are highly consistent with data inserts and high inserts.

With the new features, enhancements, and improved upgrade paths added in version 7.1.6, there is no better time to transition existing CDH or HDP clusters to CDP Private Cloud Foundation. To plan your migration, refer to the CDP upgrade and migration path for more information, or contact your Cloudera account team to discuss best practices.

These are the new features of CDP Private Cloud basic version 7.1.6 shared by Xiaobian. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.