Shulou(Shulou.com)06/01 Report--

This article introduces you to what Cloudera Management Service is. The content is very detailed. Interested friends can refer to it for reference. I hope it can help you.

I'll briefly explain how Service Monitoring (SMON) and Host Monitoring (HMON) work with CM/CDH.

As shown above, I have all CDH services in a single node and managed by a CM Agent, which reduces the number of connections between each component for ease of reading.

Before I dive into the diagram, let me explain what each component of the diagram does.

CM Server

It is an important part of Cloudera Manager, which mainly has the following functions:

Saving cluster health to database

Save configuration information for each component to a database

Cloudera Manager API

Cloudera Manager Web UI

Manage Cloudera Manager Agent Process Heartbeat

Communicates with various Cloudera Management Services to provide the following capabilities:

Generate alarm message

alarm

display the chart

trigger command

etc…

CM Agent

This service is available on every node managed by Cloudera Manager and its primary role is to:

Start or stop processes via supervisor

Decompress configuration and deploy on each host

trigger installation

monitoring host

Service Monitor (SMON)

Its main functions are:

Collect health and metrics information about services and present it on CM's Web UI.

Collect activity information from YARN services to display YARN's job list on CM's Web UI.

Collect activity information from Impala services to display Impala's job list on CM's Web UI.

Host Monitor (HMON)

Its main functions are:

Collect health and metrics information about hosts and present it on CM's Web UI.

Now let's go back to the previous diagram to see exactly how services communicate and work.

As you can see CM agent is installed on every host running CDH components, and these hosts are managed by Cloudera Manager. Agent keeps track of all processes running on the host such as NameNode, ResourceManager, JobHistory Server, etc. This is achieved through supervisord, a third-party component integrated with Cloudera Manager.

By default, CM agents send heartbeats to CM servers every 15 seconds, so CM servers always know what is happening on these hosts. For example, actions triggered on certain hosts, such as starting, stopping, or restarting services, are sent to CM servers as part of a heartbeat, CM updates these information records to backend databases, and new instructions are sent to agents telling them what to do next.

CM server saves all data to database, including configuration of each component, status of each service and host, etc. Most CM/CDH users will use MySQL, PostgreSQL or Oracle as their backend database, as these are widely used relational databases.

CM agent also periodically sends host and service status to Host Monitor and Service Monitor so CM can record health status and display it on its Web UI. If any abnormal condition is detected, an alarm is triggered in CM and the user is notified.

About what Cloudera Management Service is shared here, I hope the above content can be of some help to everyone, you can learn more knowledge. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.