Shulou(Shulou.com)06/02 Report--

Abstract: this paper introduces two feasible monitoring schemes of K8s: Prometheus and Sensu. Both schemes can provide comprehensive system-level monitoring data to help developers track the performance of key components of K8s, locate faults and receive early warning.

Extended Reading: Kubernetes Monitoring practice (1): the working principle and Monitoring practice of K8s

1. Brief introduction of Prometheus1.1 for K8s monitoring

Prometheus is an open source monitoring and early warning tool for containers and micro-services, which is robust and suitable for cloud administrators and developers in the development process. Prometheus regularly aggregates the indicator data in the configuration object, evaluates rule expressions, displays results, and sends early warning.

Prometheus can not only monitor predefined indicators, but also implement multidimensional data models, conduct in-depth analysis, and establish associations for multiple indicators to provide data support for developers and administrators from multiple perspectives.

1.2 how it works

Prometheus is an additional layer of the K8s environment. Data acquisition parameters and data acquisition interval should be defined during the first installation. Prometheus can not only monitor K8s nodes in real time, but also analyze the monitoring data regularly. Prometheus can also send early warning against node failure and other failures.

1.3 Prometheus Operator

Many users will confuse controller with operator. K8s Operator refers to the mode that Controller adds new objects, configuration management Prometheus and other applications to K8s API. In short, operator is a domain-specific controller.

PrometheusOperator simplifies the operation of Prometheus in K8s without changing the configuration of K8s. Using Prometheus Operator, you can easily monitor the services of K8s. You can run Prometheus through a predefined .yml file. Prometheus Operator can create, configure, and manage all monitoring instances on K8s. When deploying a new application, K8s creates a new pod (container). After the creation, the original pod will be destroyed. Prometheus continuously monitors API and, if inconsistencies are found, creates a new Prometheus configuration based on service or pod changes.

1.4 Core components

As a robust monitoring tool, Prometheus pulls data from the service and does not need the service to actively push data. However, Prometheus provides a push entry, but it can accept the data pushed by the service when it is unable to pull data.

In addition, Prometheus supports the association of time series with metric names and key-value pairs, simplifying the monitoring of multiple clouds. Prometheus not only monitors the overall situation of the application, but also drills into the micro-service level. Users can manage data through the query function to understand the application status; you can also use PromQL to create charts and tables to achieve data visualization and generate early warning according to specific parameters.

Prometheus's Web console shows all the features and tools. Users can create datasets and alerts through regular expressions and advanced PromQL search conditions.

1.5 advantage

The biggest advantage of Prometheus is that it is simple and flexible, and it can realize the multi-dimensional data model of monitoring. Users can build a monitoring framework for container clusters, and can also be combined with Grafana to further improve the visualization of monitoring data.

Prometheus can collect node, pod and service metrics through the local service discovery configuration of K8s. Users can directly define expressions and create early warning without switching back and forth in different monitoring systems.

The crawling capability of Prometheus can be integrated into tools such as K8s, Docker and StatsD. Users can also configure early warning and manage charts through Web GUI.

But Prometheus also has its drawbacks: the data model is limited. The default time series data acquisition model of Prometheus has both advantages and disadvantages: on the one hand, this model helps to collect data according to the standard format; on the other hand, the standardized data format leads to the limitation of the data model. Prometheus maintains communication with K8s through the discovery mechanism, so there is a delay.

II. Brief introduction of Sensu2.1 for K8s monitoring

Sensu is a monitoring tool for multi-cloud container infrastructure. Sensu agent is a cross-platform event generator. Users can monitor the health status of systems and services by executing service check, while collecting and analyzing metrics. It not only provides early warning or event management functions, monitoring API, client libraries and multi-script or programming language plug-ins, but also supports custom workflows, enriching Sensu capabilities.

Prometheus actively grabs data by pulling, while Sensu uses the communication mechanism of message bus to push or pull data through publish / subscribe (Pub/Sub).

2.2 how it works

Sensu uses local plug-ins to collect data from mainstream programs such as StatsD libraries, Prometheus exporter, Nagios plug-ins, and SNMP traps. Unlike other monitoring tools such as Prometheus, Sensu is available out of the box and supports multi-cloud environments. Users can provide high availability after configuring the framework.

Sensu provides event filter, mutator, and operator handler to support the creation and automation of model workload. In this way, the data of external monitoring tools can be integrated with the monitoring data of Sensu to form an event-based monitoring path.

2.3 Core components

Sensu optimizes auto-discovery capabilities for K8s containers. Users can easily configure monitoring check and collector for container environments such as K8s and Docker, or configure multiple check for all components of K8s and applications running on K8s.

Sensu also supports native integration and plug-ins, which can be used with logging tools and Prometheus. Users can run Sensu and Prometheus at the same time to handle different data sets. Sensu can collect StatsD metrics and write the acquisition results to Prometheus.

Sensu can also run in a K8s environment. Suppose you initially deploy Sensu in a container and then decide to migrate the entire application to another container environment. After the migration is complete, Sensu agent can still function properly in the new environment and can be discovered by Sensu because Sensu has an automatic discovery mechanism.

Application of 2.4 in K8s

Sensu is scalable and can be expanded with the increase of deployed applications and cloud environments to provide specific functional monitoring services. There is no limit to the number of objects monitored by Sensu, and it will not be further complicated by the increase in the number of monitored objects.

3. Sensu+Prometheus of K8s monitoring

Running Sensu and Prometheus simultaneously can improve the visibility of monitoring and realize the complementary advantages of their respective functions.

3.1 how it works

SensuPrometheus Collector is a Sensu Check plug-in that aggregates data fetched from Prometheus exporter or Prometheus query API. The collected data are stored in STDOUT in Influx (default), Graphite and JSON format.

SensuPrometheus Collector integrates the workload automation capabilities of Sensu and the data crawling capabilities of Prometheus. Users can design and implement the code and early warning receiving time according to their own needs. Sensu can also store the collected metrics data to external timing data sets such as InfluxDB, Graphite and Prometheus.

3.2 install Sensu PrometheusCollector

Users can discover, download and share assets through Sensu's asset index library Bonsai. Click the download button on the Bonsai information page to download the asset definition for the Sensu back-end platform and architecture. When a request for check, filter, mutator, or handler is received, Sensu downloads the verification asset based on the asset definition.

After downloading the asset definition, you can register the asset through Sensu's resource management command line tool sensuctl, and then create a monitoring workflow.

Below is the asset definition of Prometheus collector and the corresponding Linux sensuctl command:

The following is the check instance of Prometheus Collector:

3.3 advantage

Running Sensu and Prometheus at the same time has the following advantages:

It can not only monitor the health status of the K8s cluster, but also dynamically monitor the health status of the surrounding infrastructure, and collect relevant indicators. It can not only realize automatic problem discovery, but also simplify the monitoring process through automated workflow. Running Prometheus and Sensu at the same time can obtain more context information and achieve more fine-grained data capture analysis. Sensu adopts standard communication password and can safely collect and transmit data through a single agent. Easy to manage configuration monitoring settings. Be able to monitor the entire infrastructure. IV. Conclusion

Sensu can monitor the entire infrastructure with a high level of customization and richer context. The Prometheus model is robust and enables deep data drillthrough, while Sensu uses industry standard technologies and formats such as Nagios and StatsD to monitor the entire infrastructure. Sensu and Prometheus can complement each other and provide richer context information.

As we become more and more dependent on software, software availability and fault repair time are very important for the survival of the enterprise. Therefore, only a full understanding of the system can overcome these challenges, infrastructure and application monitoring is a key part. In order to make full use of the ability of K8s, we must implement a unified monitoring solution to achieve the effective convergence of new and old technologies.

Author: STEFAN THORPE

Original link: https://dzone.com/refcardz/monitoring-kubernetes

First launched in UAVStack intelligent operation and maintenance.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.