Shulou(Shulou.com)06/01 Report--

This article focuses on "what are the open source tools for monitoring K8S and Docker", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let the editor take you to learn what are the open source tools for monitoring K8S and Docker.

Kubernetes and Docker are the two most common words heard in DevOps circles. Docker is a tool that enables you to run applications in a containerized way, and Kubernetes is a platform for orchestrating and managing containers-impractical if you want to use Docker CLI to manage thousands of containers manually.

However, it is not enough to manage and run thousands of containers through Kubernetes, you also need to monitor these containers to ensure that the service is at its best. This process is called website Reliability Engineering (Site Reliability Engineering), a term coined and promoted by Google. Observability and analysis are important components of SRE. It can be subdivided into the following three parts:

Tracking: extract numerical metrics from applications and hosts that can be visualized and analyzed to show the current state of the resource. Once numerical indicators are extracted, they can be used to set alarm rules, facilitate analysis and debugging, and make better decisions

Logs: help developers troubleshoot the cause of a container failure. The container log disappears with the end of the container life cycle. Kubernetes and Docker do provide a way to browse container logs, but their capabilities are very limited. Therefore, in any environment made up of containers, centralized management logs are necessary.

Tracking: helps you debug services running on the network and track request links until you find the root cause of the problem. In a micro-service architecture, when multiple services / containers send requests to each other to complete a business task, an appropriate tracking solution is required.

This article will explain in detail the six most popular open source tools specifically for the monitoring and analysis of containerized services.

Prometheus

When discussing open source monitoring solutions, the first thing that comes to mind is Prometheus. It is very popular in the development community and is a graduate project of CNCF. Prometheus was originally created by SoundCloud and open source. Prometheus simplifies the process of extracting numerical indicators, directly from a monitoring endpoint based on time series. Suitable for monitoring highly dynamic container environments.

Prometheus consists of three parts: Prometheus server, Alertmanager and exporters. Exporters runs on the target machine as an independent process or container, generates a variety of metric data, and sends it to Prometheus server by API. Prometheus server is responsible for service discovery. You can also pull metric data directly from exporters, and then store the data in Prometheus's database for visualization or alarm services. Alertmanager is used to set alarm rules, analyze data in the Prometheus database, and send alerts to recipients when a rule is triggered. A large number of exporters can be found here, all of which are officially supported by Prometheus and maintained by the community.

Prometheus has become the industry standard for monitoring cloud native applications. Although it is known for its simplicity of service discovery, ease of use, alarm services, and integration with Kubernetes, the way that metric data can only be pulled by Prometheus server is not ideal. This means that exporters must be accessible by Prometheus server. However, a pushgateway is implemented in Prometheus that allows data to be pushed directly to Prometheus server rather than actively pulled by Prometheus server. Another disadvantage of Prometheus is that it does not scale horizontally well. A solution to this problem can be found in Thanos adaptation of Prometheus (https://improbable.io/blog/thanos-prometheus-at-scale).

Related tools and technologies: Grafana, Cortex, Thanos, Prometheus Exporters, Alert Manager, Istio, Prometheus Operator.

Figure 1:Prometheus chart

Grafana

Grafana is an open source indicator analysis and visualization suite. It allows you to create custom dashboards using data from multiple data sources such as Prometheus, Elasticsearch, MySQL, Postgres, and Redis. In addition, Grafana has its own alarm system and role-based access control (RBAC) system. As a data visualization tool, Grafana is very famous among Prometheus users. It can effectively visualize the indicator data stored in Prometheus. Grafana provides a large number of official and community-built customized dashboards for data from a variety of data sources, allowing users to easily set up dashboards and monitor data. Grafana provides another related product, Loki, which aggregates logs in Kubernetes and integrates well with Grafana UI.

Related tools and technologies: Loki, Prometheus.

Figure Kubernetes resource dashboard in 2:Grafana

Elastic Stack

Elastic Stack is a set of open source products from Elastic designed to help users search, analyze, and visualize data in any format from any type of data source in real time. The product was formerly known as ELK, and each letter represents one of the company's major products: Elasticsearch, Logstash and Kibana. Elastic Stack leverages Elasticsearch to provide monitoring and logging solutions.

In order to aggregate logs, people tend to use Elasticsearch for storage, use Logstash or Fluentd to send logs, and use Kibana for visualization. Fluentd is not a part of Elastic Stack, but it is widely used in Kubernetes environment to replace Logstash. Similarly, Metricbeat is used to capture metric data and visualize it on Kibana. The Elastic Stack Enterprise Edition comes with the XMui Pack Pack, which is a set of additional tools that support reporting, alerts, and role-based access control (RBAC). By default, Elastic Stack GUI (Kibana) does not support RBAC. You must use Elastic Stack Enterprise Edition to enable RBAC.

Related tools and technologies: X-pack, Metricbeat, Logstash, Kibana.

The discovery view in figure 3:Elasticsearch.

Sensu Go

Sensu Go is a telemetry and service health inspection solution for multi-cloud monitoring. Help you view servers, containers, services, applications, functions, and connected devices across any public or private cloud. Sensu can be run with Prometheus to get the best results from both solutions, or you can run it yourself without Prometheus. Because exporting application-level metrics data to Prometheus requires loading Prometheus SDK into the application's code base and exposing a port, Sensu works best with Prometheus. The port is then responsible for collecting data and storing it in Prometheus server. It sounds like there's a lot of work to do-sometimes it is. Sensu avoids this complexity by using sidecar. A Sensu agent is deployed next to the application. The Sensu agent constantly collects data and sends it to Prometheus server so that there is no need to change the application code.

Sensu can also work independently of Prometheus and run in Kubernetes, where Sensu has its own server to store and visualize data collected by Sensu agents.

Related tools and technologies: Prometheus.

Events in figure 4:Sensu

Sysdig Inspect

Sysdig has two open source products: Sysdig Inspect and Falco. Here, we will focus on the use of Inspect,Sysdig Inspect to monitor and capture container processes running in the system, and allow you to drill down into these processes for ex post facto forensics to help analyze application performance, troubleshoot errors, and monitor any processors that may have errors. In addition, if your system is compromised, Sysdig helps you understand how the damage occurred and what data was lost in the process. Sysdig Inspect is a very powerful tool that focuses on system performance tuning and security investigation.

Related tools and technologies: Grafana, Sysdig, Sysdig Falco.

Figure 5:Sysdig check overview

Jaeger

Jaeger is an end-to-end distributed tracking solution that is open source by Uber Engineering. Help you monitor and troubleshoot transactions in complex distributed systems. In modern micro-service architecture, most of the operational problems belong to the category of network and observability. When a service failure occurs, you don't know how the request is transferred from one service to another over the network to complete a single business request. Debugging becomes extremely difficult. Jaeger is currently incubated under CNCF, and Jaeger uses tracking techniques to find the cause of errors, performance and latency optimizations, and distributed transaction monitoring. Jaeger can be used with Istio, which is implemented by Google's open source popular service grid.

Related tools and technologies: Prometheus, Jaeger, Zipkin, Istio.

Figure 6:Jaeger trajectory tracking chart

At this point, I believe you have a deeper understanding of "what are the open source tools for monitoring K8S and Docker?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.