Shulou(Shulou.com)06/01 Report--

The app store maintained by the Rancher community recently welcomed two star projects-SPM and Logsene, from Sematext's monitoring and logging tool. If you are already familiar with Logstash,Kibana,Prometheus,Grafana monitoring or logging solutions, please read this article carefully, which will show you what features SPM and Logsene bring to Rancher users and how they differ from other solutions.

Contact Sematext Docker Agent

Sematext Docker Agent is a native Docker monitoring and log collection agent (Agent) program. It runs a very small container on each Docker host to collect logs, system metrics, and events for all cluster nodes and containers on those nodes. The Agent agent listens to all nodes managed by Rancher. After deploying Sematext Docker Agent, all logs, Docker events, and metrics can be accessed externally.

It's going to be valuable. Why? Because it means you don't have to spend hours or even days figuring out exactly what data needs to be collected or how to graph it. Also, you don't need to spend resources to maintain your own log monitoring system, your Docker system metrics and events will be output to SPM and logs will be output to Logsene (SPM is an application performance monitoring service that supports integration with many platforms, including Docker, while Logsene is a log management service, an ELK application stack hosted on Rancher, which works well with Kibana and Grafana)

Comparison of DevOps tools

There are some open source tools for Docker logging and metrics processing, such as cAdvisor and Logspout. Unfortunately, none of these tools have sufficient comprehensive capabilities. One solution is to integrate a series of tools to achieve the goal, but the consequence is to take the system to the direction of "Franken-monitoring", which will leave users with heavy technical debt, and no one will be willing to solve this complex problem. As a result, another solution is that tools such as Sematext Docker Agent simply combine cAdvisor with Logspout, with a particular focus on log management, such as format checking, log syntax analysis, data improvement (Geo-IP address geolocation information, metadata tags, etc.), and log routing.

Configure Sematext Docker Agent through Rancher Catalog

To set up Sematext Docker Agent on the Rancher platform, you only need to select the appropriate catalog template (look for "Sematext" under the community Catalog entry in Rancher). Let's take a quick look at what Sematex provides. How to set it on the Rancher platform?

Docker Compose and Kubernetes automatic log tags

All logs are tagged with metadata, which includes support for Docker/Rancher Compose projects and Kubernetes:

For Docker Container

Container ID

Container name

Image name

For Docker/Rancher Compose

Service name

Project name

Number of containers (if you set Scale=N)

For Kubernetes

Note that the logs of the Kubernetes container are not very different from those of other Docker containers, but Kubernetes users need to access the logs of the deployed pods, so it is useful to crawl Kubernetes-specific information for log search, such as:

Namespace

Pod name

Container name

Kubernetes UID

Tip: to enable the Kubernetes tag, set Kubernetes=1

Log routing

For larger deployments, you may need to index logs for different tenants or applications and output them to different paths or Logsene applications for processing (so you can also distinguish between different users' access to different logs). Let's make this very simple: simply add a Docker tag to your container, or set the LOGSENE_TOKEN environment variable (LOGSENE_TOKEN=your app token), and Sematext Docker Agent will send the log to the correct index entry! In this way you don't need a central configuration file to map containers and index entries / tokens, so log routing becomes very flexible, dynamic and flexible.

Integrated log parser

Log processing is based on Docker API and a library called logagent-js, which is open source by Sematext. This analysis framework includes different log format patterns used by the standard formal Docker container for log format detection and analysis:

Web servers such as Nginx, Apache httpd, or other log formats that use common Web servers

Search engines are similar to Elasticsearch and Solr

Message queues are similar to Apache Kafaka and nsq.io

Database is similar to MongoDB, HBase, MySQL

Detect JSON log formats, which are commonly used by Node.js programs, such as bunyan and winston log frameworks

Tip: if you want to create custom patterns, add them to a field called LOGAGENT_PATTERNS in the Rancher Catalog template.

Automatic container log Geo-IP enhancements

Getting the collected, shipped and analyzed logs from outside the Docker container has saved a lot of time, but some application logs need to obtain additional enhancement information from other data sources. A common case is to enhance the log information of Web Server (or any log that contains IP address information) to provide geolocation information for IP addresses in the log.

Sematext Docker Agent supports Geo-IP enrichment of docker logs. It uses the Maxmind Geo-IP lightweight database, which provides you with regular updates without having to stop the container or mount a new volume containing the Geo-IP database, or any other manual operation.

Tip: if you need to turn on Geo-IP enhancements, set the environment variable GEOIP_ENABLED=true.

Filter container log

In some scenarios, we only need to collect logs for important applications and ignore those lower priority or noisy service logs (such as frequent cleanup work). In this way, we can use whitelist or blacklist (for container name or image name) to deal with these containers, which are set to use regular expressions to match the corresponding metadata fields.

Whitelist containers

MATCH_BY_NAME regular expression whitelist setting for container name MATCH_BY_IMAGE regular expression whitelist setting Blacklist containers for image name

SKIP_BY_NAME regular expression blacklists the container name, ignores the container SKIP_BY_IMAGE regular expression sets the image name, and ignores the log

How to use Sematext Catalog projects

In Rancher's UI, enter the list of Catalog projects in the community and search for the keywords "sematext", "monitoring" or "logs".

Click "View Details" and enter the tokens for SPM and Logsene App in the "Configuration Options" option. You can register from https://apps.sematext.com and create your SPM and Logsene applications, and then get these access tokens (tokens). If your Rancher cluster runs behind a firewall (proxy), you need to fill in proxy URL in the HTTPS_PROXY or HTTP_PROXY environment variable. If you are also running Kubernetes in this cluster, select KUBERNETES=1.

If you want to collect all the logs, do not fill in the filter values of any containers or image names, just click "Launch".

Summary

We hope that this introduction to Sematext Docker Agent in the Rancher platform will help you start monitoring and logging Docker without using a hodgepodge of cumbersome patterns. The complete configuration parameters of the project are available for reference from Github. We believe that the new Catalog template can already cover the most commonly used options. If you find some important items missing, please submit them to Rancher community catalog (submit issue or pull request). Try Sematext Docker Agent with Rancher, all services will be easily taken over, and you will have peace of mind. Rancher Community Catalog allows the log monitoring system to set up and run instantly, making everything so easy.

Original source: Rancher Labs

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.