Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the overview and function of go monitoring scheme". In daily operation, I believe many people have doubts about the overview and function of go monitoring scheme. The editor consulted all kinds of data and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "Overview and function of go monitoring scheme". Next, please follow the editor to study!

Outline why monitoring is needed

When writing an application, a log is usually recorded for ex post analysis, and in many cases it is a static analysis after a problem has arisen. In many cases, we may need to know how the whole system is running at the present time, or at a certain time, such as how many services are provided in the current system, what is the response time of these services, what is the situation that changes over time, and how often the system goes wrong. These dynamic quasi-real-time information is very important for monitoring the health status of the whole system.

Due to the large number of business systems, a large number of system logs and business logs are generated every day. The log 400m generated by a server of single-stream business may want to view the content directly for a few minutes, and it is impossible to view the content at all. It brings a lot of inconvenience to development and operation and maintenance. Now the business is distributed and logs are distributed on each server, so it is even more inefficient to view logs and statistics. It is extremely urgent to collect logs distributed on different nodes or machines in real time for offline or online access and analysis to improve work efficiency.

These dynamic quasi-real-time information is very important for monitoring the health status of the whole system.

Profiling is very important. If you can have a particularly powerful Profiling system, you will know where the whole system is located, on which machine, and how much resources such as CPU, memory, disk IO, or network bandwidth are spent, and you can know where the optimization is most beneficial.

Where can I get the information needed for monitoring?

This information needs to be obtained directly or calculated on the basis of the data, so how to obtain the real-time data and what standard is used as the measurement standard. That is, when collecting information, how to get it. How to obtain real-time data such as the following

Response time: the time when a user sends a request to the user to receive a response from the server is the response time

Throughput: the number of client requests processed by the system per unit time

Number of concurrent users: the number of users whose colleagues submit requests to the system at a certain time

TPS: transactions per second

PV: access a URL and generate a PV

UV: all pages of the site visited by the user are counted as UV

The technologies that need to be used in the whole solution

Metrics: monitoring information collection and use in project code, supporting gc, mem and other information collection

Statsd: using statsd to transfer udp data

Telegraf: data collection outside the project using telegraf

Influxdb: use this time series data for data storage

Grafana: data presentation

Because the monitoring related technology has not been involved before, so there are all kinds of problems in doing it. Here is a record of the holes that have been stepped on in this process.

When I did it at that time, I mainly wanted to use the metrics provided by metrics as the standard for data collection, then write the data into influxdb, and then grafana to display it, but the metrics used on git only provides a way for TCP to connect to influxdb. So it doesn't meet my needs.

TCP has an impact on performance because it has to shake hands, so it wants to use UDP as the transmission mode, and then find that statsd supports TCP/UDP for transmission, but statsd supports few types, which can not fully meet these metrics supported by the collected metrics. Therefore, it is necessary to convert the complex types collected by metrics into the basic type of statsd for transmission.

The original system uses telegraf to support information collection outside docker, and can add statsd plug-ins, so that you can not only collect performance indicators outside the code, but also easily store statsd transmission data into influxdb, and then use grafana as a display panel to show the data, and then complete the whole program.

The general flow of the plan

At this point, the study on "Overview and role of go monitoring scheme" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.