Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the introduction of the relationship between Metrics, tracing and logging". In the daily operation, I believe that many people have doubts about the introduction of the relationship between Metrics, tracing and logging. The editor has consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about the relationship between Metrics, tracing and logging. Next, please follow the editor to study!

Text

Today, I am honored to attend the 2017 distributed tracking Summit (2017 Distributed Tracing Summit) and have a pleasant communication and discussion with colleagues from AWS/X-Ray, OpenZipkin, OpenTracing, Instana, Datadog, Librato, and other organizations. One of the important arguments is about the scope and definition of the monitoring project. As a distributed tracking system, should logs be managed? From a different point of view, what exactly is a journal? How to locate these various systems through a picture?

Generally speaking, I think we are entangled in some common nouns. I think we can define the scope of monitoring through charts to make the scope of each noun more clear. We use Venn diagram to describe the definitions of Metrics, tracing and logging. The three of them overlap in some cases, but I try to define their differences. As shown in the following figure:

The characteristic of Metric is that it is accumulative: they are atomic, each is a logical metering unit, or a bar chart over a period of time. For example, the current depth of the queue can be defined as a metering unit that updates statistics when writing or reading; the number of input HTTP requests can be defined as a counter for simple accumulation; and the execution time of requests can be defined as a bar chart that updates and summarizes statistics on a specified time slice.

The characteristic of logging is that it describes discrete (discontinuous) events. For example, an application outputs debug or error information through a scrolling file and stores it in Elasticsearch through a log collection system; approval details are stored in a database (BigTable) through Kafka; or the metadata information for a specific request is stripped from the service request and sent to an exception collection service, such as NewRelic.

The most important feature of tracing is that it processes information within the scope of a single request. Any data and metadata information is bound to a single transaction in the system. For example: a RPC execution procedure that invokes a remote service; an actual SQL query statement; and a business ID of an HTTP request.

According to the above definition, we can mark the overlap of the image above.

Of course, a large number of monitored applications are applications with distributed capabilities (Cloud-native), and logical processing is done within the scope of a single request. Therefore, it makes sense to discuss the context of tracking. However, we note that not all monitoring systems are tied to the life cycle of the request. They may be logical component diagnostic information, lifecycle details of the process, which are orthogonal to any discrete request time. Therefore, not all metric and log can be crammed into the concept of tracking systems, at least not without data processing. Or, we may find that the use of metric statistics is of great help to application monitoring, such as prometheus ecology, which can quantitatively display the application view in real time; accordingly, if we force metric statistics to be processed using the pipeline for log, we will lose a lot of features.

So, here, we can begin to classify known systems. For example, Prometheus, a dedicated metric statistics system, may evolve into a tracking system over time to count metrics within requests, but it is unlikely to go deep into the field of log processing. ELK Ecology provides log recording, scrolling and aggregation, and constantly accumulates and integrates more features in other areas.

In addition, I find that when showing the relationship between the three through the Venn diagram, it happens to show an additional effect. Of the three functional areas, metric tends to be more resource-efficient because it compresses data "naturally". On the contrary, logs tend to increase indefinitely and frequently exceed the expected capacity. (there is another article I wrote about this, check it out, translator's note: untranslated). So, we can draw the demand trend of capacity on the graph, metrics is low to logging is high, and trace may be in the middle between them.

Perhaps this is not the most perfect way to describe the management of the three, but from the feedback I received at the meeting, the classification is quite good: the clearer the relationship between the three, the easier it is for us to discuss other issues constructively. If you try to position the function of the product, you may also need this picture to clarify the location of the product during the discussion.

At this point, the study of "introduction to the relationship between Metrics, tracing and logging" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.