Shulou(Shulou.com)06/01 Report--

This article will explain in detail what OpenTelemetry means, the content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

OpenTelemetry is an observability project of CNCF, which aims to provide a standardization scheme in the field of observability, to solve the standardization problems of data model, collection, processing and export of observed data, and to provide services independent of tripartite vendor.

2021.02.10 the tracing spec of OpenTelemetry reached version 1.0 (link). Based on this milestone, the author explores OpenTelemetry to judge the value and development prospects in the field of observability.

Below is the author's understanding of OpenTelemetry, throwing a brick to attract jade.

What is OpenTelemetry?

From official What is OpenTelemetry? You can learn:

OpenTelemetry is a set of APIs, SDKs, tooling and integrations that are designed for the creation and management of telemetry data such as traces, metrics, and logs.

The project provides a vendor-agnostic implementation that can be configured to sent telemetry data to the backend (s) of your choice. It supports a variety of popular open-source projects including Jaeger and Prometheus.

OpenTelemetry is a collection of standards and tools designed to manage observational data, such as trace, metrics, logs, etc. (new observational data types may emerge in the future).

OpenTelemetry provides vendor-independent implementations that export observation data to different backends, such as open source Prometheus, Jaeger, or cloud vendor services, according to the needs of users.

So what isn't OpenTelemetry? As can be seen from the official description:

OpenTelemetry is not an observability back-end like Jaeger or Prometheus. Instead, it supports exporting data to a variety of open-source and commercial back-ends. It provides a pluggable architecture so additional technology protocols and formats can be easily added.

That is, OpenTelemetry does not provide back-end services related to observability, which usually provide storage, query, visualization and other services.

You can easily understand the scope of OpenTelemetry's work through the following abstract diagram:

What is the problem domain that OpenTelemetry faces?

From wikipedia: Observability to the definition of observability:

In control theory, observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs.

Consider a physical system modeled in state-space representation. A system is said to be observable if, for any possible evolution of state and control vectors, the current state can be estimated using only the information from outputs (physically, this generally corresponds to information obtained by sensors). In other words, one can determine the behavior of the entire system from the system's outputs. On the other hand, if the system is not observable, there are state trajectories that are not distinguishable by only measuring the outputs.

It is simply stated that observability is a method to deduce the internal state of the system from the external output of the system.

The following figure simplifies the composition of the system and the interaction between systems:

From the above interaction diagram, we can see that the interactive behavior of the system has the following forms:

The function of the internal components of the system is closed-loop and does not interact with other components or system interaction components.

Interaction between systems and between systems

In this way, if you want to understand the state of the system through the external output of the system, you need two forms of information:

Information of component closed loop

Information that flows between components or systems

The first form can usually be characterized by logs or metrics, while the second form needs to be characterized by trace, adding markers to the flow of information.

The difference between logs and metrics can be understood through their methods of operation.

Further abstraction, observability involves the following issues:

Data model of observed data

Collection of observation data

Processing of observed data

Derivation of observed data

Use of observed data

Etc.

The above are the problem domains and specific problems faced by OpenTelemetry, and the specific problems are limited to:

Data model of observed data

Collection of observation data

Processing of observed data

Derivation of observed data

What is the solution for OpenTelemetry?

OpenTelemetry standardizes the data model and methods of collection, processing and export of observed data through Spec, including trace, metrics, logs (new types will not be ruled out in the future), see

Opentelemetry-specification .

At the same time, for convenience, it is described by protobuf, see opentelemetry-proto.

Based on Spec,OpenTelemetry for the generation and processing of observation data, the following efforts have been made:

For the convenience of developers, language-related SDK is provided, such as opentelemetry-go, opentelemetry-java, opentelemetry-js, etc. All supported development languages can be found in the official documentation.

In order to facilitate the collection, processing and export of observable data, Collector services managed by configuration are provided, such as opentelemetry-collector for open source projects and opentelemetry-collector-contrib for third-party vendor.

The following figure provides a visual understanding of the components and workflows of OpenTelemetry:

What is the history of OpenTelemetry?

You can simply learn from A brief history of OpenTelemetry (So Far) that OpenTelemetry is made up of two open source projects that are merged:

OpenCensus standardizes data models for trace and metrics, and provides tools for acquisition, processing and export.

OpenTracing standardizes the data model for trace and provides tools for acquisition, processing and export.

In May 2019, two open source projects merged and the open source OpenTelemetry project was officially announced.

According to the official maturity Model (link), the spec of trace has reached stable level, metrics has reached beta level, and logs is still at alpha level:

What are the prospects for OpenTelemetry?

Since the launch of OpenTelemetry, more and more manufacturers have begun to pay attention to and contribute.

From

Opentelemetry-collector-contrib can see that manufacturers focus on the exporter part, which facilitates the import of observation data into their own services, which already include Aliyun's own SLS products:

For receiver and processor, it is believed that manufacturers will gradually devote more energy, such as:

Through the cooperation of receiver and exporter, the processing workflow of observed data is formed.

Standardized processing is carried out before the observation data is stored through processor

For multi-cloud scenarios, the observation data model and acquisition / processing / export standards defined by OpenTelemetry will help users to interface with multiple cloud vendors through a set of observability standards and avoid vendor locking.

Even for a single cloud (such as services within a cloud vendor), it is inevitable to consider open source and co-building with the outside world in the future. Using community observability standards can reduce open source costs. At the same time, the concepts, standards and technologies of observability are constantly iterating, and by following up the community, we can make better use of the technological dividend and influence brought by the community.

Therefore, whether facing a multi-cloud scenario or a single cloud vendor, it is necessary to adopt the observability standards of the industry.

How do I use OpenTelemetry?

Core concept

There are many concepts in OpenTelemetry. Here are some common concepts for easy understanding:

Observation data related Signal observation data types, such as trace, metrics, logsInstrument, can be considered as examples of some kind of Signal.

Formal description of APIOpenTelemetry Spec related to OpenTelemetry's own projects, such as API implementation of opentelemetry-protoSDK for different development languages Contrib Packages related to specific open source projects or vendor products

Component related ComponentsReceivers receives observation data component Processors processes received observation data component Exporters exports observation data, such as exporting to open source project Prometheus or cloud vendor service Extensions does not participate in the processing of observation data, and assists the operation of related processing components, such as health detection, service discovery, etc. Services characterizes which components of the configuration need to run For example, receivers / processors / exporters / extentionsCollector can be considered as a whole composed of receivers / processors / exporters / extentions / services. Controller is used in applications developed by developers, and its role can be equal to that of receivers / processors / exporters.

Golang demo

The author wrote a golang demo to demonstrate:

How to generate trace / metrics data in APP

Using stdout controller in APP to collect, process and print trace / metrics data

Trace / metrics data are collected by otlp controller in APP and exported to external collector.

Run an independent collector service locally to receive trace / metrics data pushed by otlp controller and export it to local files and Ali Cloud SLS

Demo see:

Https://github.com/flyer103/otel-demo

Specific usage can be found in demo's README.md, which is briefly described below.

The cmd/app/server.go file describes the logic of using OpenTelemetry and is divided into two parts:

Initialize and run the global controller to receive / process / export observation data within the APP or to push the observation data within the APP to the outside

Generate metrics and trace according to business requirements in APP

Controller and signal (trace / metrics) are encapsulated in the pkg/ directory, and the specific implementation is not described in detail:

An example of exporting observation data to SLS is provided under yaml/, including receiver for receiving observation data (the client side can push the data to this receiver through grpc client), processors for observation data conversion processing, exporters for data export, and services for opening components:

For developers, trace / metrics / logs can be generated and exported through a set of standard solutions based on OpenTelemetry, reducing the cost of using different types of observation data in the development process, and also reducing the cost of interfacing with different back-end services, such as open source project Prometheus or services of third-party cloud vendors.

For SRE, OpenTelemetry can provide a set of standard process for collecting, processing and exporting observation data, and standardize the observation data according to the needs of the team in the processing process, so as to facilitate the subsequent use of standardized programs to use observation data, such as monitoring and alarm services.

At the same time, both developers and SRE can continue to iterate through the strength of the community to understand the observability problem domain, absorb the community's technical dividend, and return the best practices generated in production to the community, so as to better promote the development of the observability field.

About what OpenTelemetry means to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.