Shulou(Shulou.com)06/01 Report--

What is the architecture design of Apache SkyWalking? I believe many inexperienced people don't know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Apache SkyWalking provides a powerful and lightweight backend. Here, I will explain why it is designed in the following ways and how it works.

Architecture diagram

For APM, agent or SDKs are just technical details of how to use libs. Manual or automatic forms have nothing to do with architecture, so in this article, we won't talk about them, but think of them as Client lib.

Basic principles

The basic principles of SkyWalking architecture design are:

1) easy to maintain

2) controllable

3) based on flow

To achieve this, the SkyWalking backend provides the following design:

1) Modular design

2) provide multiple connection methods for clients

3) Cluster discovery mechanism

4) flow mode

5) switchable storage implementation

I. Modularization

The SkyWalking collector (collector) is based on modular design, and users can change or integrate the functions of the collector according to their own needs.

2. Module

The module defines a set of features that can include technical implementations (such as grpc/jetty server management), trace analysis (such as trace segment or zipkin span parsers), or aggregation features. All in all, these are defined and implemented by modules.

Each module can define its own services through the Java interface, and the implementation class implements these services. And these implementation classes define which classes they depend on based on the functionality of the implementation. This means that even two different implementations of a module can rely on different modules.

In addition, the modular core in the collector checks the startup sequence, and if no circular dependencies or dependencies are found, the core function terminates the collector.

The collector starts all modules, which are defined in the application.yml file. The structure of this file is as follows:

1) the root node is the module name, such as: cluster,naming

2) the secondary node is the functional implementation name of this module, for example: zookeeper is the cluster module

3) the third-level node is the attribute of function implementation, such as: hostPort and sessionTimeout are the attributes required by zookeeper.

Third, multi-connection mode

First, the collector provides two types of connections, that is, support for two protocols: HTTP and gRPC.

1) name the service in HTTP and return all available collectors in the backend cluster

2) the Uplink service supports gRPC (mainly used as a local agent for SkyWalking) and HTTP, which tracks and measures collectors. Each client sends only monitoring data (tracking and metrics) to a single collector. If the connected collector is disconnected, try to connect to another collector.

Sample processing flow between client lib and Collector Cluster

Fourth, collector cluster discovery

When collectors are running in cluster mode, collectors must discover each other in some way. By default, SkyWalking uses zookeeper for coordination and uses this as the registry for discovery.

In this way, the client's lib will not use zookeeper to find the cluster. Users are advised not to do this. Because the cluster discovery mechanism is switchable, it is provided by the modular core. Based on this, the ability to switch is broken.

We hope that the community will provide more functional implementations of cluster discovery. For example, there is Eureka,Consul,Kubernate now.

Fifth, flow mode

The flow pattern favors a lightweight storm/spark implementation and allows you to use api to build a flow process diagram (DAG), as well as a data convention for the input / output of each node.

The new module can find and extend the existing process diagram.

There are three situations in the process:

1) synchronization process. Traditional method invocation.

2) Asynchronous process, a.k.a batch process based on queue buffer.

3) remote process, aggregation matrix collector, in this way, the selector is defined in the node to determine how to find the collector in the cluster. (HashCode,Rolling,ForeverFirst is supported in three ways)

With these features, the collector operates like a flowing network. Support for writing the same id at the same time through aggregation metrics and storage-independent implementation capabilities.

6. Switchable storage implementation

Because the stream pattern is responsible for concurrency, the responsibility of the storage implementation is to provide high-speed sketching and group queries.

ElasticSearch is now supported, as well as H2 preview, and the ShardingSphere project is supported for the management of MySql relational database clusters.

7. Web UI

In addition to the principles of collector design, UI is another core part of SkyWalking. It provides collector cluster discovery, query dispatch, and visualization based on React, Antd, and Zuul agents.

Web UI uses localhost:10800 to do named queries for the collector cluster.

After reading the above, have you mastered how to design the architecture of Apache SkyWalking? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.