Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how to use distributed link tracking". In daily operation, I believe that many people have doubts about how to use distributed link tracking. The editor consulted all kinds of data and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to use distributed link tracking"! Next, please follow the editor to study!

Overview of distributed link tracing

Before we introduce the distributed link tracking system in detail, we need to understand what link tracking is. In the introduction of the monitoring system earlier in this column, we can know that the observation data of the monitoring system mainly come from three aspects: statistical indicators, logs and link tracking. The data can be divided into two types: request level and aggregation level.

The data at the request level mainly comes from real requests, such as a HTTP call, a RPC call, and so on. This type of link tracing is introduced in this article. The aggregation level is the metrics requested by the API or the aggregation of some parameter data, such as QPS, CPU usage, and so on. Log and statistical metrics data can be either request-level or aggregate-level, because they may come from real requests or from information recorded during the system's own diagnosis.

For link tracking, its main logic is to record the complete behavior of the requested link, so that the distributed link tracking functions such as link query, performance analysis, dependency relationship, topology diagram and so on can be realized visually. As shown in the following figure:

In the figure above, it is assumed that a total of two micro-services participate in an interface call in the micro-service system, and the invocation relationship is A-> B-> C, where B service also has an invocation relationship with third-party services such as Redis, and C service also needs to call MySQL database service. So in fact, what link tracing does is to record in detail the detailed call information on the complete link A-> B (B-> Redis)-> C (C-> MySQL), such as interface response result, time consuming, and so on.

So how exactly is the data on this calling link recorded? Then we continue to take the above call chain as an example to analyze the specific composition and transmission form of the downlink tracking information, in order to further understand the principle and concept of the distributed link tracking system. The specific logic diagram is as follows:

As shown in the figure above, the object monitored by distributed link tracking is the link generated by repeated calls. Figure 1-8 shows a complete link (Trace), which is recorded by the system through a unique TraceId. Each dependent call in the link generates a call trace information (Span). The initial generated Span is called the root Span (Root Span), and the subsequent generated Span takes the Sid of the previous Span as the parent ID (Pid) of this Span information.

In this way, the Span information will be transmitted in context within or across processes with the execution of the link, and the trace information generated by the link calls can be concatenated through the Span data link, and the log information (Annotation) attached to each Span is the data source for our call chain monitoring and analysis. This is the basic principle of distributed link tracing.

At this point, you may have questions: monitoring such a large amount of data will consume a lot of system resources? This is true, so most link tracking systems have a setting called sampling rate (Sampling), which is used to control the proportion of link information collected by the system, thus improving system performance. Because in many cases, a large amount of link information is the same, we may only need to pay attention to the links that are relatively time-consuming and have more errors, and it is not necessary to collect 100% of the link.

Introduction to SkyWalking

Before we explained what link tracking is from the point of view of basic principles, then we will introduce the most popular distributed link tracking system-SkyWalking.

SkyWalking is an excellent open source APM (Application Performance Management) system, which not only provides distributed tracking functions such as link tracking and link analysis, but also supports a series of application performance monitoring functions, such as performance index analysis, application and service dependence analysis, service topology analysis, alarm, etc., which can help us to locate problems effectively.

In terms of data collection, SkyWalking supports a variety of different data sources and formats, including non-invasive Agent probes in different languages such as Java, .NET Core, NodeJS, PHP and Python, as well as support for Service Mesh (Service Grid) architecture. The specific structure is shown in the following figure:

As shown in the figure above, the core of SkyWalking consists of a link collection server (Receiver Cluster) and an aggregation server (AggregatorCluster). Receiver Cluster is the entrance to the access of the entire back-end service, which is specially used to collect various indicators and link information of the service.

AggregatorCluster is used to summarize and aggregate the data collected by the collector, and finally store the aggregated data in the database, and there can be a variety of specific storage methods, such as common ElasticSearch, MySQL, TIDB, etc., we can choose according to the actual needs. These aggregated data can be used for alarm settings, and can also be visually displayed after being accessed by visualization systems such as GUI/CLI in the form of HTTP.

In addition, from the perspective of data acquisition logic, SkyWalking supports multiple language probes and project protocols, and can cover most of the mainstream distributed technology stacks. Specifically, there are three main types:

Metrics System: statistical system. It supports pulling metric data directly from Prometheus to SkyWalking, and also supports the program itself to push data through micrometer.

Agents: service probe. It refers to the integration of probe services in various business systems for link tracking, that is, link data collection. SkyWalking supports probes in Java, Go, .NET, PHP, NodeJS, Python, Nginx LUA and other languages. In addition, it supports the transfer of data through gRPC or HTTP

Service Mesh:SkyWalking also supports the monitoring of the new generation of micro service architecture Service Mesh. It can collect data from the data plane and control plane through a specific Service Mesh protocol, and realize the observation of the link data of the service grid.

The above content briefly introduces the basic situation of SkyWalking, and makes a simple analysis of its system architecture. In fact, SkyWalking has developed very fast in the past two years, and the community is also very active, and it is more and more widely used in the fields of micro-service link tracking and application performance monitoring. because of the space, it is impossible to share more deeply. Interested readers can learn more about it through official documents or the community!

SkyWalking installation and deployment

The previous content introduces the basic principles of distributed link tracing, and focuses on SkyWalking! Obviously, if the writing ends here, this article will be of little value, because it's just a bunch of correct nonsense, and you'll forget it when you read it! This is obviously not in line with my sharing style, so let's play SkyWalking from an experimental point of view.

The following content needs to carry on the actual experiment operation, if it is not convenient on the subway, you can collect it first, and then play the specific experiment when you have time!

The deployment of SkyWalking mainly involves back-end OAP Server and front-end UI, which can be deployed in physical machines, virtual machines or Kubernetes clusters according to actual needs. In order to demonstrate the consistency of the environment, we choose to deploy the SkyWalking back-end service and UI to the Kubernetes cluster respectively.

The specific way to install SkyWalking can be installed by Helm through the official Kubernetes deployment file, or you can write the Kubernetes deployment file manually. Here, in order to facilitate learning, we use the latter method. The specific steps are as follows:

1) create a Namespace running the SkyWalking container separately in the Kubernetes cluster. The command is as follows:

# create the namespace command $kubectl create ns skywalking after connecting to the Kubernetes cluster through kubectl

After the command is executed, you can check whether the Namespace has been created successfully, as follows:

# View namespace creation $kubectl get ns NAME STATUS AGE default Active 10d kube-node-lease Active 10d kube-public Active 10d kube-system Active 10d kubernetes-dashboard Active 10d skywalking Active 46s

You can see that the skywalking space has been successfully created at this time!

2). Write SkyWalking-UI and OAP Server service Kubernetes deployment files

In the process of writing a specific Kubernetes deployment file, you need to specify the container images of SkyWalking-UI and OAP Server. Generally speaking, you can package them manually or use the official packaged images directly. For the convenience of demonstration, the images that have been packaged in the official image repository of Docker are used here. As shown in the figure:

As shown in the above two figures, we have found the official container image versions of SkyWalking-UI and OAP Server in the Docker Hub official image repository, and then write the specific deployment files.

Write the SkyWalking server Kubernetes deployment file (skywalking-aop.yml) as follows:

ApiVersion: apps/v1 kind: Deployment metadata: name: oap namespace: skywalking spec: replicas: 1 selector: matchLabels: app: oap release: skywalking template: metadata: labels: app: oap release: skywalking spec: containers:-name: oap # specify OAP Server container image and version information image: apache/skywalking-oap-server : 8.3.0-es7 imagePullPolicy: IfNotPresent ports:-containerPort: 11800 name: grpc-containerPort: 12800 name: rest-apiVersion: V1 kind: Service metadata: name: oap namespace: skywalking labels: service: oap spec: ports: # restful port-port: 12800 name: rest # rpc port -port: 11800 name: grpc-port: 1234 name: page selector: app: oap

The above is a standard Kubernetes deployment file, and the specific meaning of the relevant instructions in the file can be found in Kubernetes-related information.

Write a SkyWalking-UI deployment file (skywalking-ui.yml) as follows:

ApiVersion: apps/v1 kind: Deployment metadata: name: ui-deployment namespace: skywalking labels: app: ui spec: replicas: 1 selector: matchLabels: app: ui template: metadata: labels: app: ui spec: containers:-name: ui image: apache/skywalking-ui:8.3.0 ports:-containerPort: 8080 Name: page env:-name: SW_OAP_ADDRESS value: oap:12800-apiVersion: v1 kind: Service metadata: name: ui namespace: skywalking labels: service: ui spec: ports:-port: 8080 name: page nodePort: 31234 type: NodePort selector: app: ui

3) execute the Kubernetes deployment command according to the prepared deployment file

According to the Kubernetes release file written in the previous step, here we directly execute the deployment command based on the published file, as follows:

# enter the storage directory of the published files, and directly execute all the file deployment commands $kubectl apply-f. Deployment.apps/oap created service/oap created deployment.apps/ui-deployment created service/ui created

After the execution is completed, use the command to view the specific deployment, as follows:

# View Pod in skywalking space, Operation of the Service object $kubectl get all-n skywalking NAME READY STATUS RESTARTS AGE pod/oap-5f6d6bc4f6-k4mvv 1 Running 0 36h pod/ui-deployment-868c66449d-fffrt 1 Running 0 36h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGE service/oap ClusterIP 10.110.112.244 12800/TCP 11800/TCP 1234/TCP 36h service/ui NodePort 10.100.154.93 8080:31234/TCP 36h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/oap 1/1 1 1 36h deployment.apps/ui-deployment 1/1 1 1 36h NAME DESIRED CURRENT READY AGE replicaset.apps/oap-5f6d6bc4f6 1 1 1 36h replicaset.apps/ui-deployment-868c66449d 1 1 1 36h

You can see that the deployed SkyWalking services are running properly! If this is the first time to deploy, the process of pulling the image may be a little slower. If you have problems during deployment, you can also view the running log of the Pod object, such as:

# you can view the startup log of aop $kubectl logs pod/oap-5f6d6bc4f6-k4mvv-n skywalking

4) check the Web access address of SkyWalking-UI

After the above steps, we have successfully run the SkyWalking-UI and OAP Server services in the Kubernetes cluster. Next, access the Web UI through the mapped port of the SkyWalking-UI service (defined as port 31234 in the k8s deployment file), which can be accessed through http://NodeIP:31234, for example:

# the IP here is the exposed node entry IP http://10.211.55.12:31234/ of the Kubernetes cluster

If you do not know the entry IP address of the Kubernetes cluster node, you can view it with the following command:

# query the IP address of the Kubernetes cluster Node node deployed by SkyWalking-UI $kubectl describe node kubernetes Name: kubernetes Roles: master... Addresses: InternalIP: 10.211.55.12 Hostname: kubernetes...

The display effect of the accessed interface is as follows:

As shown in the figure above, you can see that SkyWalking has been running successfully, and there is no monitoring data for the time being because there is no service access.

At this point, the study on "how to use distributed link tracking" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.