Shulou(Shulou.com)06/01 Report--

This article mainly shows you "how to solve the problem of full-link monitoring of Spring clouded micro-service project based on Pinpoint". The content is easy to understand and clear. I hope it can help you solve your doubts. Let me lead you to study and learn this article "how to solve the problem of full-link monitoring of Spring clouded micro-service project based on Pinpoint".

1. The concept of full-link monitoring

With the popularity of micro-service architecture, services are split according to different latitudes, and multiple micro-service programs may be designed for one request. These services may be developed in different programming languages and developed by different teams, and many copies of songs may be deployed, so you need some tools that can help understand system behavior and analyze performance problems, so that when problems occur, you can quickly locate and solve problems. This tool is the full link monitoring component.

The full-link monitoring component can monitor a data index of the communication between each micro-service program, which can help operators to analyze how a request is forwarded to each micro-service and the scheduling between micro-services.

Full-link performance monitoring displays various indicators from overall latitude to local latitude, showing the performance information of all scheduling chains across applications, making it easy to measure the overall and local performance, and finding the source of the fault in terms of finding the source of the fault. the troubleshooting time can be greatly shortened in production.

Problems solved by full-link monitoring:

Request Link Trackin

Through monitoring, we can analyze the invocation relationship between services, draw the runtime topology information, and visualize it.

Invocation measurement

Performance analysis of each call link, such as throughput, response time, number of errors

Container planning reference

According to the analysis of monitoring data, micro services can be expanded / reduced, degraded and flow controlled in time.

Operation feedback

Quickly locate error messages through monitoring data and business logs

There are many kinds of full-link monitoring tools, which should be selected from the following aspects

Performance consumption of probe

Full-link monitoring tools need to open a probe to monitor the links of micro-services, so we need to choose tools that have as little impact on services as possible, fast data analysis and small performance consumption.

The intrusiveness of code

Some link monitoring tools also need to be integrated in the application code, so we should choose tools that can monitor micro-services while not changing the business program code as much as possible, so as to reduce the burden on developers.

Monitoring latitude

The monitoring data should be all-sided and in place, and the more monitoring indicators, the better.

Expandability

The tool can support sub-deployment and has good scalability.

The mainstream link monitoring tools are zipkin, skywalking and pinpoint.

Introduction of 2.pinpoint Link Monitoring component

Pinpoint is an APM (application performance management) tool suitable for large-scale distributed systems written by JAVA/PHP.

Features of pinpoint:

Service Map (serverMap): show the system topology by visualizing the modules of the distributed system and the interaction between them. Clicking on a node will show the details of the module, such as the current status and the number of requests.

Real-time activity Line Diagram (Realtime Active Thread Chart): real-time monitoring of active threads within the application.

Request response Distribution Map (Request/Response Scatter Chart): long-term visualization of the number of requests and response patterns to identify potential problems.

Call stack (CallStack): generate code-level views for each call in a distributed environment, locate bottlenecks and Inspector in a single view: view other details on the application, such as CPU usage, memory / garbage collection, TPS, and JVM parameters.

Pinpoint architecture

Pinpoint agent will be deployed on the program server, and the agent service will be specified when the program starts. The pinpoint collector acquires the link monitoring between the micro services, and then stores it in the Hbase database, and pinpoint web ui displays the monitoring data.

3. Deploy pinpoint monitoring components using docker

Pinpoint is hosted on github. Pulling images is very slow and can be obtained from Baidu's network disk.

Pinpoint also requires a version of docker-compose, preferably above version 1.25.

1. Download docker-compose# curl-L https://github.com/docker/compose/releases/download/1.25.0/docker-compose-Linux-x86_64-o / usr/local/bin/docker-compose # chmod astatx / usr/local/bin/docker-compose 2. Download the pinpoint docker deployment installation package # wget https://codeload.github.com/pinpoint-apm/pinpoint-docker/zip/refs/tags/1.8.53. Deploy pinpoint# unzip pinpoint-docker-1.8.5.zip# docker-compose pull & & docker-compose up-d4. If the network speed is not available, deploy it by downloading # import pinpoint image [root@binary-k8s-node2 ~] # cd pinpoint-image/ [root@binary-k8s-node2 pinpoint-image] # for i in `ls`; do docker load-I $I Done# deployment pinpoint [root@binary-k8s-node2 ~] # unzip pinpoint-docker-1.8.5.zip > / dev/null [root@binary-k8s-node2 ~] # cd pinpoint-docker-1.8.5/ [root@binary-k8s-node2 pinpoint-docker-1.8.5] # docker-compose up-dCreating pinpoint-docker-185_zoo2_1... DoneCreating pinpoint-flink-jobmanager... DoneCreating pinpoint-docker-185_zoo1_1... DoneCreating pinpoint-hbase... DoneCreating pinpoint-mysql... DoneCreating pinpoint-docker-185_zoo3_1... DoneCreating pinpoint-flink-taskmanager... DoneCreating pinpoint-collector... DoneCreating pinpoint-web... DoneCreating pinpoint-agent... DoneCreating pinpoint-quickstart... Done [root@binary-k8s-node2 pinpoint-docker-1.8.5] # docker-compose ps

The pinpoint-web container provides a page display. Port 8080 of the container has been mapped to port 8039. You can access port 8079 directly.

Pinpoint comes with a presentation template for quickapp.

4. The Tomcat access method for integrating pinpoint-agent4.1.pinpoint-agent into micro services requires modifying the bin/catalina.sh script CATALINA_OPTS= "$CATALINA_OPTS-javaagent:" $AGENT_PATH/pinpoint-bootstrap-$VERSION.jar "CATALINA_OPTS=" $CATALINA_OPTS-Dpinpoint.agentId= "$AGENT_ID" CATALINA_OPTS= "$CATALINA_OPTS-Dpinpoint.applicationName=" $APPLICATION_NAME "java-jar startup program as follows: directly added to the startup command, namely But the jar package of the java-jar-javaagent:$AGENT_PATH/pinpoint-bootstrap-$VERSION.jar-Dpinpoint.agentId=$AGENT_ID-Dpinpoint.applicationName=$APPLICATION_NAME xxx.jar-javaagent:$AGENT_PATH/pinpoint-bootstrap-$VERSION.jar # agent program-the Dpinpoint.agentId=$AGENT_ID # program counts as an id number of the pinpoint Each microservice replica node requires a unique-Dpinpoint.applicationName=$APPLICATION_NAME # application group name. For the same microservice, their application group name should remain the same.

Download the pinpoint-agent program

Click Settings-> installation

Https://github.com/pinpoint-apm/pinpoint/releases/download/1.8.5/pinpoint-agent-1.8.5.tar.gz

4.2. Configure pinpoint-agent

Configure pinpoint-agent, and then copy the pinpoint-agent directory to the program code directory of each microservice.

1. Download pinpoint-agent [root@binary-k8s-master1 ~] # wget https://github.com/pinpoint-apm/pinpoint/releases/download/1.8.5/pinpoint-agent-1.8.5.tar.gz2. Configure pinpoint [root@binary-k8s-master1 ~] # vim pinpoint-agent/pinpoint.config profiler.collector.ip=192.168.20.13 # to specify the IP address of the pinpoint collector component 3. Copy pinpoint to the code directory of each microservice [root@binary-k8s-master1 ~] # cp-rp pinpoint-agent springcloud/simple-microservice-dev1/ {eureka-service,gateway-service Portal-service} [root@binary-k8s-master1] # cp-rp pinpoint-agent springcloud/simple-microservice-dev1/product-service/product-service-biz/ [root@binary-k8s-master1] # cp-rp pinp springcloud/simple-microservice-dev1/stock-service/stock-service-biz/ [root@binary-k8s-master1] # cp-rp pinpoint-agent springcloud/simple-microservice-dev1/stock-service/stock-service-biz/4.3. Modify the Dockerfile access pinpoint-agent of each microservice program

Modify the Dockerfile of each micro-service program, copy the program directory of pinpoint to the container, and then add the pinpoint-agent parameter to the startup command of the program to access pinpoint, where the value of-Dpinpoint.agentId field is defined by the name of pod, and the value of Dpinpoint.applicationName field is defined by simple- service name.

Eureke-service service

[root@binary-k8s-master1 simple-microservice-dev1] # vim eureka-service/Dockerfile FROM 192.168.20.11/base/centos-java:v1RUN mkdir / data/simple-microservice/eureka-service-pCOPY. / target/eureka-service.jar / data/simple-microservice/eureka-service/COPY pinpoint-agent/pinpoint- agentRUN echo-e "#! / bin/bash\ njava-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=\ ${HOSTNAME} -Dpinpoint.applicationName=simple-eureka-Deureka.instance.hostname=$ {MY_POD_NAME} .eureka-service.simple-ms / data/simple-microservice/eureka-service/eureka-service.jar > / data/simple-microservice/eureka-service/eureka-service.log &\ ntail-f / data/simple-microservice/eureka-service/eureka-service.log "> / data/entrypoint.sh & & chmod axix / data/entrypoint.shEXPOSE 8080ENTRYPOINT / data/entrypoint.sh# # # java-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=$ {HOSTNAME}-Dpinpoint.applicationName=simple-eureka-Deureka.instance.hostname=$ {MY_POD_NAME}. Eureka-service.simple-ms / data/simple-microservice/eureka-service/eureka-service.jar

Gateway-service service

[root@binary-k8s-master1 simple-microservice-dev1] # vim gateway-service/Dockerfile FROM 192.168.20.11/base/centos-java:v1RUN mkdir / data/simple-microservice/gateway-service-pCOPY. / target/gateway-service.jar / data/simple-microservice/gateway-service/COPY pinpoint-agent/pinpoint- agentRUN echo-e "#! / bin/bash\ njava-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=\ ${HOSTNAME} -Dpinpoint.applicationName=simple-gateway / data/simple-microservice/gateway-service/gateway-service.jar > / data/simple-microservice/gateway-service/gateway-service.log &\ ntail-f / data/simple-microservice/gateway-service/gateway-service.log "> / data/entrypoint.sh & & chmod axix / data/entrypoint.shEXPOSE 8080ENTRYPOINT / data/entrypoint.sh## # java-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=$ {HOSTNAME}-Dpinpoint.applicationName=simple-gateway / data/simple-microservice/gateway-service/gateway-service.jar

Order-service service

[root@binary-k8s-master1 simple-microservice-dev1] # vim order-service/order-service-biz/Dockerfile FROM 192.168.20.11/base/centos-java:v1RUN mkdir / data/simple-microservice/order-service-pCOPY. / target/order-service-biz.jar / data/simple-microservice/order-service/COPY pinpoint-agent/pinpoint- agentRUN echo-e "#! / bin/bash\ njava-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar- Dpinpoint.agentId=\ ${HOSTNAME}-Dpinpoint.applicationName=simple-order / data/simple-microservice/order-service/order-service-biz.jar > / data/simple-microservice/order-service/order-service-biz.log &\ ntail-f / data/simple-microservice/order-service/order-service-biz.log "> / data/entrypoint.sh & & chmod axix / data/entrypoint.shEXPOSE 8080ENTRYPOINT / data/entrypoint.sh# # # java-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=$ {HOSTNAME}-Dpinpoint.applicationName=simple-order / data/simple-microservice/order-service/order-service-biz.jar

Portal-service service

[root@binary-k8s-master1 simple-microservice-dev1] # vim portal-service/Dockerfile FROM 192.168.20.11/base/centos-java:v1RUN mkdir / data/simple-microservice/portal-service-pCOPY. / target/portal-service.jar / data/simple-microservice/portal-service/COPY pinpoint-agent/pinpoint- agentRUN echo-e "#! / bin/bash\ njava-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=\ ${HOSTNAME} -Dpinpoint.applicationName=simple-portal / data/simple-microservice/portal-service/portal-service.jar > / data/simple-microservice/portal-service/portal-service.log &\ ntail-f / data/simple-microservice/portal-service/portal-service.log "> / data/entrypoint.sh & & chmod axix / data/entrypoint.shEXPOSE 8080ENTRYPOINT / data/entrypoint.sh## # java-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=$ {HOSTNAME}-Dpinpoint.applicationName=simple-protal / data/simple-microservice/portal-service/portal-service.jar

Product-service service

[root@binary-k8s-master1 simple-microservice-dev1] # vim product-service/product-service-biz/Dockerfile FROM 192.168.20.11/base/centos-java:v1RUN mkdir / data/simple-microservice/product-service-pCOPY. / target/product-service-biz.jar / data/simple-microservice/product-service/COPY pinpoint-agent/pinpoint- agentRUN echo-e "#! / bin/bash\ njava-jar-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5. Jar-Dpinpoint.agentId=\ ${HOSTNAME}-Dpinpoint.applicationName=simple-product / data/simple-microservice/product-service/product-service-biz.jar > / data/simple-microservice/product-service/product-service-biz.log &\ ntail-f / data/simple-microservice/product-service/product-service-biz.log "> / data/entrypoint.sh & & chmod axix / data/entrypoint.shEXPOSE 8080ENTRYPOINT / data/entrypoint.sh# # # java-jar-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar-Dpinpoint.agentId=$ {HOSTNAME}-Dpinpoint.applicationName=simple-product / data/simple-microservice/product-service/product-service-biz.jar

Stock-service service

[root@binary-k8s-master1 simple-microservice-dev1] # vim stock-service/stock-service-biz/Dockerfile FROM 192.168.20.11/base/centos-java:v1RUN mkdir / data/simple-microservice/stock-service-pCOPY. / target/stock-service-biz.jar / data/simple-microservice/stock-service/COPY pinpoint-agent/pinpoint- agentRUN echo-e "#! / bin/bash\ njava-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5.jar- Dpinpoint.agentId=\ ${HOSTNAME}-Dpinpoint.applicationName=simple-stock / data/simple-microservice/stock-service/stock-service-biz.jar > / data/simple-microservice/stock-service/stock-service-biz.log &\ ntail-f / data/simple-microservice/stock-service/stock-service-biz.log "> / data/entrypoint.sh & & chmod axix / data/entrypoint.shEXPOSE 8080ENTRYPOINT / data/entrypoint.shjava-jar-javaagent:/pinpoint-agent/pinpoint-bootstrap-1.8.5 .jar-Dpinpoint.agentId=$ {HOSTNAME}-Dpinpoint.applicationName=simple-stock / data/simple-microservice/stock-service/stock-service-biz.jar4.4. First connect product goods and services to pinpoint to observe the effect.

The Dockerfile for our access to pinpoint has been written, so you can redeploy the produce directly using a script.

Sh deploy_k8s.sh product-service [root@binary-k8s-master1 k8s] # kubectl get pod-n simple-msNAME READY STATUS RESTARTS AGEeureka-0 1 Running 1 6d17heureka-1 1 6d17heureka-2 1 Running 0 6d17heureka-2 1 6d17hgateway-7bb79fdbc4 1 Running 1 -vvnct 1 to 1 Running 0 6d11horder-6694f4c474-rg7kw 1 to 1 Running 0 6d17hportal-697fdf95dd-ml86k 1 to 1 Running 0 6d16hproduct-847bb6d9fc-cs692 1 to 6d16hproduct-847bb6d9fc-cs692 1 Running 0 10m # to redeploy stock-6cf98bb445-l8gxb 1 to Running 0 6d19h

After the product is deployed, purchase a commodity on the portal portal, generate a request for product merchandise micro-service, and observe the link status on the pinpoint.

The product inventory micro service is called after the order is made for the stock product micro service, which has not yet been connected to the pinpoint, so when you watch the link monitoring on the pinpoint, you will find that the stock will only be displayed in the form of ip.

You can see how many requests are invoked between services and how they are invoked in the link monitoring view.

4.5. Connect all the micro services to the pinpoint system

Dockerfile has been adjusted, just execute the deployment script we wrote, create the latest image of all the microservices according to the latest Dockerfile, and then redeploy in the K8S cluster.

The deploy_k8s script does not follow any location parameters to indicate that all microservices are updated and deployed.

[root@binary-k8s-master1 k8s] # sh deploy_k8s.sh

Purchase two items in the portal to observe the calls between microservices.

5.pinpoint monitoring system is easy to use

On the right side of the monitoring view, you can select different micro-service replica nodes, calls between programs, and the number of successful requests.

5.1. See how other microservices are called between program code classes

1) Click view servers to switch each replica node randomly.

2) Click the top statistical chart on the right, and then find the dot in the diagram. Selecting this dot will take you to the page called by the code.

3) jump immediately after the selection, and then click the request

Request for successful monitoring content

Monitoring content that failed to request

4) there will be richer views here than before.

5) Click mixed view to see a richer display view

The request is successful

Request failed

5.2. View the JVM monitoring information of the microservice

1) Click inspector

2) Select the specified replica node

These names take effect by adding the-Dpinpoint.agentId=$ {HOSTNAME} parameter to the startup command.

The above is all the contents of this article entitled "how to solve the problem of full-link monitoring of Spring Cloud micro-service project based on Pinpoint". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.