Shulou(Shulou.com)06/01 Report--

This article introduces you how to use Netsil to monitor microservices on Kubernetes. The content is very detailed. Interested friends can refer to it and hope to help you.

Kubernetes is the king of container orchestration and scheduling, beating competitors Docker Swarm and Apache Mesos to open up a shining future where microservices are self-healing, can scale automatically, and can federate across zones, regions, and even cloud vendors. In this new era of cloud-native applications, it's increasingly important to be able to see how services interact in a simple way--as opposed to looking for a needle in a haystack for a specific cause of performance problems.

We spent some time researching Netsil and packaging its solution as a native Kubernetes Deployment. Netsil's application, Application Operations Center (AOC), helps users observe and collect analytics data for microservices applications running across Kubernetes clusters. The service itself is agnostic because it is on the network that determines how it actually works. Over time, and in real time, it learns and discovers the user's environment, helping the user build SLA indicators, alarms, and so on.

First you need a Kubernetes cluster. I used Stackpoint.io to quickly create a cluster. Create a cluster on any major vendor such as AWS, GCE or Azure. Make sure you choose a large enough configuration for your master node--all collectors will send data here, which can be costly in terms of network, processor, and memory. A worker node can be any configuration that meets the needs of a microservices application. In my example, I used a larger instance configuration because I would push multiple services into this environment.

In our example, a cluster is built using three instances of N1 standard 4, exposed through HAProxy Ingress Controller, which is self-discovering and registers the AOC service when they are deployed. We were able to access the AOC dashboard using the public VIP of the cluster.

before the start

Install some services on the empty cluster running only Kubernetes services, using Sock Shop, a microservices reference program developed by Weaveworks. This helps simulate a real environment. Sock Shop uses 14 different services, which is the complexity that many enterprise applications reach. Now add AOC to our environment.

Here is more information about Sock Shop. Pushing it into the environment is simple, just run the following command after cloning repo:

kubectl apply -f deploy/kubernetes/manifests

Then check if the Pod is online:

$ kubectl get pods --namespace=defaultNAME READY STATUS RESTARTS AGEcart-3694116665-eccpp 1/1 Running 0 55mcart-db-2305146297-u30g8 1/1 Running 0 55mcatalogue-11453786-lkslj 1/1 Running 0 55mcatalogue-db-393939662-bn7uc 1/1 Running 0 55mfront-end-3820830240-01e6t 1/1 Running 0 55morders-3498886496-z8jun 1/1 Running 0 55morders-db-1775353731-u7dmf 1/1 Running 0 55mpayment-3012088042-vbfhw 1/1 Running 0 55mqueue-master-936560853-ocmxi 1/1 Running 0 55mrabbitmq-1897447621-2ij04 1/1 Running 0 55mshipping-1232389217-b278a 1/1 Running 0 55mspc-balancer-biilo 1/1 Running 0 1huser-3090014237-196pv 1/1 Running 0 55muser-db-1338754314-exyou 1/1 Running 0 55m Start watching it

We already have a Kubernetes 1.4 cluster running and the Sock Shop application installed, so let's start learning what's in the environment. Do we know when stock buyers are having problems?

Before deploying AOC, you need to run the following command on all hosts. This command helps avoid a known competition problem between Flannel and kube-proxy.

iptables -t nat -I POSTROUTING -o flannel.1 -s host-private-ip -j MASQUERADE

Replace host-private-ip with the private IP of each host. Once complete, clone AOC Kubernetes repo from GitHub:

git clone https://github.com/netsil/netsil-kube.git

And push it to Kubernetes using a single command:

kubectl apply -f netsil.yml

Make sure Pod and Service are online. The AOC container may take some time, but the collectors will be started and the data in the queue will be pushed in as they begin to discover your environment.

$ kubectl get po,svc - namespace=netsilNAME READY STATUS RESTARTS AGEcollector-7wpaa 1/1 Running 0 1hcollector-9o6k4 1/1 Running 0 1hcollector-rzekv 1/1 Running 0 4mnetsil-vjf5f 1/1 Running 0 1hNAME CLUSTER-IP EXTERNAL-IP PORT(S) AGEnetsil 10.200.126.143 443/TCP,2001/TCP,2003/TCP,2003/UDP 1h

The AOC topology has two main components. The first is a Pod that runs as part of a Replication Controller with a single replica. It runs AOC dashboards and data collection platforms. The second component is DaemonSet for the AOC collector. It tells Kubernetes to run a Pod with collector containers on all nodes of the environment. These collectors are configured to send information to the AOC Pod.

generate traffic

We will use more tools from Sock Shop to simulate shopping behavior on the site. This allows us to see how AOC learns traffic patterns and our general topology.

You need to know the IP address and port of the front end where Sock Shop listens and runs:

docker run weaveworksdemos/load-test -h $frontend-ip[:$port] -r 100 -c 2

As load-test runs, you can start to see AOC lit up as data is acquired:

Because AOC is deployed as a DaemonSet, if any Pod is destroyed and rescheduled elsewhere, AOC can continue to observe topology changes as Kubernetes change.

One of the things I love about AOC is that deployments are organized by services, and I'm able to observe the environment in real time and start digging into different metrics, building service-level alerts for things that might impact customers. So when the environment turns red like the one below, I can get alerts that a service is in an emergency, such as credit cards and address endpoints in the Sock Shop.

I can even go deep into the dashboard and know what the pods and containers that are under the greatest pressure are. In this example, the container with the highest network pressure is the flannel Pod. This allows us to understand which services are the busiest and helps us rethink how we configure or distribute deployments in Kubernetes.

Netsil's AOC is a great tool to help users observe their environment in real time, updating as usage patterns change. Users can mine historical data and add alerts. The application automatically expands as more nodes are added, and when a new node goes online, a collector is launched on it, so that users can get all the data from the node going online to destruction.

How to use Netsil to monitor microservices on Kubernetes is shared here. I hope the above content can be helpful to everyone and learn more. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.