Shulou(Shulou.com)06/02 Report--

This article focuses on "how to use Apache Kafka and SigNoz to achieve application observability", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to use Apache Kafka and SigNoz to achieve application observability.

SigNoz helps developers achieve their observability goals quickly with minimal effort.

SigNoz is an open source platform for application observability. Written in React and Go, SigNoz is designed from beginning to end to enable developers to achieve their observability goals as soon as possible with minimal effort.

This article describes the software in detail, including architecture, Kubernetes-based deployment, and some common SigNoz uses.

SigNoz architecture

SigNoz bundles several components together to create an extensible, loosely coupled system that is easy to use. Some of the most important components are:

OpenTelemetry Collector

Apache Kafka

Apache Druid

OpenTelemetry Collector is the tracking or metrics data collection engine. This enables SigNoz to obtain data in industry standard formats, including Jaeger, Zipkin, and OpenConsensus. The collected data is then forwarded to Apache Kafka.

SigNoz uses Kafka and stream processors to obtain large amounts of observable data in real time. The data is then passed to Apache Druid, which is good at storing the data for short-term and long-term SQL analysis.

When the data is flattened and stored in Druid, SigNoz's query service can query and pass the data to the SigNoz React front end. The front end then creates beautiful charts for the user to visualize the observability data.

SigNoz architecture

Install SigNoz

The components of SigNoz include Apache Kafka and Druid. These components are loosely coupled and work together to ensure a seamless experience for end users. Given these components, it is best to run SigNoz as a combination of micro services on Kubernetes or Docker Compose (for local testing).

This example uses a Kubernetes Helm Chart-based deployment to install SigNoz on Kubernetes. As a prerequisite, you need a Kubernetes cluster. If you don't have a Kubernetes cluster available, you can use tools such as MiniKube or Kind to create a test cluster on your local machine. Note that this machine must have at least the free space of 4GB to work.

When you have an available cluster and configure kubectl to communicate with the cluster, run:

$git clone https://github.com/SigNoz/signoz.git & & cd signoz$ helm dependency update deploy/kubernetes/platform$ kubectl create ns platform$ helm-n platform install signoz deploy/kubernetes/platform$ kubectl-n platform apply-Rf deploy/kubernetes/jobs$ kubectl-n platform apply-f deploy/kubernetes/otel-collector

This will install SigNoz and related containers on the cluster. To access the user interface (UI), run the kubectl port-forward command. For example:

$kubectl-n platform port-forward svc/signoz-frontend 3000UM3000

You should now be able to use your local browser to access your SigNoz dashboard at http://localhost:3000.

Now that your observability platform has been established, you need an application that generates observable data for visualization and tracking. For this example, you can use HotROD, a sample application developed by the Jaegar team.

To install it, run:

$kubectl create ns sample-application$ kubectl-n sample-application apply-Rf sample-apps/hotrod/ exploration function

You should now have an application that has the right instrument installed and can be run in the demo settings. Look at the metrics and tracking data on the SigNoz dashboard. When you log in to the home page of the dashboard, you will see a list of all configured applications that are sending meter data to SigNoz.

SigNoz dashboard

Index

When you click on a specific application, you will log in to the app's home page. The metrics page displays information for the last 15 minutes (this number is configurable), such as application latency, average throughput, error rate, and the interface to which the application currently accesses the highest. This gives you a general idea of the status of the application. Any spikes in errors, delays, or loads can be seen immediately.

Metrics in SigNoz

Tracking

The tracking page lists the high-level details of each request in chronological order. When you find a request of interest (for example, something that takes longer than expected), you can click track to see a separate time span for each action that occurs in the request. The drilldown mode provides a thorough examination of each request.

Tracing in SigNoz

Tracing in SigNoz

Usage Resource Manager

Most metrics and tracking data are very useful, but only for a certain period of time. Over time, the data is no longer useful in most cases. This means that it is important to plan an appropriate retention time for the data. Otherwise, you will pay more for storage. The usage Resource Manager provides an overview of getting data on an hourly, daily, and weekly basis.

Add instrument

So far, you've been looking at the metrics and tracking of HotROD apps. Ideally, you will want to test your application so that it sends observable data to SigNoz. Refer to the meter overview on the SigNoz website.

SigNoz supports a vendor-independent instrument library, OpenTelemetry, as the main way to configure instruments. OpenTelemetry provides instrument libraries in various languages and supports both automatic and manual instruments.

At this point, I believe you have a deeper understanding of "how to use Apache Kafka and SigNoz to achieve application observability". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.