Shulou(Shulou.com)06/02 Report--

How to deploy Pulsar to Kubernetes, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

prerequisite

Kubernetes server 1.14.0 +

Kubectl 1.14.0 +

Helm 3.0 +

Tip

For the next steps, steps 2 and 3 apply to developers, and steps 4 and 5 apply to administrators.

Step 0: prepare a Kubernetes cluster

Before installing Pulsar Helm chart, you must create a Kubernetes cluster. You can follow the instructions to prepare a Kubernetes cluster.

We use Minikube in this quick start guide. To prepare a Kubernetes cluster, follow these steps:

Create a Kubernetes cluster on Minikube.

Minikube start-memory=8192-cpus=4-kubernetes-version=

It can be any Kubernetes version supported by the Minikube installer, such as v1.16.1.

Set up kubectl to use Minikube.

Kubectl config use-context minikube

To use Kubernetes Dashboard with the local Kubernetes cluster on Minikube, enter the following command:

Minikube dashboard

This command will automatically open a web page with your browser.

Step 1: install Pulsar Helm chart

Add a Pulsar Helm repository.

Helm repo add apache https://pulsar.apache.org/charts

Helm repo update

Clone Pulsar Helm chart's warehouse.

Git clone https://github.com/apache/pulsar-helm-chartcd pulsar-helm-chart

Run the script prepare_helm_release.sh to create the secrets required to install Apache Pulsar Helm chart. The username pulsar and password pulsar can be used to log in to Grafana dashboard and Pulsar Manager.

. / scripts/pulsar/prepare_helm_release.sh\-n pulsar\-k pulsar-mini\-c

Use Pulsar Helm chart to install a Pulsar cluster to Kubernetes.

NOTE

You need to specify-- set initialize=true when installing Pulsar the first time. This command installs and starts Apache Pulsar.

Helm install\-values examples/values-minikube.yaml\-set initialize=true\-namespace pulsar\ pulsar-mini apache/pulsar

Check the status of all pod.

Kubectl get pods-n pulsar

If all pod starts successfully, you will see that STATUS becomes Running or Completed.

Output

NAME READY STATUS RESTARTS AGEpulsar-mini-bookie-0 1/1 Running 0 9m27spulsar-mini-bookie-init-5gphs 0/1 Completed 0 9m27spulsar-mini-broker-0 1/1 Running 0 9m27spulsar-mini-grafana-6b7bcc64c7-4tkxd 1 + 1 Running 0 9m27spulsar-mini-prometheus-5fcf5dd84c-w8mgz 1 + + 1 Running 0 9m27spulsar-mini-proxy-0 1 + 1 Running 0 + 9m27spulsar-mini-pulsar-init-t7cqt 0 + + 1 Completed 0 9m27 spulsarmuri * Htpcs 1/1 Running 0 9m27spulsar-mini-toolset-0 1/1 Running 0 9m27spulsar-mini-zookeeper-0 1/1 Running 0 9m27s

Check the status of all services under the namespace pulsar.

Kubectl get services-n pulsar

Output

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGEpulsar-mini-bookie ClusterIP None 3181 AGEpulsar-mini-bookie ClusterIP None TCP 11mpulsar-mini-broker ClusterIP None 8080/TCP 6650/TCP 11mpulsar-mini-grafana LoadBalancer 10.106.141.246 3000:31905/TCP 11mpulsar-mini-prometheus ClusterIP None 9090/TCP 11mpulsar-mini-proxy LoadBalancer 10.97.240.109 80:32305/TCP 6650:31816/TCP 11mpulsar-mini-pulsar-manager LoadBalancer 10.103.192.175 9527:30190/TCP 11mpulsar-mini-toolset ClusterIP None 11mpulsar-mini-zookeeper ClusterIP None 2888/TCP,3888/TCP,2181/TCP 11m

Step 2: use pulsar-admin to create Pulsar tenants / namespaces / themes

Pulsar-admin is the CLI (command line client) tool of Pulsar. In this step, you can use pulsar-admin to create resources, including tenants, namespaces, and themes.

Enter the toolset container.

Kubectl exec-it-n pulsar pulsar-mini-toolset-0-/ bin/bash

In the toolset container, create a tenant named apache.

Bin/pulsar-admin tenants create apache

Then you can list all the tenants to see if the tenant has been created successfully.

Bin/pulsar-admin tenants list

You should see output similar to the following: the tenant apache has been successfully created.

"apache"public"pulsar"

In the toolset container, create a namespace called pulsar under tenant apache.

Bin/pulsar-admin namespaces create apache/pulsar

You can then list all the namespaces of the tenant apache to see if the namespace has been created successfully.

Bin/pulsar-admin namespaces list apache

You should see output similar to the following: the namespace apache/pulsar has been successfully created.

"apache/pulsar"

In the toolset container, create a theme test-topic with four partitions under the namespace apache/pulsar.

Bin/pulsar-admin topics create-partitioned-topic apache/pulsar/test-topic-p 4

In the toolset container, list all partition topics in the namespace apache/pulsar.

Bin/pulsar-admin topics list-partitioned-topics apache/pulsar

Then you can see all the partition themes under the namespace apache/pulsar.

"persistent://apache/pulsar/test-topic"

Step 3: use the Pulsar client to produce and consume messages

You can use the Pulsar client to create producers and consumers and to produce and consume messages.

By default, Pulsar Helm chart exposes the Pulsar cluster through Kubernetes LoadBalancer. In Minikube, you can check the proxy service using the following command.

Kubectl get services-n pulsar | grep pulsar-mini-proxy

You will see output similar to the following:

Pulsar-mini-proxy LoadBalancer 10.97.240.109 80:32305/TCP,6650:31816/TCP 28m

This output shows the node port to which both the binary port and HTTP port of the Pulsar cluster are mapped. 80: the back port is the HTTP port, while the 6650: the back port is the binary port.

Then you can find the IP address and expose your Minikube server by running the following command.

Minikube service pulsar-mini-proxy-n pulsar

Output

|-| | NAMESPACE | NAME | TARGET PORT | URL | |-|- -| | pulsar | pulsar-mini-proxy | http/80 | http://172.17.0.4:32305 | | pulsar/6650 | http://172.17.0.4:31816 | |- -|? The starting channel for the pulsar-mini-proxy service. | |-| NAMESPACE | NAME | TARGET PORT | URL | |-|-| -| |-|-- | | pulsar | pulsar-mini-proxy | | http://127.0.0.1:61853 | | http://127.0.0.1:61854 | |- -|-|

At this point, you can get the service URL that connects to your Pulsar client. Here are some examples of URL:

WebServiceUrl= http://127.0.0.1:61853/brokerServiceUrl=pulsar://127.0.0.1:61854/

Then you can continue with the following steps:

Download Apache Pulsar tarball from the download page.

Extract tarball based on your download file.

Tar-xf .tar.gz

Expose PULSAR_HOME.

(1) enter the directory of the unzipped download file.

(2) PULSAR_HOME was exposed as an environmental variable.

````bash export PULSAR_HOME=$ (pwd) ```

Configure the Pulsar client.

In the ${PULSAR_HOME} / conf/client.conf file, replace webServiceUrl and brokerServiceUrl with the service URL you got from the above step.

Create subscriptions to consume messages from apache/pulsar/test-tope.

Bin/pulsar-client consume-s sub apache/pulsar/test-topic-n 0

Open a new terminal. In the new terminal, create a producer and send 10 messages to the test-topic topic.

Bin/pulsar-client produce apache/pulsar/test-topic-m "- hello apache pulsar-"-n 10

Verify the results.

For producers

Output

The message has been successfully produced.

18 messages successfully produced 15 messages successfully produced 15.489 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool-10

For consumers,

Output

At the same time, you can receive the following message.

-got message-hello apache pulsar- got message-hello apache pulsar -got message-hello apache pulsar- got message-hello apache Pulsar- got message-hello apache pulsar- got message-hello apache pulsar-

Step 4: use Pulsar Manager to manage the cluster

Pulsar Manager is a management tool based on network GUI, which is used to manage and monitor Pulsar.

By default, Pulsar Manager is exposed to the outside world as a separate LoadBalancer. You can open the Pulsar Manager interface using the following command:

Minikube service-n pulsar pulsar-mini-pulsar-manager

Pulsar Manager UI will open in your browser. You can log in to Pulsar Manager using the username pulsar and password pulsar.

In Pulsar Manager UI, you can create an environment.

Click the New Environment button in the upper left corner.

Enter pulsar-mini in the Environment Name column in the pop-up window.

Enter http://pulsar-mini-broker:8080 in the Service URL column in the pop-up window.

Click the Confirm button in the pop-up window.

After successfully creating the environment, you will be redirected to the tenant page of the new environment. You can then use Pulsar Manager to create tenants, namespaces, and themes.

Step 5: use Prometheus and Grafana to monitor the cluster

Grafana is an open source visualization tool for visually visualizing time series data into dashboards.

By default, Grafana is exposed to the outside world as a separate LoadBalancer. You can open the Grafana interface using the following command:

Minikube service pulsar-mini-grafana-n pulsar

Grafana UI will open in your browser. You can log in to the Grafana dashboard with the username pulsar and password pulsar.

You can view the dashboards of different components of the Pulsar cluster.

This is the answer to the question about how to deploy Pulsar to Kubernetes. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.