Shulou(Shulou.com)05/31 Report--

This article mainly introduces the use of RocketMQ Operator, a sharp weapon controlled by RocketMQ operations and maintenance in the original cloud era, which can be used for reference. Interested friends can refer to it. I hope you can learn a lot after reading this article.

1. RocketMQ

During 2012-2013, Alibaba middleware team independently developed and opened up the third generation distributed messaging engine RocketMQ. Its high performance, low latency and anti-stacking characteristics steadily supported Alibaba's double 11 trillion data flood peak business, and its cloud product Aliware MQ shines in numerous operating conditions such as micro service, flow computing, IoT, asynchronous decoupling, data synchronization and so on.

In 2016, Alibaba donated RocketMQ to the Apache Software Foundation. The following year, RocketMQ successfully graduated from the foundation and became a top open source project for Apache, bringing good news to developers in the globally distributed and big data field together with Apache Hadoop,Apache Spark. However, in today's cloud native era, RocketMQ, as a stateful distributed service system, how to achieve extremely simple operation and maintenance in large-scale clusters is a very challenging and valuable problem.

RocketMQ supports multiple deployment methods, taking the basic dual-master and dual-slave architecture as an example, as shown in the following figure.

RocketMQ dual master and double slave architecture

This includes a total of 7 RocketMQ service instances: 3 name server instances, 2 master broker instances, and 2 slave broker instances.

Traditional deployment requires manual or scripting to configure the environment and files on each node. In addition, with the increase of user services, there is a need for seamless expansion of the cluster. The traditional way is that the operation and maintenance personnel visit different nodes and rely on the operation manual and script to complete the operation step by step, which is labor-consuming and there is the possibility of misoperation. Some companies may use platforms and tools such as Ansible to help automate operations and maintenance, and more and more companies are beginning to integrate and use Kubernetes-based cloud native ecology.

The management of stateless applications can be well solved by using native resources such as Deployment and StatefulSet provided by Kubernetes, but there are many limitations for stateful applications such as database and RocketMQ. For example, for RocketMQ, capacity expansion is not only completed by pulling up a new instance Pod, but also needs to synchronously replicate the status information of Broker, including Topic information and subscription relationship metadata, and correctly configure the config parameters of the new Broker, including brokerName and NameServer IP List, to make the new expanded Broker available, which cannot be achieved only by users writing StatefulSet, modifying size or replicas, and then apply.

In fact, Kubernetes developers also found these problems, so they introduced the concept of custom resources and controllers, so that developers can directly use Go language to call Kubernetes API, write custom resources and corresponding controller logic to solve the management problems of complex stateful applications. This kind of code component that provides custom resources related to specific applications is called Operator. Operator is written by experts with RocketMQ domain knowledge, shielding the application domain expertise, so that users only need to care about and define the cluster final state they want to achieve, which is also the design philosophy of Kubernetes declarative API.

2. Kubernetes Operator

Operator is based on Kubernetes by extending Kubernetes API to create, configure and manage complex stateful applications, such as distributed databases. Based on the concept of custom controllers introduced since Kubernetes 1.7, Operator is built on custom resources and controllers, while at the same time containing application-specific domain knowledge. The key to implementing an Operator is the design of CRD (custom resources) and Controller (controller).

Operator stands in the perspective of Kubernetes, opening the door to a new world for the application of Yunyuan Biochemistry. Custom resources allow developers to add new functions, update existing functions, and automatically perform some management tasks. These custom controllers are like native components of Kubernetes, and Operator can be developed directly using Kubernetes API. In other words, they can create and change Pods / Services and scale up running applications according to the custom rules written by these controllers.

Quick start

This article uses RocketMQ Operator 0.2.1 to show how to quickly create and deploy a RocketMQ service cluster on Kubernetes using RocketMQ Operator.

To prepare the K8s environment, you can use the K8s that comes with docker desktop, or minikube

Clone the rocketmq-operator repository to your K8s node

$git clone https://github.com/apache/rocketmq-operator.git$ cd rocketmq-operator

Run the script to install RocketMQ Operator

$. / install-operator.sh

Check whether RocketMQ Operator is installed successfully

$kubectl get podsNAME READY STATUS RESTARTS AGErocketmq-operator-564b5d75d-jllzk 1 move 1 Running 0 108s

When successfully installed, rocketmq-operator pod is in a running state similar to the example above.

Create a RocketMQ cluster using custom resources of Broker and NameService

Use the rocketmq_v1alpha1_rocketmq_cluster.yaml file in rocketmq-operator / example to quickly deploy a RocketMQ cluster. The rocketmq_v1alpha1_rocketmq_cluster.yaml file is as follows:

ApiVersion: rocketmq.apache.org/v1alpha1kind: Brokermetadata: # name of broker cluster name: brokerspec: # size is the number of the broker cluster, each broker cluster contains a master broker and [replicaPerGroup] replica brokers. Size: 1 # nameServers is the [ip:port] list of name service nameServers: "" # replicationMode is the broker replica sync mode Can be ASYNC or SYNC replicationMode: ASYNC # replicaPerGroup is the number of each broker cluster replicaPerGroup: 1 # brokerImage is the customized docker image repo of the RocketMQ broker brokerImage: apacherocketmq/rocketmq-broker:4.5.0-alpine # imagePullPolicy is the image pull policy imagePullPolicy: Always # resources describes the compute resource requirements and limits resources: requests: memory: "2048Mi" cpu: "250m" limits: memory: "12288Mi" cpu: "500m" # allowRestart defines Whether allow pod restart allowRestart: true # storageMode can be EmptyDir HostPath StorageClass storageMode: EmptyDir # hostPath is the local path to store data hostPath: / data/rocketmq/broker # scalePodName is broker- [broker group number]-master-0 scalePodName: broker-0-master-0 # volumeClaimTemplates defines the storageClass volumeClaimTemplates:-metadata: name: broker-storage spec: accessModes:-ReadWriteOnce storageClassName: rocketmq-storage resources: requests: storage: 8Gi---apiVersion Rocketmq.apache.org/v1alpha1kind: NameServicemetadata: name: name-servicespec: # size is the the name service instance number of the name service cluster size: 1 # nameServiceImage is the customized docker image repo of the RocketMQ name service nameServiceImage: apacherocketmq/rocketmq-nameserver:4.5.0-alpine # imagePullPolicy is the image pull policy imagePullPolicy: Always # hostNetwork can be true or false hostNetwork: true # Set DNS policy for the pod. # Defaults to "ClusterFirst". # Valid values are 'ClusterFirstWithHostNet',' ClusterFirst', 'Default' or' None'. # DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy. # To have DNS options set along with hostNetwork, you have to specify DNS policy # explicitly to 'ClusterFirstWithHostNet'. DnsPolicy: ClusterFirstWithHostNet # resources describes the compute resource requirements and limits resources: requests: memory: "512Mi" cpu: "250m" limits: memory: "1024Mi" cpu: "500m" # storageMode can be EmptyDir, HostPath StorageClass storageMode: EmptyDir # hostPath is the local path to store data hostPath: / data/rocketmq/nameserver # volumeClaimTemplates defines the storageClass volumeClaimTemplates:-metadata: name: namesrv-storage spec: accessModes:-ReadWriteOnce storageClassName: rocketmq-storage resources: requests: storage: 1Gi

Note that storageMode: EmptyDir in this example indicates that the storage is using EmptyDir, and the data will be erased as the Pod is deleted, so this method is only used for development testing. HostPath or StorageClass is generally used for persistent storage of data. When using HostPath, you need to configure hostPath to declare the directory mounted on the host. When using storageClass, you need to configure volumeClaimTemplates and declare the PVC template. For details, please refer to the RocketMQ Operator documentation.

Apply the yaml file above and enter the command:

$kubectl apply-f example/rocketmq_v1alpha1_rocketmq_cluster.yamlbroker.rocketmq.apache.org/broker creatednameservice.rocketmq.apache.org/name-service created

Check the cluster Pod status:

$kubectl get pods-owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbroker-0-master-0 1 owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbroker-0-master-0 1 Running 0 27s 10.1.2.27 docker-desktop broker-0-replica-1-0 1 shock 1 Running 0 27s 10.1.2.28 docker-desktop name-service-0 1/1 Running 0 27s 192.168.65.3 docker-desktop rocketmq-operator-76b4b9f4db-x52mz 1/1 Running 0 3h35m 10.1.2.17 docker-desktop

Using the default rocketmq_v1alpha1_rocketmq_cluster.yaml file configuration, we see one name server service (name-service-0) and two broker services (one master and one slave) pulled up in the cluster.

All right! At this point, you have successfully deployed a RocketMQ service cluster through the custom resources provided by Operator.

Access the Pod in this RocketMQ cluster to verify that the cluster is working properly

Run Producer example using RocketMQ's tools.sh script:

$kubectl exec-it broker-0-master-0 bashbash-4.4# sh. / tools.sh org.apache.rocketmq.example.quickstart.ProducerOpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m Support was removed in 8.006 sendStatus=SEND_OK 56 queueId=0 29.145 [main] DEBUG i.n.u.i.l.InternalLoggerFactory-Using SLF4J as the default logging frameworkSendResult [sendStatus=SEND_OK, msgId=0A0102CF007778308DB1206383920000, offsetMsgId=0A0102CF00002A9F0000000000000000, messageQueue=MessageQueue [topic=TopicTest, brokerName=broker-0, queueId=0], queueOffset=0]... 06V 51.120 [NettyClientSelector_1] INFO RocketmqRemoting-closeChannel: close the connection to remote address [10.1.2.207 Using SLF4J as the default logging frameworkSendResult] result: truebash-4.4#

Run Consumer example on another node:

$kubectl exec-it name-service-0 bashbash-4.4# sh. / tools.sh org.apache.rocketmq.example.quickstart.ConsumerOpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m Support was removed in 8.007 DEBUG i.n.u.i.l.InternalLoggerFactory-Using SLF4J as the default logging frameworkConsumer Started.ConsumeMessageThread_1 Receive New Messages: [MessageExt [queueId=0, storeSize=273, queueOffset=19845, sysFlag=0, bornTimestamp=1596768410268, bornHost=/30.4.165.204:53450, storeTimestamp=1596768410282, storeHost=/100.81.180.84:10911, msgId=6451B45400002A9F000014F96A0D6C65, commitLogOffset=23061458676837, bodyCRC=532471758, reconsumeTimes=0, preparedTransactionOffset=0, toString () = Message {topic='TopicTest', flag=0, properties= {MIN_OFFSET=19844, TRACE_ON=true, eagleTraceId=1e04a5cc15967684102641001d0db0, MAX_OFFSET=19848, MSG_REGION=DefaultRegion CONSUME_START_TIME=1596783715858, UNIQ_KEY=1E04A5CC0DB0135FBAA421365A5F0000, WAIT=true, TAGS=TagA, eagleRpcId=9.1}, body= [72,101,108,108,111,32, 77, 101,116,97,81,32,48], transactionId='null'}]] ConsumeMessageThread_4 Receive New Messages: [MessageExt [queueId=1, storeSize=273, queueOffset=19637, sysFlag=0, bornTimestamp=1596768410296, bornHost=/30.4.165.204:53450, storeTimestamp=1596768410298, storeHost=/100.81.180.84:10911, msgId=6451B45400002A9F000014F96A0D7141, commitLogOffset=23061458678081, bodyCRC=1757146968, reconsumeTimes=0, preparedTransactionOffset=0, toString () = Message {topic='TopicTest', flag=0, properties= {MIN_OFFSET=19636 TRACE_ON=true, eagleTraceId=1e04a5cc15967684102961002d0db0, MAX_OFFSET=19638, MSG_REGION=DefaultRegion, CONSUME_START_TIME=1596783715858, UNIQ_KEY=1E04A5CC0DB0135FBAA421365AB80001, WAIT=true, TAGS=TagA, eagleRpcId=9.1}, body= [72, 101, 108, 108, 111, 32, 77, 101, 116, 97, 81, 32, 49], transactionId='null'}]]...

Delete the cluster and clean up the environment

Clear the RocketMQ service cluster instance:

$kubectl delete-f example/rocketmq_v1alpha1_rocketmq_cluster.yaml

Clear RocketMQ Operator:

$. / purge-operator.sh install RocketMQ Operator according to the instructions of OperatorHub official website

Search for RocketMQ Operator on the OperatorHub.io page

Select the Streaming & Messaging category and click RocketMQ Operator:

Go to the RocketMQ Operator page and click the Install button

Follow the instructions to install OLM and RocketMQ Operator

Install OLM locally to use RocketMQ Operator

Install and move OLM (Operator Lifecycle Manager) console locally

Reference: OLM installation documentation.

Launch the UI interface console locally

$make run-console-local

Visit the http://localhost:9000 View console

OperatorHub

Search RocketMQ or click Streaming & Messaging in the All Items category to find RocketMQ Operator and install it

After installing RocketMQ Operator, you can find RocketMQ Operator in Installed Operators.

Installed Operators interface

RocketMQ Operator introduction interface

Create NameService custom resources through the UI interface

You can create NameService and Broker instances under the specified Namespace in UI, and browse and manage the created instances. We can also check the Pod status in the current K8s cluster with the command, for example:

$kubectl get pods-ANAMESPACE NAME READY STATUS RESTARTS AGEdocker compose-78f95d4f8c-8fr5z 1 Running 1 Running 0 32hdocker compose-api-6ffb89dc58-nv9rh 1 32hkube-system coredns-5644d7b6d9-hv6r5 1/1 Running 0 32hkube-system coredns-5644d7b6d9-mkqb6 1/1 Running 0 32hkube-system etcd-docker-desktop 1/1 Running 0 32hkube-system kube-apiserver-docker-desktop 1/1 Running 0 32hkube-system kube-controller-manager -docker-desktop 1 to 1 Running 1 32hkube-system kube-proxy-snmxh 1 to 1 Running 0 32hkube-system kube-scheduler-docker-desktop 1 to 1 Running 1 32hkube-system storage-provisioner 1 to Running 1 32hkube-system vpnkit-controller 1 Running 0 32hmarketplace broker-0-master-0 1 5h4mmarketplace broker-0-replica-1 1 Running 0 5h4mmarketplace broker-0-replica-1-0 1 Running 0 5h4mmarketplace name-service-0 1 Running 0 5h4mmarketplace marketplace-operator-69756457d8-42chk 1 42chk 1 Running 0 32hmarketplace rocketmq-operator-0.2.1-c9fffb5f-cztcl 1 Running 0 32hmarketplace rocketmq-operator-84c7bb4ddc-7rvqr 1 Running 0 32hmarketplace upstream-community-operators-5b79db455f-7t47w 1 / 1 Running 1 32holm catalog-operator-7b788c597d-gjz55 1/1 Running 0 32holm olm-operator-946bd977f-dhszg 1/1 Running 0 32holm operatorhubio-catalog-fvxp9 1/1 Running 0 32holm packageserver-789c7b448b-7ss7m 1/1 Running 0 32holm packageserver-789c7b448b-lfxrw 1/1 Running 0 32h

You can see that the corresponding name server and broker instances are also successfully created in marketplace, the namespace.

The above cases are based on the use of RocketMQ Operator in OperatorHub and OLM installations. We will continue to push and maintain new versions of RocketMQ Operator to this platform to facilitate users to get the latest updates or select the appropriate Operator version.

Thank you for reading this article carefully. I hope the article "what is the use of RocketMQ Operator, a sharp weapon of RocketMQ operation and maintenance control in the cloud original era" shared by the editor, will be helpful to everyone. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.