Shulou(Shulou.com)06/02 Report--

First, know RabbitMQ and its use.

Message Queue (MQ)

Relative to some larger companies will use this MQ, such as kafka,rabbitmq,activeMQ,rocketMQ is mainly used for system-to-system, or program-to-program data transmission, then as a message middleware, we can use it as a message proxy server, the most important function is to "knot" the information between systems, MQ is an important guarantee of data reliability.

So before we talk about the role of mq, for example, what problems will we encounter before?

1) for example, now we have two systems, one system An and one system B, then these two systems can understand it as an order system and a background payment system, so between these two systems, you call mine. I call yours, that is, a mutual transfer of data. Well, according to the previous practice, there will be problems with the transmission back and forth. For example, whether there is a problem with either system an or system b, our message will not be sent. Then, in this case, we need to write some additional programs to ensure the successful delivery of our messages, such as increasing the cache and increasing the number of retries, which requires additional coding. This requires a lot of coding work.

2) in addition, in the process of data transfer, the two systems are both developed by java, for example, one is developed by java and the other is developed by python, then there is a lot of trouble in the process of data transfer between the two systems, where the platform is different and the interface for transmission is also different, which virtually brings us a lot of problems.

3) if there is a problem between system an and system b, we don't even know what caused it. For example, the interface of system a has changed, which can not be received by system b, and system b does not know what problems there are. We can't cut through the problem, so it is also an inevitable and catastrophic consequence in the process of online debugging and operation and maintenance.

To sum up, the direct call is called serious coupling, and neither of the two can work properly, so how to solve it? we need mq to solve it at this time.

So mq is called a message proxy server. We can see that the basic architecture diagram of mq above, system an is no longer in direct communication with system b, and a MQ message proxy server is added in the middle. All messages are sent to this message proxy server. Then this message proxy server itself has some characteristics of high availability and high concurrency, so it can be estimated that it is very stable. At least it is much more stable than the system we developed. There is no need to worry about the crash and the service cannot be provided. On this basis, all systems release the message, and then the message server will forward or distribute the message separately according to the data situation.

Well, the purpose of our data transmission, then there is such an advantage in doing so.

First of all, our systems no longer communicate with each other, then it completes the work of saving the lotus root, which is completely oriented to a middleman.

Second, MQ is an international standard. At the bottom, it follows an advanced message queuing protocol called amqp. Well, at this bottom, because it is an international standard, system an and system b, no matter what kind of system is developed, as long as we follow the specification of data transmission, then we can complete the data transmission through mq, which is the protocol of data unification. In addition, there is the process of data completion, so in the process of data completion, you will find that we estimate and think that mq is a stable service. System a transfers data with system B. if system a sends the data to mq, then the task of system an is completed. as for system b to take away our data, that is the matter of system b, then system an is done, so if there is a real problem, such as the operation and maintenance engineer finds that the program has not been executed, then watching system a send successfully, then the remaining problem is associated with mq. Go to system b for processing and tracking, then mq and provide a powerful graphical management page, we can easily track and manage the data sent before. This is also very important.

So when it comes to mq, it has such a good feature. In a word, mq solves the coupling problem of data transfer between this system. Using mq is suitable for almost all of our system calls or distributed projects.

What is RabbitMQ?

There are all kinds of mq in our market, but rabbitmq is the most popular open source messaging proxy server in the world, with more than 35000 projects deployed in rabbitmq around the world, using a large number of people, and solving some related problems very quickly. This rabbit translated into rabbit, this is also a leader of the whole message middleware, that is, we do not have to think about a lot of solutions that we use rabbitmq. Because engineers all over the world have contributed to this rabbitmq, then the latest version of this rabbitmq is 3.8.2.

And rabbitmq supports almost all operating systems and programming languages, provides mature solutions with high concurrency and high availability, supports multiple message protocols, and is easy to deploy and use.

Official address: https://www.rabbitmq.com/

Comparison between Rabbitmq and other MQ

For example, activemq and kafka are both sub-members of apache, while rocketmq is a message middleware promoted by Alibaba and also a sub-project of apache.

From this table, we can see that this rabbitmq is the most widely used. There is no doubt that the problems encountered in general use can be solved in their forums and Baidu. In addition, kafka has gradually replaced activemq due to big data's more applications and more applications. There are generally many applications like rocketmq in China, and they all support saving these data to disk. Concurrent throughput is how much data per second can be processed. In different designs and different backgrounds, its results are completely different. The highest here is kafka, because it is big data's field, and also because high performance has attracted the attention of many big data engineers. Concurrency is also very high in rocketmq, because rocket as large as Singles Day has withstood, and its underlying optimization is also done very well. In the concurrent throughput rabbitmq is slightly inferior, rabbitmq has made great efforts on the issue of data reliability, it ensures that the data here is very reliable, but also very high, in addition, the ecological integrity, do some related around, such as UI or routing distribution, etc., the total number of users is also rabbitmq, followed by kafka, application scenarios, due to high data reliability Rabbitmq is mainly used in highly reliable trading systems, activemq is used in some more traditional messaging, kafka is generally mostly used to deal with big data applications, while rocketmq is mainly used inside Alibaba, such as rocketmq used in Singles Day holiday, which is mainly suitable for high concurrency and high availability of the Internet, but this piece, such as dubbo, has also been opened up in Ali. At that time, it also stopped updating and then restored, like this piece of our community's integrity and open source is extremely disadvantageous. For example, Ali has a distributed database called OceanBase, which is jointly completed by Taobao core system research and development department, operation and maintenance, DBA, advertising, application development and other departments. It is also a secondary development for the bottom layer of the mysql database, but it is used to close the source, so I am worried that Ali will close the source of rabbitmq. In terms of domestic open source software, there is not much that can be done very well now. As to whether the rocketmq will be closed for commercial use or continue to open source in the future, it still belongs to a wait-and-see attitude. Now we can see that the star of rocketmq in apache is more than 5000 stars, which can only be said to be regular.

Application scenarios of RabbitMQ

Data transfer of heterogeneous systems: what is a heterogeneous system, that is, there are two systems that want to transfer data to each other, one using java, one developed with c, one in Tianjin and one in Beijing. Systems like this, which are not in one region but are developed in different languages, are called heterogeneous systems. It is a good choice for heterogeneous systems to use mq for data transfer, but they encounter huge amounts of data. However, it is not recommended to use mq for transmission. Generally speaking, mq is good at passing small-scale data.

Flow control of highly concurrent programs: for example, when Ali's Singles Day holiday is a second kill or when the system is busiest, when flow control is carried out, peak cutting and Pinggu are certainly no problem for applications like this. Mq uses a queue mechanism, so only how many people are allowed to enter at the same time, then mq is used here for control.

Based on p2pforce p2ppp program: mq itself supports subscription and release mode or peer-to-peer consumption mode, such as using mq for peer-to-peer development, such as self-written qq, like Wechat, peer-to-peer communication, we can use mq for distribution, in addition, if it is a p2ppp program, it is also very suitable, like when we were at the bottom of our program, we saw iqiyi or something. For example, if you like a person's channel and click to follow it directly, then when this person's program is updated, people who subscribe immediately can receive the latest news about it, that is, through mq subscription and release, which are all inherent in mq.

Transaction consistency of distributed system TCC: distributed transaction is the most head-scratching problem in our development process. For example, what is a distributed transaction? now there are five systems. If these five systems want to complete a large business, then this large business, each of these five systems has completed a small piece of data. Now there is a demand, or all of these five systems can be processed successfully. Either there is a problem in the middle, and I want to roll back all the previous system information and do nothing, then this is the distributed transaction we are talking about, then our design at the bottom of the distributed transaction is called tcc, then the core of this is to distribute messages based on mq to inform our application that it will be submitted and rolled back.

Highly reliable trading system: in a highly reliable trading system, rabbitmq is most proud of its high reliability. For applications, mq itself is highly reliable. At the same time, it also provides high reliable support for data, so there is almost no loss of data at the level of mq. Then in this highly reliable trading system, rabbitmq is the most suitable for any two or more projects with multiple data interaction. It is more appropriate to use mq as a server for data transfer.

3. Install the latest RabbitMQ on Linux platform

RabbitMQ is developed using Erlang

Erlang is a general concurrency-oriented programming language. Erlang is a structured, dynamically typed programming language with built-in parallel computing support.

It is much easier to use Erlang to write distributed applications. The Erlang runtime environment is a virtual machine, a bit like a Java virtual machine, so that once the code is compiled, it can also run anywhere.

Download address: https://www.rabbitmq.com/download.html

Download the latest rabbitmq directly to the local

Rabbitmq also needs to rely on the running environment of erlang

Https://www.erlang-solutions.com/resources/download.html

Select centos, find the latest version of 22.2.5 and download it locally.

[root@zhaocheng ~] # mkdir rabbitmq

[root@zhaocheng rabbitmq] # ls

Esl-erlang_22.2.5-2~centos~7_amd64.rpm rabbitmq-server-3.8.2-1.el7.noarch.rpm

-- nodeps does not check dependencies and removes dependencies

[root@zhaocheng rabbitmq] # rpm-ivh-- nodeps esl-erlang_22.2.5-2~centos~7_amd64.rpm

[root@zhaocheng rabbitmq] # rpm-ivh-- nodeps rabbitmq-server-3.8.2-1.el7.noarch.rpm

[root@zhaocheng rabbitmq] # rabbitmq-plugins enable rabbitmq_management

Start the console

[root@zhaocheng rabbitmq] # rabbitmq-server

Visit http://ip:15672

User can only log in via localhost appears when logging in, because it is forbidden to use guest/guest permission to access other than localhost.

Find this file, rabbit.app.

Vim / usr/lib/rabbitmq/lib/rabbitmq_server-3.7.7/ebin/rabbit.app

Will: {loopback_users, []}

Change to: {loopback_users, []}

Restart the service

[root@zhaocheng ~] # rabbitmqctl stop

[root@zhaocheng ~] # rabbitmq-server

Login: guest

Password: guest

RabbitMQ common commands

Start and shut down

Rabbitmq-server foreground start

Rabbitmq-server-detached background startup

Rabbitmqctl stop out of service / pid for kill-9 rbbitmq

Terminate and start the application

Rabbitmqctl start_app launches the application

Rabbitmqctl stop_app terminates the application

That is, to start or terminate our application, which will not affect our process, but just suspend our queue service. For example, a power plant regards this power plant as a process, for example, the power plant is upgraded and needs to be pulled, then it is equivalent to using this rabbitmqctl stop_app. If it is restored, use start_app.

User management

Create a user, such as kubernetes

Format: rabbitmqctl add_user {username} {password}

[root@zhaocheng ~] # rabbitmqctl add_user kubernetes 123456

Adding user "kubernetes"...

Delete user

Rabbitmqctl delete_user {username}

Reset the password, generally speaking, change the password for guest users when deploying rabbitmq, because this permission is very large.

[root@zhaocheng ~] # rabbitmqctl change_password guest 123456789

Changing password for user "guest"...

Format: rabbitmqctl change_password {username} {newpassword}

To grant user roles (Tag), there are four basic roles in rabbitmq with different permissions to operate on mq. These four roles are called tag, and you can give them certain permissions for the users we create. This tag contains certain policy information.

Format: rabbitmqctl set_user_tags {username} {tag}

[root@zhaocheng ~] # rabbitmqctl set_user_tags kubernetes administrator

Setting tags for user "kubernetes" to [administrator]...

Check that permissions have been set for our kubernetes users as Super Admin

Set the vhost that the user is allowed to access: the virtual host that the user is allowed to access is equivalent to the corresponding database in the mysql. Mysql can put the data in different corresponding data and classify the data, then vhost is the same. Different virtual hosts can be set for different systems, which virtual hosts can be accessed through set_permissions, and the following star can perform all permissions. The first is to configure permissions, read permissions, write permissions,-p is the path of the virtual host

Format: rabbitmqctl set_permissions-p / user_admin'.'

Give kubernetes users the right to operate a default virtual host. In general, multiple virtual hosts will be created in a production environment to carry out some related permissions.

[root@zhaocheng] # rabbitmqctl set_permissions-p / kubernetes'.

Setting permissions for user "kubernetes" in vhost "/".

You can add users directly on the page

Four kinds of tag for rabbitmq users

Super Admin (administrator): that is, to manage all users

You can log in to the management console (with management plugin enabled), you can view all the information, and you can operate on users, policies (policy).

Monitoring: you can view the relevant information of the rabbitmq node, which manages the running status of our running node.

Log in to the admin console (with management plugin enabled) and view the

Information about rabbitmq nodes (number of processes, memory usage, disk usage, etc.)

Policy maker (policymaker): this cannot view the relevant information about our node, but we can customize some policies, such as which queues can be replicated when managing the cluster, or the rules for running, which specify how our node works.

You can log in to the administrative console (with management plugin enabled), and you can also access the

Managed by policy. However, you cannot view the information about the node (the part identified in the red box above).

General Manager (management)

You can only log in to the administrative console (with management plugin enabled), and you cannot see node information or manage policies.

Another is that when you don't write anything, you can't even log in to the console without a role.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.