Shulou(Shulou.com)06/03 Report--

1. Introduction

Regular readers will remember that I shared an article entitled "Technical practical Information: teaching you to Design a million-level message push system from scratch" before National Day last year. Although I posted some pseudo code in the article, there are still some friends who want to share some source code that can run directly. Well, I have nothing to say to question my poverty (because I am really poor. ), it is absolutely impossible to doubt my ability to code, so this time I am going to pull up the keyboard and do a big job-- unarmed distributed IM! ^ _ ^!

This article records an IM system I developed for IM learners-CIM (full name: CROSS-IM), and provides some components to help developers build an IM that belongs to their own horizontal scalability.

By learning this article and the CIM code, you can gain the following knowledge:

1) how to develop an IM from scratch (CIM customers are a little weak, I beg your pardon)

2) how to design distributed IM architecture

3) how to implement your distributed IM architecture with code and related technologies.

The CIM source code address for this article:

Primary image: https://github.com/crossoverJie/cim

Alternate image: https://github.com/52im/cim

* friendly reminder: reading this article and CIM source code, you need to have some knowledge of network programming, IM theory and so on. If you don't have these, please read the "beginner's introduction to Mobile IM from Zero Development" first.

This article is published simultaneously on: http://www.52im.net/thread-2775-1-1.html

2. About the author

CrossoverJie (Chen Jie): post-90s, graduated from Chongqing Institute of Information Engineering, now works for Chongqing Zhu Bajie Network Co., Ltd.

3. Run the demo

This time, we specially recorded two video presentations (group chat and private chat). Click the link below to view the video version of Demo.

CIM private chat video demo: https://www.bilibili.com/video/av39405821

CIM group chat video demo: https://www.bilibili.com/video/av39405501

4. Architecture design

Let's take a look at the specific architectural design:

Architecture description:

1) each component in CIM is built with SpringBoot.

2) using Netty + Protocol Buffer to build the underlying communication

3) Redis stores each client's routing information, account information, online status, etc.

4) Zookeeper is used for registration and discovery of IM-server services.

The whole is mainly composed of the following modules:

1) cim-server--IM server: it is used to receive client connection, transparent message transmission, message push and other functions. Support for cluster deployment

2) cim-forward-route-- message routing server: used to handle message routing, message forwarding, user login, user offline and some operational tools (obtaining the number of online users, etc.)

3) cim-client--IM client: for the message terminal used by the user, a command can start and initiate communication to others (group chat, private chat); at the same time, some common commands are built-in for easy use.

5. Logic flow chart

The overall process is also relatively simple, the flow chart is as follows:

The process is explained as follows:

1) client initiates login to route

2) Log in successfully, select available IM-server from Zookeeper to return to the client, and save login and routing information to Redis

3) the client initiates a long connection to the IM-server and keeps the heartbeat after success

4) clear the status information through route when the client goes offline.

So when we deploy ourselves, we need the following steps:

1) build basic middleware such as Redis and Zookeeper

2) deploy cim-server, which is a real IM server. Horizontal scaling is supported in order to meet performance requirements. You only need to register with the same Zookeeper.

3) deploy cim-forward-route, which is the routing server that all messages need to pass through. Because it is stateless, Nginx agents can also be used to improve availability

4) cim-client is really a user-oriented client; after startup, it will automatically connect to the IM server to send and receive messages in the console.

For more instructions, please refer to Quick launch.

The next chapter will focus on the specific detailed design and implementation, such as group chat, private chat message flow; IM server load balancing; service registration discovery and so on.

6. IM server

First, let's take a look at the server: mainly to achieve the client on and off the line, message distribution and other functions.

The first is the service startup:

Since it is built in SpringBoot, the Netty service needs to be started when the application is started.

It can be seen from pipline that Protobuf codec is used (for specific messages are analyzed in the client, please see "detailed explanation of Protobuf communication protocol: code demonstration, detailed principle introduction, etc.").

7. Registration discovery

It needs to meet the horizontal scaling requirements of the IM server, so cim-server needs to publish its own data to the registry. Here is a specific introduction to "getting Service Registration and Discovery" shared earlier.

Therefore, after the application is launched successfully, you need to register your own data in Zookeeper:

The main purpose is to register the ip + cim-server-port+ http-port of the current application:

The figure above shows two cim-server instances I registered in the demo environment (only with different ports because they are on the same server). In this way, you can know the currently available service information in real time on the client side (listening on the Zookeeper node).

8. Log in

When the client requests the login interface in cim-forward-route (see below) to complete business authentication (just like logging in to other websites on a daily basis), the client initiates a long connection to the server.

As shown in the previous process:

At this point, the client will send a special message indicating that it is currently login information. After receiving it, the server needs to save the userID and the current Channel channel relationship of the client.

It also caches the user's information, that is, userID and user name.

9. Offline messages

When the client is disconnected, you also need to erase the cached information.

At the same time, you also need to call the route API to clear the relevant information (see below for more information).

10. IM routing

As you can see from the architecture diagram, the routing layer is a very important link; it provides a series of HTTP services that host both the client and the server.

At present, there are mainly the following interfaces.

10.1 Registration Interface

Since every client needs to log in to use it, the first step is naturally to register.

The design here is relatively simple, directly using Redis to store user information; user information is only ID and userName. Just to make it easier to query the KV in Redis, in turn, it stores a copy of VK, so that both ID and userName must be unique.

10.2 login interface

The login here is different from the login in cim-server, which is of a business nature:

Specific process:

1) after a successful login, you need to determine whether the login is repeated (a user can only run one client)

2) after a successful login, you need to obtain the service list (cim-server) from Zookeeper and select a service to return to the client according to a certain algorithm

3) after a successful login, you also need to save the routing information, that is, the service instance assigned by the current user is saved to the Redis.

In order to enable only one user to log in, set in Redis is used to save login information; using userID as key, repeated logins will fail.

Similar to HashSet in Java, it can only be resaved.

It is also easy to obtain an available routing instance:

1) get all the service instances from Zookeeper to do an internal cache

2) polling to select a server (currently there is only one algorithm, which will be added later).

Of course, before getting the service instance in Zookeeper, you need to listen to the node that cim-server registered before.

The specific code is as follows:

It also listens to the routing nodes in the Zookeeper after the application is started, and updates the internal cache as soon as it changes. Guava's cache is used here, which is based on ConcurrentHashMap, so the atomicity of clearing and adding caches can be guaranteed.

10.3 Group chat Interface

This is a real messaging interface, and the effect is that one of the clients sends a message and all the other clients can receive it! The process must be that the client sends a message to the server. After receiving it, the server traverses all the Channel (channels) in the SessionSocketHolder described above and then sends the message. The server can be used as a stand-alone machine, but now it is a cluster design. So all clients are assigned to different cim-server instances according to the previous polling algorithm.

So the routing layer is needed to play a role.

After receiving the message, the routing interface first traverses the relationship between all clients and service instances.

Routing relationships are stored in Redis as follows:

Because of the single-threaded nature of Redis, when the amount of data is large, once keys is used to match all cim-route:* data, Redis will not be able to process other requests. So instead use the scan command to iterate through all the cim-route:*.

The HTTP interface of the server where each client is located is then called one by one to push messages.

The implementation in cim-server is as follows:

After receiving the message, cim-server queries the internal cache for the channel of the userID, and then only needs to send a message.

10.4 online user interface

This is an auxiliary interface that can query current online user information.

The implementation is also very simple, that is, to save the "user login status of the de-set" before the query.

10.5 Private chat interface

The reason why it is said that access to online users is an auxiliary interface, in fact, is used to assist private chat. Generally speaking, when we use the premise of private chat, we must know which users are online, and then you will know who you want to talk to in private.

Something like this:

In our scenario, the premise of private chat is to obtain the userID of online users:

Therefore, after receiving the message, the private chat API needs to query the cim-server instance information of the recipient, and the subsequent steps are the same as the group chat. Call the HTTP API of the instance where the receiver resides to send messages. It's just that group chat is the difference between traversing all online users and sending only one private chat.

10.6 offline interface

Once the client is offline, we need to delete some of the information previously stored in the Redis (routing information, login status).

11. IM client

Some of the logic in the client has actually been mentioned above.

11.1 Login

The first step is to log in. You need to call the login interface of route at startup to obtain cim-server information and then create a connection.

During the login process, the route API will determine whether the login is repeated, and repeated login will directly exit the program.

The next step is to create a connection using the cim-server instance information (ip+port) returned by the route interface. The final step is to send a login flag to the server to maintain the relationship between the client and the Channel.

11.2 Custom Protocol

Some of the login messages and real message messages mentioned above can actually be distinguished in our custom protocol. Since you are using Protocol Buffer codec, take a look at the original format first.

In fact, there are only three fields in this agreement:

1) requestId can be understood as userId

2) reqMsg is the real news

3) type is the message category mentioned above.

At present, there are mainly three types, corresponding to different businesses:

11.3 heartbeat

In order to maintain the connection between the client and the server, it is necessary to send a heartbeat automatically every time a message is not sent.

The current strategy is to send a heartbeat packet to the server every other minute:

In this way, when the server does not receive a business message every other minute, it will receive the heartbeat packet of ping:

11.4 built-in command

The client also has some basic commands built in for ease of use.

For example, typing: Q exits the client and shuts down some system resources.

When you type: olu (short for onlineUser), it calls route to get all the online user interfaces.

11.5 Group chat

The use of group chat is very simple, just enter the message in the console and enter. At this time, the group chat interface of route will be called.

11.6 talk in private

The same is true for private chat, but only if the keyword is triggered; using a format like userId;; message content will send a message to a user, so you usually need to use the

The acquisition of the lu command makes it easy for online users to use.

11.7 message callback

In order to meet some customized requirements, such as messages that need to be saved. So after the client receives the message, it calls back an interface in which the implementation can be customized.

Therefore, we first create a bean for caller, which contains a CustomMsgHandleListener interface, which needs to be handled by ourselves and only needs to be implemented.

11.8 Custom Interface

Because I can't write the interface myself, but I'm sure no other Daniel can write it. Therefore, the client services such as group chat, private chat, access to online users, message callback (and subsequent business) are all provided in the form of interfaces.

It is also convenient to do page integration later, you only need to adjust these interfaces; you don't need to care about the specific implementation.

12. Summary of this article

At present, cim is only the first version, with more BUG and less features (only a few friends have been tested); but it will be improved later, at least this version will bring some ideas to those who have no relevant experience.

Follow-up plan:

This article is published simultaneously on: http://www.52im.net/thread-2775-1-1.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.