Shulou(Shulou.com)06/03 Report--

  Protobuf is widely used, especially as a network communication protocol. This article details several eye-catching protobuf protocol designs that will inspire developers who are ready to use or have already used protobuf, and even can be used directly. The protocols described here are designed for instant messaging, buried data collection, messaging, redis, and mysql data brokers in production environments.

  Bwar has been using protobuf since 2013, designing protobuf for mysql data proxies in 2014, and protobuf for instant messaging in 2015. Nebula https://github.com/Bwar/Nebula, a high-performance C++ IoC network framework, takes these protobuf protocol designs to the extreme.

1. TCP Communication Protocol Design

  This protocol was designed in 2015 for IM and buried point data collection and real-time analysis in a production environment, and extended in 2016 to protobuf3-based versions for the open source network framework Nebula. Protobuf2 and protobuf3 are less different, and the two versions of the protocol design are explained separately here.

1.1. protobuf version 2.5 Msg

  There is no release version of protobuf3 in 2015, and the fixed32 type of protobuf2 version is fixed to occupy 4 bytes, which is very suitable for network communication protocol design. The protocol Bwar designed for IM systems consists of two protobuf messages: MsgHead and MsgBody, defined as follows:

```C++

syntax = "proto2";

/**

@brief Message header

*/

message MsgHead

{

required fixed32 cmd = 1 ; ///

< 命令字（压缩加密算法占高位1字节） required fixed32 msgbody_len = 2; ///< 消息体长度（单个消息体长度不能超过65535即8KB） required fixed32 seq = 3; ///< 序列号 } /** @brief 消息体@note 消息体主体是body，所有业务逻辑内容均放在body里。session_id和session用于接入层路由，两者只需要填充一个即可，首选session_id，当session_id用整型无法表达时才使用session。 */ message MsgBody { required bytes body = 1; ///< 消息体主体 optional uint32 session_id = 2; ///< 会话ID（单聊消息为接收者uid，个人信息修改为uid，群聊消息为groupid，群管理为groupid） optional string session = 3; ///< 会话ID（当session_id用整型无法表达时使用） optional bytes additional = 4; ///< 接入层附加的数据（客户端无须理会） }   解析收到的字节流时先解固定长度（15字节）的MsgHead（protobuf3.0之后的版本必须在cmd、msgbody_len、seq均不为0的情况下才是15字节），再通过MsgHead里的msgbody_len判断消息体是否接收完毕，若接收完毕则调用MsgBody.Parse()解析。MsgBody里的设计在下一节详细说明。   MsgHead在实际的项目应用中对应下面的消息头并可以相互转换： ```C++ #pragma pack(1) /** @brief 与客户端通信消息头 */ struct tagClientMsgHead { unsigned char version; ///< 协议版本号（1字节） unsigned char encript; ///< 压缩加密算法（1字节） unsigned short cmd; ///< 命令字/功能号（2字节） unsigned short checksum; ///< 校验码（2字节） unsigned int body_len; ///< 消息体长度（4字节） unsigned int seq; ///< 序列号（4字节） }; #pragma pack()   转换代码如下：```C++E_CODEC_STATUS ClientMsgCodec::Encode(const MsgHead& oMsgHead, const MsgBody& oMsgBody, loss::CBuffer* pBuff){ tagClientMsgHead stClientMsgHead; stClientMsgHead.version = 1; // version暂时无用 stClientMsgHead.encript = (unsigned char)(oMsgHead.cmd() >

> 24); stClientMsgHead.cmd = htons((unsigned short)(gc_uiCmdBit & oMsgHead.cmd())); stClientMsgHead.body_len = htonl((unsigned int)oMsgHead.msgbody_len()); stClientMsgHead.seq = htonl(oMsgHead.seq()); stClientMsgHead.checksum = htons((unsigned short)stClientMsgHead.checksum); ...} E_CODEC_STATUS ClientMsgCodec::Decode(loss::CBuffer* pBuff, MsgHead& oMsgHead, MsgBody& oMsgBody){ LOG4_TRACE("%s() pBuff->ReadableBytes() = %u", __FUNCTION__, pBuff->ReadableBytes()); size_t uiHeadSize = sizeof(tagClientMsgHead); if (pBuff->ReadableBytes() >= uiHeadSize) { tagClientMsgHead stClientMsgHead; int iReadIdx = pBuff->GetReadIndex(); pBuff->Read(&stClientMsgHead, uiHeadSize); stClientMsgHead.cmd = ntohs(stClientMsgHead.cmd); stClientMsgHead.body_len = ntohl(stClientMsgHead.body_len); stClientMsgHead.seq = ntohl(stClientMsgHead.seq); stClientMsgHead.checksum = ntohs(stClientMsgHead.checksum); LOG4_TRACE("cmd %u, seq %u, len %u, pBuff->ReadableBytes() %u", stClientMsgHead.cmd, stClientMsgHead.seq, stClientMsgHead.body_len, pBuff->ReadableBytes()); oMsgHead.set_cmd(((unsigned int)stClientMsgHead.encript , =, LT = 3; ///

< 小于< GE = 4; ///< 大于等于>

= LE = 5; ///

< 小于等于, =, LT = 3; ///< 小于< GE = 4; ///< 大于等于>

= LE = 5; ///<<=

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.