Shulou(Shulou.com)06/02 Report--

  HTTPS is a security-oriented HTTP channel, which is simply the secure version of HTTP. That is, the SSL layer is added under HTTP, and the security basis of HTTPS is SSL, so the details of encryption need SSL. Nebula is a high-performance event-driven network framework that provides developers with a high-performance event-driven network framework for rapidly developing highly concurrent network service programs or building highly concurrent distributed service clusters. It is very important for Nebula to provide HTTPS support as a general network framework. Nebula can be used as both a https server and a https client. In this paper, SSL programming based on openssl will be described in detail with the https implementation of Nebula framework. If you think this article is useful to you, help to Nebula's Github or Code Cloud to give a star, thank you. Nebula is not only a framework, but also provides a series of applications based on this framework, the goal is to create a high-performance distributed service cluster solution. The main application areas of Nebula: instant messaging (successfully applied to an IM), message push platform, real-time data analysis and computing (success stories), etc. Bwar also plans to develop crawler applications based on Nebula.

1. SSL encrypted communication

   HTTPS communication adds a SSL layer between the TCP communication layer and the HTTP application layer. If the application layer is not HTTP protocol, you can also use SSL to encrypt the communication, such as WebSocket protocol WS plus SSL layer after WSS. The Nebula framework can change the communication protocol without modifying the code by changing the Codec. After Nebula adds SSL support, all the communication protocols supported by Nebula have SSL encrypted communication support, and the business code based on Nebula does not need to be modified.

The SSL connection establishment process after    Socket connection is established:

2. OpenSSL API

There are many API of    OpenSSL, but not all of them will be used. If you need to see the details of how to use an API, you can read the API documentation.

2.1 initialize OpenSSL

   OpenSSL must be initialized before it can be used. Before establishing a SSL connection, specify the protocol and its version for this connection for Client and Server, respectively. Currently, the available protocol versions include SSLv2, SSLv3, SSLv2/v3, and TLSv1.0. For a SSL connection to be established properly, Client and Server must use a mutually compatible protocol. Below    is the code for the Nebula framework SocketChannelSslImpl::SslInit () function to initialize OpenSSL, which is initialized by calling different API according to different versions of OpenSSL.

# if OPENSSL_VERSION_NUMBER > = 0x10100003L if (OPENSSL_init_ssl (OPENSSL_INIT_LOAD_CONFIG, NULL) = = 0) {pLogger- > WriteLog (neb::Logger::ERROR, _ _ FILE__, _ _ LINE__, _ FUNCTION__, "OPENSSL_init_ssl () failed!"); return (ERR_SSL_INIT) } / * * OPENSSL_init_ssl () may leave errors in the error queue * while returning success * / ERR_clear_error (); # else OPENSSL_config (NULL); SSL_library_init (); / / initialize the SSL algorithm library function (load the algorithm to be used). This function must be called SSL_load_error_strings () before calling the SSL function / / initialize the error message OpenSSL_add_all_algorithms (); # endif2.2 create CTX

   CTX is a SSL session environment where different protocols are used to establish a connection, and the CTX is also different. Create related OpenSSL functions for CTX:

/ / both client and server need to call SSL_CTX_new (); / / apply for SSL session environment / / call SSL_CTX_set_verify () if there is a need to verify the other party's certificate; / / specify the certificate verification method SSL_CTX_load_verify_location () / / load the list of trusted CA certificates that should be used for the SSL session environment / / if there is a need to load a certificate, call int SSL_CTX_use_certificate_file (); / / load the certificate int SSL_CTX_use_certificate_chain_file () of this application for the SSL session; / / load the certificate chain int SSL_CTX_use_PrivateKey_file () to which the certificate of this application belongs for the SSL session / / load the private key int SSL_CTX_check_private_key () of this application for the SSL session; / / verify whether the loaded private key and certificate match 2.3.Create a SSL socket

Before    creates a SSL socket, it creates a Socket socket and establishes a TCP connection. Create SSL socket related functions:

SSL * SSl_new (SSL_CTX * ctx); / create a SSL socket int SSL_set_fd (SSL * ssl, int fd); / / bind stream socket int SSL_set_rfd (SSL * ssl, int fd) in read-write mode; / / bind stream socket int SSL_set_wfd (SSL * ssl, int fd) in read-only mode; / / bind stream socket 2.4 in write-only mode to complete SSL handshake

   in this step, we need to establish a SSL connection based on a normal TCP connection. Similar to the process of establishing a connection with a normal stream socket: Client uses the function SSL_connect () [similar to connect () used in stream sockets] to initiate a handshake, while Server uses the function SSL_ accept () [similar to accept () used in stream sockets] to respond to the handshake, thus completing the handshake process. The prototype of the two functions is as follows:

Int SSL_connect (SSL * ssl); int SSL_accept (SSL * ssl)

After the    handshake process is complete, the Client usually asks the Server to send certificate information to authenticate the Server. The following two functions are used in its implementation:

X509 * SSL_get_peer_certificate (SSL * ssl); / / obtain the certificate information of the other party from the SSL socket X509_NAME * X509_get_subject_name (X509 * a); / / get the name of the certificate holder 2.5 data transfer

After a series of previous procedures,    is ready for secure data transfer. In the data transfer phase, SSL_read () and SSL_write () need to be used to replace the read () and write () functions used by ordinary stream sockets to complete the read and write operation of SSL sockets. The prototypes of the two new functions are as follows:

Int SSL_read (SSL * ssl,void * buf,int num); / / read data from SSL socket int SSL_write (SSL * ssl,const void * buf,int num); / / write data to SSL socket 2.6.session end

   when the communication process between Client and Server is complete, use the following function to release the SSL resources requested in the previous procedure:

Int SSL_shutdown (SSL * ssl); / / close SSL socket void SSl_free (SSL * ssl); / / release SSL socket void SSL_CTX_free (SSL_CTX * ctx); / / release SSL session environment 3. SSL and TLS

   HTTPS uses two protocols, SSL (Secure Socket Layer) and TLS (Transport LayerSecurity). SSL technology was first advocated by browser developer Netscape Communications, which developed a previous version of SSL3.0. At present, the leadership has been transferred to IETF (Internet Engineering Task Force,Internet Engineering Task Force).

   IETF takes SSL3.0 as the benchmark, followed by TLS1.0, TLS1.1 and TLS1.2. TSL is a protocol based on SSL, which is sometimes referred to as SSL. The current mainstream versions are SSL3.0 and TLS1.0.

   did not actually put into use because the SSL1.0 protocol was found to have problems at the beginning of its design. SSL2.0 was also found to be problematic, so many browsers simply abolished the version of the protocol.

4. Implementation of SSL communication in Nebula

The    Nebula framework supports both SSL server applications and SSL client applications. Initialization of openssl only needs to be initialized once (SslInit () only needs to be called once). The SSL-related code of the Nebula framework (including client-side and server-side implementations) is encapsulated in the SocketChannelSslImpl class. Nebula's SSL communication is based on asynchronous non-blocking socket communication and does not use openssl's BIO (because it is unnecessary and the code is more complex).

   SocketChannelSslImpl is a derivative of SocketChannelImpl, adding a SSL communication layer to SocketChannelImpl regular TCP communication, and there is little difference between the calls of the two classes. The SocketChannelSslImpl class is declared as follows:

Class SocketChannelSslImpl: public SocketChannelImpl {public: SocketChannelSslImpl (SocketChannel* pSocketChannel, std::shared_ptr pLogger, int iFd, uint32 ulSeq, ev_tstamp dKeepAlive = 0.0); virtual ~ SocketChannelSslImpl (); static int SslInit (std::shared_ptr pLogger); static int SslServerCtxCreate (std::shared_ptr pLogger); static int SslServerCertificate (std::shared_ptr pLogger, const std::string& strCertFile, const std::string& strKeyFile); static void SslFree (); int SslClientCtxCreate () Int SslCreateConnection (); int SslHandshake (); int SslShutdown (); virtual bool Init (E_CODEC_TYPE eCodecType, bool bIsClient = false) override; / / overrides the Send () method of the base class, realizing that after the non-blocking socket connection is established, continue to establish the SSL connection, and send and receive data virtual E_CODEC_STATUS Send () override; virtual E_CODEC_STATUS Send (int32 iCmd, uint32 uiSeq, const MsgBody& oMsgBody) override Virtual E_CODEC_STATUS Send (const HttpMsg& oHttpMsg, uint32 ulStepSeq) override; virtual E_CODEC_STATUS Recv (MsgHead& oMsgHead, MsgBody& oMsgBody) override; virtual E_CODEC_STATUS Recv (HttpMsg& oHttpMsg) override; virtual E_CODEC_STATUS Recv (MsgHead& oMsgHead, MsgBody& oMsgBody, HttpMsg& oHttpMsg) override; virtual bool Close () override;protected: virtual int Write (CBuffer* pBuff, int& iErrno) override; virtual int Read (CBuffer* pBuff, int& iErrno) override Private: E_SSL_CHANNEL_STATUS E_SSL_CHANNEL_STATUS SslChannelStatus; / / add the SSL channel state to the base class m_ucChannelStatus channel state; SSL* mSecretbIsClientConnectionsSslConnectionmSecretpServerSslCtx; / / when you turn on the ssl option to compile and start the Nebula service, the static SSL_CTX* m_pClientSslCtx is automatically created. / / default is empty. Create} when you turn on the ssl option to compile and initiate a connection to another SSL service for the first time (such as accessing a https address)

The methods with the override keyword in the    SocketChannelSslImpl class override the methods of the base class SocketChannelImpl with the same name, and are also the key to the transparency of SSL communication and non-SSL communication calls. Methods without the override keyword are SSL communication-related methods, in which there are function calls to openssl. Methods without override can be divided into static and non-static methods, and static methods are called only once in the process, regardless of the specific Channel object. There is no need to call non-static ssl-related methods outside of SocketChannel.

Because    is non-blocking socket,SSL_do_handshake () and SSL_write (), SSL_read () return values can not fully determine whether there is an error, but also need SSL_get_error () to obtain the error code. Both SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE are normal.

Most of the openssl example programs on    simply call openssl functions in order to realize synchronous ssl communication. In non-blocking IO applications, ssl communication is much more complex. SocketChannelSslImpl implements non-blocking ssl communication. From the implementation of this class, the whole communication process is not completely linear. The following SSL communication diagram illustrates more clearly how SSL communication is implemented in the Nebula framework:

Static methods in    SocketChannelSslImpl need to be called only once during the lifetime of the process, which can also be understood as SSL_CTX_new (), SSL_CTX_free (), and so on. Further understand that the SSL_CTX structure needs to be created only once within the process (one for Server and one for Client in Nebula) to be used by all SSL connections; of course, it is okay to create a separate SSL_CTX for each SSL connection (it has been measured in Nebula 0.4 to create a separate SSL_CTX for each Client), but it is generally not done because it consumes more memory resources and is less efficient.

When    establishes a SSL connection, the client invokes SSL_connect () and the server invokes SSL_accept (), as many openssl demo uses. SSL_do_handshake () is used in Nebula, which is suitable for both client and server, and SSL_do_handshake () is more suitable for services with both client and server functions. Note that before calling SSL_do_handshake (), you need to call SSL_set_connect_state () on the client side and SSL_set_accept_state () on the server side. In non-blocking IO, SSL_do_handshake () may need to be called several times to complete the handshake. The specific time to call is to get the error code SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE based on SSL_get_error () to determine whether to listen for read events or write events, and call SSL_do_handshake () again when the corresponding events are triggered. For more information, please refer to the Send and Recv methods of SocketChannelSslImpl.

When    closes the SSL connection, first call SSL_shutdown () to close the SSL layer connection normally (SSL_shutdown () in the non-blocking IO may also need to be called multiple times), then call SSL_free () to release the SSL connection resources, and finally close the socket connection. SSL_CTX does not need to be released. The whole SSL communication has been completed successfully. When Nebula 0.4 opened multiple terminals to call curl simple pressure test with shell script in an endless loop, SSL client and SSL server functions were normal:

While: do curl-v-k-H "Content-Type:application/json"-X POST-d'{"hello": "nebula ssl test"} 'https://192.168.157.168:16003/test_ssl done

The    test method is shown below:

   checks the resource usage. The memory usage on the SSL Server side has been growing, and it is suspected that there is a memory leak. However, when pmap-d checks that a certain anon memory reaches near 18MB, it will no longer grow, indicating that it may not be a memory leak, but that part of the memory is used by openssl as a cache. There is no solution to this problem on the Internet. Find clues from the definition of the struct ssl_ctx_st structure, and then find SSL_CTX_remove_session () in the nginx source code, so add SSL_CTX_remove_session () before SSL_free (). Session multiplexing can improve the efficiency of SSL communication, but Nebula is not needed for the time being.

   this testing method takes NebulaInterface as the SSL server and NebulaLogic as the SSL client. At the same time, it completes the Nebula framework SSL server and client functional testing, simple stress testing. The SSL communication test of the Nebula framework has passed, and it can also be put into production applications, and it will certainly continue to be improved in the follow-up applications. Openssl is really hard to use, no wonder there are so many complaints, maybe the soon-to-be version of Nebula will replace openssl with other ssl libraries.

5. End

   plus the Nebula framework supported by SSL passed the test, which is not too complicated, but the process is tortuous and time-consuming. Here we share Nebula using openssl to develop SSL communications, hoping to be useful to developers who are ready to use openssl. If you think this article is useful to you, don't forget to go to Nebula's Github or Code Cloud to give a star, thank you.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.