Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what are the knowledge points of the tgt program framework". In the daily operation, I believe that many people have doubts about the knowledge points of the tgt program framework. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "what are the knowledge points of the tgt program framework?" Next, please follow the editor to study!

Tgt program framework

Tgt is the iscsi target implemented in user mode, while iet (iscsi enterprise target) is implemented in kernel mode target,tgt compared with iet, because of its user mode implementation, convenient debugging, adding some new functions, etc., but the performance is slightly worse than iet. The following is to introduce the program framework of tgt (skip the parsing part of iscsi protocol) in order to grasp the code of tgt as a whole and facilitate some subsequent modifications.

1. Overall framework

The operation of the command in tgt and the communication with the initiator are implemented through epoll, which are described below.

1.1 interaction between tgtadm and tgtd

When the tgtd process is started, a unix socket is initialized, the fd is added to the epoll, the EPOLLIN event is listened for, and each subsequent tgtadm command is executed, the unix socket is used to communicate with tgtd.

When you type a tgtadm command on the command line, initiating the request through unix socket will trigger the EPOLLIN event of epoll, accept on the tgtd side, add the connection fd to the epoll, register the callback function mtask_recv_send_handler, and return the response of the connection establishment to the tgtadm.Then the tgtadm command triggers the EPOLLIN event, and the tgt side calls mtask_recv_send_handler Different handlers are called for processing according to different mode (here is sys_mgmt/target_mgmt/portal_mgmt, etc.). After processing, change the event to EPOLLIN | EPOLLOUT, and then send the response to the tgtadmside.

1.2 interaction between initiator and tgtd

When the tgtd process starts, it creates a socket to listen for the request of the initiator (specifically the processing in the iscsi_tcp_init_portal function), adds the socket to the epoll, and registers the callback function accept_connection.

When the initiator request arrives, add the fd of the new connection to the epoll and register the callback function iscsi_tcp_event_handler. Then, when the EPOLLIN event is received, iscsi_rx_handler is called to parse the iscsi request, and the API of backing store is called to perform the io operation. When the operation is completed, the event is modified to EPOLLLOUT, which triggers the call to iscsi_tx_handler to send the response to initiator.

1.3 function call stack

Here are some of the function call stack relationships of iscsi_rx_handler, which involve the call relationship with the io processing interface in backing store. 2. Backing store

Tgt supports a variety of backend storage, such as rdwr,aio,sg,rbd,sheepdog. The default is rdwr. You can specify flag (O_SYNC | O_DIRECT). With a clear understanding of the processing mode of tgt's back-end storage, you can add new back-end storage to support custom features.

The main interface is bs_open,bs_init,bs_cmd_submit,bs_close,bs_exit.

Bs_open and bs_init do some initialization operations when creating lun, such as opening device files, creating processing threads, registering callback functions, and so on.

This function is called for processing when the bs_cmd_submit:io request arrives.

Bs_close and bs_exit just do some destruction when deleting the lun.

Synchronous and asynchronous io modes are supported in BS (rdwr is synchronous and aio is asynchronous). These two modes are described below.

2.1 synchronous io

In synchronous io, each lun corresponds to a request queue (pending_list), and multiple threads are created during bs_init (the current default of 16 threads for a new version of lun). When a new request arrives, bs_cmd_submit is called to add the request to the pending_list, and multiple threads share the pending_list, fetching the request from the pending_list for processing. When there is no request, these threads will wait on the pending_list, and when a new request is added to the pending_lsit, the waiting thread will be awakened for processing.

2.2 Asynchronous io (aio)

Tgt is implemented using linux native aio, and the fd created by eventfd is used to associate the aio context with epoll. (you can refer to some information on how to use aio in combination with epoll.) Here are some specific processing logic.

1) bs_aio_open:io_setup establishes an asynchronous io context, then afd=eventfd (), adds afd to the epoll (callback function bs_aio_get_completions), and the afd is associated with the context of the aio.

2) when bs_aio_cmd_submit:IO arrives, add the request to the cmd_wait_list, then traverse the list to determine the number of io that can be submitted this time and initialize the iocb (because the maximum number of aio processing is limited to 128and some io is still being processed, the number of aio currently submitted is limited), and then use io_submit to submit a batch of asynchronous io.

3) after IO processing, the EPOLLIN event will be triggered through afd, and the callback function bs_aio_get_completions will be called for processing. First, read the number of io currently completed, and then call the

Io_getevents () fetches the information of the completed io, and then does some end processing (changing the processing state) for each completed io call bs_aio_comlete_one. Target_cmd_io_done is called in this function, and the event of epoll is changed to EPOLLIN in target_cmd_io_done. This triggers the EPOLLOUT event, calls iscsi_tcp_event_handler, and judges if it is EPOLLOUT in this function. Iscsi_tx_handler is called to send the response.

At this point, the study of "what are the knowledge points of the tgt program framework" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.