Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to understand the processing flow of request in RGW". In daily operation, I believe many people have doubts about how to understand the processing flow of request in RGW. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubt of "how to understand the processing flow of request in RGW"! Next, please follow the editor to study!

Source code interpretation of request processing flow in RGW request processing flow chart

Take civetweb as an example

Rgw_main.cc is the entry of the entire radosgw service. The main () function selects different front-end types according to the rgw frontends parameter settings in ceph.conf, and then executes the corresponding run () method to start the entire frontend service. Note that different handler will be generated by implementing S3, swift, admin and other types of interfaces according to the setting of rgw_enable_apis in ceph.conf. The specific code is as follows

# src/rgw/rgw_main.ccget_str_list (gencrypted confession-> rgw_enable_apis, apis); # get the interface type list map apis_map; for (list::iterator li = apis.begin (); li! = apis.end (); + + li) {apis_map [* li] = true;}. If (apis_map.count ("S3") > 0 | | s3website_enabled) {if (! Swift_at_root) {rest.register_default_mgr (set_logging (new RGWRESTMgr_S3 (s3website_enabled); # set the default handler of S3 interface to RGWRESTMgr_S3... If (apis_map.count ("swift") > 0) {do_swift = true; swift_init (g_ceph_context); RGWRESTMgr_SWIFT* const swift_resource = new RGWRESTMgr_SWIFT;# sets the swift interface default handler to RGWRESTMgr_SWIFT.

Then in the corresponding rgw_civetweb_fronted.cc, according to the civetweb startup process described earlier, set the corresponding startup parameters, and then use mg_start () to complete the startup of civetweb. (note that callback is set to civetweb_callback in the parameter)

# src/rgw/rgw_civetweb_frontend.ccint RGWMongooseFrontend::run () {char thread_pool_buf [32]; snprintf (thread_pool_buf, sizeof (thread_pool_buf), "% d", (int) gencrypted confession-> rgw_thread_pool_size); string port_str; map conf_map = conf- > get_config_map (); conf- > get_val ("port", "80", & port_str) Conf_map.erase ("port"); std::replace (port_str.begin (), port_str.end (),'+','); conf_map ["listening_ports"] = port_str; # civetweb default launch listening port set_conf_default (conf_map, "enable_keep_alive", "yes"); # keep_alive parameter setting set_conf_default (conf_map, "num_threads", thread_pool_buf) # default threads setting set_conf_default (conf_map, "decode_url", "no");... Struct mg_callbacks cb; memset ((void *) & cb, 0, sizeof (cb)); cb.begin_request = civetweb_callback; # callback function sets cb.log_message = rgw_civetweb_log_callback; cb.log_access = rgw_civetweb_log_access_callback; ctx = mg_start (& cb, & env, (const char * *) & options); # start service if (! ctx) {return-EIO;} return 0 } / * RGWMongooseFrontend::run * /

After setting up in the previous step, every request request in civetweb_callback needs to be processed by process_request (). Note that each request request is bound to a set of RGWRados (responsible for data reading and writing of the underlying Librados) / RGWREST (processing corresponding to request and Response) / OpsLogSocket (log message recording)

# src/rgw/rgw_civetweb_frontend.cc

Static int civetweb_callback (struct mg_connection* conn) {struct mg_request_info* req_info = mg_get_request_info (conn); RGWMongooseEnv* pe = static_cast (req_info- > user_data)

{/ / hold a read lock over access to pe- > store for reconfiguration RWLock::RLocker lock (pe- > mutex)

RGWRados* store = pe- > store; RGWREST* rest = pe- > rest; OpsLogSocket* olog = pe- > olog

RGWRequest req (store- > get_new_req_id ()); RGWMongoose client_io (conn)

Int ret = process_request (pe- > store, rest, & req, & client_io, olog); # each request request is bound to a set of previous RGWRados, RGWREST, OpsLogSocket. ```

Then call process_request () in rgw_process.cc, where rest- > get_handler obtains the corresponding handler type based on whether the requested URL contains bucket and object information, and then calls handler- > get_op (store) to obtain the final handler according to the request_method corresponding to the handler previously obtained, and then triggers the pre_exec (), execute () and complete () corresponding to handler to complete the processing of the entire request request. The code is as follows:

# src/rgw/rgw_process.ccint process_request (RGWRados* store, RGWREST* rest, RGWRequest* req,GWStreamIO* client_io, OpsLogSocket* olog) {int ret = 0 RGWHandler_REST handler = rest- > get_handler (store, s, client_io, & mgr,&init_error) # based on whether the URL contains bucket and Object fields, the corresponding handler type if (init_error! = 0) {abort_early (s, NULL, init_error, NULL) {abort_early (s, NULL, init_error, NULL); goto done;} dout (10) op); op = handler- > get_op (store); # the handler type for the final processing of the request request is obtained here based on request_method. Req- > log (s, "pre-executing"); op- > pre_exec () # request preprocessing req- > log (s, "executing"); op- > execute (); # specific implementation of specific request req- > log (s, "completing"); op- > complete (); # complete request processing

# src/rgw/rgw_process.ccRGWHandler_REST* RGWRESTMgr_S3::get_handler (struct req_state * s) {bool is_s3website = enable_s3website & & (s-> prot_flags & RGW_REST_WEBSITE); int ret = RGWHandler_REST_S3::init_from_header (s, is_s3website? RGW_FORMAT_HTML: RGW_FORMAT_XML, true); if (ret

Init_state.url_bucket.empty () {handler = new RGWHandler_REST_Service_S3Website;} else if (s-> object.empty ()) {handler = new RGWHandler_REST_Bucket_S3Website;} else {handler = new RGWHandler_REST_Obj_S3Website;}} else {if (s-> init_state.url_bucket.empty ()) {handler = new RGWHandler_REST_Service_S3 If # bucket is empty, switch to RGWHandler_REST_Service_S3} else if (s-> object.empty ()) {handler = new RGWHandler_REST_Bucket_S3; # obj if empty, switch to RGWHandler_REST_Bucket_S3} else {handler = new RGWHandler_REST_Obj_S3; # bucket and Object are not empty, switch to RGWHandler_REST_Obj_S3}} ldout (s-> cct, 20) handler procedure

Understand the whole URL conversion process of handler, can feel the request information quickly locate the specific op operation, facilitate debug, the whole process is summarized in the following picture.

At this point, the study on "how to understand the processing flow of request in RGW" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.