Shulou(Shulou.com)05/31 Report--

This article mainly introduces the Ceph OSD processing client write operation processing flow example analysis, the article introduces in great detail, has a certain reference value, interested friends must read it!

1. The processing flow of the request received by OSD from the client

OSD::ms_fast_dispatch ()

| | _ OSD::dispatch_session_waiting () |

| | _ OSD::dispatch_op_fast () |

| | _ OSD::handle_op () |

| | _ OSD::get_pg_or_queue_for_pg () |

| | _ OSD::enqueue_op () |

| | _ PG::queue_op () |

| | _ _ OSD::op_wq.queue () writes the request of client to the queue |

The processing flow of the master OSD processing the request from the client side

OSD::ShardOpWQ::_process ()

| | _ OSD::dequeue_op () |

| | _ ReplicatedPG::do_request () |

| | _ ReplicatedPG::do_op () |

| | _ ReplicatedPG::execute_ctx () |

| | _ ReplicatedPG::preapare_transcation () |

| | _ ReplicatedPG::do_osd_ops () |

| | _ ReplicatedPG::issue_repop () |

| | _ ReplicateBackend::submit_transaction () |

| | _ _ ReplicateBackend::issue_op () Master OSD sends write request to slave OSDs |

| | _ _ ReplicatedPG::queue_transcations () writes the main OSD FileJournal |

| | _ ObjectStore::queue_transactions () |

| | _ FileStore::queue_transactions () |

| | _ JournalingObjectStore::_op_journal_transactions () |

| | _ _ FileJournal::submit_entry () writes logs to the writeq queue of FileJournal |

| | _ ReplicatedPG::eval_repop () |

The FileJournal write thread of the main OSD

FileJournal::write_thread_entry () is responsible for getting logs to be written from the writeq queue

| | _ _ FileJournal::prepare_multi_write () constitutes a log |

| | _ _ FileJournal::do_write () actually writes logs |

| | _ FIleJournal::queue_completions_thru () |

| | _ _ completion_peek_front () gets the C_JournalAhead class instance |

| | _ finisher- > queue () |

| | _ _ finisher_cond.Signal () starts FileJournal's finisher thread to process C_JournalAhead class instances |

C_JournalAhead class processing of the main OSD

C_JournalAhead::finish ()

| | _ FileStore::_journaled_ahead () |

| | _ FileStore::queue_op () |

| | _ _ FileStore::op_wq.queue () writes data to the FileStore writer thread queue |

| | _ _ ondisk_finisher.queue (ondisk) calls ReplicatedBackend- > op_commit () callback function |

FileStore write data thread class processing of main OSD

FileStore::op_wq

| | _ FileStore::op_wq._process () |

| | _ FileStore::_do_op () |

| | _ FileStore::_do_transactions () |

| | _ FileStore::_do_transaction () |

| | _ _ FileStore::_write () is stored in FileStore (written to the specified file) |

| | _ FileStore::op_wq._process_finish () |

| | _ FileStore::_finish_op () |

| | _ _ op_finisher.queue () calls ReplicatedPG::op_applied () callback function |

2. Process the request sent by the main OSD from the OSD

OSD::ms_fast_dispatch ()

| | _ OSD::dispatch_session_waiting () |

| | _ OSD::dispatch_op_fast () |

| | _ OSD::handle_replica_op () |

| | _ OSD::get_pg_or_queue_for_pg () |

| | _ OSD::enqueue_op () |

| | _ PG::queue_op () |

| | _ _ OSD::op_wq.queue () writes the request of the master OSD to the slave OSD processing queue |

Processing workqueue from OSD

OSD::ShardOpWQ::_process ()

| | _ OSD::dequeue_op () |

| | _ ReplicatedPG::do_request () |

| | _ ReplicatedBackend::handle_message () |

| | _ ReplicatedBackend::sub_op_modify () |

| | _ ReplicatedBackend::sub_op_modify_impl () |

| | _ _ register commit callback function class C_OSD_RepModifyCommit () |

| | _ _ register Apply callback function class C_OSD_RepModifyApply () |

| | _ _ ReplicatedPG::queue_transcations () writes FileJournal from OSD |

| | _ ObjectStore::queue_transactions () |

| | _ FileStore::queue_transactions () |

| | _ JournalingObjectStore::_op_journal_transactions () |

| | _ _ FileJournal::submit_entry () writes logs to the writeq queue of FileJournal |

| | _ ReplicatedPG::eval_repop () |

Write thread from FileJournal of OSD

FileJournal::write_thread_entry () is responsible for getting logs to be written from the writeq queue

| | _ _ FileJournal::prepare_multi_write () constitutes a log |

| | _ _ FileJournal::do_write () actually writes logs |

| | _ FIleJournal::queue_completions_thru () |

| | _ _ completion_peek_front () gets the C_JournalAhead class instance |

| | _ finisher- > queue () |

| | _ _ finisher_cond.Signal () starts FileJournal's finisher thread to process C_JournalAhead class instances |

From the C_JournalAhead class of OSD

C_JournalAhead::finish ()

| | _ FileStore::_journaled_ahead () |

| | _ FileStore::queue_op () |

| | _ _ FileStore::op_wq.queue () writes data to the FileStore writer thread queue |

| | _ _ ondisk_finisher.queue (ondisk) calls ReplicatedBackend- > op_commit () callback function |

The processing after writing the log from OSD

C_OSD_RepModifyCommit ()

| | _ finish () |

| | _ ReplicatedBackend::sub_op_modify_commit () |

| | _ _ generate MOSDRepOpReply message CEPH_OSD_FLAG_ONDISK |

| | _ _ ReplicatedPG::send_message_osd_cluster () sends the reply message from OSD to the master OSD |

| | _ OSD::send_message_osd_cluster () |

Write data thread class processing from FileStore of OSD

FileStore::op_wq

| | _ FileStore::op_wq._process () |

| | _ FileStore::_do_op () |

| | _ FileStore::_do_transactions () |

| | _ FileStore::_do_transaction () |

| | _ _ FileStore::_write () is stored in FileStore (written to the specified file) |

| | _ FileStore::op_wq._process_finish () |

| | _ FileStore::_finish_op () |

| | _ _ op_finisher.queue () calls ReplicatedPG::op_applied () callback function |

The processing after the completion of the setting of the OSD

C_OSD_RepModifyApply ()

| | _ finish () |

| | _ ReplicatedBackend::sub_op_modify_applied () |

| | _ _ generate MOSDRepOpReply message CEPH_OSD_FLAG_ACK |

| | _ _ ReplicatedPG::send_message_osd_cluster () sends the reply message from OSD to the master OSD |

| | _ OSD::send_message_osd_cluster () |

3. Master OSD processing sends Reply processing flow from OSD

Processing requests sent from the main OSD side from the OSD

OSD::ms_fast_dispatch ()

| | _ OSD::dispatch_session_waiting () |

| | _ OSD::dispatch_op_fast () |

| | _ OSD::handle_replica_op () |

| | _ OSD::get_pg_or_queue_for_pg () |

| | _ OSD::enqueue_op () |

| | _ PG::queue_op () |

| | _ _ OSD::op_wq.queue () writes the reply of the slave OSD to the master OSD processing queue |

The main OSD handles the reply sent from the OSD side

OSD::ShardOpWQ::_process ()

| | _ OSD::dequeue_op () |

| | _ ReplicatedPG::do_request () |

| | _ ReplicatedBackend::handle_message () |

| | _ ReplicatedBackend::sub_op_modify_reply () |

| | _ ip_op.waiting_for_commit.erase () |

| | _ ip_op.waiting_for_applied.erase () |

| _ _ is empty for ip_op.waiting_for_applied, then ip_op.on_applied- > complete (0) is called, that is, on_all_applied callback function is called for processing. The on_all_applied callback function is registered in the ReplicatedPG.cc::execute_ctx () function and is used to process all copies after they have been dropped. The on_all_applied () function creates a MOSDOpReply message and sets the flags=CEPH_OSD_FLAG_ACK of the message and sends the message to the client

| _ _ is empty for ip_op.waiting_for_commit, then ip_op.on_commit- > complete (0) is called, that is, on_all_commit callback function is called for processing. The on_all_commit callback function is registered in the ReplicatedPG.cc::execute_ctx () function and is used to process all copies after writing the FileJournal. The on_all_commit () function creates a MOSDOpReply message and sets the flags=CEPH_OSD_FLAG_ACK of the message | CEPH_OSD_FLAG_ONDISK and sends the message to the client

The above is all the contents of the article "sample Analysis of Ceph OSD processing client write Operation processing flow". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.