Shulou(Shulou.com)06/02 Report--

This article introduces how to understand the communication architecture between Apache Mesos modules. The content is very detailed. Interested friends can use it for reference. I hope it will be helpful to you.

1. Introduction

Mesos is mainly composed of four components, namely mesos-master,mesos-save,scheduler and Executor, and each component uses actor model based on protocal buffer to communicate (using open source library libprocess). In other words, each module is a server (actually a socket server) that listens to messages from other modules, and once a message is received, the corresponding message handling function is called for processing.

two。 Overview of Communication Framework

For a computing framework (such as Hadoop,Spark, etc.), if you want to connect to Mesos, you need to write two components, FrameworkSchduler and FrameworkExecutor, which implement Scheduler and Executor interfaces respectively, and connect to Mesos through SchedulerDriver and ExecutorDriver, respectively. For example, the black dotted line in the figure indicates that the relationship between these components is generated through function calls.

Other components, namely mesos-master,mesos-save,SchedulerProcess and ExecutorProcess, communicate through a messaging mechanism (using the libprocess open source library).

(1) [SchedulerProcess and mesos-master]: mesos-master allocates resources for each framework, which are passed directly to SchedulerProcess, and then SchedulerProcess calls the relevant functions of FrameworkScheduler. FrameworkScheduler allocates these resources to tasks in the framework and returns them to mesos-master, which is forwarded by mesos-master to the corresponding mesos-slave.

(2) [SchedulerProcess and Mesos-slave]: if the address of the Mesos-slave is saved in the SchedulerProcess, the relevant message is sent directly to the Mesos-slave and does not have to be forwarded by the mesos-master.

(3) [Mesos-master and Mesos-slave]: Mesos-master manages mesos-slave, such as monitoring the health status of slave.

(4) [Mesos-slave and ExecutorProcess]: Mesos-slave is responsible for managing the executor of each framework and allocating resources to executor, etc.

The details are shown in the following figure.

3. Message types and their handling functions

3.1 message types and handling functions of Mesos-Slave

3.2message types and handling functions of Mesos-master

Message types and handling functions of 3. 3 scheduler

Located in sched/sched.cpp and implemented by SchedulerProcess/MesosSchedulerDriver

3.4 message types and handling functions of Executor

Located in the file exec/exec.cpp and implemented in ExectorProcess

On how to understand the communication architecture between Apache Mesos modules is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.