Shulou(Shulou.com)06/01 Report--

There is no need to say much about the rapid development of Docker. There are enough tools for managing Docker and orchestrating Docker, but they always feel that they have not touched the pain point, so there is a need for the existence of mesos in the data center. What is mesos and what mesos can do will be briefly introduced in this article.

Here first put a piece of my own understanding of mesos and the use of ideas, if biased also hope to correct.

According to the picture above, we incorporate all the physical machines into the mesos publishing system, so that we abstract all the physical machine resources together, so that we do not have to worry about the use of CPU, memory, and hard disk, because when an application needs more resources, we can add them directly and dynamically. I think this alone is enough to make us eager to try, and more importantly, we don't need to purchase servers with the same hardware configuration.

Of course, the previous point is not enough to explain what mesos is and what it can do. in the author's opinion, it is absolutely impossible to understand one thing on paper, so let's first put the environment together and use it, so I think we can have a deeper understanding of Mesos.

Installation:

System environment: Centos7

Mesos cluster

172.16.51.72 mesos-master

172.16.51.72 mesos-slaver

172.16.51.71 mesos-slaver

172.16.51.73 mesos-slaver

Zookeeper cluster

172.16.51.72 1

172.16.51.71 2

172.16.51.73 3

Note: iptables,selinux is closed, of course.

Installation method rpm package

Rpm source

Rpm-Uvh http://repos.mesosphere.com/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm

Install mesos,marathon,zookeeper

Yum-yinstall mesos marathonyum-yinstall mesosphere-zookeeper

Configure zookeeper Cluster

The myid of 72pence71pm73 is configured in turn as 1jcm2jp3jmid file does not exist by default.

Cat / var/lib/zookeeper/myid1

Then configure the zookeeper profile

Each zookeeper node adds the following three lines

Cat / etc/zookeeper/conf/zoo.cfgserver.1=172.16.51.72:2888:3888server.2=172.16.51.71:2888:3888server.3=172.16.51.73:2888:3888

Then start zookeeper in turn

Systemctlstart zookeeper

Check to see if you are listening

Lsof-iRu 2181

Configure zookeeper information for mesos (both master and slaver)

Zk://172.161.51.72:2181172.16.51.71:2181172.16.51.73:2181/mesos

Then start the masterside

Service mesos-master restart

Start the slaver terminal

Service mesos-slave restart

It's very simple.

Then verify that mesos,marathon is installed successfully

Access ports 5050 and 8080 of the mesos-master host, as shown in the following figure

Note: if the first installation is completed, the corresponding web page should not have any tasks, such as the test above

There may be questions here, and then what?

Let's first take a look at the architecture of Mesos.

According to the above diagram and combined with our installed environment, we have established three mesos-slaver nodes, one mesos-master end node (in fact, the mesos-master side can also make a cluster)

Let's put aside the Hadoop,MPI scheduler above.

Zookeeper is used for service discovery (here we make it in cluster mode for high availability), so both mesos-slaver and mesos-master nodes need to set the configuration information of zookeeper. Through zookeeper,mesos-master, they will ask which hosts mesos-slaver is and what its related information is, and vice versa.

Then mesos-master will only mesos-slaver what they need to do, and how do they know what mesos-slaver can do?

Let's look at the picture below.

According to the figure above, mesos-slaver will tell mesos-master how many resources it has, such as how much cpu, memory, etc., and then master-slaver decides how to allocate resources and tasks, so how do we request resources and release tasks for mesos? then we need to know what a framework is. Let's explain the two main frameworks, marathon,chronos.

What is a frame? Is used to ask for the source, look at the specific examples, you will have a certain understanding.

Marathon is a private PaaS platform built on Mesos. It can automatically handle hardware or software failures, and ensure that each application is "always online" (for example, if you set up three instances of an application in marathon, then one of them dies, marathon will immediately start an instance and maintain the three instances). Personally, it overlaps with kubernetes. Of course, kubernetes is more powerful, because the ability of choreography is stronger.

Default listening port: 8080

Main functions: create resident tasks, task expansion, etc. (such as the increase in the number of instances)

Create JOB

As shown in the figure above, we create a task with ID as test, and allocate 0.1% CPU (default), 16MB memory (default), OMB hard disk (default), an instance (default), and then this instance executes the command python-m SimpleHTTPServer, and then click "create". After the creation is successful, mesos-master randomly asks mesos-slaver to create an instance.

To facilitate verification, we can select the number of instances as 3, and then check to see if we are listening.

And verify it.

Note: the name is arbitrary, but it does not conflict with the existing ID name

Then take a look at the basic settings

ID Settin

The name is optional and does not conflict with the existing ID name

CPU, memory, hard disk, number of instances (custom)

CPU, memory, hard disk, number of instances (custom)

Executed command

For example, using Python to set up a simple web service

Python-m SimpleHTTPServer

Settings related to the Docker container

Network, data volume, etc.

Environmental variables such as

JAVA_HOME= xxx

Label

Custom tags for classification

Health monitoring

There are three ways to detect command,http,tcp.

Other options (optional settings)

Such as port

There is one last restriction, constraint.

For example, limit which node starts this task.

After the creation is successful, click on the specific task and have relevant information, such as host port, etc.

The mesos-master side also has corresponding job information.

Here is the information about marathon

Then the three main actions are to extend (for example, one instance to three instances), restart, suspend, and destroy

Note: because of the test task, the written command is python-m SimpleHTTPServer, so port 8000 is listened by default. Since there are only three nodes, a maximum of three instances can be enabled. Although the scale (size) can be set to more than 3, new instances will continue to be activated, and then continue to fail and keep starting.

Therefore, starting more than three instances should ensure that the ports occupied are different.

For example, ports are random

Python-m SimpleHTTPServer $PORT

Because the creation task supports Docker, we might as well imagine that we can directly mirror it in an existing docker environment.

Chronos is a distributed job scheduler that supports complex topography structures. It can be a substitute for cron.

Products to provide better fault tolerance.

Chronos is just a planned task, but distributed.

Installation

Yum-y install chronos

Servicechronos start

Default listening 4400

such as

Cat / tmp/test2.txtSat Dec26 17:08:10 CST 2015

Conclusion: since it is the beginning, let's come here for a long time, and the author is still studying it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.