Shulou(Shulou.com)06/01 Report--

Background

Container technology, represented by Docker and Rocket, is becoming more and more popular. It changes the way companies and users create, publish and run distributed applications, and will bring its due value to the cloud computing industry in the next five years. Its allure lies in:

1) Resource independence and isolation

Resource isolation is the most basic requirement of cloud computing platform. Docker restricts hardware resources and software running environment through Linux namespace and cgroup, which is isolated from other applications on the host and does not affect each other. Different applications or services load or unload "ships" in units of "container". On "container ships" (hosts or clusters running container), tens of thousands of "containers" are arranged neatly, and different companies and different types of "goods" (programs, components, operating environment, dependencies) for running applications remain independent.

2) consistency of environment

After the development engineer completes the application development, build a docker image, and the container created based on this image looks like a container with various "bulk cargo" (programs, components, running environment, dependencies) needed to run the application. No matter where the container is: development environment, test environment, production environment, you can ensure that the type and number of "goods" in the container are exactly the same, the software package will not be missing in the test environment, the environment variables will not forget to configure in the production environment, and the development environment and the production environment will not cause the application to run abnormally because of the dependency of different versions installed. This consistency benefits from the fact that the "build docker image" has been sealed into the "container", and each link is transporting this complete "container" that does not need to be split and merged.

3) lightweight

Compared with the traditional virtualization technology (VM), the performance loss of using docker on cpu, memory, disk IO, network IO has the same level or even better. Container has received a lot of praise for its rapid creation, startup and destruction.

4) Build Once, Run Everywhere

This feature really attracts a lot of people. When "goods" (applications) migrate and exchange between "cars", "trains" and "ships" (private cloud, public cloud and other services), they only need to move "containers" (docker container) that meet the standard specifications and loading and unloading methods, thus reducing the time-consuming and laborious manual "loading and unloading" (online and offline applications), resulting in huge time and labor cost savings. This makes it possible for only a few container clusters used on the super-large-scale loading line for operation and maintenance personnel in the future, just as a few machine operators can load and unload a 10,000-class container ship in a few hours after the 1960s.

Container technology is also widely used in the field of database. Its "Build Once, Run Everywhere" feature greatly reduces the time spent installing a configuration database environment, because even for DBA, which has been working on databases for many years, installing a configuration database environment is still a seemingly simple but often unsmooth task. Of course, other advantages of container technology are also well applied in the use of databases.

As an excellent domestic distributed NewSQL database, SequoiaDB has been recognized by more and more users. Taking Docker as an example, this paper focuses on how to use Dockerfile to quickly build SequoiaDB images, and how to use containers to quickly build and start SequoiaDB clusters for application systems.

Build a SequoiaDB image

How to install docker and configure the image warehouse is not the focus of this article, there are many related technical articles online. It should be noted that the Aliyun image repository is used in this article, because the speed of uploading the image to the official Docker repository is really unflattering. For more information on how to register and use the Ali Cloud image repository, please refer to the article (http://www.jb51.net/article/123101.htm).

STEP 1: create a Dockerfile with just a few lines of simple instructions as follows

* # Sequoiadb DOCKERFILES PROJECT--This is the Dockerfile for Sequoiadb 2.8.4REQUIRED FILES TO BUILD THIS IMAGE-- (1) sequoiadb-2.8.4-linuxx8664-enterprise-installer.run (2) installSDB.shHOW TO BUILD THIS IMAGE- -Put all downloaded files in the same directory as this DockerfileRun: $sudo docker build-t sequoiadb:2.8.4. Pull base imageFROM ubuntuEnvironment variables required for this buildENV INSTALLBINFILE= "sequoiadb-2.8.4-linuxx8664-enterprise-installer.run"\ INSTALLSDBSCRIPT= "installSDB.sh"\ INSTALLDIR= "/ opt/sequoiadb" Copy binariesADD $INSTALLBINFILE $INSTALLSDBSCRIPT $INSTALLDIR/Install SDB software binariesRUN chmod $INSTALLDIR/$INSTALLSDBSCRIPT\ & $INSTALLDIR/$INSTALLSDBSCRIPT\ & & rm $INSTALLDIR/$INSTALLSDBSCRIPT**

The installSDB.sh script is as follows:

Chmod 755 $INSTALL_DIR/$INSTALL_BIN_FILE$INSTALL_DIR/$INSTALL_BIN_FILE-- mode unattended rm $INSTALL_DIR/$INSTALL_BIN_FILEecho 'service sdbcm start' > > / root/.bashrc

It is important to note that this example uses Sequoiadb Enterprise Edition 2.8.4. You can also download the community version (select the tar package, download and decompress it) from the Giant Sequoia official website to replace the media name in this example.

Download address of Giant Sequoia official website: http://download.sequoiadb.com/cn/

STEP 2: create a mirror

Root users execute:

Docker build-t sequoiadb:2.8.4.

If you are an ordinary user, you need to use sudo:

Sudo docker build-t sequoiadb:2.8.4.

STEP3: log in to Ali Cloud Image Warehouse

Docker login-username=xxx registry.cn-hangzhou.aliyuncs.com

Xxx is your account registered with Aliyun.

STEP4: view local sequoiadb image id

Docker images

STEP5: Mark the local image and put it in the Aliyun repository

Docker tag 04dc528f2a6f registry.cn-hangzhou.aliyuncs.com/508mars/sequoiadb:latest

04dc528f2a6f is the author's local sequoiadb image id, and the new tag format has certain requirements. Registry.cn-hangzhou.aliyuncs.com is the address of Aliyun's warehouse, 508mars is the author's user name in Aliyun, sequoiadb is the image name, and latest is tag.

STEP6: submit sequoiadb images to the mirror library

Docker push registry.cn-hangzhou.aliyuncs.com/508mars/sequoiadb:latest

Start SequoiaDB Cluster using Container

The network of Docker adopts bridge mode by default. Containers with bridge mode have the following characteristics:

1) containers of the same host can communicate with each other by ping

2) the containers of different hosts differ from each other in ping

However, the SequoiaDB cluster requires that all nodes can communicate with each other, so if the containers running SequoiaDB are running on different hosts, the default network mode of docker is obviously not appropriate. There are many ways to solve the connectivity problem between different host containers. This article only introduces the solution of weave virtual network, because weave also provides a function of DNS server. With this feature, you no longer need to modify the / etc/hosts inside each container when deploying SequoiaDB clusters using containers, which greatly simplifies the steps of automatic deployment.

STEP1: installing weave network

Curl-s-L git.io/weave-o / usr/local/bin/weavechmod aquix / usr/local/bin/weave

Need to be installed in all hosts, the author uses three virtual machines as hosts: sdb1, sdb2 and sdb3.

STEP2: start the weave network

Weave launch

The weave image is downloaded the first time it starts.

STEP3: download sequoiadb image from Ali Cloud Warehouse

Docker pull registry.cn-hangzhou.aliyuncs.com/508mars/sequoiadb

STEP4: create mount volumes for docker on all hosts

Cd / home/sdbadminmkdir-p data/disk1 data/disk2 data/disk3mkdir-p conf/localchmod-R 777 datachmod-R 777 conf

The location of the mount volume can be customized, but generally speaking, you need to create two types of mount volume, one for storing collection data, such as data/disk1, data/disk2, and data/disk3 in this case, and one for storing node configuration information, such as conf/local in this example. In this way, even if the container is mistakenly deleted, you can still start a new container to play the role of the mistakenly deleted container.

STEP5: launch the container

Sdb1:

Weave stopweave launcheval $(weave env) docker run-dit-- name sdbserver1-p 11810 purl 11810-v / home/sdbadmin/data:/data-v / home/sdbadmin/conf/local:/opt/sequoiadb/conf/local registry.cn-hangzhou.aliyuncs.com/508mars/sequoiadb

Sdb2:

Weave stopweave launch 192.168.17.132eval $(weave env) docker run-dit-- name sdbserver2-p 11810 purl 11810-v / home/sdbadmin/data:/data-v / home/sdbadmin/conf/local:/opt/sequoiadb/conf/local registry.cn-hangzhou.aliyuncs.com/508mars/sequoiadb

Sdb3:

Weave stopweave launch 192.168.17.132eval $(weave env) docker run-dit-- name sdbserver3-p 11810 purl 11810-v / home/sdbadmin/data:/data-v / home/sdbadmin/conf/local:/opt/sequoiadb/conf/local registry.cn-hangzhou.aliyuncs.com/508mars/sequoiadb

Where 192.168.17.132 is the IP address of sdb1 and 11810 is the exposed cluster access port. The volume where the host stores the node configuration information must be mounted to the / opt/sequoiadb/conf/local directory of the container, and the volume storing table data can be mounted to a user-defined directory, but once the cluster is created, it cannot be changed. You must specify the machine name when starting the container, because after building the cluster, the machine name will be saved in the system table of SequoiaDB, and the inconsistency between the machine name of the node and the system table will make it impossible to join the cluster. In scenarios where weave is used, it is recommended to use the-- name option instead of-- hostname to set the machine name. The latter prevents weave from adding the machine name to the DNS server, and weave automatically sets the machine name according to the value of-- name, and adds the weave.local domain name after the machine name and adds it to the DNS server.

STEP6: copy the script to create the SequoiaDB cluster into the container

Docker cp create_cluster.js sdbserver1:/data

The create_cluster.js content is as follows:

Var array_hosts = ["sdbserver1.weave.local", "sdbserver2.weave.local", "sdbserver3.weave.local"]; var array_dbroot = ["/ data/disk1/sequoiadb/database", "/ data/disk2/sequoiadb/database", "/ data/disk3/sequoiadb/database"]; var port_sdbcm = "11790"; var port_temp_coord = "18888" Var cataloggroup = {gname: "SYSCatalogGroup", gport: "11820", ghosts: ["sdbserver1.weave.local", "sdbserver2.weave.local", "sdbserver3.weave.local"]}; var array_coordgroups = [{gname: "SYSCoord", gport: "11810", ghosts: ["sdbserver1.weave.local", "sdbserver2.weave.local", "sdbserver3.weave.local"]}] Var array_datagroups = [{gname: "dg1", gport: "11830", ghosts: ["sdbserver1.weave.local", "sdbserver2.weave.local", "sdbserver3.weave.local"], goptions: {transactionon:true}}, {gname: "dg2", gport: "11840", ghosts: ["sdbserver1.weave.local", "sdbserver2.weave.local", "sdbserver3.weave.local"], goptions: {transactionon:true}}, {gname: "dg3" Gport: "11850", ghosts: ["sdbserver1.weave.local", "sdbserver2.weave.local", "sdbserver3.weave.local"], goptions: {transactionon:true}}] Var array_domains = [{dname: "allgroups", dgroups: ["dg1", "dg2", "dg3"], doptions: {AutoSplit:true}}]; println ("start interim coordination node"); var oma = new Oma (array_coordgroups [0] .ghosts [0], port_sdbcm); oma.createCoord (port_temp_coord, array_dbroot [0] + "/ coord/" + port_temp_coord); oma.startNode (port_temp_coord) Println ("create cataloged node group:" + cataloggroup.ghosts [0] + "+ cataloggroup.gport+"+ array_dbroot [0] +" / cata/ "+ cataloggroup.gport); var db = new Sdb (array_coordgroups [0] .ghosts [0], port_temp_coord); db.createCataRG (cataloggroup.ghosts [0], cataloggroup.gport, array_dbroot [0] +" / cata/ "+ cataloggroup.gport); var cataRG = db.getRG (" SYSCatalogGroup ") For (var i in cataloggroup.ghosts) {if (iCatalog 0) {continue;} println ("create catalog node:" + cataloggroup.ghosts [I] + "+ cataloggroup.gport+"+ array_dbroot [0] +" / cata/ "+ cataloggroup.gport); var catanode = cataRG.createNode (cataloggroup.ghosts [I], cataloggroup.gport, array_dbroot [0] +" / cata/ "+ cataloggroup.gport); catanode.start () } println ("create orchestration node group"); var db = new Sdb (array_coordgroups [0] .ghosts [0], port_temp_coord); var coordRG = db.createCoordRG () For (var i in array_coordgroups) {for (var j in array_ coordgroups [I] .ghosts) {println ("create a coordination node group:" + array_ coordgroups [I] .ghosts [j] + "+ array_ coordgroups [I] .gport +"+ array_dbroot [0] +" / coord/ "+ array_ coordgroups [I] .gport) CoordRG.createNode (array_ coordgroups [I] .ghosts [j], array_ coordgroups [I] .gport, array_dbroot [0] + "/ coord/" + array_ coordgroups [I] .gport);} coordRG.start (); println ("delete temporary coordination nodes") var oma = new Oma (array_coordgroups [0] .ghosts [0], port_sdbcm); oma.removeCoord (port_temp_coord) Println ("create a data node group") var db = new Sdb (array_coordgroups [0] .ghosts [0], array_coordgroups [0] .gport); var kryptonite (var i in array_datagroups) {var dataRG = db.createRG (array_ datagroups [I] .gname) For (var j in array_ datagroups [I] .ghosts) {println ("create data nodes:" + array_ datagroups [I] .gname + "+ array_ datagroups [I] .ghosts [j] +" + array_ datagroups [I] .gport + "+ array_ dbrootks [k] +" / data/ "+ array_data groups [I] .gport +" + array_ datagroups [I] .goptions) DataRG.createNode (array_ datagroups [I] .ghosts [j] Array_ datagroups [I] .gport, array_ dbroot [k] + "/ data/" + array_ datagroups [I] .gport, array_ datagroups [I] .goptions) } dataRG.start (); kryptonite;} println ("create domain"); var db = new Sdb (array_coordgroups [0] .ghosts [0], array_coordgroups [0] .gport) For (var i in array_domains) {println ("create domain:" + array_ domains.dname + "" + array_ domains.dgroups + "" + array_ domains.doptions) db.createDomain (array_ domains.dname, array_ domains.dgroups, array_ domains.doptions);}

STEP7: create a SequoiaDB cluster

Docker exec sdbserver1 su-sdbadmin-c "sdb-f / data/create_cluster.js"

At this point, the SequoiaDB cluster is created and started, and the cluster will start automatically when the container is started later.

Conclusion

SequoiaDB uses container technology to achieve rapid cluster deployment, which greatly simplifies the difficulty of installation and deployment for beginners. Later, the author will do some optimization in the production of SequoiaDB image, because the current image is a bit large, the main source is that using ADD or COPY command to copy the installation media to the Docker container will generate a new image image1. Although the installation media is deleted in the resulting image image2, it is on top of the image1, so the size of the image2 still includes the installation media. It is best to use ADD to copy the tar package (ADD will extract it automatically) or something like this: RUN mkdir-p / usr/src/things\ & & curl-SL http://example.com/big.tar.xz\ | tar-xJC / usr/src/things\ & & make-C / usr/src/things all

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.