Shulou(Shulou.com)06/01 Report--

Brief introduction

MogileFS is an open source distributed file storage system developed by DangaInteractive, a company owned by LiveJournal. The Danga team has developed a number of well-known open source projects, including Memcached, MogileFS, Perlbal and so on.

Characteristics

1. Application layer: user space file system without special core components

2. No single point: (tracker, mogstore, database (MySQL))

3. Automatic file replication: the smallest unit of copy is not a file, but class

4. Transport neutrality, no special protocol: you can communicate through NFS or HTTP

5. Simple namespace: each file corresponds to a key: used for domain to define namespaces

component

MogileFS:

Tracker: tracking metadata

Database: storing metadata

Storage: storing data

Tracker:

Mogilefsd (daemon), responsibilities:

Replication: file replication between nodes

Deletion: deleting fil

Queryworker: file metadata access requests in response to customer requests

Reaper: replaces file replication requests in the queue after storage failure

Monitor: check the health status of hosts and devices

Database:

Mysql is generally used to store the metadata of mogilefs. It is recommended to use redundancy scheme to ensure availability (MMM,MHA).

Mogilefs provides a data structure management tool, mogdbsetup.

Storage:

Mogstored (process name), a ready mogstored node can be added to the current cluster through the mogadm command

Storage nodes need to define "device dev" as storage space; each device needs to be identified by a unique DevID in the current cluster.

Client:

The client is used to communicate with mogilefs resumes to complete data access.

Install the package:

MogileFS-Server-2.46-2.el6.noarch.rpm

MogileFS-Server-mogilefsd-2.46-2.el6.noarch.rpm

MogileFS-Server-mogstored-2.46-2.el6.noarch.rpm

MogileFS-Utils-2.19-1.el6.noarch.rpm

Perl-MogileFS-Client-1.14-1.el6.noarch.rpm

Perl-Net-Netmask-1.9015-8.el6.noarch.rpm

Perl-Perlbal-1.78-1.el6.noarch.rpm

Architecture:

1. Install mogilefs and database

# ls

MogileFS-Server-2.46-2.el6.noarch.rpm perl-MogileFS-Client-1.14-1.el6.noarch.rpm

MogileFS-Server-mogilefsd-2.46-2.el6.noarch.rpm perl-Net-Netmask-1.9015-8.el6.noarch.rpm

MogileFS-Server-mogstored-2.46-2.el6.noarch.rpm perl-Perlbal-1.78-1.el6.noarch.rpm

MogileFS-Utils-2.19-1.el6.noarch.rpm

# yum localinstall * .rpm-y

# yum install mysql-server mysql- y

two。 The database authorizes remote access and inherits authorization permissions

Mysql > grant all on *. * to root@'192.168.95.%' identified by 'liaobin' with grant option

Mysql > flush privileges

3. Create the database user and authorization for mofilefs

Mysql > grant all on *. * to mguser@'192.168.95.%' identified by 'liaobin'

Mysql > flush privileges

4. Initialize mogilefs database with mogdbsetup, a tool that comes with mogilefs

# mogdbsetup-dbhost=192.168.95.30-dbname=mogilefs-dbrootuser=root-dbrootpass=liaobin-dbuser=mguser-dbpass=liaobin-yes

View the initialized database

Mysql > show databases

+-+

| | Database |

+-+

| | information_schema |

| | mogilefs |

| | mysql |

| | test |

+-+

4 rows in set (0.01sec)

5. Modify the tracker (mogilefsd) configuration file

# vim / etc/mogilefs/mogilefsd.conf

# Enable daemon mode to work in background and use syslog

Daemonize = 1

# Where to store the pid of the daemon (must be the same in the init script)

Pidfile = / var/run/mogilefsd/mogilefsd.pid

# Database connection information

Db_dsn = DBI:mysql:mogilefs:host=192.168.95.30 # configure connection information for database

Db_user = mguser # configure the database user password for mogilefs

Db_pass = liaobin

# IP:PORT to listen on for mogilefs client requests

Listen = 0.0.0.0 7001 # configure the address and port for listening

# Optional, if you don't define the port above.

Conf_port = 7001

# Number of query workers to start by default.

Query_jobs = 10 # how many worker processes are generated for queries

# Number of delete workers to start by default.

Delete_jobs = 1 # how many worker processes are generated for deletion

# Number of replicate workers to start by default.

Replicate_jobs = 5 # how many worker processes are generated for deletion

# Number of reaper workers to start by default.

# (you don't usually need to increase this)

Reaper_jobs = 1

# Number of fsck workers to start by default.

# (these can cause a lot of load when fsck'ing)

# fsck_jobs = 1

# Minimum amount of space to reserve in megabytes

# default: 100

# Consider setting this to be larger than the largest file you

# would normally be uploading.

# min_free_space = 200

# Number of seconds to wait for a storage node to respond.

# default: 2

# Keep this low, so busy storage nodes are quickly ignored.

# node_timeout = 2

# Number of seconds to wait to connect to a storage node.

# default: 2

# Keep this low so overloaded nodes get skipped.

# conn_timeout = 2

# Allow replication to use the secondary node get port

# if you have apache or similar configured for GET's

# repl_use_get_port = 1

6. Start the mogilefsd service

[root@localhost mgfs] # / etc/init.d/mogstored start

7. Add a new partition for each node as storage

# fdisk / dev/sda

.

# partx-a / dev/sda

# mkfs.ext4 / dev/sda5

# mkdir / mgfs

# vim / etc/fstab

# add the following entries

/ dev/sda5 / mgfs ext4 defaults 0 0

# mount-a

# mount

/ dev/sda5 on / mgfs type ext4 (rw)

8. The three servers create storage directories and modify permissions

95.30:

# mkdir / mgfs/dev1

# chown-R / mgfs/dev1

95.34:

# mkdir / mgfs/dev2

# chown-R / mgfs/dev2

95.35:

# mkdir / mgfs/dev3

# chown-R / mgfs/dev3

9. Modify the configuration file of store (mogstored)

# vim / etc/mogilefs/mogstored.conf

Maxconns = 10000

Httplisten = 0.0.0.0pur7500

Mgmtlisten = 0.0.0.0purl 7501

# Note here, just write to the superior directory of dev (1 ~ 2 ~ 3).

Docroot = / mgfs

10. Start the mogstored service

# / etc/init.d/mogstored start

11. Manage with mogadm

Create a storage host:

# mogadm-trackers=192.168.95.30:7001 host add store1-ip=192.168.95.30

# mogadm-trackers=192.168.95.30:7001 host add store2-ip=192.168.95.34

# mogadm-trackers=192.168.95.30:7001 host add store3-ip=192.168.95.35

View the created host:

# mogadm-trackers=192.168.95.30:7001 host list

Create a storage device:

# mogadm-trackers=192.168.95.30:7001 device add store1 1-status=alive

# mogadm-trackers=192.168.95.30:7001 device add store2 2-status=alive

# mogadm-- trackers=192.168.95.30:7001 device add store3 3-- status=alive

View the created storage device:

# mogadm-trackers=192.168.95.30:7001 device list

Create a domain domain:

Mogadm-trackers=192.168.95.30:7001 domain add img

Mogadm-trackers=192.168.95.30:7001 domain add conf

View the created domain:

Mogadm-trackers=192.168.95.30:7001 domain list

Upload file test:

Mogupload-trackers=192.168.95.30:7001-domain=img-key= "1.jpg"-file= "/ root/1.jpg"

View the uploaded file:

# mogfileinfo-trackers=192.168.95.30:7001-domain=img-key= "1.jpg"

-file: 1.jpg

Class: default

Devcount: 2

Domain: img

Fid: 9

Key: 1.jpg

Length: 34892

-http://192.168.95.35:7500/dev3/0/000/000/0000000009.fid

-http://192.168.95.30:7500/dev1/0/000/000/0000000009.fid

Access the test with fid's url:

Normal access

Nginx reverse proxy mogilefs

1. Install nginx and mogilefs modul

Install the development environment:

# yum groupinstall "Development tools"Server Platform Development"-y

# yum install pcre-devel-y

Download and extract the nginx package and the mogilefs package:

Start compiling and installing:

# cd nginx-1.6.2

#. / configure\

-- prefix=/usr/local/nginx\

-- sbin-path=/usr/local/nginx/sbin/nginx\

-- conf-path=/etc/nginx/nginx.conf\

-- error-log-path=/var/log/nginx/error.log\

-- http-log-path=/var/log/nginx/access.log\

-- pid-path=/var/run/nginx/nginx.pid\

-- lock-path=/var/lock/nginx.lock\

-- user=nginx\

-- group=nginx\

-- with-http_ssl_module\

-- with-http_flv_module\

-- with-http_stub_status_module\

-- with-http_gzip_static_module\

-- http-client-body-temp-path=/var/tmp/nginx/client/\

-- http-proxy-temp-path=/var/tmp/nginx/proxy/\

-- http-fastcgi-temp-path=/var/tmp/nginx/fcgi/\

-- http-uwsgi-temp-path=/var/tmp/nginx/uwsgi\

-- http-scgi-temp-path=/var/tmp/nginx/scgi\

-with-pcre-add-module=../nginx_mogilefs_module-1.0.4

# make & & make install

Create a service script for nginx

# vim / etc/init.d/nginx

#! / bin/sh

#

# nginx-this script starts and stops the nginx daemon

#

# chkconfig:-85 15

# description: Nginx is an HTTP (S) server, HTTP (S) reverse\

# proxy and IMAP/POP3 proxy server

# processname: nginx

# config: / etc/nginx/nginx.conf

# config: / etc/sysconfig/nginx

# pidfile: / var/run/nginx.pid

# Source function library.

. / etc/rc.d/init.d/functions

# Source networking configuration.

. / etc/sysconfig/network

# Check that networking is up.

["$NETWORKING" = "no"] & & exit 0

Nginx= "/ usr/local/nginx/sbin/nginx"

Prog=$ (basename $nginx)

NGINX_CONF_FILE= "/ etc/nginx/nginx.conf"

[- f / etc/sysconfig/nginx] & &. / etc/sysconfig/nginx

Lockfile=/var/lock/subsys/nginx

Make_dirs () {

# make required directories

User= `nginx-V 2 > & 1 | grep "configure arguments:" | sed's / [^ *] *-- user=\ ([^] *\). * /\ 1Uniple g'- `

Options= `$ nginx-V 2 > & 1 | grep 'configure arguments:' `

For opt in $options; do

If [`echo $opt | grep'. *-temp-path' `]; then

Value= `echo $opt | cut-d "="-f 2`

If [!-d "$value"]; then

# echo "creating" $value

Mkdir-p $value & & chown-R $user $value

Fi

Fi

Done

}

Start () {

[- x $nginx] | | exit 5

[- f $NGINX_CONF_FILE] | | exit 6

Make_dirs

Echo-n $"Starting $prog:"

Daemon $nginx-c $NGINX_CONF_FILE

Retval=$?

Echo

[$retval-eq 0] & & touch $lockfile

Return $retval

}

Stop () {

Echo-n $"Stopping $prog:"

Killproc $prog-QUIT

Retval=$?

Echo

[$retval-eq 0] & & rm-f $lockfile

Return $retval

}

Restart () {

Configtest | | return $?

Stop

Sleep 1

Start

}

Reload () {

Configtest | | return $?

Echo-n $"Reloading $prog:"

Killproc $nginx-HUP

RETVAL=$?

Echo

}

Force_reload () {

Restart

}

Configtest () {

$nginx-t-c $NGINX_CONF_FILE

}

Rh_status () {

Status $prog

}

Rh_status_q () {

Rh_status > / dev/null 2 > & 1

}

Case "$1" in

Start)

Rh_status_q & & exit 0

, 1

Stop)

Rh_status_q | | exit 0

, 1

Restart | configtest)

, 1

Reload)

Rh_status_q | | exit 7

, 1

Force-reload)

Force_reload

Status)

Rh_status

Condrestart | try-restart)

Rh_status_q | | exit 0

*)

Echo $"Usage: $0 {start | stop | status | restart | condrestart | try-restart | reload | force-reload | configtest}"

Exit 2

Esac

Give executive authority:

# chmod + x / etc/init.d/nginx

Create nginx users and required directories:

# useradd-r nginx

# mkdir / var/tmp/nginx/client-pv

Modify the nginx configuration file to support mogilefs reverse proxy (red font is an addition):

# vim / etc/nginx/nginx.conf

# user nobody

Worker_processes 1

# error_log logs/error.log

# error_log logs/error.log notice

# error_log logs/error.log info

# pid logs/nginx.pid

Events {

Worker_connections 1024

}

Http {

Include mime.types

Default_type application/octet-stream

# log_format main'$remote_addr-$remote_user [$time_local] "$request"'

#'$status $body_bytes_sent "$http_referer"'

#'"$http_user_agent"$http_x_forwarded_for"'

# access_log logs/access.log main

Sendfile on

# tcp_nopush on

# keepalive_timeout 0

Keepalive_timeout 65

# gzip on

Upstream stores {

Server 192.168.95.30:7001

Server 192.168.95.34:7001

Server 192.168.95.35:7001

}

Server {

Listen 80

Server_name localhost

# charset koi8-r

# access_log logs/host.access.log main

Location / {

Root html

Index index.html index.htm

}

Location / img/ {

Mogilefs_tracker stores

Mogilefs_domain img

Mogilefs_pass {

Proxy_pass $mogilefs_path

Proxy_hide_header Content-Type

Proxy_buffering off

}

}

Location / upload/ {

Mogilefs_tracker stores

Mogilefs_domain img

Mogilefs_methods PUT GET DELETE

Mogilefs_pass {

Proxy_pass $mogilefs_path

Proxy_hide_header Content-Type

Proxy_buffering off

}

}

# error_page 404 / 404.html

}

Test the nginx agent with path access:

Upload another file to test:

# mogupload-trackers=192.168.95.30:7001-domain=img-key= "2.jpg"-file= "/ root/2.jpg"

Test delete files:

# curl-X DELETE http://192.168.95.34/upload/1.jpg

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.