Shulou(Shulou.com)05/31 Report--

This article will explain in detail the case study of the problems encountered by rbdmap in ceph. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Rbdmap running on centos6.5:

[root@mon0 ceph] # cat / etc/init.d/rbdmap #! / bin/bash## rbdmap Ceph RBD Mapping## chkconfig: 2345 70 7 hours description: Ceph RBD Mapping### BEGIN INIT INFO# Provides: rbdmap# Required-Start: $network $remote_fs# Required-Stop: $network $remote_fs# Should-Start: ceph# Should-Stop: ceph# X-Start-Before: $x-display-manager# Default-Start: 234 "Default-Stop: 01" Short-Description: Ceph RBD Mapping# Description: Ceph RBD Mapping### END INIT INFODESC= "RBD Mapping:" RBDMAPFILE= "/ etc/ceph/rbdmap". / lib/lsb/init-functionsdo_map () {if [!-f "$RBDMAPFILE"]; then # log_warning_msg "$DESC: No $RBDMAPFILE found." Exit 0 fi # Read / etc/rbdtab to create non-existant mapping RET=0 while read DEV PARAMS; do case "$DEV" in "" |\ # *) continue;; * / *) *) DEV=rbd/$DEV Esac # log_action_begin_msg "${DESC}'${DEV}'" newrbd= "" MAP_RV= "" RET_OP=0 OIFS=$IFS IFS=',' for PARAM in ${PARAMS [@]} Do CMDPARAMS= "$CMDPARAMS-- $(echo $PARAM | tr'='')" done IFS=$OIFS if [!-b / dev/rbd/$DEV]; then MAP_RV=$ (rbd map $DEV $CMDPARAMS 2 > & 1) if [$?-eq 0] Then newrbd= "yes" else RET=$ (${RET} + $?) RET_OP=1 fi fi # log_action_end_msg ${RET_OP} "${MAP_RV}" if ["$newrbd"] Then # # Mount new rbd MNT_RV= "" mount-- fake / dev/rbd/$DEV > > / dev/null 2 > & 1\ & & MNT_RV=$ (mount-v / dev/rbd/$DEV 2 > & 1) [- n "${MNT_RV}" ] & & log_action_msg "mount: ${MNT_RV}" # # post-mapping if [- x "/ etc/ceph/rbd.d/$ {DEV}"] Then # log_action_msg "RBD Running post-map hook'/ etc/ceph/rbd.d/$ {DEV}'" / etc/ceph/rbd.d/$ {DEV} map "/ dev/rbd/$ {DEV}" fi fi done

/ dev/null 2 > & 1; then for DEV in / dev/rbd [0-9] *; do # # pre-unmapping for L in $(find / dev/rbd-type l) Do LL= "${L##/dev/rbd/}" if ["$(readlink-f $L)" = "${DEV}"]\ & & [- x "/ etc/ceph/rbd.d/$ {LL}"] Then log_action_msg "RBD pre-unmap:'${DEV} 'hook' / etc/ceph/rbd.d/$ {LL}'" / etc/ceph/rbd.d/$ {LL} unmap "$L" break fi Done # log_action_begin_msg "RBD un-mapping:'${DEV}'" UMNT_RV= "" UMAP_RV= "" RET_OP=0 MNT=$ (findmnt-- mtab-- source ${DEV}-- noheadings | awk | '{print $1'}) if [- n "${MNT}"] Then # log_action_cont_msg "un-mounting'${MNT}'" UMNT_RV=$ (umount "${MNT}" 2 > & 1) fi if mountpoint-Q "${MNT}"; then # # Un-mounting failed. RET_OP=1 RET=$ (${RET} + 1) else # # Un-mapping. UMAP_RV=$ (rbd unmap $DEV 2 > & 1) if [$?-ne 0]; then RET=$ ((${RET} + $?)) RET_OP=1 fi fi # log_action_end_msg ${RET_OP} "${UMAP_RV}" [- n "${UMNT_RV}"] & & log_action_msg "${UMNT_RV}" done fi Exit ${RET}} case "$1" in start) do_map ; stop) do_unmap;; restart | force-reload) $0 stop $0 start;; reload) do_map;; status) rbd showmapped;; *) log_success_msg "Usage: rbdmap {start | stop | restart | force-reload | reload | status}" exit 1;; esac

After modifying some log that are not available on centos, this script is still problematic to use, as described below:

1. After only rbdmap one block to get / dev/rbd0, you can use the rbdmap script to map/unmap / dev/rbd0 normally.

two。 When / dev/rbd0 is formatted and mounted to a directory, and then rbdmap is used, the system will hang on unmounting filesystem when shutting down, and can only be forcibly powered off. After rebooting, the do_map () function of rbdmap is executed, and everything is fine.

After troubleshooting, it is found that when / dev/rbd0 is mounted to the directory, rbdmap will not execute the do_unmap () function, even if an explicit umount operation is added to the function.

Think of a compromise, explicitly add the umount operation to the stop function in the rbdmap stop high priority service, and everything will be fine when you restart.

Let's first take a look at the start and stop order of ceph and rbdmap:

Head rbdmappings network network # rbdmap Ceph RBD Mapping## chkconfig: 2345 70 7 accounting description: Ceph RBD Mappinghead CEPHALUMUBING sh # Start/stop ceph daemons# chkconfig:-60 80 accounts.BEGIN INIT INFO# Provides: ceph# Default-Start:# Default-Stop:# Required-Start: $remote_fs $named $network $time# Required-Stop: $remote_fs $named $network $time

You can see that ceph starts before rbdmap, and rbdmap stops before ceph. If you use nfs and use the block devices mapped by rbdmap, take a look at the start and stop order of nfs:

Head / EtcUnip init.dAccord nfssteps incense binds # nfs This shell script takes care of starting and stopping# the NFS services.## chkconfig:-30 6 months description: NFS is a popular protocol for file sharing across networks.# This service provides NFS server functionality, which is\ # configured via the / etc/exports file.# probe: true

Nfs is the first of the three to start and stop. So add the umount command to the stop function of nfs:

Umount / mnt/nfsumount / mnt/nfs2

Add the mount command to rbdmap:

Mount / dev/rbd0-o rw,noexec,nodev,noatime,nobarrier,discard / mnt/nfsmount / dev/rbd1-o rw,noexec,nodev,noatime,nobarrier,discard / mnt/nfs2

The settings of / etc/ceph/rbdmap are as follows:

Backup1/backup.imgbackup2/backup.img

Remember that there was no problem with the test at ceph- 0.80, and the above problem occurred at 0.87.1.

This is the end of the article on "case study of rbdmap problems in ceph". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.