Ceph-deploy add / remove mon nodes
1. Introduction
I really have relatively few mon operations. I have never performed mon add and delete operations in an online environment anyway. Ha, in order to prepare for a rainy day, I should practice my hands to understand it. This article is based on the ceph-deploy tool (I have tried it manually anyway-it is not difficult, you can try it yourself if you are interested, and add the manual operation document later if you need to leave a message)
1.1 operating system # # this article operating system Ubuntu 16.04amd64# lsb_release-a1.2 ceph version
Ceph Luminous 12.2.5
2. Delete mon
Why delete mon first? because my environment has just deployed three mon. Delete two mon first.
2.1 Delete # ceph-deploy-- username cstor mon destroy mon2 [ceph _ deploy.conf] [DEBUG] found configuration file at: / home/cstor/ .cephdeploy.conf[ ceph _ deploy.cli] [INFO] Invoked (1.5.39): / usr/bin/ceph-deploy-- username cstor mon destroy mon2 [ceph _ deploy.cli] [INFO] ceph-deploy options: [ceph _ deploy.cli] [INFO] username: cstor [ Ceph_deploy.cli] [INFO] verbose: False[ceph _ deploy.cli] [INFO] overwrite_conf: False[ceph _ deploy.cli] [INFO] subcommand: destroy[ceph _ deploy.cli] [INFO] quiet: False[ceph _ deploy.cli] [INFO] cd_conf : [ceph_deploy.cli] [INFO] cluster: CEO [ceph _ deploy.cli] [INFO] mon: ['mon2'] [ceph_deploy.cli] [INFO] func: [ceph_deploy.cli] [INFO] ceph_conf : None[ceph _ deploy.cli] [INFO] default_release: False[ceph _ deploy.mon] [DEBUG] Removing mon from mon2cstor@mon2's password: [mon2] [DEBUG] connection detected need for sudocstor@mon2's password: [mon2] [DEBUG] connected to host: cstor@mon2 [mon2] [DEBUG] detect platform information from remote host [mon2] [DEBUG] detect machine type [mon2] [DEBUG] find the location of an executable [mon2] [DEBUG] get Remote short hostname [mon2] [INFO] Running command: sudo ceph-- cluster=ceph-n mon. -k / var/lib/ceph/mon/ceph-mon2/keyring mon remove mon2 [mon2] [WARNIN] removing mon.mon2 at XXX.XXX.XXX.XXX:6789/0, there will be 2 monitors [mon2] [INFO] polling the daemon to verify it stopped [mon2] [INFO] Running command: sudo systemctl stop ceph-mon@mon2.service [mon2] [INFO] Running command: sudo mkdir-p / var/lib/ceph/mon-removed [mon2] [DEBUG] move old monitor data2.2 configuration modification
I don't know if there will be any impact if I don't modify the configuration (it's better to modify it)-- mon_initial_members and mon_host (to modify all the configurations in the cluster-although troublesome, but still need to be operated)
# cat / etc/ceph/ceph.conf...... Mon_initial_members = mon0, mon1 mon_host = XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX.. 3, add mon
Ha, after deleting mon, now start to add mon (start burying pit)
3.1 configuration modification # # when you modify the configuration file, you need to unify the configuration file of the cluster # cat / etc/ceph/ceph.conf. Mon_initial_members = mon0, mon1,mon2 mon_host = XXX.XXX.XXX.XXX,XXX.XXX.XXX.XXX XXX.XXX.XXX.XXX... 3.2 add mon# ceph-deploy-- username cstor mon create mon2 [ceph _ deploy.conf] [DEBUG] found configuration file at: / home/cstor/ .cephdeploy.conf[ ceph _ deploy.cli] [INFO] Invoked (1.5.39): / usr/bin/ceph-deploy-- username cstor mon create mon2 [ceph _ deploy.cli] [INFO] ceph-deploy options: [ceph _ deploy.cli] [ INFO] username: cstor[ceph _ deploy.cli] [INFO] verbose: False[ceph _ deploy.cli] [INFO] overwrite_conf: False[ceph _ deploy.cli] [INFO] subcommand: create[ceph _ deploy.cli] [INFO] quiet: False [ceph _ deploy.cli] [INFO] cd_conf: [ceph_deploy.cli] [INFO] cluster: CEO [ceph _ deploy.cli] [INFO] mon: ['mon2'] [ceph_deploy.cli] [INFO] func: [ceph_deploy .cli] [INFO] ceph_conf: None[ceph _ deploy.cli] [INFO] keyrings: None[ceph _ deploy.cli] [INFO] default_release: False[ceph _ deploy.mon] [DEBUG] Deploying mon Cluster ceph hosts mon2[ceph _ deploy.mon] [DEBUG] detecting platform for host mon2... cstor@mon2's password: [mon2] [DEBUG] connection detected need for sudocstor@mon2's password: [mon2] [DEBUG] connected to host: cstor@mon2 [mon2] [DEBUG] detect platform information from remote host [mon2] [DEBUG] detect machine type [mon2] [DEBUG] find the location of an executable[ceph _ deploy.mon] [INFO] distro info: Ubuntu 16.04 xenial [mon2] [DEBUG] determining If provided host has same hostname in remote [mon2] [DEBUG] get remote short hostname [mon2] [DEBUG] deploying mon to mon2 [mon2] [DEBUG] get remote short hostname [mon2] [DEBUG] remote hostname: mon2 [mon2] [DEBUG] write cluster configuration to / etc/ceph/ {cluster} .conf [mon2] [DEBUG] create the mon path if it does not exist [mon2] [DEBUG] checking for done path: / var/lib/ceph/mon/ceph-mon2/done [mon2] [DEBUG] done path does not exist / var/lib/ceph/mon/ceph-mon2/done [mon2] [INFO] creating keyring file: / var/lib/ceph/tmp/ceph-mon2.mon.keyring [mon2] [DEBUG] create the monitor keyring file [mon2] [INFO] Running command: sudo ceph-mon-cluster ceph--mkfs-I mon2-keyring / var/lib/ceph/tmp/ceph-mon2.mon.keyring-setuser 64045-setgroup 64045 [mon2] [INFO] unlinking keyring file / var/lib / ceph/tmp/ceph-mon2.mon.keyring [mon2] [DEBUG] create a done file to avoid re-doing the mon deployment [mon2] [DEBUG] create the init path if it does not exist [mon2] [INFO] Running command: sudo systemctl enable ceph.target [mon2] [INFO] Running command: sudo systemctl enable ceph-mon@mon2 [mon2] [INFO] Running command: sudo systemctl start ceph-mon@mon2 [mon2] [INFO] Running command: sudo ceph--cluster=ceph-- admin-daemon / var/ Run/ceph/ceph-mon.mon2.asok mon_ status [mon2] [DEBUG] * [mon2] [DEBUG] status for monitor: mon.mon2 [mon2] [DEBUG] {[mon2] [DEBUG] "election_epoch": 0 [mon2] [DEBUG] "extra_probe_peers": [[mon2] [DEBUG] "XXX.XXX.XXX.XXX:6789/0", [mon2] [DEBUG] "XXX.XXX.XXX.XXX:6789/0" [mon2] [DEBUG]] [mon2] [DEBUG] "feature_map": {[mon2] [DEBUG] "mon": {[mon2] [DEBUG] "group": {[mon2] [DEBUG] "features": "0x1ffddff8eea4fffb", [mon2] [DEBUG] "num": 1 [mon2] [DEBUG] "release": "luminous" [mon2] [DEBUG]} [mon2] [DEBUG]} [mon2] [DEBUG]}, [mon2] [DEBUG] "features": {[mon2] [DEBUG] "quorum_con": "0", [mon2] [DEBUG] "quorum_mon": [], [mon2] [DEBUG] "required_con": "0" [mon2] [DEBUG] "required_mon": [[mon2] [DEBUG] "kraken", [mon2] [DEBUG] "luminous" [mon2] [DEBUG]] [mon2] [DEBUG]}, [mon2] [DEBUG] "monmap": {[mon2] [DEBUG] "created": "2019-01-04 1543 purge 39.644529", [mon2] [DEBUG] "epoch": 3 [mon2] [DEBUG] "features": {[mon2] [DEBUG] "optional": [], [mon2] [DEBUG] "persistent": [[mon2] [DEBUG] "kraken", [mon2] [DEBUG] "luminous" [mon2] [DEBUG] [mon2] [DEBUG]}, [mon2] [DEBUG] "fsid": "79fabbdf-b031-4621-952a-3fe880068501" [mon2] [DEBUG] "modified": "2019-02-22 15 Brave 47mon2 19.502869", [mon2] [DEBUG] "mons": [[mon2] [DEBUG] {[mon2] [DEBUG] "addr": "XXX.XXX.XXX.XXX:6789/0", [mon2] [DEBUG] "name": "mon0" [mon2] [DEBUG] "public_addr": "XXX.XXX.XXX.XXX:6789/0", [mon2] [DEBUG] "rank": 0 [mon2] [DEBUG]} [mon2] [DEBUG] [mon2] [DEBUG]}, [mon2] [DEBUG] "name": "mon2", [mon2] [DEBUG] "outside_quorum": [] [mon2] [DEBUG] "quorum": [], [mon2] [DEBUG] "rank":-1, [mon2] [DEBUG] "state": "probing" [mon2] [DEBUG] "sync_provider": [] [mon2] [DEBUG]} [mon2] [DEBUG] * [mon2] [INFO] Monitor: mon.mon2 is currently at the state of probing [mon2] [INFO] Running command: sudo ceph--cluster=ceph-- admin-daemon / var/run/ceph/ceph-mon.mon2.asok mon_ status [mon2] [WARNIN] monitor mon2 does not exist in monmap [mon2] [WARNIN] neither `public_ addr`nor `public_ network` keys are defined for monitors [mon2] [WARNIN] monitors may not be able to form quorum4, Mon status query # ceph- s# ceph quorum_status [--format json-pretty] # ceph--cluster=ceph-- admin-daemon / var/run/ceph/ceph-mon. {mon_name} .asok mon_status
