Shulou(Shulou.com)05/31 Report--

This article mainly introduces the ceph-deploy source code structure and cli is how, the article introduces in great detail, has a certain reference value, interested friends must read!

Ceph-deploy Source Code Analysis (1)-- Source Code structure and cli

Ceph-deploy is a tool for deploying Ceph clusters. Through SSH, you can install Ceph software packages on remote hosts, create clusters, add monitors, collect (or destroy) keys, add OSD and metadata servers, configure management hosts, and even dismantle clusters.

Ceph-deploy is developed using Python, and GitHub is https://github.com/ceph/ceph-deploy. The ceph-deploy version of this source code analysis is 1.5.37.

Source structure

Ceph-deploy-1.5.37 source directory

├── ceph_deploy # ceph-deploy source directory ├── ceph_deploy.egg-info # egg-info directory ├── CONTRIBUTING.rst # contribution Guide ├── LICENSE # MIT LICENSE ├── MANIFEST.in # Packaging rules ├── PKG-INFO # PKG-INFO file ├── README.rst # ceph-deploy introduction to ├── scripts # startup script directory ├── setup.cfg # setup.py configure ├── setup.py # ceph-deploy security Install script ├── tox.ini # standardized Test └── vendor.py

Ceph_deploy source code directory file

├── admin.py # subcommand admin module, add ceph.conf and client.admin key push to remote host ├── calamari.py # subcommand calamari module, connect calamari master ├── cli.py # CLI entry ├── cliutil.py # to decorator function add priority ├── conf # ceph.conf and cephdeploy.conf read and write related operation directory ├── config.py # subcommand config module, push ceph.conf file to remote host From the remote host pull ceph.conf file ├── connection.py # connect the local host, remote host ├── exc.py # exception handling Error ├── forgetkeys.py # subcommand forgetkeys module, remove the authentication keys ├── gatherkeys.py # subcommand gatherkeys module locally Pull the operation of authentication keys ├── hosts # ceph-deploy on different operating systems (centos, debian, fedora, rhel, suse) from the mon host ├── _ _ init__.py # initialize version number information, current version 1.5.37 ├── install.py # subcommand install module, install and uninstall ceph package, clear data ├── lib # vendor class library ├── mds.py # subcommand mds module Mds management ├── misc.py # other tool classes, such as: mon host assembly tuples ├── mon.py # subcommand mon module, mon management ├── new.py # subcommand new module, deployment cluster ├── osd.py # subcommand osd module, osd management ├── pkg.py # subcommand pkg module, comma-separated package installation and uninstall ├── repo.py # subcommand repo module Add yum repo ├── rgw.py # subcommand rgw module, rgw manages ├── tests # test file directory ├── util # util directory └── validate.py # parameter check function

Source entry

The ceph-deploy file in the script directory is the entrance to ceph-deploy, followed by / usr/bin/ceph-deploy after installation.

The _ _ main__ function of ceph-deploy calls the main function of ceph_deploy.cli

.from ceph_deploy.cli import mainif _ _ name__ ='_ _ main__': sys.exit (main ())

Cli module

Cli.py is a command line operation module.

The main function calls the _ main function

Def main (args=None, namespace=None): try: _ main (args=args Namespace=namespace) finally: # This block is crucial to avoid having issues with # Python spitting non-sense thread exceptions. We have already # handled what we could, so close stderr and stdout. If not os.environ.get ('CEPH_DEPLOY_TEST'): try: sys.stdout.close () except: Pass try: sys.stderr.close () except: Pass

_ main function

Set up logs: Console Logger and File Logger are added to root_logger

Call argparse module to parse cli parameters

Call the ceph-deploy module set_overrides function in the conf directory to obtain ceph-deploy-global from the cephdeploy.conf or ~ / .cephdeploy.conf file in the current directory, and write the ceph-deploy- [subcmd] configuration entry to args.

Call the corresponding module that executes subcmd

Catches ((KeyboardInterrupt, RuntimeError, exc.DeployError,), handle_all=True) def _ main (args=None, namespace=None): # Set console logging first with some defaults, to prevent having exceptions # before hitting logging configuration. The defaults can/will get overridden # later. # Console Logger # Command Line console logs sh = logging.StreamHandler () # different levels of logs are distinguished by different colors: DEBUG blue, WARNIN yellow, ERROR red INFO White sh.setFormatter (log.color_format ()) # sets the log level to WARNING sh.setLevel (logging.WARNING) # because we're in a module already _ _ name__ is not the ancestor of # the rest of the package Use the root as the logger for everyone # root_logger log root_logger = logging.getLogger () # allow all levels at root_logger Handlers control individual levels # set the root_logger log level to DEBUG root_logger.setLevel (logging.DEBUG) # add sh to root_logger root_logger.addHandler (sh) # get the argparse that parses cli Call the argparse module parser = get_parser () if len (sys.argv) < 2: Parser.print_help () sys.exit () else: # parsing to get the ceph- deployment subcommands and parameters in sys.argv Args= parser.parse_args (args=args Namespace=namespace) # set log level console_loglevel = logging.DEBUG # start at DEBUG for now if args.quiet: Console_loglevel = logging.WARNING if args.verbose: console_loglevel = logging.DEBUG # Console Logger sh.setLevel (console_loglevel) # File Logger # File log fh = logging.FileHandler ('ceph-deploy- {cluster} .log' .format (cluster=args.cluster)) fh.setLevel (logging.DEBUG) fh.setFormatter (logging.Formatter (log.FILE_FORMAT)) # add fh to root_logger root_logger.addHandler (fh) # Reads from the config file and sets values for the global # flags and the given sub-command # the one flag that will never work regardless of the config settings is # logging because we cannot set it before hand since the logging config is # not ready yet. This is the earliest we can do. # obtain the ceph-deploy configuration override command line parameter args = ceph_deploy.conf.cephdeploy.set_overrides (args) LOG.info ("Invoked") from the current directory's cephdeploy.conf or ~ / .cephdeploy.conf file (% s):% s "% (ceph_deploy.__version__ '' .join (sys.argv) log_flags (args) # args.func is the subcmd subcommand in cli Call the corresponding module return args.func (args)

Get_parser function

[ceph_deploy.cli] is configured in the following ways:

Module name = module package name: execute function

For example: new = ceph_deploy.new:make

New, as a subcommand of ceph-deploy, executes the make function when executing the ceph-deploy new command

Other modules are similar:

Mon = ceph_deploy.mon:make

Osd = ceph_deploy.osd:make

Rgw = ceph_deploy.rgw:make

Mds = ceph_deploy.mds:make

Config = ceph_deploy.config:make

one

two

three

four

five

six

seven

eight

nine

ten

eleven

twelve

thirteen

fourteen

fifteen

sixteen

seventeen

eighteen

nineteen

twenty

twenty-one

twenty-two

twenty-three

twenty-four

twenty-five

twenty-six

twenty-seven

twenty-eight

twenty-nine

thirty

thirty-one

thirty-two

thirty-three

thirty-four

thirty-five

thirty-six

thirty-seven

thirty-eight

thirty-nine

forty

forty-one

forty-two

forty-three

forty-four

forty-five

forty-six

forty-seven

forty-eight

forty-nine

fifty

fifty-one

fifty-two

fifty-three

fifty-four

fifty-five

fifty-six

fifty-seven

fifty-eight

fifty-nine

sixty

sixty-one

sixty-two

sixty-three

sixty-four

sixty-five

sixty-six

sixty-seven

sixty-eight

sixty-nine

seventy

seventy-one

seventy-two

seventy-three

seventy-four

seventy-five

seventy-six

seventy-seven

seventy-eight

Def get_parser ():

# call argparse module

Parser = argparse.ArgumentParser (

Prog='ceph-deploy'

Formatter_class=argparse.RawDescriptionHelpFormatter

Description='Easy Ceph deployment\ n\ n% s'% _ _ header__

)

Verbosity = parser.add_mutually_exclusive_group (required=False)

Verbosity.add_argument (

'- vested,'-- verbose'

Action='store_true', dest='verbose', default=False

Help='be more verbose'

)

Verbosity.add_argument (

'- QQ,'-- quiet'

Action='store_true', dest='quiet'

Help='be less verbose'

)

Parser.add_argument (

'--version'

Action='version'

Version='%s' ceph_deploy.__version__

Help='the current installed version of ceph-deploy'

)

Parser.add_argument (

'--username'

Help='the username to connect to the remote host'

)

Parser.add_argument (

'--overwrite-conf'

Action='store_true'

Help='overwrite an existing conf file on remote host (if present)'

)

Parser.add_argument (

'--cluster'

Metavar='NAME'

Help='name of the cluster'

Type=validate.alphanumeric

)

Parser.add_argument (

'--ceph-conf'

Dest='ceph_conf'

Help='use (or reuse) a given ceph.conf file'

)

Sub = parser.add_subparsers (

Title='commands'

Metavar='COMMAND'

Help='description'

)

Sub.required = True

# get the entry_points under ceph_deploy.cli

Entry_points = [

(ep.name, ep.load ())

For ep in pkg_resources.iter_entry_points ('ceph_deploy.cli')

]

# sort by priority

Entry_points.sort (

Key=lambda name_fn: getattr (name_fn [1], 'priority', 100)

)

# add a module to a subcommand

For (name, fn) in entry_points:

P = sub.add_parser (

Name

Description=fn.__doc__

Help=fn.__doc__

)

If not os.environ.get ('CEPH_DEPLOY_TEST'):

P.set_defaults (cd_conf=ceph_deploy.conf.cephdeploy.load ())

# flag if the default release is being used

P.set_defaults (default_release=False)

Fn (p)

P.required = True

Parser.set_defaults (

Cluster='ceph'

)

Return parser

Argparse module

The parsing of the cli command uses the argparse.py module. Argparse is a parser for command-line options, parameters, and subcommands in the Python standard library to replace outdated optparse modules, and argparse was introduced in Python2.7.

For argparse module, please refer to

Https://docs.python.org/2.7/library/argparse.html

Http://python.usyiyi.cn/translate/python_278/library/argparse.html

Set_overrides function

The set_overrides function of ceph-deploy module under conf directory

Call the load () function

Judge the ceph-deploy configuration file ceph-deploy-global, ceph-deploy- [subcommand] configuration items, and call the override_subcommand () function to write args.

Def set_overrides (args, _ conf=None):

"

Read the configuration file and look for ceph-deploy sections

To set flags/defaults from the values found. This will alter the

``args`` object that is created by argparse.

"

# Get the subcommand name to avoid overwritting values from other

# subcommands that are not going to be used

Subcommand = args.func.__name__

Command_section = 'ceph-deploy-%s'% subcommand

# load ceph-deploy configuration

Conf = _ conf or load ()

For section_name in conf.sections ():

If section_name in ['ceph-deploy-global', command_section]:

# ceph-deploy-global, ceph-deploy- [subcommand] configuration items are written to args

Override_subcommand (

Section_name

Conf.items (section_name)

Args

)

Return args

Load function, calling the location () function

one

two

three

four

five

Def load ():

Parser = Conf ()

# read and parse ceph-deploy configuration file

Parser.read (location ())

Return parser

Location function, calling the _ locate_or_create () function

one

two

three

four

five

six

Def location ():

"

Find and return the location of the ceph-deploy configuration file. If this

File does not exist, create one in a default location.

"

Return _ locate_or_create ()

_ locate_or_create function to determine whether the cephdeploy.conf or ~ / .cephdeploy.conf file of the current directory exists.

If none exists, call the create_stub function to create a ~ / .cephdeploy.conf file. This file is created from the template and the content is empty.

If there is a (pre-created) cephdeploy.conf or ~ / .cephdeploy.conf file, you can configure public_network, cluster_network, overwrite-conf, and so on.

Def _ locate_or_create ():

Home_config = path.expanduser ('~ / .cephdeploy.conf')

# With order of importance

Locations = [

Path.join (os.getcwd (), 'cephdeploy.conf')

Home_config

]

For location in locations:

If path.exists (location):

Logger.debug ('found configuration file at:% s'% location)

Return location

Logger.info ('could not find configuration file, will create one in $HOME')

Create_stub (home_config)

Return home_config

The above is all the content of this article "what is the source code structure and cli in ceph-deploy?" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.