Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you about how to solve the problems found in OpenStack Overcloud with Python scripts. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

OpenStack stores and manages a stack of log files on its Overcloud nodes and Undercloud hosts. Therefore, it is not easy to use OSP log files to troubleshoot problems encountered, especially if you don't even know what caused the problem.

If you are in this situation, then LogTool can make your life easier! It will save you the time and effort you need to manually troubleshoot problems. LogTool is based on the fuzzy string matching algorithm and provides all unique errors and warnings that have occurred in the past. You can export this information for a specific time period (for example, 10 minutes ago, an hour ago, a day ago, etc.) according to the time stamp in the log.

LogTool is a set of Python scripts whose main module PyTool.py is executed on the Undercloud host. Some modes of operation use other scripts that are executed directly on the Overcloud node, such as exporting errors and warnings from the Overcloud log.

LogTool supports Python2 and Python3, and you can change the working directory: LogTool_Python2 or LogTool_Python3 as needed.

Mode of operation 1. Export errors and warnings from Overcloud logs

This mode is used to extract errors and warnings from Overcloud nodes that have occurred in the past. As a user, you will be prompted to provide a "start time" and "debug level" to extract error or warning messages. For example, if something goes wrong in the past 10 minutes, you can extract only error and warning messages during that time period.

This mode of operation generates a directory containing the result files for each Overcloud node. The resulting file is a compressed simple text file (* .gz) to reduce the time required to download from the Overcloud node. You can use zcat or similar tools to convert compressed files to regular text files. In addition, some versions of Vi and any latest version of Emacs support reading compressed data. The result file is divided into sections and contains directories at the bottom.

LogTool can instantly detect two types of log files: standard and non-standard. In the standard file, each log line has a known and defined structure: timestamp, debug level, information, and so on. In non-standard files, the structure of the log is unknown. For example, it might be a third-party log. In the directory, you can find the "name-> line number" of each section, for example:

Raw data-error / warning messages extracted from the standard OSP log: this section contains all extracted error / warning messages without any modifications or changes. These messages are the raw data used by LogTool for fuzzy matching analysis.

Statistics-number of errors / warnings per standard OSP log: in this section, you will find the number of errors and warnings per standard log file. This information can help you understand the potential components used to troubleshoot the root cause of the problem.

Statistics-unique messages for each standard OSP log file: this section provides unique error and warning messages within the specified timestamp. For more details about each unique error or warning, look for the same message in the original data section.

Statistics-unique messages for each non-standard log file at any time: this section contains unique messages in the non-standard log file. Unfortunately, LogTool cannot handle these log files in the same way as standard log files. As a result, you will be ignored when you extract log information from a "specific time", and you will see all unique error / warning messages created in the past. So, first, scroll down to the directory at the bottom of the result file and view its section-use the line index in the directory to jump to the relevant section, where the information on lines 3, 4, and 5 is the most important.

2. Download all logs from the Overcloud node

The logs of all Overcloud nodes will be compressed and downloaded to the local directory on the Undercloud host.

3. Search strings in all Overcloud logs

The pattern "grep" (search) is a string provided by the user on all Overcloud logs. For example, you might want to view all the log messages for a particular request, such as the failed request ID for "Create VM".

4. Check the current CPU, RAM, and disk usage on Overcloud

This mode displays the current CPU, RAM, and disk information on each Overcloud node.

5. Execute user script

This mode allows users to run their own scripts on the Overcloud node. For example, if the Overcloud deployment fails, you need to perform the same process on each controller node to fix the problem. You can implement the "alternative" script and use this mode to run it on the controller.

6. Download the related log only according to the given timestamp

This mode only downloads the Last modified time log for the given timestamp on Overcloud. For example, if an error occurred 10 minutes ago, it has nothing to do with the old log file, so there is no need to download it. In addition, you cannot (or should not) attach larger files to some error reporting tools, so this model may be helpful in writing error reports.

7. Export errors and warnings from the Undercloud log

This is the same as mode 1 above.

8. Check for abnormal docker on Overcloud

This mode is used to search for abnormal Docker on the node.

9. Download the OSP log and run LogTool locally

This mode allows you to download OSP logs (for example, cougar11.scl.lab.tlv.redhat.com) from Jenkins or Log Storage and analyze them locally.

10. Analyze the deployment log on Undercloud

This pattern can help you understand what went wrong with Overcloud or Undercloud deployment. For example, in overcloud_deploy.sh scripts, deployment logs are generated when you use the-- log option; the problem with such logs is "unfriendly", and it's hard to understand what went wrong, especially when the detail level is set to vv or higher, making the data in the log difficult to read. This mode provides detailed information about all failed tasks.

11. Analyze the log of Gerrit (Zuul) failure

This mode is used to analyze Gerrit (Zuul) log files. It automatically downloads all files (HTTP downloads) from the remote Gerrit door and analyzes them locally.

Installation

LogTool is available on GitHub. Clone it to your Undercloud host using the following command:

Git clone https://github.com/zahlabut/LogTool.git

The tool also uses some external Python modules:

Paramiko

By default, the SSH module is usually installed on Undercloud. Use the following command to verify that it is installed:

Ls-a / usr/lib/python2.7/site-packages | grep paramiko

If you need to install the module, execute the following command on Undercloud:

Sudo easy_install pipsudo pip install paramiko==2.1.1BeautifulSoup

This HTML parser module is used only in the mode of downloading log files using HTTP. It is used to parse the Artifacts HTML page to get all the links in it. To install BeautifulSoup, enter the following command:

Pip install beautifulsoup4

You can also install all required modules using the requirements.txt file by executing the following command:

Pip install-r requirements.txt configuration

All required parameters are set directly in the PyTool.py script. The default value is:

Overcloud_logs_dir ='/ var/log/containers'overcloud_ssh_user = 'heat-admin'overcloud_ssh_key =' / home/stack/.ssh/id_rsa'undercloud_logs_dir ='/ var/log/containers'source_rc_file_path='/home/stack/' usage

This tool is interactive, so to start it, simply type:

Cd LogToolpython PyTool.py troubleshooting LogTool

Two log files are created at run time: Error.log and Runtime.log. Please add both to the description of the problem you want to open.

Limitation

LogTool is hard-coded to handle files up to 500 MB.

LogTool_Python3 script

Get it at github.com/zahlabut/LogTool.

This is how to solve the problems found in OpenStack Overcloud with Python script. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.