Shulou(Shulou.com)06/01 Report--

This article mainly explains "Python how to obtain the Nginx log corresponding to the task request in real time". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "Python how to obtain the Nginx log corresponding to the task request in real time".

Requirement description

In the process of project requirements testing, we need to send some use case requests to the Nginx server, and then check the corresponding Nginx log to determine whether there is characteristic content to determine whether the task is successful or not. In order to improve efficiency, this process needs to be automated.

Practical environment

Python 3.6.5

Code design and implementation

#! / usr/bin/env python#-*-coding:utf-8-*-"@ CreateTime: 2021-06-26 9:05@Author: shouke"import timeimport threadingimport subprocessfrom collections import dequedef collect_nginx_log (): global nginx_log_queue global is_tasks_compete global task_status args =" tail-0f / usr/local/openresty/nginx/logs/access.log "while task_status! =" req_log_got ": with subprocess.Popen (args, stdout=subprocess.PIPE) Stderr=subprocess.PIPE, shell=True, universal_newlines = True) as proc: log_for_req = "" outs, errs = "", "try: outs, errs = proc.communicate (timeout=2) except subprocess.TimeoutExpired: print (" get nginx log timeout Retrying ") proc.kill () try: outs, errs = proc.communicate (timeout=5) except subprocess.TimeoutExpired: print (" getting nginx log timeout, timeout again Stop retry ") break finally: for line in outs.split ("): flag = "client_ip": "10.118.0.77"# feature if flag in line: # find the log log_ containing the feature content For_req + = line if task_status = = "req_finished": nginx_log_queue.append (log_for_req) task_status = "req_log_got" def run_tasks (task_list): "" running tasks: param task_list task list "" global nginx_log_queue global is_tasks_compete Global task_status for task in task_list: thread = threading.Thread (target=collect_nginx_log Name= "collect_nginx_log") thread.start () time.sleep (1) # before performing a task Let the log collection thread prepare print ("executing task:% s"% task.get ("name")) # execute Nginx task request #. Task_status = "req_finished" time_to_wait = 0.1while task_status! = "req_log_got": # nginx log collection triggered by the request is not completed time.sleep (time_to_wait) time_to_wait + = 0.01 else:# get the nginx log if nginx_log_queue triggered by the use case request: Nginx_log = nginx_log_queue.popleft () task_status = "req_ready" # parsing log # do something here #... Else: print ("queue for storing request log is empty") # do something here #... if _ _ name__ = = "_ _ main__": nginx_log_queue = deque () is_tasks_compete = False # whether all tasks are completed task_status = "req_ready" # req_ready,req_finished Req_log_got # stores some status of executing subtask print ("# # Task start # #") tast_list = [{"name": "test_task" "other": "..."}] run_tasks (tast_list) is_tasks_compete = True current_active_thread_num = len (threading.enumerate ()) while current_active_thread_num! = 1: time.sleep (2) current_active_thread_num = len (threading.enumerate ()) print ("# # # Task completion # # ")

Note:

1. Why not put the above code in place in one step and directly tail-0f / usr/local/openresty/nginx/logs/access.log | grep "feature content"? This is because you cannot get Nginx's log by doing so.

2. In practice, it is found that when proc.communicate (timeout=2) is executed for the first time to obtain the log, it is always impossible to obtain the log, which times out and requires a second acquisition. Moreover, if the timeout setting is too small (set to 1 second in practice), the Nginx log cannot be obtained during the second execution.

Thank you for your reading. The above is the content of "how Python obtains the Nginx log corresponding to task request in real time". After the study of this article, I believe you have a deeper understanding of how Python obtains the Nginx log corresponding to task request in real time, and the specific usage needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.