Shulou(Shulou.com)05/31 Report--

This article introduces how to use the pytest-xdist distributed plug-in to ensure that the fixture of scope=session can only run once in the case of multi-process running, the content is very detailed, interested friends can refer to, I hope it can be helpful to you.

Background

Using pytest-xdist distributed plug-ins can speed up the run and take full advantage of machine multi-core CPU

Putting common functions into fixture can improve reusability and maintainability.

When doing interface automation testing, we usually put the login interface into fixture, and scope will be set to session so that it can only run once globally.

But when using pytest-xdist, scope=session 's fixture cannot be guaranteed to run only once, and officials have reported the problem.

Official description

Pytest-xdist is designed so that each worker process will execute its own test set and all test subsets, which means that fixture requiring high-level scope (such as session) will be executed multiple times in different testing processes, which exceeds expectations and may not be desirable in some cases

Although pytest-xdist does not have built-in support to ensure that scope=session 's fixture is executed only once, it can be achieved by using locked files for interprocess communication

Official solution (apply directly) import jsonimport pytestfrom filelock import FileLock@pytest.fixture (scope= "session") def session_data (tmp_path_factory, worker_id): if worker_id = = "master": # not executing in with multiple workers Just produce the data and let# pytest's fixture caching do its jobreturn produce_expensive_data () # get the temp directory shared by all workersroot_tmp_dir = tmp_path_factory.getbasetemp (). Parent fn = root_tmp_dir / "data.json" with FileLock (str (fn) + ".lock"): if fn.is_file (): data = json.loads (fn.read_text ()) else: data = produce_expensive_data () fn.write_text (json.dumps (data)) return data

If a scope = session fixture needs to be run only once, you can use the above method, apply it directly, and then change the part that needs to be changed (this will be explained in more detail later)

Official words: this technique may not work in every case, but in many cases it should be a starting point, in which case it is important to execute fixture only once for scope = session

The following code project structure of Chestnut: xdist+fixture (folder) │ tmp (store allure data folder) │ conftest.py │ test_1.py │ test_2.py │ test_3.py │ _ _ init__.py │ test_1.py Code import osdef test_1 (test): print ("os environment variable", os.environ ['token']) print ("test1 test case" Test) test_2.py code import osdef test_2 (test): print ("os environment variable", os.environ ['token']) print ("test2 test case", test) test_3.py code import osdef test_3 (test): print ("os environment variable", os.environ [' token']) print ("test3 test case" Test) unresolved chestnut conftest.py code import osimport pytestfrom random import random@pytest.fixture (scope= "session") def test (): token = str (random ()) print ("fixture: request login API Get token ", token) os.environ ['token'] = tokenreturn token run command pytest-n 3-- alluredir=tmp run result

Scope=session 's fixture is obviously executed three times, and the data of the three test cases under the three processes are different, which is obviously not the result we want.

Using the official solution, chestnut rtincture raceme python#-*-coding: utf-8-*-"" _ _ title__ = _ _ Time__ = 2021-4-27 1128 pineapple test notes _ _ Blog__ = https://www.cnblogs.com/poloyy/"""import jsonimport osimport pytestfrom random import randomfrom filelock import FileLock@pytest.fixture (scope= "session") def test (tmp_path_factory) Worker_id): # if it is running on a stand-alone machine, run the code block here [cannot be deleted or modified] if worker_id = = "master": "" [custom code block] here write the actions you should do yourself. For example, login requests, new data, emptying database history data, etc. "" token = str (random ()) print ("fixture: request login interface to obtain token", token) os.environ ['token'] = token# if the test case is needed, you can return the corresponding data, such as tokenreturn token# if it is a distributed operation # to get a temporary directory shared by all child nodes There is no need to modify [cannot delete, modify] root_tmp_dir = tmp_path_factory.getbasetemp (). Parent# [cannot delete, modify] fn = root_tmp_dir / "data.json" # [cannot delete, modify] with FileLock (str (fn) + ".lock"): # [cannot delete, modify] if fn.is_file (): # read data in cache file For login operations such as token [cannot be deleted, modified] token = json.loads (fn.read_text ()) print (f "read cache file, token is {token}") else: "[custom code block] is the same as the code block of if above" token = str (random ()) print ("fixture: request login API" Get token ", token) # [cannot be deleted or modified] fn.write_text (json.dumps (token)) print (f" execute for the first time, token is {token} ") # it is best to store the data that needs to be retained later somewhere, for example, here is the environment variable os.environ ['token'] = tokenreturn token of os run the command pytest-n 3-- alluredir=tmp run result

You can see that the fixture is executed only once, and the test cases under different processes share a data token

Highlight

Read cache file is not read by every test case, it is read according to the process

For example, if-n 3 specifies three processes to run, one process will execute fixture (random) once, and the other two processes will read the cache once.

Assuming that each process has many use cases, it only reads the cache file once, not multiple times.

Therefore, it is best to save the data read from the cache file in a specific place. For example, the os.environ of the above code can save the data in the environment variable.

Two processes run three test case files

Or the code of the chestnut above.

Run the command pytest-n 2-- alluredir=tmp run result

You can see that there is no read cache file in the test_3 test case, and each process will only read the cache file once, remember!

On how to use the pytest-xdist distributed plug-in to ensure that the fixture of scope=session can still be shared here only once in the case of multi-process running, I hope the above content can be helpful to everyone and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.