Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about how to use appium to grab app application data in Python, many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.

What is selenium?

Selenium was originally an automated testing tool for Web applications. The Selenium test runs directly in the browser, just like a real user is working on it. Supported browsers include IE (7, 8, 9, 10, 11), Mozilla Firefox,Safari,Google Chrome,Opera, etc. We crawler engineers use selenium to crawl dynamic web pages.

In today's Python tutorial, we introduce another automated testing tool, Appium.

Appium official documentation

Https://github.com/DoctorQ/appium/blob/master/docs/en/about-appium/intro.md

What is Appium?

Appium is a mobile automation framework that can be used to test native applications, mobile web applications, and hybrid applications across platforms. Operating systems available for IOS and Android as well as firefox.

Native applications are those written in android or ios's sdk

Mobile web application refers to web application, which is similar to safari application or Chrome application or browser-like application in ios.

Hybrid application refers to an application that wraps webview, which is native to the interactive application of web content.

The important thing is that Appium is cross-platform, what is cross-platform, which means that a set of api can be used to write test cases for different platforms.

The same Appium was originally used as an automated testing tool for app applications, and we crawler engineers can also use it to grab app application data.

Appium is equivalent to a server, we can send some operation instructions to Appium, Appium will drive the mobile device according to different instructions and complete different actions.

1. The purpose of this Python course:

We take JD.com, a mobile phone on the Android platform, as an example to demonstrate the method of starting and operating App by Appium, and finally capture the product data of JD.com on the mobile phone.

II. Preparatory work

Make sure that PC has installed Appium. This instance is installed in windows environment, and the version of appium installed is Appium-Desktop.

Install the Android development environment, uiautomatorviewer tools (required), python development environment and Python version of Appium API (Appium-Python-Client)

Android Simulator (Night God Simulator) install the mobile phone JD.com app

Ensure that the mitmproxy package grab tool (required) and fiddler package grab tool (optional) are installed on the PC

The database for data storage is Mongodb

3. Obtain the key parameters used by Appium-Desktop (Desired Capabilities parameter)

Desired Capabilities parameters: they are platformName, deviceName, appPackage, appActivity, respectively.

PlatformName: it is the name of the platform and needs to distinguish between Android and iOS. We are using the android platform, so it is Android.

DeviceName: it is the name of the device. Here is the specific type of phone. DeviceName can be obtained through the adb devices command.

Note here: the simulator displays 127.0.0.1. The display of the real mobile phone is different from that of the simulator, and the real mobile phone is a combination of alphabetic data.

AppPackage: this is the name of the App package, and this parameter can be obtained using logcat in adb shell.

Enter after the order.

Then, open the mobile phone JD.com app in the android simulator

In adb shell, we can get appPackage and appActivity

AppActivity: it is the Activity name of the entry, which is usually required here with. In the beginning, this parameter can be obtained using logcat in adb shell.

Where:

AppPackage is: com.jingdong.app.mall

AppActivity is: com.jingdong.app.mall.MainFrameActivity

After getting the key parameters (Desired Capabilities parameter) used by Appium-Desktop, we can start Appium-Desktop

Start the Appium-Desktop server

After confirming that appium-desktop starts successfully, we can write code

5. Write app startup code

From appium import webdriverdesired_caps = {} desired_caps ['platformName'] =' Android'desired_caps ['platformVersion'] =' 4.4.2'desired_caps ['deviceName'] =' 192.168.54.56:62001'desired_caps ['appPackage'] =' com.jingdong.app.mall'desired_caps ['appActivity'] =' com.jingdong.app.mall.MainFrameActivity'desired_caps ['unicodeKeyboard'] = Truedesired_caps [' resetKeyboard'] = True# start appium-desktop server Server IP according to the actual fill in driver = webdriver.Remote ('http://192.168.54.56:4723/wd/hub', desired_caps)

Run the code to start app

After confirming that the mobile phone JD.com app starts successfully and there are no errors, we can write automatic code to control the app behavior.

How to locate the app internal control node?

The answer is to use uiautomatorviewer

By getting the xpath of the control, we can navigate to the app control

7. Write automation code

# wait for app to start time.sleep (5) # turn off ads If there is close_ad = driver.find_element_by_xpath ("/ / android.widget.TextView [@ resource-id='com.jingdong.app.mall:id/sq']") if close_ad: close_ad.click () # Click the category select_fenlei = driver.find_element_by_xpath ("/ / android.widget.RadioGroup [@ resource-id='com.jingdong.app.mall:id/tf'] / android.widget.RadioButton [2]") .click () # Click search select_search = driver.find_element_by_xpath ("/ / android.widget.ImageView [@ resource-id='com.jingdong.app.mall:id/xo']") .click () # enter the keyword key = 'laptop' # to determine whether to enter the text while True: if driver.find_element_by_xpath ("/ / android.widget.EditText [@ resource-id='com.jingdong.app.mall:id/xp']") .text ! = key: select_keyword = driver.find_element_by_xpath ("/ / android.widget.EditText [@ resource-id='com.jingdong.app.mall:id/xp']") .send_keys (key) else: break# Click to search for driver.find_element_by_xpath ("/ / android.widget.Button [@ resource-id='com.jingdong.app.mall:id/el1']") .click () time.sleep (1) start_x = 500start_y = 900distance = 80mm analog mobile phone end slide while True: driver.swipe (start_x Start_y,start_x,start_y-distance) time.sleep (0.2) if Sorry There are no more goods.'in driver.page_source: break

After the automated code has been written, run it to see if there is a problem. If there is no problem, you can proceed to the next step.

8. Crawling application packets

Here, we get the data packet of the mobile phone JD.com through the fiddler packet grabbing tool.

Look at the response, which is really the data we want, and then we can write code to parse the response data.

IX. Write code for parsing response data

Import jsonfrom save_data import mongo_infodef response (flow): if 'client.action?functionId=search&clientVersion=5.3.0' in flow.request.url: response_dict = json.loads (flow.response.text) if' wareInfo' in response_dict: for i in response_dict ['wareInfo']: mongo_info.insert_data (I)

Write logic code for data storage

Import pymongofrom pymongo.collection import Collectionclass Connect_mongo (object): def _ init__ (self): self.client = pymongo.MongoClient (host='192.168.54.41',port=27017,connect=False) self.db_data = self.client ['jingdong'] def insert_data (self,item): collections = Collection (self.db_data,'jingdong_data') collections.insert (item) mongo_info = Connect_mongo ()

Start mitmdump and parse the data!

Note: the mobile phone or simulator must set the proxy of the mitmdump server IP.

Start mitmdump:

Mitmdump-p 8889-s decode_jingdong.py

12. Next, start the automatic control script to grab the data

Python jingdong.py

XIII. View data

After reading the above, do you have any further understanding of how to use appium to grab app application data in Python? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.