Shulou(Shulou.com)06/01 Report--

This article shows you how to solve the last kilometer problem of UI automation by 4399AT. The content is concise and easy to understand, which will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Why is it the last kilometer? Two aspects:

For Android APP, there is no solution to the positioning acquisition of suspension window controls realized by WindowManager, because it does not belong to the category of native app and webiview, so it cannot be operated and can only use coordinates rudely, but this can not solve the problem.

For wap mode, the prompt box or permission box cannot be clicked, as shown in the following figure

So I want to solve this problem through image recognition technology (opencv).

At present, 4399AT has opened the last kilometer, through image recognition technology to solve this problem.

Increased api for 4399AT image recognition:

PicClick: click on the picture

PicCondition: picture condition judgment

PicCheck: picture check

PicInput: picture input

PicSlide: slide the picture up and down

PicAlert: used for prompt box clicks, mainly used in wap mode, some clicks that prompt authorization, to solve the difficulties in wap mode testing

Api usage:

Action: "Action"

Paths: [picture path] can be more than one, and the path cannot be in Chinese

The examples are as follows:

Apply scenarios:

Click on the player of Rabbit ear Story APP: (the player is implemented by windowManager)

Take a screenshot, then paste it on the window10 drawing panel, and then resize

Adjust the pixel level to 450 (be sure to be 450), so there is no limit to the height.

Then capture the feature image to be operated, such as the player, and I will take a screenshot of the two corners of the player, do not include other pictures, otherwise it will affect the success rate of recognition, and then save it as xx chart, name and saved path, not in Chinese.

Intercept the green box and save it. Pictures can be in png and jpg formats.

After these are done, the path to save the picture is written in the api of the image recognition operation, such as the image is placed in the root directory of disk D.

Multiple images can be saved, depending on the features of the picture, and then the rest will be written according to your use case configuration 4399AT's api, and the execution method is the same.

The above is how 4399AT solves the last kilometer problem of UI automation. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.