Shulou(Shulou.com)06/03 Report--

Editor to share with you about Wechat jump python auxiliary software ideas and image recognition source code example analysis, I believe that most people do not know much, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to understand it!

First of all, let's see the effect.

Core thought

Get the distance from the chess piece to the center of the next square

Calculate the time to touch the screen

Click on the screen

Important method

Calculate the distance from the chess piece to the center of the next square

Use the adb shell screencap-p command to get the current screen screen of the phone.

Then use the information on the image to find out the coordinates of the pieces and the coordinates of the center point of the next square.

Then the distance is calculated by the formula of distance between two points.

Calculate the time to touch the screen

Tanya * S

Where S is the pixel distance calculated in the previous step, T is the pressing time (ms), and An is a coefficient that varies with the screen resolution, which is 1.35 in 1920 to 1080 and 1.475 in 2560 to 1440.

Click on the screen

Adb shell input swipe x y x y time (ms)

This command can click on the phone screen xpeny location time (ms)

Analysis of partial source code of image processing

Part of the image processing code is in the find_piece_and_board (im) method.

The coordinate points of the chess piece and the center of the next square are calculated from the input image im.

As soon as you enter the find_piece_and_board method, there are two nested for loops:

For i in range (int (h / 3), int (h * 2 / 3), 50): last_pixel = im_pixel [0, I] for j in range (1, w): pixel = im_pixel [j, I] # is not a solid color line, then record the value of scan_start_y Prepare to jump out of the loop if pixel [0]! = last_pixel [0] or pixel [1]! = last_pixel [1] or pixel [2]! = last_pixel [2]: scan_start_y = I-50 break if scan_start_y: break

The purpose of this code is to look down for lines that are not solid colors from the position of screen 2 to 3. And will find the position of the ordinate-50 as to find the pieces and squares of the starting coordinates. This can simplify the workload of the future search, because there is nothing above this Abscissa.

Next is the code to find the coordinates of the chess pieces.

# find the pawn coordinates # the total number of piece_x_sum Abscissa piece_x_c points the maximum number of piece_y_max ordinates # scan down from scan_start_y, the pieces should be in the upper half of the screen It is tentatively set to be no more than 2 for i in range 3 for i in range (scan_start_y, int (h * 2 + 3)): for j in range (scan_x_border, w-scan_x_border): # Abscissa also reduces part of the scanning cost pixel = im_pixel [j, I] # according to the color judgment of the lowest row of the chess piece, find the average of those points in the last row. This color should be OK. Don't bring up if for the time being (50

< pixel[0] < 60) and (53 < pixel[1] < 63) and (95 < pixel[2] < 110): piece_x_sum += j piece_x_c += 1 piece_y_max = max(i, piece_y_max) if not all((piece_x_sum, piece_x_c)): return 0, 0, 0, 0 # 平均横坐标 piece_x = int(piece_x_sum / piece_x_c) # 纵坐标最大值-底座一半的高度 piece_y = piece_y_max - piece_base_height_1_2 # 上移棋子底盘高度的一半 查找棋子的重要依据就是棋子的颜色较为单一并且和方块的颜色有较大差距。如果一个像素点的RGB像素值在B(50, 60), G(53, 63), R(95, 110)范围内那么就认为这个像素点是属于棋子的。根据以上信息就能计算出棋子的平均横坐标，以及最大的纵坐标值。 所以不难计算出棋子坐标（棋子平均横坐标， 棋子最大纵坐标 - 底座一半的高度）其中底座一半的高度因手机分辨率而异。需要提前配置好。 最后是查找下一个方块中心点的坐标的代码 # 寻找最高的棋盘 # 棋盘不会和棋子在同一侧 # 限制棋盘扫描的横坐标，避免音符 bug if piece_x < w / 2: board_x_start = piece_x board_x_end = w else: board_x_start = 0 board_x_end = piece_x for i in range(int(h / 3), int(h * 2 / 3)): last_pixel = im_pixel[0, i] if board_x or board_y: break board_x_sum = 0 board_x_c = 0 for j in range(int(board_x_start), int(board_x_end)): pixel = im_pixel[j, i] # 下一个棋盘紧贴着棋子 # 修掉脑袋比下一个小格子还高的情况的 bug if abs(j - piece_x) < piece_body_width: continue # 修掉圆顶的时候一条线导致的小 bug，这个颜色判断应该 OK，暂时不提出来 if abs(pixel[0] - last_pixel[0]) + abs(pixel[1] - last_pixel[1]) + abs(pixel[2] - last_pixel[2]) >

10: board_x_sum + = j board_x_c + = 1 if board_x_sum: # average Abscissa of the highest chessboard board_x = board_x_sum / board_x_c last_pixel = im_ pixel [board _ x, I]

The beginning of the code limits the width of the search by where the pieces are on the screen, and if the pieces are on the left side of the screen, search for squares on the right side of the screen, and vice versa. Because squares and pieces are not on the same side of the screen.

Then there is a top-down search for the top vertex of the square.

Vertex coordinates on the square (average Abscissa, ordinate of the current row)

Then the position of the lower ordinate + 247 begins to look up for a point with the same color as the upper vertex, which is the lower vertex.

Of course, this method has some limitations for solid color planes, but for non-solid color planes. May make a mistake in judgment.

If the previous hop hits the middle, there will be a point of r245 g245 b245 in the next target center. Use this attribute to make up for the judgment errors that may exist in the previous code.

If the previous hop does not jump to the middle for some reason, and the next hop happens to be unable to correctly identify the pattern, the game may fail. Because the pattern area is usually large, the probability of failure is low.

Improvable scheme

First of all, for multi-resolution, the current scheme requires multiple profiles to record the coefficients at different resolutions and the height of half of the chessboard chassis. Six mobile phones were randomly tested, two of which could not function properly due to lack of accessories.

The first is the coefficient A, the observation equation is a trainable quantity, and the unary first order equation is fitted by a machine learning framework such as TensorFlow.

Observe the role of half the height of the chessboard chassis in the code. It is not difficult to find out the ordinate of the center of the chessboard chassis. The position of the center of the chessman chassis is precisely the widest part of the chess piece. Therefore, the ordinate of the center of the chassis can be found by finding the ordinate of the widest part of the pawn. This frees you from the dependence on configuration files and allows the code to work properly on any phone.

Secondly, the error rate of the method for judging the location of the central coordinates of the box is high, although there are central white points to make up for, but there will still be errors in the process of a large number of jumps. 3Tai's mobile phone has been running all afternoon, and the highest score is only 2009.

The reason for the high error rate of the current method is the use of pure color method to judge, but there are also a lot of colorful squares in the actual game. If you want to change, you can not rely on the color method to judge, but should use the shape of the geometric image to calculate the position of the square. It is not difficult to find that there are only two shapes of squares in the game: prisms and circles.

Firstly, the contours of the image are extracted by canny or other contour search operators, and then the central coordinates of circles and prisms are extracted by Hough transform.

The above is "Wechat jump python auxiliary software ideas and image recognition source code example analysis" all the content of this article, thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.