Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge about "how to use Python and OpenCV to realize online table tennis". In the actual case operation process, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!

capture screen

The first thing is to capture the screen. I wanted to make sure my frame rate was as fast as possible, and for that I found MSS to be a great python package. With this, I easily hit a top speed of 60 frames per second, compared to PIL, where I can only get about 20 frames per second. It is returned as a numpy array.

Paddle detection

For simplicity, we need to define the position of the paddle. This can be done in several different ways, but I think the most obvious is to mask the area of each Paddle and then run the connected component to find the Paddle object. Here is a code:

def get_objects_in_masked_region(img, vertices, connectivity = 8): ''':return connected components with stats in masked region [0] retval number of total labels 0 is background [1] labels image [2] stats[0] leftmostx, [1] topmosty, [2] horizontal size, [3] vertical size, [4] area [3] centroids ''' mask = np.zeros_like(img) # fill the mask cv2.fillPoly(mask, [vertices], 255) # now only show the area that is the mask mask = cv2.bitwise_and(img, mask) conn = cv2.connectedComponentsWithStats(mask, connectivity, cv2.CV_16U) return conn

Above,"vertices" is simply a list of coordinates that define the masked area. Once I have objects in each region, I can get their centroid positions or bounding boxes. One thing to note is that OpenCV treats the background as the zeroth object in any connected component list, so in this case I always get the second largest object. The results are as follows-the green centroid racket on the right is the player/soon to be AI controlled racket.

Move the paddle.

Now that we have an output, we need an input. For this, I turned to a useful package and someone else's code.

It uses ctypes to simulate keyboard presses, in which case the game is played with the "k" and "m" keys. I have the scan code here. After testing that it just randomly moves up and down, we can start tracking it.

Table tennis testing

The next step is to identify and track ping-pong balls. Again, this can be handled in several ways--one of which might be object detection by using templates, however, I've used connected components and object properties again, i.e., the area of the ping-pong ball, because it's the only object with size.

I knew I would have problems whenever the ping-pong ball passed through or touched other white objects, but I also thought it would be fine as long as I could track it most of the time. After all, it moves in a straight line. If you watch the video below, you will see how the red circle marking the ping-pong ball flashes. This is because it only finds one in every 2 frames. At 60 frames per second, it doesn't matter.

Bounce Predicted Ray Casting

At this point, we already have a working AI. If we just move the player's paddle so it's in the same y-axis position as the ping-pong ball, it works pretty well. However, it does run into problems when the ping-pong ball gets a good bounce. The racquet is too slow to keep up, and needs to predict where the ping-pong ball will be instead of just moving to its current position. This has been done in the clip above, and here is a comparison of the two approaches.

The difference isn't big, but if you choose the right AI, it's definitely a more stable victory. To do this, I first created a list of locations for table tennis. To be fair, I've limited the list to five, which is pretty much achievable. The list should not be too long, or it will take longer to discover that it has changed direction. After getting the list of locations, I use simple vector averaging to smooth out and get the direction vector--as shown by the green arrow. This is also normalized to a unit vector, which is then multiplied by a length to facilitate visualization.

Casting rays is just an extension of this--making the forward projection longer. Then I checked if the future location was outside the boundary of the top and bottom zones. If so, it simply projects the location back into the game area. For the left and right sides, it calculates the position where it intersects the x position of the paddle and fixes the x and y positions to that point. This ensures that the paddle points to the correct position. Without that, it usually goes too far. Here is the code that defines the ray that predicts the future position of the ping-pong ball:

def pong_ray(pong_pos, dir_vec, l_paddle, r_paddle, boundaries, steps = 250): future_pts_list = [] for i in range(steps): x_tmp = int(i * dir_vect[0] + pong_pos[0]) y_tmp = int(i * dir_vect[1] + pong_pos[1]) if y_tmp > boundaries[3]: #bottom y_end = int(2*boundaries[3] - y_tmp) x_end = x_tmp elif y_tmp

r_paddle[0]: #right x_end = int(boundaries[1]) y_end = int(pong_pos[1] + ((boundaries[1] - pong_pos[0])/dir_vec[0])*dir_vec[1]) elif x_tmp < boundaries[0]: #left x_end = int(boundaries[0]) y_end = int(pong_pos[1] + ((boundaries[0] - pong_pos[0]) / dir_vec[0]) * dir_vec[1]) else: x_end = x_tmp end_pos = (x_end, y_end) future_pts_list.append(end_pos) return future_pts_list

In the above, perhaps less obvious calculation is to determine the intercept of the paddle with respect to the left or right position of the target. We basically do this by using similar triangles, and the picture and equations are shown below. We know the intercept at the x position of a given paddle in the boundary. We can then calculate how far the ping-pong ball will move and add it to the current y position.

Although the paddle looks straight, it actually has a curved bounce surface. That is, if you hit the ball at both ends with a racket, it will bounce, just like the racket has angles. So I allowed the racket to hit the edge, which increased the aggressiveness of the AI and made the ping-pong ball fly around.

"How to use Python and OpenCV to achieve online table tennis" content is introduced here, thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.