Shulou(Shulou.com)06/02 Report--

This article is about how to use Java+Selenium+OpenCV simulation to achieve web page sliding verification, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article. Let's take a look at it.

At present, many web pages have sliding verification, the purpose is to prevent bad crawlers from scraping the data of their websites. This time, I used Java and selenium to solve the problem of sliding verification. I spent a week (copying code), and finally succeeded in the sliding verification of a certain tone!

Effect display:

I. demand analysis

There are only two steps to simulate sliding verification:

1. Find the slider

2. Press and hold the slider and slide for a distance

The first step is very simple, found directly through xPath, what is more important and difficult is the problem of distance in the second step. I spent so much time in this study, mainly in calculating the distance to slide.

In the process of programming for Baidu, I saw a lot of learning materials, basically the same method: using opencv computer vision tool to compare two processed images to calculate the sliding distance.

Second, simulation step 1. Open a web page with selenium.

Open it directly

2. Find the small slider and the background image where the small slider is located

Open the front-end tuning tool, F12, locate the slider and the background image, copy xpath, and then use selenium to find the element

Eg: driver.findElement (By.xpath ("xpath of the slider"))

3. Calculate the distance that the small slider needs to slide

This part is the most important, so you need to focus on recording and learning once, and you can solve the same problem immediately in the future.

Steps:

1. Save the image and background image of the slider.

As shown in the picture, these two pictures can be easily obtained by using selenium.

2. Clip the background image to the specified scale and area.

There are two more important parameters in this step:

1. The top value of the slider

2. The size ratio of the image currently displayed on the web page to the original image needs to be used to calculate the sliding distance.

Clipping uses BufferedImage's getSubimage method, which has a total of four parameters

Image = image.getSubimage (x, y, width, height)

X and y are the coordinate values of the upper left corner of the image after the screenshot. If x and y are both 0, then start from the upper left corner of the original image. Width and height are the length and width of the image after the screenshot, respectively.

In the sliding verification of a note, x is set to the width of the small slider, and y is set to the top,top of the small slider, that is, the pixel of the small slider from the boundary on the background image.

Width is set to the original width of the background image-the width of the small slider

Height is set to the height of the small slider

The final cut image is similar to this. Be sure to include the gap in the background image.

3. Binarize the small slider image.

Opencv (open source computer visual library) will be used from here.

First of all, convert the saved small slider picture to grayscale, and then quickly binarize the grayscale slide, which is either black or white. After understanding it, we know that the principle used in many machine recognition is similar to this.

The code is as follows:

/ / small slider Mat object Mat s_mat = Imgcodecs.imread (sFile.getPath ()); / / convert grayscale image Mat s_newMat = newMat (); Imgproc.cvtColor (s_mat, s_newMat, Imgproc.COLOR_BGR2GRAY); / / binary image binaryzation (s_newMat) Binaryzation is a method with Imgcodecs.imwrite (sFile.getPath (), s_newMat) in the source code.

4. Compare the small binary slider with the clipped background image.

I copied the code. I can't understand it, so I won't put it here.

I have studied it for a long time, because I have not learned opencv, I do not quite understand several methods of calculating procedure calls, but the final return value needs to be adjusted according to the actual situation, otherwise the success rate of verification is almost zero.

4. Press and hold the slider and slide

The sliding process should not allow the program to finish in one step, otherwise the page will think that you are a crawler and will fail to verify even if you can slide to the specified location. The sliding process should simulate manual operation as much as possible.

/ * simulated moving slider * @ param driver * @ param ele slider * @ param distance sliding distance * / public void move (ChromeDriver driver,WebElement ele,int distance) {int randomTime = 0; if (distance > 90) {randomTime = 250 } else if (distance > 80 & & distance 80 & & distance ucThre_new) {mat.put (j, I, WHITE); nWhite++;} else {mat.put (j, I, BLACK); nBlack++ } / / make sure the black word if (nBlack > nWhite) {for (j = 0; j < height; + + j) {for (I = 0; I < width; + + I) {nValue = (int) (mat.get (j, I) [0]) If (nValue = = 0) {mat.put (j, I, WHITE);} else {mat.put (j, I, BLACK) Delay the loading of the element and return the element when it appears NOSuchElement exception * @ param driver * @ param by * @ param count * @ return WebElement * @ throws Exception * / private static WebElement waitWebElement (WebDriver driver, By by, int count) throws Exception {WebElement webElement = null is thrown if there is no response after more than 500 seconds. Boolean isWait = false; for (int k = 0; k < count; kits +) {try {webElement = driver.findElement (by); if (isWait) System.out.println ("ok!"); return webElement;} catch (org.openqa.selenium.NoSuchElementException ex) {isWait = true If (k = = 0) System.out.print ("waitWebElement (" + by.toString () + ")"); else System.out.print ("."); Thread.sleep (50);} if (isWait) System.out.println ("outTime!") Return null;}} above is how to use Java+Selenium+OpenCV to simulate how to achieve web page sliding verification. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.