Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to prevent interface theft in Spring Boot anti-crawler". In daily operation, I believe many people have doubts about how to prevent interface theft in Spring Boot anti-crawler. Xiaobian consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubt of "how to prevent interface theft in Spring Boot anti-crawler". Next, please follow the editor to study!

System requirements

Development based on spring-boot (either spring-boot1.x or spring-boot2.x)

Need to use redis

Work flow

Kk-anti-reptile uses Filter based on Servlet specification to filter requests, instantiates a Filter internally through spring-boot 's extension point mechanism, and injects it into the Spring container FilterRegistrationBean, and then injects it into the Servlet container through Spring, thus realizing the filtering of requests.

Within the filtering Filter of kk-anti-reptile, various filtering rules are woven through the responsibility chain pattern, and an abstract interface is provided, which can be extended by the caller.

When Filter is called, the chain filters the request. If the filter fails, it intercepts the request, returns the status code 509, and outputs the verification code input page. After the output verification code is correct, the filtering rule chain is called to reset the rules.

At present, there are two rules in the rule chain

Ip-rule

Ip-rule counts the number of requests in the current time window through the time window. If the number of requests is less than the specified maximum, it can be passed, otherwise it will not be passed. The time window, maximum number of requests, ip whitelist, etc., can be configured.

Ua-rule

By judging the User-Agent carried by the request, ua-rule can get the operating system, device information, browser information, etc., and can configure various dimensions to filter the request.

After hitting the rule

After hitting the crawler and anti-theft brushing rule, the request is blocked and a verification code is generated to remove the blocking. The verification code can be combined in a variety of ways. If the client can enter the verification code correctly, it can continue to access

There are three forms of CAPTCHA: Chinese, English letters + numbers, and simple arithmetic. Each form also has two picture formats: static picture and GIF dynamic picture. At present, there are six kinds of CAPTCHA, and all types of CAPTCHA will appear randomly. At present, it is very difficult to identify by technical means, which can effectively prevent reptiles from crawling data on a large scale.

Access and use

Access to the backend is very simple. You only need to refer to kk-anti-reptile 's maven dependency and configure to enable kk-anti-reptile to join the maven dependency.

Cn.keking.project kk-anti-reptile 1.0.0-SNAPSHOT

Configure to enable kk-anti-reptile

Anti.reptile.manager.enabled=true

The frontend needs to intercept the ajax where the request is sent uniformly. After intercepting the request return status code 509, a new page pops up, and the response content is transferred to the page, and then the backend API baseUrl parameter is passed to the page. Take the axios request as an example:

Import axios from 'axios'; import {baseUrl} from'. / config'; axios.interceptors.response.use (data = > {return data;}, error = > {if (error.response.status = 509) {let html = error.response.data; let verifyWindow = window.open ("", "_ blank", "height=400,width=560"); verifyWindow.[ XSS _ clean] (html); verifyWindow.document.getElementById ("baseUrl"). Value = baseUrl }}); export default axios

Be careful

Bootstrap is required for apollo-client

For users using apollo configuration center, since @ ConditionalOnProperty is used internally in the component, to add the following sample configuration to application.properties/bootstrap.properties (apollo-client requires version 0.10.0 or above), see apollo bootstrap instructions for more information.

Apollo.bootstrap.enabled = true

Need to have Redisson

Connection if Redisson,kk-anti-reptile is used in the project, the RedissonClient instance object will be obtained automatically. If not, you need to add the following Redisson connection configuration in the configuration file:

Spring.redisson.address=redis://192.168.1.204:6379 spring.redisson.password=xxx

Configuration list

In spring-boot, all configurations are automatically prompted and described in the configuration file, as shown in the following figure:

All configurations are prefixed with anti.reptile.manager. The following are all configuration items and instructions:

At this point, the study on "how to prevent interface theft in Spring Boot anti-crawler" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.