Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to intercept specific user agents in Nginx". In daily operation, I believe many people have doubts about how to intercept specific user agents in Nginx. Xiaobian consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts of "how to intercept specific user agents in Nginx". Next, please follow the editor to study!

Blacklist specific user agents in nginx

To configure the user agent blocking list, open your site's nginx configuration file and find the server definition section. The file may be placed in a different place, depending on your nginx configuration or linux version (e.g., / etc/nginx/nginx.conf,/etc/nginx/sites-enabled/,/usr/local/nginx/conf/nginx.conf,/etc/nginx/conf.d/).

The copy code is as follows:

Server {

Listen 80 default_server

Server_name xmodulo.com

Root / usr/share/nginx/html

....

}

After opening the configuration file and finding the server section, add the following if declaration somewhere within that section.

The copy code is as follows:

Server {

Listen 80 default_server

Server_name xmodulo.com

Root / usr/share/nginx/html

# case-sensitive matching

If ($http_user_agent ~ (antivirx | arian) {

Return 403

}

# case-independent matching

The copy code is as follows:

If ($http_user_agent ~ * (netcrawl | npbot | malicious)) {

Return 403

}

....

}

As you might expect, these if declarations use regular expressions to match any bad user string and return a 403 http status code to the matching object. Http_user_agent is a variable in a http request that contains a user agent string.' The ~ 'operator performs case-sensitive matching for user agent strings, while the' ~ * 'operator performs case-independent matching.' | the operator is logical or, so you can put a number of user agent keywords in the if declaration and block them all.

After modifying the configuration file, you must reload nginx to activate blocking:

$sudo / path/to/nginx-s reload

You can test user agent blocking by using wget with the "--user-agent" option.

$wget-user-agent "malicious bot" http://

Manage the blacklist of user agents in nginx

So far, I've shown how to block http requests from some user agents in nginx. What if you have many different types of web crawler robots to block?

Because the blacklist of user agents can grow a lot, it's not a good idea to put them in the server section of nginx. Instead, you can create a separate file that lists all blocked user agents. For example, let's create / etc/nginx/useragent.rules and define a graph that defines all blocked user agents in the following format.

$sudo vi / etc/nginx/useragent.rules

The copy code is as follows:

Map $http_user_agent $badagent {

Default 0

~ * malicious 1

~ * backdoor 1

~ * netcrawler 1

~ antivirx 1

~ arian 1

~ webbandit 1

}

Similar to the previous configuration,'~ * 'will match keywords in a case-insensitive manner, while' ~ 'will use case-sensitive regular expressions to match keywords. The "default 0" line means that user agents not listed in any other file will be allowed.

Next, open your site's nginx configuration file, find the section that contains http, and add the following line to a location in the http section.

The copy code is as follows:

Http {

.

Include / etc/nginx/useragent.rules

}

Note that the include declaration must appear before the server section (which is why we added it to the http section).

Now, open the section where the nginx configuration defines your server and add the following if declaration:

The copy code is as follows:

Server {

....

If ($badagent) {

Return 403

}

....

}

Finally, reload the nginx.

$sudo / path/to/nginx-s reload

Now, any user agent that contains the keywords listed in / etc/nginx/useragent.rules will be automatically disabled by nginx.

At this point, the study on "how to intercept specific user agents in Nginx" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.