Shulou(Shulou.com)06/01 Report--

This article is about how to implement the dynamic blocking of ip blacklist in nginx. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

When a website is maliciously requested, blocking IP is an important means. It would be too low to configure it on nginx every time it is blocked. We need to control the nginx IP blacklist more easily.

1. Scheme

The blacklist is persisted to mysql (the common scheme is redis, but it is not good for control, such as: different IP settings for different validity periods, IP CRUD, statistics, etc.)

Through lua-nginx-module, a piece of memory (lua_shared_dict) is opened up in nginx, and lua regularly refreshes the blacklist from mysql to lua_shared_dict.

All requests go to IP check in lua_shared_dict.

two。 Installation

2.1 install luajit

Cd LuaJIT-2.0.5makemake install PREFIX=/usr/local/luajit

2.2. When installing nginx, compile the lua module into

Export LUAJIT_LIB=/usr/local/luajit/libexport LUAJIT_INC=/usr/local/luajit/include/luajit-2.1. / configure-- prefix=/nginx\-- with-ld-opt= "- Wl,-rpath,/usr/local/luajit/lib"\-- add-module=/opt/ngx_devel_kit-0.3.1rc1\-- add-module=/opt/lua-nginx-module-0.10.14rc3 make-j2make installln-s / nginx/sbin/nginx / usr/sbin/nginx

3. Configuration

3.1 nginx configuration

Http {server_tokens off; lua_package_path "/ usr/local/lib/lua/?.lua;;"; lua_shared_dict ip_blacklist 4m;} server {set $real_ip $remote_addr; if ($http_x_forwarded_for ~ "^ (\ d+\.\ d+)") {set $real_ip $1 } # Management information. Visit the URL to view the IP blacklist information location / get-ipblacklist-info {access_by_lua_file conf/lua/get_ipblacklist_info.lua;} # synchronous URL in nginx. Call the URL through a scheduled task to realize the regular refresh of the IP blacklist from mysql to nginx location / sync-ipblacklist {access_by_lua_file conf/lua/sync_ipblacklist.lua } # production domain configuration. All location that need to be controlled by IP blacklist should include the following statement location / {access_by_lua_file conf/lua/check_realip.lua;}}

The nginx server configures the following crrontab

* / usr/bin/curl-o / dev/null-s http://127.0.0.1/sync-ipblacklist > / dev/null 2 > & 1

3.2 lua script

Sync_ipblacklist.lua

Local mysql_host = "ip of mysql server" local mysql_port = 3306local database = "dbname" local username = "user" local password = "password"-- update ip_blacklist from mysql once every cache_ttl secondslocal cache_ttl = 1local mysql_connection_timeout = 1000 local client_ip = ngx.var.real_iplocal ip_blacklist = ngx.shared.ip_blacklistlocal last_update_time = ip_blacklist:get ("last_update_time") If last_update_time = = nil or last_update_time < (ngx.now ()-cache_ttl) then local mysql = require "resty.mysql"; local red = mysql:new (); red:set_timeout (mysql_connect_timeout) Local ok, err, errcode, sqlstate = red:connect {host = mysql_host, port = mysql_port, database = database, user = username, password = password, charset = "utf8", max_packet_size = 1024 * 1024,} if not ok then ngx.log (ngx.ERR, "mysql connection error while retrieving ip_blacklist:". Err); else new_ip_blacklist, err, errcode, sqlstate = red:query ("select ip_addr from ip_blacklist where status = 0 order by create_time desc limit 10000", 100) if not new_ip_blacklist then ngx.log (ngx.ERR, "bad result. Errcode:". Errcode.. "sqlstate:" Sqlstate.. "err:" Err); return end ip_blacklist:flush_all (); for K1, v1 in pairs (new_ip_blacklist) do for K2, v2 in pairs (v1) do ip_blacklist:set; endend ip_blacklist:set ("last_update_time", ngx.now ()); endend ngx.say ("sync successful")

Get_ipblacklist_info.lua

-- call URL to view blacklist information-10,000 IP consumes less than 1.5m ngx.shared memory-- getting all KEY will block other normal requests' access to ngx.shared memory, so only a few key can be used to display require "resty.core.shdict" ngx.say ("total space:". Ngx.shared.ip_blacklist:capacity ().. Ngx.say ("free space:". Ngx.shared.ip_blacklist:free_space ().. Ngx.say ("last update time:". Os.date ("% Y%m%d_%H:%M:%S", ngx.shared.ip_blacklist:get ("last_update_time")).. Ngx.say ("first 100keys:"); ngx.say ("- -"); ip_blacklist = ngx.shared.ip_blacklist:get_keys; for key, value in pairs (ip_blacklist) do ngx.say (key. ":" Value.. "); end

Check_realip.lua

If ngx.shared.ip_blacklist:get (ngx.var.real_ip) then return ngx.exit (ngx.HTTP_FORBIDDEN); end

3.3 Database design

CREATE TABLE `ip_ blacklist` (`id` int (11) NOT NULL AUTO_INCREMENT, `ip_ addr` varchar (15) COLLATE utf8mb4_bin DEFAULT NULL, `status` int (11) DEFAULT'0' COMMENT'0: valid is valid, 1: invalid is invalid', `effective_ hour` decimal (11J 2) DEFAULT '24' COMMENT' is valid Unit: hour', 'ip_ source` varchar' COLLATE utf8mb4_bin DEFAULT NULL COMMENT 'blacklist source', `create_ time` datetime DEFAULT CURRENT_TIMESTAMP, `modify_ time` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `remark` varchar (255) COLLATE utf8mb4_bin DEFAULT NULL COMMENT 'remarks', PRIMARY KEY (`id`) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin CREATE PROCEDURE proc_ip_blacklist_status_update ()-change the expired IP state to invalid begin update ip_blacklist set status=1 where date_add (create_time,INTERVAL effective_hour hour) < now (); commit;end; CREATE EVENT job_ip_blacklist_status_updateON SCHEDULE EVERY 1 MINUTEON COMPLETION PRESERVEENABLEDOcall proc_ip_blacklist_status_update ()

4 CRUD

Blacklists are generated either manually, automatically, or both.

The automatic way is to analyze the elk log through python and automatically write malicious IP to mysql, which is a big topic and is not covered here.

You can manually view the elk request log by human flesh, find malicious IP, and manually fill in mysql. Here we recommend an open source CRUD tool. The user experience is very nice (much better than direct navicat). Of course, you can also write it yourself.

Project address: https://github.com/jonseg/crud-admin-generator

The power of the project is that all the tables help you generate menus, and then the CRUD of these tables is used directly.

See the official instructions for the specific operation, so I won't repeat it.

Thank you for reading! On "how to achieve ip blacklist dynamic ban in nginx" this article is shared here, I hope the above content can be of some help to you, so that you can learn more knowledge, if you think the article is good, you can share it out for more people to see it!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.