Shulou(Shulou.com)06/02 Report--

This article refers to: https://www.cnblogs.com/yanshicheng/articles/9436373.html

Https://www.cnblogs.com/fawaikuangtu123/articles/10360264.html

1.logstash installation (jdk pre-installation 1.8)

Rpm-ivh logstash-6.6.2.rpm

two。 Upload database-driven jar package

Wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.tar.gz

Mkdir-p / usr/share/logstash/vendor/jar/jdbccd / usr/share/logstash/vendor/jar/jdbc [root@localhost soft] # cd / usr/share/logstash/vendor/jar/jdbc [root@localhost jdbc] # lltotal 984 RWMUM. 1 logstash logstash 1006959 Jul 11 19:43 mysql-connector-java-5.1.48-bin.jar [root@localhost jdbc] #

3. Database creates a library and authorizes users

Create database nginxlog;use nginxlog;CREATE TABLE `refererlog` (`log_ ip` varchar) DEFAULT NULL, `log_ time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `status` int (6) DEFAULT NULL, `http_ referer`text, `AgentVersion`varchar (512) DEFAULT NULL,) grant all on nginxlog.* to nginxlog@'%' identified by '123456'

4. Install the logstash-output-jdbc plug-in

Vim / usr/share/logstash/Gemfile# source "https://rubygems.org" replace foreign source comments with domestic source" https://gems.ruby-china.com/"/usr/share/logstash/bin/logstash-plugin install logstash-output-jdbcValidating logstash-output-jdbcInstalling logstash-output-jdbcInstallation successful / usr/share/logstash/bin/logstash-plugin list | grep jdbclogstash-input-jdbclogstash-output-jdbc

5.nginx log format settin

Log_format access_log_json'{"client_ip": "$remote_addr", "log_time": "$time_local", "request": "$request", "status": "$status", "body_bytes_sent": "$body_bytes_sent", "http_referer": "$http_referer", "AgentVersion": "$http_user_agent", "upstream_addr": "$upstream_addr", "request_time": "$request_time" "upstream_response_time": "$upstream_response_time"}

6.

[root@localhost conf.d] # cat consumer_log.conf

/ etc/logstash/conf.d

Input {file {path = > "/ usr/local/tengine-2.1.2/logs/sxt-consumer.log" start_position = > "beginning" stat_interval = > "2" codec = > "json"}} filter {if [status]! = "200" {drop {}} output {jdbc {connection_string = >" jdbc:mysql://192.168.14.61/nginxlog?user=nginxlog&password=123456&useUnicode=true " & characterEncoding=UTF8 "statement = > [" insert into consumerlog (client_ip) Status,http_referer,AgentVersion) VALUES (?) "," client_ip "," status "," http_referer "," AgentVersion "]}}

7. Start logstash

Systemctl start logstash

8. Validate data

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.