Shulou(Shulou.com)06/02 Report--

Log format: log_format usgateway'$http_clientip\ t$http_ServiceName\ t$http_uid\ t$http_sid\ t [$time_local]'\ t$request\ t$status\ t$body_bytes_sent\ tlog connectionrequests'\ t$remote_addr\ t$http_referer\ taccounthttpuseraccountagent` '\ t$request_body\ t$request_time\ troommsec' Log example: 10.10.45.152 Murray-[23/Jun/2017:17:37:42 + 0800] POST / sg HTTP/1.14055765910.10.130.100-Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1) Trident/6.0) {\ x22mbrNetName\ x22:\ X22\ xE9\ X87\ xE6\ X98\ X9F\ X22,\ x22nameCn\ X22:\ X22\ xE8\ X8A\ x92\ xE6\ X9e\ X9C\ xE7\ xBD\ x91,\ x22gender\ x22:\ x2211\ x22,\ x22birthday\ X22:\ x222010-10-01\ X22,\ x22mbrId\ X22:\ x2235954629\ X22,\ x22emailAddr\ X22:\ x22334740263@qq.com\ X22 \ x22mobileNo\ x22:\ x2215902074059\ x22} 0.0011498210662.427logstash configuration file: input {file {type = > "uSG_gateway_access" path = > ["/ usr/local/elk/elklog/nginxlog/log0/uSG_gateway_elk.log"]}} filter {ruby {init = > "@ kname = ['http_clientip','http_ServiceName','http_uid','http_sid','time_local' 'request','status','body_bytes_sent','connection_requests','remote_addr','http_referer','http_user_agent','request_body','request_time' 'msec'] "code = >" new_event = LogStash::Event.new (Hash [@ kname.zip (event.get (' message'). Split ('')]) new_event.remove ('@ timestamp') event.append (new_event) "} if [request] {ruby {init = >" @ kname = ['method','uri' 'verb'] "code = >" new_event = LogStash::Event.new (Hash [@ kname.zip (event.get (' request'). Split ('')]) new_event.remove ('@ timestamp') event.append (new_event) "} mutate {convert = > [" body_bytes_sent "," integer "," content_length "," integer "," upstream_response_time "," float "," request_time " "float"]} grok {match = > ["message", "% {IPORHOST:clientip}% {USER}% {USER}% {USER}\ [% {HTTPDATE:timestamp}\]]} date {match = > [" timestamp " "dd/MMM/YYYY:HH:mm:ss Z"] locale = > "en"} geoip {source = > "clientip"} useragent {source = > "http_user_agent" target = > "useragent"}} output {elasticsearch {hosts = > "10.10.45.200 output 8201" index = > "logstash-gateway-frontend-% {+ YYYY.MM.dd}"} Note: the log delimiter is the table key.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.