Shulou(Shulou.com)06/02 Report--

Common expressions for grok rules

● USERNAME or USER

User name, a string consisting of numbers, case, and special characters (. _ -)

For example: 1234, Bob, Alex.Wong, etc.

● EMAILLOCALPART

The email user name section, the first bit consists of uppercase and lowercase letters, and the other bits are a string of numbers, uppercase and lowercase, and special characters (_. +-=:). Note that domestic QQ pure digital mailbox accounts cannot be matched, and the rules need to be modified.

For example: stone, Gary_Lu, abc-123, etc.

● EMAILADDRESS

E-mail

For example: stone@abc.com, Gary_Lu@gmail.com, abc-123@163.com, etc.

● HTTPDUSER

The user of the Apache server, which can be EMAILADDRESS or USERNAME

● INT

Integers, including 0 and positive and negative integers

For example: 0,123,43987, etc.

● BASE10NUM or NUMBER

Decimal numbers, including integers and decimals

For example: 0,18,5.23, etc.

● BASE16NUM

Hexadecimal number, integer

For example: 0x0045fa2d,-0x3F8709, etc.

● BASE16FLOAT

Hexadecimal numbers, integers and decimals

● WORD

String, including numbers and uppercase and lowercase letters

For example: String, 3529345, ILoveYou, etc.

● NOTSPACE

A string without any spaces

● SPACE

Space string

● QUOTEDSTRING or QS

Quoted string

For example: "This is an apple", "What is your name?'" Etc.

● UUID

Standard UUID

For example: 550E8400-E29B-11D4-A716-446655440000

● MAC

MAC address, which can be the MAC address in the Cisco device, or the MAC address of the general purpose or Windows system.

● IP

IP address, IPv4 or IPv6 address

For example: 127.0.0.1, FE80:0000:0000:0000:AAAA:0000:00C2:0002, etc.

● HOSTNAME

Host name

● IPORHOST

IP or host name

● HOSTPORT

Hostname (IP) + Port

For example, 127.0.0.1 3306, api.stozen.NET:8000, etc.

● PATH

Path, path format in Unix or Windows systems

For example: / usr/local/nginx/sbin/nginx, c:\ windows\ system32\ clr.exe, etc.

● URIPROTO

URI protocol

For example: http, ftp, etc.

● URIHOST

URI host

For example: www.stozen.Net, 10.0.0.1VR 22, etc.

● URIPATH

URI path

For example: / / www.stozen.net/abc/, / api.PHP, etc.

● URIPARAM

GET parameter in URI

For example:? a=1&b=2&c=3

● URIPATHPARAM

URI path + GET parameter

For example: / / www.stozen.net/abc/api.php?a=1&b=2&c=3

● URI

Complete URI

For example: http://www.stozen.net/abc/api.php?a=1&b=2&c=3

Date-time expression

● MONTH

Month name

For example: Jan, January, etc.

● MONTHNUM

Monthly figures

For example: 03, 9, 12, etc.

● MONTHDAY

Date number

For example: 03, 9, 31, etc.

● DAY

Name of day of the week

For example: Mon, Monday, etc.

● YEAR

Year number

● HOUR

Hourly number

● MINUTE

Minute number

● SECOND

Second digit

● TIME

time

For example, 00:01:23

● DATE_US

Us date format

For example, 10-15-1982, 10-15-15, 1982, etc.

● DATE_EU

European date format

For example, 15-10-1982, 15-10-10, 1982, 15.10.1982, etc.

● ISO8601_TIMEZONE

ISO8601 time format

For example: + 10:23,-1023, etc.

● TIMESTAMP_ISO8601

ISO8601 timestamp format

For example: 2016-07-03T00:34:06+08:00

● DATE

Date, US date% {DATE_US} or European date% {DATE_EU}

● DATESTAMP

Full date + time

For example: 07-03-2016 00:34:06

● HTTPDATE

Http default date format

For example: 03/Jul/2016:00:36:53 + 0800

● Log expression

LOGLEVEL

Log level

For example: Alert, alert, ALERT, Error, etc.

Grok rule configuration instance

Grok rule:% {expression: custom name}, where the custom name will be displayed in the kibana interface and used for filtering.

Take the nginx log rule as an example, and list the comparison table of grok rule and nginx log format:

Grok rule

Nginx log format

% {IPORHOST:http_host}

In IPORHOST format, corresponding to IP address

$remote_addr

-

-

-

$remote_user

\ [% {HTTPDATE:timestamp}\]

In HTTPDATE format, [and] characters need to be escaped with\

[$time_local]

\ "(?:% {WORD:http_verb}% {NOTSPACE:http_request} HTTP/% {NUMBER:http_version}\"

% {NUMBER:http_status_code} (?:% {NUMBER:bytes_read} | -))

Http_verb (string), http_request (string without any spaces) and http_version (number) correspond to $request,http_status_code (number) corresponds to $status,bytes_read (number) corresponds to $body_bytes_sent

"$request" $status

$body_bytes_sent

% {QS:referrer}

QS is a quoted string

"$http_referer"

% {QS:agent}

"$http_user_agent"

% {QS:forwarded}

"$http_x_forwarded_for"

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.