Shulou(Shulou.com)06/01 Report--

This article mainly explains "the source code analysis of nginx log module". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "nginx log module source code analysis" it!

Introduction to ngx_http_log_module

The ngx_http_log_module module writes access logs in the specified format.

At the end of the request processing, the log is accessed according to the configuration context of the request path. If an internal jump occurs during request processing (see another nginx jump article), the path at the end of the request may be different from the original request path.

Official module usage instructions http://nginx.org/en/docs/http/ngx_http_log_module.html

Configuration example description

The relevant configuration instructions in nginx.conf are:

Log_format proxyformat "$time_iso8601 $remote_addr:$remote_port $server_addr:$server_port $upstream_addr $request_time $upstream_response_time $status $upstream_status $request_leng

Th $body_bytes_sent\ "$request_method $scheme://$http_host$request_uri $server_protocol\"\ "$http_referer\"\ "$http_user_agent\"

Access_log syslog:facility=local3,severity=info,server=127.0.0.1:514,tag=tengine proxyformat

Log_format gzip'$remote_addr-$remote_user [$time_local]'"$request" $status $bytes_sent''"$http_referer"$http_user_agent"$gzip_ratio"; access_log / spool/logs/nginx-access.log gzip buffer=32k; syntax format: access_logpath [format [buffer=size]]

Access_logoff; default: access_log logs/access.log combined; configuration takes effect space context: http, server, location, if in location, limit_except

Set the path, format, and buffer size for the access log (nginx access log supports caching).

Multiple logs can be specified at the same configuration level.

A specific value, off, cancels all access_log instructions at the current configuration level.

If no log format is specified, the predefined combined format is used.

The size of the buffer cannot exceed the size of atomic writes to disk files.

For FreeBSD, the buffer size is unlimited.

The path to the log file can contain variables (0.7.6 +), but there are some restrictions for this type of log:

You should have permission to create files in the directory.

Invalid write buffer.

Each log write opens and closes the file. Frequently used file descriptors can be stored in the cache, and write operations can continue to write to the old file for the time specified by the valid parameter of the open_log_file_cache instruction.

Each log write operation checks whether the requested root directory exists (that is, ngx root configuration), and if it does not exist, the log will not be created. It is a good idea to specify both root and access_log at the same level:

Server {root / spool/vhost/data/$host; access_log / spool/vhost/logs/$host;.} syntax: log_formatname string...; default: log_format combined "..."; configuration effective space context: http

Specifies the format of the log.

The log format allows you to include normal variables and variables that exist only when the log is written:

$body_bytes_sent

The number of bytes sent to the client, excluding the size of the response header; this variable is compatible with the "% B" parameter in the Apache module mod_log_config.

$bytes_sent

Total number of bytes sent to the client.

$connection

The serial number of the connection.

$connection_requests

The number of requests currently obtained through one connection.

$msec

Log write time. The unit is seconds and the precision is milliseconds.

$pipe

If the request is sent through the HTTP pipeline (pipelined), the pie value is "p", otherwise it is ".".

$request_length

The length of the request (including the request line, the request header and the request body).

$request_time

Request processing time, in seconds, precision milliseconds; from reading the first byte of the client to log writing after the last character is sent to the client.

$status

Response status.

$time_iso8601

Local time in ISO8601 standard format.

$time_local

Local time under the common log format.

The response header sent to the client has a "sent_http_" prefix, such as $sent_http_content_range.

The configuration always contains a predefined "combined" log format:

Log_format combined'$remote_addr-$remote_user [$time_local]'"$request" $status $body_bytes_sent''"$http_referer"$http_user_agent"; syntax: open_log_file_cachemax=N [inactive=time] [min_uses=N] [valid=time]

Open_log_file_cacheoff; default: open_log_file_cacheoff; configuration takes effect space context: http, server, location

Define a cache to store log file descriptors that contain variables in frequently used file names. The directive contains the following parameters:

Max

Sets the maximum number of descriptors in the cache; if the cache is full, the least recently used (LRU) descriptor will be turned off.

Inactive

Sets how long the cache file descriptor is turned off without being accessed; the default is 10 seconds.

Min_uses

Sets the minimum number of visits to keep the file descriptor in the cache at the time specified by the inactive parameter; the default is 1.

Valid

Set a period of time to check whether the file still exists under the same name after the timeout; the default is 60 seconds.

Off

Disable caching.

Examples of use:

Open_log_file_cache max=1000 inactive=20s valid=1m min_uses=2;nginx log_module source code analysis

First pay attention to the initialization of the log module

Static ngx_http_module_t ngx_http_log_module_ctx = {

NULL, / * preconfiguration * /

Ngx_http_log_init, / * postconfiguration * /

Ngx_http_log_create_main_conf, / * create main configuration * /

NULL, / * init main configuration * /

NULL, / * create server configuration * /

NULL, / * merge server configuration * /

Ngx_http_log_create_loc_conf, / * create location configuration * /

Ngx_http_log_merge_loc_conf / * merge location configuration * /

}

Static void *

Ngx_http_log_create_main_conf (ngx_conf_t * cf)

{

Ngx_http_log_main_conf_t * conf

Ngx_http_log_fmt_t * fmt

Conf = ngx_pcalloc (cf- > pool, sizeof (ngx_http_log_main_conf_t))

If (conf = = NULL) {

Return NULL

}

/ / initialize format

If (& conf- > formats, cf- > pool, 4, sizeof (ngx_http_log_fmt_t))! = NGX_OK)

{

Return NULL

}

Return conf

}

Static ngx_int_t ngx_http_log_init (ngx_conf_t * cf)

{

/ / this block has the logic to determine whether to use the default combine. In tengine, there is a custom wave, so it does not care.

/ / add handler to the log phase

Cmcf = ngx_http_conf_get_module_main_conf (cf, ngx_http_core_module)

H = ngx_array_push (& cmcf- > phases [NGX _ HTTP_LOG_PHASE] .handlers)

If (h = = NULL) {

Return NGX_ERROR

}

* h = ngx_http_log_handler

Return NGX_OK

}

In the handler of the log_format instruction, save the fmt

Static char * ngx_http_log_set_format (ngx_conf_t * cf, ngx_command_t * cmd, void * conf)

{

/ / obtain loc_main_conf

Ngx_http_log_main_conf_t * lmcf = conf

/ / initialize fmt

Fmt = lmcf- > formats.elts

Fmt = ngx_array_push (& lmcf- > formats)

If (fmt = = NULL) {

Return NGX_CONF_ERROR

}

/ / assign name

Fmt- > name = value [1]

/ / initialize flushes and ops

Fmt- > flushes = ngx_array_create (cf- > pool, 4, sizeof (ngx_int_t))

If (fmt- > flushes = = NULL) {

Return NGX_CONF_ERROR

}

Fmt- > ops = ngx_array_create (cf- > pool, 16, sizeof (ngx_http_log_op_t))

If (fmt- > ops = = NULL) {

Return NGX_CONF_ERROR

}

/ / finally compile the whole log format

Return ngx_http_log_compile_format (cf, fmt- > flushes, fmt- > ops, cf- > args, 2)

}

Static char * ngx_http_log_compile_format (ngx_conf_t * cf, ngx_array_t * flushes,ngx_array_t * ops, ngx_array_t * args, ngx_uint_t s)

{

/ / traverse all args

For (/ * void * /; s

< args->

Nelts; slots +) {

/ / check whether there is a specified log format in the existing ngx_http_log_vars

For (v = ngx_http_log_vars; v-> name.len; vault +) {

If (v-> name.len = = var.len & & ngx_strncmp (v-> name.data, var.data, var.len) = = 0)

{

/ / assign the corresponding method and length to the corresponding op, waiting for writing

Op- > len = v-> len

Op- > getlen = NULL

Op- > run = v-> run

Op- > data = 0

Goto found

}

}

/ / if not, call this function

If (ngx_http_log_variable_compile (cf, op, & var)! = NGX_OK) {

Return NGX_CONF_ERROR

}

If (len) {

Op- > len = len

Op- > getlen = NULL

/ / if the length is small, record it directly with data.

If (len run = ngx_http_log_copy_short

Op- > data = 0

/ / record data

While (len--) {

Op- > data run = ngx_http_log_copy_long

P = ngx_pnalloc (cf- > pool, len)

If (p = = NULL) {

Return NGX_CONF_ERROR

}

Ngx_memcpy (p, data, len)

Op- > data = (uintptr_t) p

}

}

}

Static ngx_int_t ngx_http_log_variable_compile (ngx_conf_t * cf, ngx_http_log_op_t * op, ngx_str_t * value)

{

Ngx_int_t index

Index = ngx_http_get_variable_index (cf, value)

Op- > len = 0

Op- > getlen = ngx_http_log_variable_getlen

Op- > run = ngx_http_log_variable

Op- > data = index

}

/ / and these parameters that can be entered into the log are the ngx_http_variables_add_core_vars function in the core_preconfiguration function.

Ngx_int_t ngx_http_variables_add_core_vars (ngx_conf_t * cf)

{

Cmcf- > variables_keys = ngx_pcalloc (cf- > temp_pool)

Sizeof (ngx_hash_keys_arrays_t))

For (cv = ngx_http_core_variables; cv- > name.len; cv++) {

V = ngx_palloc (cf- > pool, sizeof (ngx_http_variable_t))

If (v = = NULL) {

Return NGX_ERROR

}

* v = * cv

Rc = ngx_hash_add_key (cmcf- > variables_keys, & v-> name, v, NGX_HASH_READONLY_KEY)

If (rc = = NGX_OK) {

Continue

}

If (rc = = NGX_BUSY) {

Ngx_conf_log_error (NGX_LOG_EMERG, cf, 0, "conflicting variable name\"% V\ "", & v-> name)

}

Return NGX_ERROR

}

Next, you can take a look at the entire ngx_http_core_variables.

Static ngx_http_variable_t ngx_http_core_variables [] = {

{ngx_string ("http_host"), NULL, ngx_http_variable_header

Offsetof (ngx_http_request_t, headers_in.host), 0,0}

{ngx_string ("http_user_agent"), NULL, ngx_http_variable_header

Offsetof (ngx_http_request_t, headers_in.user_agent), 0,0}

{ngx_string ("http_referer"), NULL, ngx_http_variable_header

Offsetof (ngx_http_request_t, headers_in.referer), 0,0}

# if (NGX_HTTP_GZIP)

{ngx_string ("http_via"), NULL, ngx_http_variable_header

Offsetof (ngx_http_request_t, headers_in.via), 0,0}

# endif

# if (NGX_HTTP_X_FORWARDED_FOR)

{ngx_string ("http_x_forwarded_for"), NULL, ngx_http_variable_headers

Offsetof (ngx_http_request_t, headers_in.x_forwarded_for), 0,0}

# endif

{ngx_string ("http_cookie"), NULL, ngx_http_variable_cookies

Offsetof (ngx_http_request_t, headers_in.cookies), 0,0}

{ngx_string ("content_length"), NULL, ngx_http_variable_content_length

0, 0, 0}

{ngx_string ("content_type"), NULL, ngx_http_variable_header

Offsetof (ngx_http_request_t, headers_in.content_type), 0,0}

{ngx_string ("host"), NULL, ngx_http_variable_host, 0,0,0}

{ngx_string ("binary_remote_addr"), NULL

Ngx_http_variable_binary_remote_addr, 0, 0, 0}

{ngx_string ("remote_addr"), NULL, ngx_http_variable_remote_addr, 0,0,0}

{ngx_string ("remote_port"), NULL, ngx_http_variable_remote_port, 0,0,0}

{ngx_string ("proxy_protocol_addr"), NULL

Ngx_http_variable_proxy_protocol_addr, 0, 0, 0}

{ngx_string ("server_addr"), NULL, ngx_http_variable_server_addr, 0,0,0}

{ngx_string ("server_port"), NULL, ngx_http_variable_server_port, 0,0,0}

{ngx_string ("server_protocol"), NULL, ngx_http_variable_request

Offsetof (ngx_http_request_t, http_protocol), 0,0}

{ngx_string ("scheme"), NULL, ngx_http_variable_scheme, 0,0,0}

{ngx_string ("https"), NULL, ngx_http_variable_https, 0,0,0}

{ngx_string ("full_request"), NULL, ngx_http_variable_full_request

0, 0, 0}

{ngx_string ("normalized_request"), NULL, ngx_http_variable_normalized_request

0, 0, 0}

{ngx_string ("request_uri"), NULL, ngx_http_variable_request

Offsetof (ngx_http_request_t, unparsed_uri), 0,0}

{ngx_string ("raw_uri"), NULL, ngx_http_variable_request

Offsetof (ngx_http_request_t, raw_uri), 0,0}

{ngx_string ("uri"), NULL, ngx_http_variable_request

Offsetof (ngx_http_request_t, uri)

NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("document_uri"), NULL, ngx_http_variable_request

Offsetof (ngx_http_request_t, uri)

NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("request"), NULL, ngx_http_variable_request_line, 0,0,0}

{ngx_string ("document_root"), NULL

Ngx_http_variable_document_root, 0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("realpath_root"), NULL

Ngx_http_variable_realpath_root, 0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("query_string"), NULL, ngx_http_variable_request

Offsetof (ngx_http_request_t, args)

NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("args")

Ngx_http_variable_set_args

Ngx_http_variable_request

Offsetof (ngx_http_request_t, args)

NGX_HTTP_VAR_CHANGEABLE | NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("is_args"), NULL, ngx_http_variable_is_args

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("request_filename"), NULL

Ngx_http_variable_request_filename, 0

NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("server_name"), NULL, ngx_http_variable_server_name, 0,0,0}

{ngx_string ("request_method"), NULL

Ngx_http_variable_request_method, 0

NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("remote_user"), NULL, ngx_http_variable_remote_user, 0,0,0}

{ngx_string ("bytes_sent"), NULL, ngx_http_variable_bytes_sent

0, 0, 0}

{ngx_string ("body_bytes_sent"), NULL, ngx_http_variable_body_bytes_sent

0, 0, 0}

{ngx_string ("pipe"), NULL, ngx_http_variable_pipe

0, 0, 0}

{ngx_string ("request_completion"), NULL

Ngx_http_variable_request_completion

0, 0, 0}

{ngx_string ("request_body"), NULL

Ngx_http_variable_request_body

0, 0, 0}

{ngx_string ("request_body_file"), NULL

Ngx_http_variable_request_body_file

0, 0, 0}

{ngx_string ("request_length"), NULL, ngx_http_variable_request_length

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("request_time"), NULL, ngx_http_variable_request_time

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("status"), NULL

Ngx_http_variable_status, 0

NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("sent_http_content_type"), NULL

Ngx_http_variable_sent_content_type, 0, 0, 0}

{ngx_string ("sent_http_content_length"), NULL

Ngx_http_variable_sent_content_length, 0, 0, 0}

{ngx_string ("sent_http_location"), NULL

Ngx_http_variable_sent_location, 0, 0, 0}

{ngx_string ("sent_http_last_modified"), NULL

Ngx_http_variable_sent_last_modified, 0, 0, 0}

{ngx_string ("sent_http_connection"), NULL

Ngx_http_variable_sent_connection, 0, 0, 0}

{ngx_string ("sent_http_keep_alive"), NULL

Ngx_http_variable_sent_keep_alive, 0, 0, 0}

{ngx_string ("sent_http_transfer_encoding"), NULL

Ngx_http_variable_sent_transfer_encoding, 0, 0, 0}

{ngx_string ("sent_http_cache_control"), NULL, ngx_http_variable_headers

Offsetof (ngx_http_request_t, headers_out.cache_control), 0,0}

{ngx_string ("limit_rate"), ngx_http_variable_request_set_size

Ngx_http_variable_request_get_size

Offsetof (ngx_http_request_t, limit_rate)

NGX_HTTP_VAR_CHANGEABLE | NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("connection"), NULL

Ngx_http_variable_connection, 0, 0, 0}

{ngx_string ("connection_requests"), NULL

Ngx_http_variable_connection_requests, 0, 0, 0}

{ngx_string ("nginx_version"), NULL, ngx_http_variable_nginx_version

0, 0, 0}

{ngx_string ("hostname"), NULL, ngx_http_variable_hostname

0, 0, 0}

{ngx_string ("pid"), NULL, ngx_http_variable_pid

0, 0, 0}

{ngx_string ("msec"), NULL, ngx_http_variable_msec

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("dollar"), NULL, ngx_http_variable_dollar

0, 0, 0}

{ngx_string ("host_comment"), NULL, ngx_http_variable_host_comment

0, 0, 0}

{ngx_string ("unix_time"), NULL, ngx_http_variable_unix_time

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("year"), NULL, ngx_http_variable_year

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("year2"), NULL, ngx_http_variable_year2

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("month"), NULL, ngx_http_variable_month

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("day"), NULL, ngx_http_variable_day

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("hour"), NULL, ngx_http_variable_hour

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("hour12"), NULL, ngx_http_variable_hour12

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("minute"), NULL, ngx_http_variable_minute

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("second"), NULL, ngx_http_variable_second

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("time_iso8601"), NULL, ngx_http_variable_time_iso8601

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("time_local"), NULL, ngx_http_variable_time_local

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("time_http"), NULL, ngx_http_variable_time_http

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

# if (NGX_HAVE_TCP_INFO)

{ngx_string ("tcpinfo_rtt"), NULL, ngx_http_variable_tcpinfo

0, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("tcpinfo_rttvar"), NULL, ngx_http_variable_tcpinfo

1, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("tcpinfo_snd_cwnd"), NULL, ngx_http_variable_tcpinfo

2, NGX_HTTP_VAR_NOCACHEABLE, 0}

{ngx_string ("tcpinfo_rcv_space"), NULL, ngx_http_variable_tcpinfo

3, NGX_HTTP_VAR_NOCACHEABLE, 0}

# endif

{ngx_null_string, NULL, NULL, 0, 0, 0}

}

This block is initialized in ngx_http_variables_init_vars

Ngx_int_t ngx_http_variables_init_vars (ngx_conf_t * cf)

{

Cmcf = ngx_http_conf_get_module_main_conf (cf, ngx_http_core_module)

V = cmcf- > variables.elts

Key = cmcf- > variables_keys- > keys.elts

For (I = 0; I)

< cmcf->

Variables.nelts; iTunes +) {

For (n = 0; n)

< cmcf->

Variables_keys- > keys.nelts; nasty +) {

Av = key [n] .value

If (v [I] .name.len = = key [n] .key.len & & ngx_strncmp (v [I] .name.data, key [n] .key.data, v [I] .name.len) = 0)

{

/ / assign get_handler and data

V [I] .get _ handler = av- > get_handler

V [I] .data = av- > data

}

At this point, I believe that everyone on the "nginx log module source code analysis" have a deeper understanding, might as well to the actual operation of it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.