Shulou(Shulou.com)06/02 Report--

This article mainly shows you "how to use Nginx pure configuration to achieve real-time log reporting", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "how to use Nginx pure configuration to achieve real-time log reporting" this article.

Preface

Nginx is a commonly used load balancing gateway. A large number of logs will be generated. However, because the configuration file of Nginx is a declarative programming paradigm, it is not convenient to describe the flow control, so it is impossible to report the log through simple instructions.

Usually the log report of Nginx needs to write a shell script or other language script to regularly parse the log file of Nginx, and then report it.

Real-time log reporting can be realized by using NJS module.

However, due to the limitation of the instructions supported by the NJS module, it is impossible to report the log well through a single instruction. Non-blocking real-time log reporting can be achieved through the combination of multiple instructions.

The scheme is implemented in Nginx and does not rely on other processes such as Node, Python and so on.

Realization idea

There are many Nginx instructions, and the following is a recently explored implementation. If there is a more elegant way to achieve it, you are welcome to leave a message.

Although there is a powerful Njs module that can write JS scripts, the instructions of the NJS module have many limitations and can not achieve arbitrary functions like Node.

If you want to achieve real-time log reporting, you need to meet the following two capabilities:

Each request can be triggered

Report in the background without blocking the processing process of the current request

Common js_set instructions can be triggered on every request, but only support synchronous operations. Cannot use fetch, subrequest method.

The fetch function can be used in js_content instructions. But it can only be used in location. Therefore, you can use other instructions to forward the request to the path of js_content, and complete the log reporting in this instruction.

The auth_request instruction of the http_auth_request_module module is used to verify the permission of the request, such as jwt check. This instruction is triggered at each request, creates a subrequest, and determines the result of permission verification according to the return result of the request.

Therefore, we can combine these two modules to achieve log reporting.

Implement step 1. Compile Nginx

To realize this function, Nginx is required to support ngx_http_js_module and ngx_http_auth_request_module modules. These two modules are not installed by default. You need to compile and implement it yourself.

For NJS module installation, please refer to when JS encounters Nginx

The http_auth_request_module module only needs to add the parameter-- with-http_auth_request_module at compile time.

Compile

. / configure-- add-module= [NJS module path] / NJS/nginx-- with-http_auth_request_module make & & make install2. The configuration file is as follows: http {js_import http.js; # introduces the js file server {listen 80; auth_request / proxy_report; # this directive is triggered at the beginning of each request, creating a subrequest to be forwarded to location / {index index.html index.htm on the proxy_report path } location / proxy_report {internal; # restricts the acceptance of internal requests only # saves the originally requested uri and method data in header. Because the auth_request request modifies the data. Proxy_set_header X-Original-URI $request_uri; proxy_set_header X-Original-METHOD $request_method; # forwarded to another server proxy_pass http://localhost:8080/report;}} server {listen 8080 # the reporting API is placed in another server, and there is no auth_request instruction in this server to avoid cyclic trigger request location / report {# introduce a js processing script through the js_content instruction to complete the reporting operation js_content http.report;}} / / http.js file import qs from "querystring" Async function report (r) {let args = {/ / fetching the original uri and method request data from header uri: r.headersIn ['Xmura OriginalMyURI'], method: r.headersIn ['Xmura OriginalMutual Metah'], remoteAddress: r.remoteAddress, status: r.status, headersIn: JSON.stringifry (r.headersIn),} / / issue an asynchronous request Do not block the currently requested process, and complete the reporting of ngx.fetch (`qs.stringify (args)} `, {method: 'GET',}`) / / return a status code of 200 to make the verification instruction successful r.return (http://[)} export default {report} above are all the contents of the article "how to use Nginx pure configuration to achieve real-time log reporting" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.