Shulou(Shulou.com)06/02 Report--

This article mainly analyzes the relevant knowledge of the usage of the tail command for you, the content is detailed and easy to understand, the operation details are reasonable, and has a certain reference value. If you are interested, you might as well follow the editor to have a look, and follow the editor to learn more about "what is the use of the tail command".

The tail command can see the log scrolling, which is very convenient. So xjjdog thought, since we can use this command to see all the logs, can we use the tail command to do log collection?

Imagine that tail is a great tool if you want a fast real-time log collection tool. It is much faster than flume, logstatsh, filebeat and so on. In fact, I did this in the old days when tools were scarce, and it worked very well.

Here is a piece of code written in the Java language. We can read the log by line and do anything in our favorite language.

Import java.io.BufferedReader;import java.io.InputStreamReader; public class TailReader {public static void main (String [] args) throws Exception {ProcessBuilder ps = new ProcessBuilder ("tail", "- f", "/ tmp/tail0"); / / print the error output as ps.redirectErrorStream (true); Process process = ps.start () / / continuously read the output try of tail (BufferedReader in = new BufferedReader (new InputStreamReader (process.getInputStream () {String line; while ((line = in.readLine ())! = null) {setLogToKafka (line) / / Note that no exception is generated here, otherwise the while loop will be interrupted}} / / the simulation will be sent to kafka, and we will simply print out static void setLogToKafka (String line) {System.out.println (line);}}

The main idea is to start a child tail process using Java's Process to monitor the output of the file all the time. Then the standard output and standard error stream are all directed to BufferedReader. Next, you can do whatever you want.

There is a certain risk that if the tail command is killed, our Java program will be useless.

The program is simple, but what xjjdog is talking about here is not this simple collector, but some interesting features of the tail command, from which you can get a glimpse of the special handling of files by some log collection tools.

Do you know the difference between tail-f and tail-F?

Before we answer this question, let's recall the common logging framework of Java, the handling of logs.

TruelogFile.%d {yyyy-MM-dd} .log303GB%-4relative [% thread]%-5level% logger {35} -% msg%n

The above configuration will scroll to form a new file in the wee hours of each night.

So how is this scrolling done? We can finish work and simulate the process.

Mv run.log run.2020-11-02.logtouch run.log

Test it

If the file scrolls, a new file will be generated. Can the tail command still trace it?

Let's test it.

The first step is to create a file to monitor

Touch / tmp/tail0

Step two, start our Java code

Third, generate an uninterrupted stream

Watch-n 1 'echo `date` > > / tmp/tail0'

The above command prints the current date into our file every second, and you can see that the Java side has received the data.

The fourth step is to simulate file scrolling

Mv / tmp/tail0 / tmp/tail.baktouch / tmp/tail0

At this point, we can see that the Java side can no longer accept data at this time.

Why?

To see why, let's use two commands to take a look at some of the state of the process.

First, use the ps command to view the current tail process.

Ps-ef | grep tail 501 21374 21373 01: 51PM? 0grep tail 00.01 tail-f / tmp/tail0

That's our order.

We use the lsof command to view the files associated with this process.

Lsof-p 21374 | awk'{print $4 "\ t" $9}'FD NAMEcwd / tmp/txt / usr/bin/tailtxt / usr/lib/dyld3r / private/tmp/tail.bak

We can see that the files monitored by the tail process are actually tail.bak files, which have nothing to do with the tail command.

Let's try to type something like tail.bak.

Echo ": xjjdog, i am from tail.bak" > > / tmp/tail.bak

At this point, as we expected, the Java process responded and printed this sentence normally.

What shall I do?

Just replace tail-f with tail-F, as we asked in our question.

Tail-f means tracking based on the file descriptor.

Tail-F means that it will retry when it is tracked by the file name.

Therefore, our log collector, no doubt tracked by the log name, should change f to F.

End

Now that we know these small differences, we have an explanation for some of the psychic problems we encounter in our daily work.

Everyone knows the rm command, which can delete a file. If there is this file, which is being used by other processes, it looks like you deleted it, but its contents are not released.

Lsof | grep deleted

With the above command, you can see these out-of-control files. Usually when you kill the corresponding process, these handles are released. But your intention to delete these files is to avoid restarting the app, which is really confusing.

Cat / dev/null > logpath

So when we delete files, we generally do not use rm, but should use redirection symbols. Send everything empty / dev/null to them.

This is the end of the introduction on "what is the use of the tail command". More related content can be searched for previous articles, hoping to help you answer questions and questions, please support the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.