Shulou(Shulou.com)06/02 Report--

This article shows you how to use grep conditions to search large files in linux, the content is concise and easy to understand, it will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

1. Grep search

Grep parameter file name | head / / find the grep parameter file name from scratch | wc- l / / check the number of lines of cat file names that meet the criteria | grep parameter $/ / output the line content ending with this parameter

2. Examples

(1) search for the number of rows according to specific parameters

Cat / data/weblogs/xxx.access.log | grep "GET / pixel.jpg?" | wc-l 4102386

(2) partial regular query

Cat / data/weblogs/em.evony.com.access.log | grep "25/Nov/2019:15: [00-59]" | wc-l 120

If you query all the data in 25/Nov/2019:15, the minutes after 15:00 are 00-59.

(3) Pipeline connection can be used between multiple conditions to query the number of rows that meet both conditions.

Cat / data/weblogs/xxx.log | grep "25/Nov/2019:15: [00-59]" | grep "GET / pixel.jpg?" | wc-l 120

Query the number of rows that meet condition 1 or condition 2

Cat / data/weblogs/xxx.log | grep-E "25/Nov/2019:15: [00-59] | GET / pixel.jpg?" | wc-l 4098135 abbreviation: grep-E "exp1 | exp2 | exp3" | wc-l reference: https://blog.csdn.net/lijing742180/article/details/84959963

3. Grep is a fuzzy query

When using grep to search for port numbers, the results are not satisfactory, and all ghosts and ghosts have been found. Examples are as follows:

Netstat-anp | grep-I '80' (Not all processes could be identified, non-owned process info will not be shown You would have to be root to see it all.) tcp 0 0127.0.0.1 LISTEN 80 0.0.0.0 LISTEN-tcp 00 10.17.2.50 LISTEN 80 0.0.0.0 LISTEN- Tcp 00 216.66.17.189 tcp 80 0.0.0.0 TIME_WAIT-tcp 00 10.17.2.50 TIME_WAIT-

Recommend a more useful one. Query the usage of port 80 and use the command:

Netstat-apn | awk'{split ($4jinarr, ":"); if (arr [2] = = "80") print $0}'

One step in place, found out is the process of port 80, very easy to use.

3. Search the ip address in the file

1. Match ip

Grep-Eo'([^ 0-9] |\ b) (1 [0-9] {2} | 2 [0-4] [0-9] | 25 [0-5] | [1-9] [0-9] | [0-9])\.) {3} (1 [0-9] [0-9] | 2 [0-4] [0-9] | 25 [0-5] | [1-9] [0-9] | [0-9]) (^ [0-9] | \ B) 'xxx.log | sed-nr's / ([^ 0-9] |\ b) (([0-9] {1Power3}\.) {3} [0-9] {1Magne3}) ([^ 0-9] |\ b) /\ 2lapp' | wc-l31116275

2. Query the number of occurrences of each ip

Grep-E-o "(25 [0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?)\. (25 [0-5] | 2 [0-4] [0-9]?)\. (25 [0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?)\. 0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?) " Xxx.log | sort | uniq-c 2 99.203.87.103 2 99.203.87.142 4 99.203.87.145 8 99.203.87.153

The front is the number of occurrences, followed by ip

3. More accurate matching ip

Grep-E-o "(25 [0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?)\. (25 [0-5] | 2 [0-4] [0-9]?)\. (25 [0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?)\. 0-5] | 2 [0-4] [0-9] | [01]? [0-9] [0-9]?) " Xxx.log | wc-l32929372

4. Fuzzy matching ip

Grep-E-o "([0-9] {1jue 3} [\.]) {3} [0-9] {1J 3}" xxx.log | wc-l32930309

5. Query ip with multiple conditions, first get the specified number of rows according to the qualification, and then search the number of ip

Cat xxx.log | grep "25/Nov/2019:15: [00-59]" | grep "GET / pixel.jpg?" | grep-E-o "([0-9] {1jue 3} [\.]) {3} [0-9] {1jue 3}" | wc-l 1110 the above is how to use grep conditions to search for large files in linux. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.