Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the common tools of Linux". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn what are the common tools of Linux.

01

Log tool

one

Logdissect

The CLI utility and Python API used by logdissect to analyze log files and other data. It can parse, merge, filter, and export data (log files or json format)

two

Installation

There are two ways to install, via github and PyPI

Github

Need: python, python-setuptools

(and/or python3, python3-setuptools)

Wget https://github.com/dogoncouch/logdissect/archive/v3.1.1.tar.gztar-xzf v3.1.1.tar.gzcd logdissect-3.1.1sudo make all

Note: during the test, the author found that when there is only python2 on the computer, it is necessary to comment out the following lines about python3 in the logdissect/Makefile file before it can be installed normally, otherwise an error will be reported.

PyPI

Need: pip

Sudo pip install logdissect

three

Introduction to options

Usage:

Logdissect.py [- h] [--dhost DHOST] [--grep PATTERN] [--last LAST] [--process PROCESS] [--protocol PROTOCOL] [--range RANGE] [--utc] [--rdhost DHOST] [--rgrep PATTERN] [--rprocess PROCESS] [--rprotocol PROTOCOL] [--rshost SHOST] [--rsource SOURCE] [--shost SHOST] [--source SOURCE] [--linejson LINEJSON] [--outlog OUTLOG] [--label LABEL] [--sojson SOJSON] [--pretty] [--version] [--verbose] [- s] [--list-parsers] [- p PARSER] [- z] [- t TZONE] [file [file.]]

Location parameters:

File specifies the input file

Option parameters:

-h,-- help view help information-- version view program version number-- terminal output with detailed verbose settings-s silent output-- list-parsers view available parsers-p PARSER select parser (default: syslog)-z,-- unzip contains gzip compressed files-t TZONE specifies UTC time zone offset (for example,'+ 0500')

Filtering options:

-- grep PATTERN matching destination host-- grep PATTERN matching pattern-- last LAST matching previous time period (for example, 5m/3h/2d/etc)-- process PROCESS matching source process-- protocol PROTOCOL matching protocol-- range RANGE matching event range (YYYYMMDDhhmm-YYYYMMDDhhmm) (time of year, month, day)-- utc uses UTC for range matching-- rdhost DHOST filters out target hosts-- rgrep PATTERN filters patterns-- rprocess PROCESS filters. Drop source process-rprotocol PROTOCOL filter out protocol-rshost SHOST filter off source host-rsource SOURCE filter out log source-shost SHOST match source host-source SOURCE match log source

Output options:

-- linejson LINEJSON sets the output file of JSON output line by line-- output file of outlog OUTLOG setting standard log output-- label LABEL sets the label type of output log (fname | fpath)-- sojson SOJSON sets the output file of single object JSON output-- pretty beautifies the sojson output format.

four

Parser

-- list-parsers output = available parsing module: = ciscoios: Cisco ios parsing module emerge:gentoo emerge log parsing module linejson:logdissect per line object JSON parsing module sojson:logdissect single object JSON parsing module syslog: syslog (standard timestamp) parsing module syslogiso:syslog (ISO timestamp) parsing module syslognohost:syslog (no standard timestamp of the host) parsing module tcpdump:tcpdump terminal output parsing module webaccess:web access Log parsing module windowsrsyslog:windows rsyslog agent log parsing module

five

Simple usage

Check the login log for the past hour:

Logdissect-last 1h secure

Check the logs of the systemd and cron processes over the past 30 minutes:

Logdissect-last 30m-process systemd-process CRON messages

View messages and dmesg logs for a specified time range in detail

Logdissect-verbose-range 20190428120000-20190428121000-label fpath messages dmesg

A file that outputs the access records of 183.238.151.209 over the past 30 days

Logdissect-s-outlog myaccess.log-- grep 183.238.151.209-- last 30d-- label fname / var/log/secure

02

Process tool

one

Linux Rootkit (vfs hook)

The hidden process detection tool, a linux kernel module, is a small demo for detecting rootkit. By reading the virtual files created by this kernel module, you can detect processes hidden by the Hook vfs function.

Download link:

Https://security.tencent.com/index.php/opensource/down/16

two

Installation method

Unzip process_list.zipcd process_listmakemake install

Use effect:

03

Network analysis tool

one

Tcpdump

It is necessary to explain Tcpdump as a classic package grabbing tool for Linux.

Note: packet capture is only a packet capture process, the most important thing is the analysis of the results. In the process of analysis, it is necessary to master the packet structure of the mainstream protocols, otherwise everything is futile.

two

Basics

Common options:

-I: specify the network interface (such as eth0,eth2). The network interface uses the ifconfig command to view)-nn: does not perform DNS anti-parsing of the IP address, and does not convert the port to characters-vv: detailed output packet information-w: save the captured results in the specified file, followed by the custom file name-r: read the results stored in the file for analysis

Important: in the process of grabbing packages, it is strongly recommended to use the-n option. If the-n option is not added, the tcpdump program will do DNS anti-parsing of the IP address, which will take a considerable amount of time. Because the libcap buffer is involved, packets captured by tcpdump are first put into the buffer and then extracted by the upper layer. However, when the upper layer tries to do DNS anti-parsing, the buffer will overflow due to too many packets, and the direct result of the overflow is that some packets are drop, and the drop of the packets will have a great impact on our analysis process-actually no packet loss, but the data display is "packet loss". In order not to affect the results of our analysis, I suggest adding the-n option.

Qualifier:

A filter expression consists of one or more primitives. The primitive usually consists of id (name or number) preceded by one or more qualifiers. Tcpdump provides three qualifiers, namely type, direction, and protocol.

Type (type):

Host / / host 192.168.1.1net / / net 192.168port / / port 80portrange / / portrange 22-801.2.2 direction (packet Direction) srcdstsrc or dstsrc and dst1.2.3 protocal (Protocol) etheriparprarptcpudpicmp

three

Basic grammar

Filter hosts:

Tcpdump-I ehtN-n host IPADDRtcpdump-n-I eth2 host 192.168.1.1 crawls all packets passing through eth2 with a destination or source address of 192.168.1.1 tcpdump-I eth2-n src host 192.168.1.1 specify source address tcpdump-I eth2-n dst host 192.168.1.1 specify destination address

Filter Port:

Tcpdump-I eth2-n port PortNumtcpdump-I eth2-n port 25 crawls all network data passing through eth2, destination or source port 25 tcpdump-I eth2-n src port 25 specifies the source port

Web filtering:

Tcpdump-I eth2-n net 192.168tcpdump-I eth2-n src net 192.168

Protocol filtering:

Tcpdump-I ethN-n Protocoltcpdump-I eth2-n icmptcpdump-I eth2-n ip

Common expressions:

Non:! Or not.

And: & & or and

Or: | | or or

Tcpdump-I eth2 tcp and port 80 and (dst host 192.168.1.254 or dst host 192.168.1.200) crawl all passing eth2, destination address is 192.168.1.254 or 192.168.1.200 TCP data with port 80 or tcpdump-I eth2'((tcp) and (port 80) and ((dst host 192.168.1.254) or (dst host 192.168.1.200)'/ / the preceding parentheses need to be escaped, and the latter is to enclose the expression in quotation marks

four

Packet head filtering

How to filter information from the header? First of all, you should be familiar with the header structure of IP, TCP and other protocols, which will not be discussed here. The syntax for header filtering is as follows:

'protocol [x]': it starts with the x byte of the protocol protocol Take the following y bytes (bytes numbered from 0) 'proto[ x: y] & z = 0': proto[ x: y] and z = 0 'proto[ x: y] & z! = 0': protox: y] and z is not 0 'proto[ x: y] & z = z': protox: y] and z as z'proto [x: y] = z': proto [x : y] equals z operator: > =, 182.118.126.96.80: Flags [S], cksum 0xe729 (correct), seq 3087933325, win 14600, options [mss 1460 female sackok Magi TS val 796071570 ecr 0recoveryWScale 7], length 017correct 00.159427 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52) 182.118.126.96.80 > 218.8.51.194.51003: Flags [S.], cksum 0xce85 (correct), seq 1946096042, ack 308793326, win 14600, options [mss 1460] Nop,nop,sackOK,nop,wscale 7], length 017 IP 00.190017 IP (tos 0x0, ttl 54, id 2336, offset 0, flags [DF], proto TCP (6), length 40) 218.8.51.194.51003 > 182.118.126.96.80: Flags [.], cksum 0x47ed (correct), seq 1, ack 1, win 115,00.190354 IP (tos 0x0, ttl 54, id 2337, offset 0, flags [DF], proto TCP (6) Length 40) 218.8.51.194.51003 > 182.118.126.96.80: Flags [F.], cksum 0x47ec (correct), seq 1, ack 1, win 115, length 017 Vlad 00.190402 IP (tos 0x0, ttl 64, id 57183, offset 0, flags [DF], proto TCP (6), length 40) 182.118.126.96.80 > 218.8.51.194.51003: Flags [F.], cksum 0x47eb (correct), seq 1, ack 2, win 115 Length 017 ttl 41V 00.221104 IP (tos 0x0, ttl 54, id 2338, offset 0, flags [DF], proto TCP (6), length 40) 218.8.51.194.51003 > 182.118.126.96.80: Flags [.], cksum 0x47eb (correct), seq 2, ack 2, win 115, length 0

Result analysis:

In the above result, 2-3 lines of information is the process of TCP three-way handshake, and offset represents the offset.

The meaning of the logo after Flags:

S (SYN)

F (FIN)

P (PUSH)

R (RST)

W (ECN CWR)

E (ECN-Echo)

. (no flags)

Cksum indicates checksum, where correct indicates that checksum is correct

Grab the message whose destination port is equal to 80:

Tcpdump-I eth2-nn-vv'((tcp [2:2] = 80))'/ / 3-4 bytes of the TCP header is the destination port

Capture a message with a source port equal to 80:

Tcpdump-I eht1-nn-vv'((tcp [0:2] = 80))'/ / 1-2 bytes of the TCP header is the source port

* * TCP's eight tags, which are in the 14 bytes of the TCP header-tcp [13]. The tag used by TCP sets the value of the tag to 1. For example, the binary representation of the SYN package is: 00000010, and decimal is 2.

+-+ +

| | C | E | U | A | P | R | S | F | |

| | W | C | R | C | S | S | Y | I |

| | R | E | G | K | H | T | N | N |

+-+ +

Grab only the SYN package during the TCP handshake:

Tcpdump-I eth2-nn-vv'((tcp [13] = 2))'at this time the value of the identification bit is 00000010

Grab the package marked SYN or ACK:

Tcpdump-I eth4-nn-vv'((tcp [13] = 18))'at this time the identification bit is up to 00010010

Other identification bit packet capture methods are analogous at one time. This numerical calculation may be troublesome, so tcpdump supports another way of writing, the above two writing methods are as follows:

Tcpdump-I eth4-n-vv "TCP [tcpflags] & (tcp-syn)! = 0"

Tcpdump-I eth4-nn-vv'(TCP [tcpflags] & tcp-syn! = 0) and (tcp [tcpflags] & tcp-ack! = 0))'

[root@tw13c150 ~] # tcpdump-I eth4-n-vv "TCP [tcpflags] & (tcp-ack)! = 0" and\ (host 218.8.51.194\)

Tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 65535 bytes

13 ttl 46 id 26.019271 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)

182.118.126.96.http > 218.8.51.194.58137: Flags [S.], cksum 0xfbc7 (correct), seq 769939001, ack 1488795467, win 14600, options [mss 1460

13 ttl 46 IP 26.049499 IP (tos 0x0, ttl 56, id 34444, offset 0, flags [DF], proto TCP (6), length 40)

218.8.51.194.58137 > 182.118.126.96.http: Flags [.], cksum 0x752f (correct), seq 1488795467, ack 769939002, win 115, lengt

Other protocols capture packets (DNS, HTTP, etc.)

HTTP:tcpdump-I eth4-nn-vv 'tcp [20:2] = 0x4745 or tcp [20:2] = 0x4854'//tcp [20:2] = 0x4745 for' GET' 's' GE',tcp [20:2] = 0x4854 for HTTP's' HT'DNS:tcpdump-I eth2 udp dst port 53

Finally, if you need to view the contents of the data, you can use tcpdump-s 0-w filename to save all the packets (- s 0 is to grab the complete packet, otherwise only 68 bytes are captured by default. And then analyze it with wireshark

Packet capture techniques for network packet loss analysis:

There are many ping programs on our company's server, and we can grab packets of specific length in order to avoid the impact of other packets.

Take ICMP as an example:

A side specifies that the data byte is 1024 bytes pingping-c 400-I 0.01-s 1024-f tw04008s2.sandai.net//, then the length of the IP packet captured by B side should be 1024 (data length) + 8 (8-byte ICMP header) + 20 (byte IP header) tcpdump-I eth2 "(ip [2:2] = 1052) and src host 123.59.127.2"-w loss.cap// where ip [2:2] refers to filtering the number of two bytes starting at the third byte. So what I filtered out was 3 or 4 bytes.

In fact, it is not difficult to capture packets, but we need to learn to analyze and clean data, which requires us to be very familiar with various protocols, especially TCP,IP,HTTP, etc.

04

Memory service tool

one

Volatility

Volatility is a memory forensics and analysis tool, which can analyze the memory out of dump and extract the files in memory. Support for multi-platform operation (requires a python environment), the tool supports Windows and Linux,Kali have been installed by default.

Download address: http://www.volatilityfoundation.org/#!releases/component_71401

two

Usage

To see the available options, run "python vol.py-h" or "python vol.py-info"

Root@kali:~/volatility-master# python vol.py-hVolatility Foundation Volatility Framework 2.6.1Usage: Volatility-A memory forensics analysis platform.

Options:

-h,-- help / / list all available options and their default values.Default values may be set in the configuration file (/ etc/volatilityrc)-- conf-file=/root/.volatilityrc / / User based configuration file-d -- debug / / Debug volatility--plugins=PLUGINS / / Additional plugin directories to use (colon separated)-- info / / Print information about all registered objects--cache-directory=/root/.cache/volatility / / Directory where cache files are stored--cache / / Use caching--tz=TZ / / Sets the (Olson) timezone for displaying timestamps.using pytz (if installed) or tzset-f FILENAME,-- filename=FILENAME / / Filename to use when opening an image--profile=WinXPSP2x86 / / Name of the profile to load (use-info to see a list of supported profiles)-l LOCATION -- location=LOCATION / / A URN location from which to load an address space-w,-- write / / Enable write support--dtb=DTB / / DTB Address--shift=SHIFT / / Mac KASLR shift address--output=text / / Output in this format (support is module specific, see the Module Output Options below)-- output-file=OUTPUT_FILE / / Write output in this file-v,-- verbose / / Verbose information--physical_shift=PHYSICAL_SHIFT / / Linux kernel physical shift address--virtual_shift=VIRTUAL_SHIFT / / Linux kernel virtual shift address-g KDBG -- kdbg=KDBG / / Specify a KDBG virtual address (Note: for 64-bit Windows 8 and above this is the address of KdCopyDataBlock)-- force / / Force utilization of suspect profile-k KPCR,-- kpcr=KPCR / / Specify a specific KPCR address--cookie=COOKIE / / Specify the address of ntasking ObHeaderCookie (valid for Windows 10 only)

The following plug-ins are supported:

Linux_apihooks-check userland apihookslinux_arp-print ARP table linux_aslr_shift-automatically detect Linux ASLR conversion linux_banner-print Linux banner information linux_bash-restore bash history from bash process memory linux_bash_env-restore process dynamic environment variable linux_bash_hash-recover bash hash table linux_check_afinfo from bash process memory-verify network protocol operation function Linux _ check_creds-check for any processes sharing credential structure linux_check_evt_arm-check exception vector table to find syscall table hook linux_check_fop-check rootkit modified file operation structure linux_check_idt-check if IDT has been changed linux_check_inline_kernel-check inline kernel hook linux_check_modules-compare module list with sysfs information such as Linux_check_syscall-check if the system call table has been changed linux_check_syscall_arm-check whether the system call table has been changed linux_check_tty-check the hook linux_cpuinfo of the tty device-print information about each active processor linux_dentry_cache-collect files linux_dmesg from the dentry cache-collect dmesg buffer linux_dump_map-map selected memory Write to disk linux_dynamic_env-dynamic environment variable linux_elfs for the recovery process-find ELF binaries linux_enumerate_files in the process map-list files referenced by the file system cache linux_find_file-list and restore files in memory linux_getcwd-list the current working directory of each process linux_hidden_modules-Burn memory to find hidden kernel module linux _ ifconfig-collect the active interface linux_info_regs-just like the 'info registers'' in GDB. It prints out all linux_iomem-provides output linux_kernel_opened_files similar to / proc / iomem-lists files opened from the kernel linux_keyboard_notifiers-parses keyboard notification program call chain linux_ldrmodules-compares the output of proc mapping with the list of libraries in libdl-lists libraries loaded into the process linux_librarydump-dumps shared libraries in process memory to Disk linux_list_raw-list applications with promiscuous sockets linux_lsmod-collect loaded kernel modules linux_lsof-list file descriptors and their path linux_malfind-find suspicious process mapping linux_memmap-dump linux task memory mapping linux_moddump-extract loaded kernel module linux_mount-collect installed fs / device linux_mount_cache-from kmem _ cache collect installed fs / deviceslinux_netfilter-list Netfilter hook linux_netscan-for network connection fabric linux_netstat-list open socket linux_pidhashtable-enumerate process linux_pkt_queues through PID hash table-write packet queue for each process to disk linux_plthook-scan PLT of ELF binaries to hook to non-NEEDED image linux_proc_maps-collect process memory image Shoot linux_proc_maps_rb-process map linux_procdump to collect linux by mapping red-black tree-dump executable image of process to disk linux_process_hollow-check for signs of process holes linux_psaux-collect process and complete command line and start time linux_psenv-collect process and its static environment variable linux_pslist-collect by traversing task_struct- > task list Active task linux_pslist_cache-collect task linux_psscan from kmem_cache-scan physical memory linux_pstree of processes-show parent / child relationships between processes linux_psxview-find hidden process linux_recover_filesystem containing various process lists-recover entire cached file system linux_route_cache from memory-restore routing cache linux_sk_buff_cache from memory-from Recovery packet linux_slabinfo in sk_buff kmem_cache-Mimics / proc / slabinfolinux_strings on the running machine-matches the physical offset to the virtual address (may take some time (very detailed) linux_threads-Thread linux_tmpfs of the print process-restore tmpfs file system linux_truecrypt_passphrase from memory-recover cached Truecrypt password linux_vma_cache-collect VMAlinux_volshell from vm_area_struct cache-Shelllinux_yarascan in memory image-shell in Linux memory image

The following is a brief introduction to the analysis of memory files in Linux dump

View network connection information:

View process information

05

Information collection tool

one

FastIR Collector

FastIR Collector Linux, can quickly complete the information collection of Linux server data, without the need to master a variety of complex linux commands and parameters. The tool can collect: system information (kernel version, kernel module, network interface, hostname, release version), user login information, network connection, user data, automatic process, disk information, file system information, etc., and output the results as csv files. FastIRcollector also has a version of Windows, which can be used to collect Windows system information.

Download address: git clone https://github.com/SekoiaLab/Fastir_Collector_Linux.git

Running environment: the code runs in python2 environment, python 2.4 or above. Must run under a root account

two

Usage

Cd Fastir_Collector_Linuxpython fastIR_collector_linux.py www.gendan5.com

After the command is executed, the output directory is generated, and all collected data is packaged and placed in that directory.

Use the sz command to download the package to your local environment. (yum install-y lszrz, rz and sz can be installed)

You can also use ftp or sftp transport

Parameters:

At this point, I believe you have a deeper understanding of "what are the common tools of Linux?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.