Shulou(Shulou.com)06/01 Report--

Compression and decompression

Compressed format: gz bz2 xz zip z

With different compression algorithms, the compression ratio is also different.

Compress: filename.Z

Uncompress:

You cannot compress a folder. The files in the folder will be compressed individually.

Gzip: .gz

Delete the source file after gzip filename compression is completed

Gzip-d equals gunzip

-#: 1-9 specifies that the compression ratio defaults to 6

Gunzip:

Gunzip filename

Zcat temporarily view compressed text files without decompression

Bzip2: .bz2

Has a higher compression ratio than gzip.

Use the same format

Bzip2 filename

-d-# both

-k can keep the source file

Bunzip2: same understanding pressure

Bzcat:

Xz: .xz ditto

Unxz decompression

Xzcat

Xzdec decompression

Zip: the path to the compressed file name after zip compression

You can compress the folder without deleting the source file

Archive: archiving itself does not mean compression

Unzip: decompression

Tar: archiving tool

-c: create an archive

-f file.tar: archive file of the operation

-x: expand the archive

-- xattrs archive while retaining its extended attribute information

-t: do not expand the archive to view internal files

-zxf: decompress the archive and call gzip

-zcf: call gzip for compressed archive

-jcf calls bzip2

-jxf

-Jcf calls xz

-Jxf

-z-j-J can be omitted during decompression, but not during compression.

Cpio: archiving tool

Read:

-p give a hint

-t timeout

While loop:

Suitable for scenarios where the number of loops is unknown, with exit conditions

Syntax:

While condition; do

Run command

Done

Use colors to control fonts

Echo-e\ 033 [xtteryterzm display text\ 033 [

X font

Y foreground color

Z background color

64,32

/ lib

/ lib32

IDE 133Mbps parallelism

SATA 300m 600m 6G serial

USB 3 480m Serial

SCSI SMALL COMPUTER SYSTEM INTERFACE 320m parallel summary

SAS 2.5in Serial

RAID:

Level, which only represents the different ways in which the disks are organized, and there is no difference between upper and lower.

Stripe: 0

Performance improvement, read and write

No redundancy.

Space utilization n

At least 2 yuan

Mirror image: 1

Write performance decreases, read performance increases

Have redundant ability

Spatial utilization nmax 2

At least 2 yuan

Validation code: 4 plates 1, 2, 3, 6, a bad piece (1-3) can all be used with disk 6.

Take turns check code: 5 take turns check code storage location

Both reading and writing are improved.

Have redundant ability

Space utilization (nMul) / n

At least 3 yuan

1: 0: stripe before mirroring

At least 4 yuan

0room1: mirror the image before the stripe

At least 4 yuan

JOBD: simply add multiple hard drives

No performance improvement

Non-redundant ability

Space utilization n

At least 2 yuan

MD multi dsiks simulates a RAID

Logical RAID

/ dev/md#

Mdadm: any block device is made into RAID

Model command

Create a pattern

-C

Dedicated options:

-l: level

-n: number of Devic

-a:yes/no automatically creates device files

-c: specifies the block chunk size default 64KB

-x: specify the number of space disks

Management mode

-- add,--remove,--fail

Monitoring mode

-F

Growth model

-G

Assembly mode

A

-D-- detail views the details of RAID

Stop the entire column

-S-- stop

Save the current RAID information to the configuration file

Mdadm-D-scan > / etc/mdadm.conf

Use mdadm-A / dev/md# to assemble later.

Watch periodically executes the specified command and displays the results in full screen mode

-n specifies the cycle length. The default unit is seconds, and the default is 2 seconds.

Watch-n 'command'

Lsmod lists kernel modules

Mdadm user Space tool

MD:MULTI DEVICE multiple Devic

DM:device mapper device mapping

Logic equipment

Lvm2

Snapshot

Multipath

Vgcreate vgname equipment

-s: the size of pe. Default is 4mb.

Lvcreate-n lv name-L space size vg name

Extended logical Volume

Lvextend

-L size logical volume

Resize2fs

Resize2fs2fs logical volume expands to size

-p is the same size as the physical boundary

Reduce logical Volume

Resize2fs logical volumes are reduced to size

Cannot be reduced online. Uninstall it first.

Ensure that the reduced space size can still store the original data.

Files should be checked straightly before reduction to ensure file system consistency.

Lvreduce-L size logical Volume

Remount

Snapshot volume

The lifecycle is the entire length of data, during which the growth of data cannot exceed the size of the snapshot volume

Snapshot volume read-only

In the same volume group as the original volume

Lvcreate

-s snapshot volume

-p r | w permissions

Lvcreate-s-L size-n logical volume name-pr / path

Until =! while

Until who | grep "hadoop" > > / dev/null; do

Sleep 5

Done

Echo "hadoop is logged in"

For ((variable assignment; variable range; variable change); do

Cycle operation

Done

Ping

-c times

-w timeout

Awk condition {Action}

Df-h | awk'{print $1} 'awk-F delimiter' {print $field serial number}'

$NF is the last field, and NF is the number of fields

-F is followed by a delimiter

CSMA/CD carrier sense multi access colision detection

Carrier sense multiple access conflict detection

Socket socket ip binds with port

Three-way handshake disconnects tcp finite state set four times

A three-way handshake to establish TCP connection B

1. Send SYN=1,SN=100

two。 Accept SYN=1,ACK=1,AN=101,SN=300

3. Send ACK=1,SN=101,AN=301

A disconnects TCP connection B four times

1. Send FIN=1

two。

3.

4.CLOSED

Host access to the network

Ip

Mask

Gateway

Hostname

Dns

Route

Dhcp:dynamie host configuration protocol

169.254.x.x automatic address

Linux network is a kernel function.

Lo: local loopback

Eth {0-9}: Ethernet card

Ppp {0-9}: point to point

5.8 / etc/modprobe.conf

Alias

6.0 / etc/udev/rules.d/70-persistent-net.rules

Ifconfig

-a displays configuration information for all interfaces

Ifconfig eth {0-9} ip/mask up | down

The configuration takes effect immediately, but fails after restart

Network Services:

5.8 / etc/init.d/network start | stop | restart | status

6 / etc/init.d/networkmanager ditto

Gateway

Route

Add add

-host host

-net network

-net 0.0.0.0 default route

Route add-net network gw next-hop address

Route add-net default gw next address

Del deletion

Same as above

Route del-net 0.0.0.0

Route-n displays host information digitally

Network profile

/ etc/sysconfig/network

Network card profile

/ etc/sysconfig/network-scripts/icfg-eth {0-9}

The device name associated with the DEVICE=, which should be consistent with the second half of the file name

BOOTPROTO= {static | none | dchp | bootp}: boot protocol

Static and dynamic state

IPADDR IP address

NETMASK subnet mask

GATEWAY Gateway

ONBOOT is automatically enabled

HWADDR mac address is the same as hardware

USERCTL {YES | NO} whether to allow ordinary users to control the interface

PEERDNS {YES | NO} whether to accept the configuration information of DNS in DHCP

It will not take effect immediately, and it is guaranteed to be valid forever.

Routing Profil

/ etc/sysconfig/network-scripts/route-eth {0-9}

Add format

Format, 1

DEST via NEXTHOP

Format, 2

ADDRESS0=

NETMASK0=

GATEWAY0=

ADDRESS1=

NETMASK1=

GATEWAY1=

DNS server specifies up to 3

/ etc/resolv.conf

Nameserver ip

Local address resolution

/ etc/hosts

Host IP host name host alias

Configure Hostnam

Hostname name

Effective immediately. Restart fails.

/ etc/sysconfig/network

HOSTNAME=

Permanently valid

Use setup to set the same effect, and you need to restart

Ifconfig old command

Iproute2

Ip

Link: configure Nic properties

Ip-s link show time-limited statistics

Ip link set device statu

Addr: protocol address

Ip addr add address dev device

Ip addr del address dev device

Ip addr show

Route: routin

Tc flow control

One network card with multiple addresses

Network devices can be aliased

Aliases cannot use DHCP

Ifconfig eth0:0

/ etc/sysconfig/network-scripts/icfg-eth {0-9}: {0-9}

DEVICE=eth {0-9} {0-9}

Program composition

Binaries / bin,/sbin

Library / lib

Configuration file / etc

Help file / usr/share/man

/ etc,/bin,/sbin,/lib

They are all directory programs that the system needs to start, and additional partitions cannot be mounted separately.

Must be on the root partition

/ usr/

/ bin

/ sbin

/ lib

Core functions of the operating system

Can be partitioned separately

/ usr/local

/ bin

/ sbin

/ lib

/ etc

/ man

Third-party software used by the user

/ opt

Early third-party software installation location

/ proc

/ sys

Cannot be partitioned separately. The kernel manages itself.

/ dev

Devices cannot be partitioned separately

Udev

/ home user data directory

Separate partitions are recommended

/ root administrator home directory, no need to partition separately

/ var has more standardized system management with him.

Log information, and operation information, separate partitions are recommended, which are getting bigger and bigger over time.

/ boot: kernel initrd {initramfs} as long as it can be found. Separate partitioning is recommended.

POST-BIOS-MBR (BOOTLOADER)-find the kernel on disk-find the location of the root

Package installer

Packaged into one file: binary file, library file, configuration file, help file

Generate a database: track every software installed

1. Make a software package

2. Install, uninstall, upgrade query and check

Redhat suse: rpm

Redhat package manager

Rpm is package manager

Redhat and suse cannot be mixed.

Debian: dpt

Dependencies:

Front-end tool: yum,apt-get

Back-end tool: rpm,dpt

Yum:yellowdog update modifier

Rpm command:

Rpm:

/ var/lib/rpm

Rpmbulid:

Install, uninstall, upgrade, query, verify, rebuild database, verify packets

Rpm naming:

Package composition:

Main package:

Bind- version

Subpackage:

Bind-libs

Bind-utils

Package name

Package name format:

Name-version-issue number. Architecture. Platform .rpm

Bind-major.minor.release-release.arch.rpm

The first release is the release number, and the second release is revised by the rpm package maker

Major version number: major improvement

Minor version number: a major change has taken place in a sub-function

Issue number: modified part of bug and adjusted some functions

Noarch is not targeted for version

Umame-a check the kernel version

Rpm package:

Binary format: used directly, mostly

Source code format: need to compile

Rpm:

1, install:

Rpm-I path / file name

-h has # shown installation read, 50 #, 2% each

-v detailed process

-vv for more detail

Rpm-ivh is commonly used

-- nodeps: ignore dependencies, can be installed but cannot be used properly

-- replacepkgs: reinstall and replace the original installation

-- force: reinstall, overwrite, downgrade and force installation

-- test test installation

2, query:

Rpm-Q package name: query the specified package

-qa: query all installed packages

-qi: query the description information of the specified package

-ql: query the list of files generated after the specified package is installed

-qc: query the configuration file of the specified package

-qd: query the installation help file of the specified package

The qf query specifies that the file is generated by which rpm package installation

-Q-- scripts queries the scripts in the specified package

If a RPM is not installed, you want to query

-qpi

-qpa

Same as above

3, upgrade:

Rpm-Uvh upgrade if there is an older version, otherwise install

-Fvh upgrade if there is an older version, otherwise exit

Rpm-Uvh-- oldpackage package name degraded installation package

4, uninstall

Rpm-e package name

-- nodeps

5, check

Rpm-V package name

6. Rebuild the database

Rpm

-- rebuliddb rebuilds the database, it will certainly be rebuilt.

-- initdb initializes the database. If there is no database, it will not be built until it is created.

7. Verify the validity of the source and software integrity

Ls / etc/pki/rpm-gpg/

RPM-GPG-KEY-redhat-release

Rpm-K package name

Dsa,gpg verifies source validity

Sha1,md5 verifies the integrity of the package

Rpm--import / etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

Import the key of redhat

HTML HYPERTEXT MARK LANGUAGGE

XML EXTENDED MARK LANGUAGGE

XML,JSON semi-structured data

Metadata File in yum Warehouse

Primary.xml.gz

List of all rpm packages:

Dependency relationship

List of files generated by each rpm

Filelist.xml.gz

List of all files for the current warehouse RPM package

Other.xml.gz

Additional information, rpm modification log

Repomd.xml

Record the timestamp and checksum of the previous file above

Xml at the beginning of comps

Packet information for rpm packages

Define repo / etc/yum.repos.d/ files ending in .repo for yum

[marked ID]

Name= name

Baseurl= warehouse path

Ftp://

Http://

File:///

Whether enable= is available

Gpgcheck= checks signature integrity

The location of the key checked by gpgkey=

Yum

List: list

All all

Available available

Installed has been installed

Updates available updates

Clean: cleaning cache

Repolist: displaying repoe list

Install:

-y automatically reply to yes

-- nogpgcheck does not check for integrity

Update:

Upgrade to the latest version

Update to:

Upgrade to the specified version

Remove | erase:

Unloading

Info: information

Provides:

Check which yum installed the specified file or feature

Groupinstall:

Grouplist

Groupinfo

Createrepo needs to use it to build a yum warehouse manually

Create a repodata in the specified directory

You also need to copy the XML file

Rpm installation

Binary format

Source program-compile-binary format

Some features are selected by the compiler and cannot be used if it is not selected by the compiler

The version of the rpm package lags behind the source package, even far behind.

Customization: manual compilation and installation

Compilation environment, development environment

Development tools

Linux: c

Gun: c

C environment, C++

Gcc: compiler

Glossary:

Make: project management tool

Makefile: a program that defines the order in which make (gcc,g++) compiles source files

Automake, create-makefile.in

Autoconf, create-configure

The combination of the above two forms makefile

Make install

Perl+java,python

Three steps of compilation and installation

Premise: prepare the development environment {compilation environment}

Install Development Tools and development libraries

First decompress the binary package with tar

Cd

. / configure (let the user choose the compilation feature and check the compilation environment)

Use configure-- help to view options

Use configure-- prefix to specify the installation directory

-- sysconfdir specifies the configuration directory for installation

Confirm the installation with make in the installation directory

Enter make install again

1. Modify the PATH environment variable to identify the binary file path of the subprogram

1. Modify / etc/profile

two。 Or create a file with the suffix .sh in the / etc/profile.d directory

Define the export PATH=$PATH:/ path in it

2. By default, the system searches the path / lib,/usr/lib of the database file and adds an additional search path

Create a file with the suffix .conf in / etc/ld.so.conf.d/, and then write the path to be added directly to the secondary file.

Ldconfig: rerouting library files

-v: shows the process of re-searching

3. Header file (parameter standard information called by the library): output to the system

Default: / usr/include

Add a header file search path, using a link

/ usr/local/tengine/include/ / usr/include/

The first ln-s / usr/local/tengine/include/* / usr/include/

The second ln-s / usr/local/tengine/include/ / usr/include/tengine

4 man file: installed in-- prefix specified directory, man directory, / usr/share/man

1the path to the man file of man-M

2, add a manpath to / etc/man.config

Netstat: network status command

-r display routing table-rn digital display

-t display tcp connection

-u shows udp connection

-l connection for listening statu

-p listens for the process number and process name of the specified socket

Source / etc/profile reload configuration file

Kill process number kills the process

While, until,for

Break: exit the cycle ahead of time to end the task

Continue: exit the current cycle ahead of time and continue the cycle

Special usage of while

While:; do

Done

Infinite cycle

The while read line; do loop reads the file and stores every line in the line

Done

< 输入重定向文件 函数,功能, function 代码重用: 定义函数 2种方法 1. function 函数名 { 函数体 } 2. 函数名() { 函数体 | `` 反引号,应用程序执行结果 执行结果状态,看命令的最后一条运行结果 return # 0-255 定义函数状态返回 接受参数的函数 函数名 参数1 参数2 $1 调用参数1 $2 调用参数2 进程管理 CPU中 MMU 内存管理单元 MEMORY MANAGEMENT UNIT 进程切换,上下文环境切换 rss 常驻内存,位于物理内存不被交换出去的 vsz 虚拟内存大小 ,栈,堆,库,初始和为初始化数据,命令 thrend 线程 由进程拆分的多个小的组成部分, 可以节省内存的使用空间 进程状态 uninterruptible 不可中断的睡眠 interruptible 可中断的睡眠 init: 内核之后的第一个进程,其他均为其子进程 优先级 0-139 共140个 进程越小越高级 100-139 :用户控制 0-99: 内核调整 高优先级的优势, 1,获得更多的CPU运行时间 2,获得优先的运行时间 每个进程都有个 nice值:友好的,优雅的 范围-20到19 对应 100到139 nice值越小优先级越高 默认都是0 普通用户仅能够调大自己进程的nice值 管理员可以随意调整 PID:process id init:所有进程的父进程 PID 为 1 进程的相关属性在目录 /proc/中对应的进程号 ps:process status BSD 风格 sysv 风格 - a:所有和终端有关的进程 u: x:所有和终端无关的进程 -e:所有进程 -l: -f: ps -o 字段 显示指定字段 进程的分类: 跟终端相关的进程 跟终端无关的进程 进程状态: D:不可中断的睡眠 R:运行或就绪 S:可中断的睡眠 T:停止的 Z:僵尸 重定向 /proc/sys/文件 sysctl -w kernel.hostname= 能立即生效,不能永久有效 永久有效,但不能立即生效,/etc/sysctl.conf 使用 sysctl -p 使内核在修改后生效 sysctl -a 显示所有内核参数选项 内核模块管理 lsmod 列出内核模块 modprobe 模块名 装载模块 modprobe -r 模块名 卸载模块名 modinfo 模块名 查看模块具体信息 insmod 模块路径 装载模块 rmmod 模块名 卸载模块 depmod 模块路径 设置模块的目录 内核中的功能除了核心功能之外,在编译时,大多数功能都有三种选择 1,不使用功能 2,编译成内核模块 3,编译进内核 如何手动编译内核 make gconfig:ghome 桌面环境使用,需要安装图形开发库 yum grouplist 安装 make kconfig:kde 桌面使用 make menuconfig 文本方式 make make modules_install make install screen命令 screen -ls 列出已经建立的会话 screen 新建屏幕 screen -r ID 可以还原屏幕 ctrl+a,d 拆除屏幕 exit 退出 二次编译时清理,清理前,如果需要,请备份配置文件.config make clean make mrproper mkinitrd initrd文件路径 内核版本号 mkinitrd /boot/initrd-`uname -r`.img `umane -r` file=/etc/init.d/acpid echo ${file##*/} 从左向右最后一次匹配 / 删除 acpid echo ${file#*.} 从左向右第一次匹配 . 删除 d/acpid echo ${file##*/*/} acpid echo ${file%.*} 从右向左 第一次匹配 . 删除 /etc/init echo ${file%%/*} 从右向左 最后一次匹配 / 删除 空 系统启动流程 post---bios(boot device)---bootloader(mbr)--- kernel(initrd|initramfs)---init(/etc/inittab|)-----rootfs/sbin/init /etc/inittab 设置默认运行级别 系统初始化 /etc/rc.d/rc.sysinit 运行指定级别的脚本 /etc/rc.d/init.d /etc/rc.d/rc#.d k s 00-99:运行次数 启动虚拟终端 启动图形终端 /etc/rc.d/rc.sysinit 检测并以读写方式挂载根文件系统 设定主机名 检测并挂载fstab中的文件系统 启动SWAP分区 初始化外围硬件的设备驱动 根据/etc/sysctl.conf 设定内核参数 激活udv和selinux 激活udev和selinux 清理过期的锁和pid文件 状态键映射 RELE 6 UPSTART --init /etc/inittab /etc/init/*.conf 内核初始化 硬件探测 装载驱动 挂载根文件系统 启动用户空间的第一进程init 1,关机和重启 shuntdown halt reboot poweroff 2,主机名 3,运行对应的服务脚本 4,启动终端 5,运行用户 6,定义单用户级别 7,状态网卡驱动 8,提供一个WEB服务器 busybox kernel 制定安装 自动化安装 定制引导盘 mount -n 挂载时不更新/etc/mtab文件 cat /proc/mounts 可以查看当前系统挂载的fs mingetty 创建终端登录 运行后会调用 /bin/longin 登录提示符 脚本编程 变量中字符的长度,$(#变量名) 终端提示信息 /etc/issue 文件的内容 设定内核参数 /etc/sysctl.conf sysctl -p 用户 PAM /etc/pan.d/* 绕过PAM: /bin/login: login:验证 nsswitch NETWORK SERVICE SWITCH 名称解析开关 框架: 库:libnss_file.so, 配置文件:/etc/nsswitch.conf 确定了去×××用户密码 /etc/passwd /etc/shandow /root/.bash_profile 中的环境变量 PS1= '[ \u @ \h \W]\$' 单用户模式 exec init S 内核编译 busybox kernel+rootfs kernel+initrd(busybox)+rootfs(busybox) 查看本机硬件信息 1. cat /proc/cpuinfo 2. lsusb 3. lspci 4. hal-device 内核配置 make menuconfig make gconfig make kconfig make config 报错为.config 文件 make make modules install make install 模块安装位置,/lib/modules/内核版本号/ 部分编译 1,只编译某子目录下的相关代码: make dir/ make arch/ make drivers/net/ 2,只编译部分模块 make M=drivers/net/ 3,只编译某一模块 make /drivers/net/pcnet32.ko 4,将编译完成的结果放置别的目录 make O=/tmp/kernel 如何编译busybox: 交叉编译 make arch=平台格式 $ {变量:-对应值} 如果变量为空或者未定义,则变量展开为"对应值",否则,展开为变量的值 $ {变量:+对应值} 如果变量为空或者未定义,无操作,否则,展开为"对应值"的值 $ {变量:=对应值} 如果变量为空或者未定义,则变量展开为"对应值"并将展开为变量的值 A=HELLOWORD ECHO ${A:2:3} 第一个数字表示略过几个,第二个数字表示之后的长度 LLO /etc/rc.d/init.d/服务脚本 服务脚本支持配置文件: /etc/sysconfig/同名服务脚本 mktemp 创建零时文件或目录 mktemp /tmp/file.XX X是随机生成的数值 mktemp -d 零时目录 信号: kill -信号 PID 1:HUP 2:INT 9:KILL 15:TERM 脚本中的信号捕捉,但是9和15无法捕捉 ctrl + c: 2 INT trap命令 trap `command` 信号列表 一行中执行多个语句,命令中间加: 即可 任务计划 在未来的某一点执行一次某任务 at batch at 时间 at>

Command

At > ctrl + d

The method of specifying the time

10:20

Now+3m

Noon noon, midnight midnight, teatime 4: 00 p.m.

At-d = atrm deletion

The execution result of the command will be sent to the user who scheduled the task in the form of an email

Automatic execution of batch system when idle

/ etc/at.allow / etc/at.deny

Periodic execution

Cron: itself is an uninterrupted service

A supplement to anacron:cron, which enables cron to resume normal execution of tasks that should have been performed in the past for various reasons.

Cron:

System cron tasks

/ etc/crontab

User tasks in time, day, month and week

Task of sharing time, day, moon and week

The validity period of time

Time wildcard representation

*: deal with all valid values

3 * 3 minutes per hour

04:22 every Sunday

Discrete point in time

10 minutes 40 * 10 minutes and 40 minutes per hour, half-hour task

10 02 * 1-5 02:10 of 1-5 per week

/: how often does the corresponding value range

* / 3 * every 3 minutes

Executed every 2 hours

Executed every 2 days

0 0 * / 2 * *

* / 3 * cat / etc/fstab & > / dev/null

Cron environment variable: all commands executed go to the path specified by the PATH environment variable

PATH / bin / sbin / usr/bin / usrsbin

Manage user tasks

Crontab

-r: remove all tasks

-u: user name-e manages cron tasks for other users

Remove a task and open it to edit and delete.

Anacron:

/ etc/anacrontab

How many days does it take to run the comment command after boot?

Make sure that crond service is turned on when in use

Make acrh/

Location of kernel files compiled by arch/x86/boot/bzImage

Hardware driver: initrd

Initrd: only need to provide the driver that accesses the device where the real root file system resides

Storage device and file system related modules

System initialization rc.sysinit: provides drivers for other hardware:

Run level is not supported for ROOTFS:busybox 's INIT

/ etc/inittab format is also different

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.