Shulou(Shulou.com)06/02 Report--

Linux's experience of web service at the beginning of love! (I think this is the first and important service to enter the Linux world.)

The article is called long, if you understand it, please take a look at what may be wrong, please correct it. Xiaobai can also understand what is written in vernacular!

Notes on http Hypertext transfer Protocol

Transport layer: tcp/udp provides process address

Port number:

Tcp: transmission control protocol, connection-oriented protocol; virtual link needs to be established before communication; link is removed after completion; data flow communication protocol

Number of ports in 0-65535 Linux

Udp:User Datagram Protocol, connectionless protocol; connectionless Datagram communication protocol

0-65535

IANA:

0-1023: it is well known that only administrators have the right to register for permanent allocation to fixed applications, privileged ports, 22/tcp (ssh), 80/tcp (http), 443/tcp (https).

1024-41951: also a registered port, but the requirements are not particularly strict, assigned to the program to register for an application, 11211/tcp, 11211/udp (memcached), 3306/tcp (mysql)

41952 ports: ports randomly used by client programs; dynamic ports, or private ports; definition of its range: / proc/sys/net/ipv4/ip_local_port_range

The client accesses the port randomly generated on the server side and accesses a fixed port on the server side.

Socket Socket: an implementation of IPC that allows communication between different processes on different hosts (or even the same host) to complete data exchange; also known as the Socket API library invocation interface, first appeared on 4.2 BSD systems (Berkeley system distribution) in 1983.

SOCK_STREAM:tcp socket

SOCK_DGRAM:udp socket

SOCK_RAW: bare sockets (communicate without the interface of UDP/TCP)

IPv4:

Classification:

ARV 1-127

BRV 128-191

Cpurs 192-223

D: multicast, 224-239

ERV 240-254

Private address:

A:10.0.0.0/8

BRV 172.16.0.0Compact 16-172.31.0.0Universe 16

CRV 192.168.0.0Compact 24-192.168.255.0Universe 24

Features of the TCP protocol:

Establish a connection: three-way handshake

Packaging data into segments: checksum (CRC-32) cyclic redundancy check

Confirmation, retransmission, and timeout:

Message sorting: logical serial number

Flow control: sliding window algorithm (with receiving cache control, pausing reception when almost full) when the sender's flow is small.

Congestion control: slow start and congestion avoidance algorithm

Socket Domain (according to the address it uses): socket is divided into three categories

AF_INET:Address Family,IPv4

AF_INET6:IPv6

AF_UNIX: used when communicating between different processes on the same host

Each type of socket provides at least two types of socket: stream, Datagram

Stream: reliable delivery, connection-oriented, borderless

Datagram: delivered unreliably, bounded, connectionless; (for example, sending letters with envelopes, multiple envelopes are required for encapsulation)

Socket-related system calls:

Socket (): create a socket

Bind (): bind

Listen (): monitoring

Accept (): receives a request

Connect (): request connection establishment

Write (): send

Read (): receive

Send (), recv (), send and receive

Sendto (), recvfrom () send and receive

HTTP protocol

Http: hyper text transfer protocol, 80/tcp

Html: programming language, hypertext markup language

TITLE

Blabla... Blabal

CSS: Cascading Style Sheet cascading style sheet

Js: javascript

MIME: Multipurpose Internet Mail Extesion Multipurpose Internet Mail extension

Working mechanism:

Http request

Http response

The requested content is called: Web resource: web resource

Static files: .jpg, .gif, .html, .txt, .js, .css, .mp3, .avi

Dynamic files: .php, .jsp, (client requests are executed on the server instead of feedback directly to the client)

(for example, the Windows browser needs to work with an external plug-in, the browser itself does not have the ability to open pictures or videos, etc., and decides to open it with the help of a plug-in on the client based on the suffix name of the file.)

Media: determines how the client browser opens with the help of that plug-in

Media type (MIME type): major/minor (primary type / secondary type)

Text/html hypertext document

Text/plain plain text document

P_w_picpath/jpeg

P_w_picpath/gif

URI: Uniform Resource Identifier uniform Resource Identifier

URL: Uniform Resorce Locator, used to describe the location of a specific resource on a server

Scheme://Server:Port/path/to/resource

Http://www.magedu.com/p_w_picpaths/logo.jpg

URN: Uniform Resource Naming uniform resource naming

Http protocol version:

HTTP/0.9: prototype version with rudimentary functions

HTTP/1.0: the first widely used version that supports MIME

HTTP/1.1: enhanced caching

Spdy

HTTP/2.0:

Reading the rfc document will help you understand 1.1 2.0

A complete http request processing process:

(1) establish or process a connection: receive or reject a request

(2) receive the request:

The process of receiving a request for a resource in a request message from a network

Concurrent access response model (Web IhampO):

Single-process Iripo structure: start a process to process user requests, and only one at a time; multiple requests are responded to serially; (multiple requests are queued up and processed in turn)

Multi-process Icano structure: start multiple processes in parallel, each responding to a request

Reusing Ipicuro structure: one process responds to n requests; (borrows a reusable Iripple O processor based on event-driven mechanism)

Multithreaded model: a process generates N threads, each responding to a user request

Event-driven: event-driven (a process maintains a similar event monitor, which can monitor multiple Icano so that a server process can connect multiple requests at the same time) aiming at how to know the connection and end of each request, the server programming completes the tracking of the connection status based on the event callback mechanism internally, and completes the response of the connection and termination based on the callback state.

Reused multi-process / thread-- Ithumb O structure: starts multiple (m) processes, each responding to n requests

(3) processing the request: parsing the request message and obtaining the requested resources, method and other related information

Metadata: request message header

/ / method of request / / Resource / / version

Host: name of the host requested by www.magedu.com

Connection:// indicates how this connection will be used

(4) access resources: obtain the requested resources in the request message

The web server, that is, the server that stores web resources, is responsible for providing the requestor with static resources requested by the other party, or resources generated after dynamic operation; these resources are placed under a path of the local file system, which is often called DocRoot

/ var/www/html/

P_w_picpaths/1.jpg

Http://www.magedu.com/p_w_picpaths/1.jpg

Web server resource path mapping method:

(a) docroot

(B) alias

(C) Virtual host docroot

(d) user's home directory docroot

(5) construct response message

MIME type of resource: tagged method

Explicit classification

Magic classification

Negotiation classification

URL redirection:

The response built by the web service is not the resource requested by the client, but another access path to the resource

(6) send a response message

(7) keep a log

Http server program:

Httpd (apache)

Nginx

Lighttpd / / Germans develop lightweight similar Nginx

Application server (container class): both movement and movement can be parsed

IIS

Tomcat, jetty, jboss, resin

Webshpere, weblogic, oc4j

Www.netcraft.com / / View the market share occupied by some web servers

Installation, configuration and use of httpd:

Httpd: apache

A patchy server = apache

ASF: apache software foundation / / Software Foundation

Features of httpd:

Highly modular: core + modules

DSO: Dynamic Shared Object dynamic module mechanism

MPM:Multipath Processing Modules implements a variety of different Istroke O model mechanisms

Prefork: a multi-process model in which each process responds to a request; each main process is only responsible for listening to sockets, receiving requests and handing over requests to child processes.

A main process: responsible for generating n child processes, also known as worker processes, each of which processes a user request; even if there is no user request, multiple idle processes are generated in advance, waiting for the request to arrive at any time; the maximum is no more than 1024

Worker: a multithreaded model in which each thread responds to a request

One main process: generate multiple child processes, each child process responsible for generating multiple threads, each thread responding to a request

Event: event-driven model where each thread responds to n requests

One main process: generate m child processes, each with n requests directly

M-process, n-thread: M-thread

Httpd-2.2: event is used for testing

Httpd-.24:event can be produced and used.

CentOS 6:

Program environment

Configuration file:

/ etc/httpd/conf/httpd.conf

/ etc/httpd/conf.d/*.conf / / Supplementary profile

Service script:

/ etc/rc.d/init.d/httpd

Service script's own configuration file: / etc/sysconfig/httpd

Main program file:

/ usr/sbin/httpd / / prefork used by default

/ usr/sbin/httpd.event / / event

/ usr/sbin/httpd.worker / / worker

Log file directory:

/ var/log/httpd

Access_log: access log

Error_log: error log / / start stop access error and other information

Site documentation directory:

/ var/www/html

Module file path:

/ usr/lib64/httpd/modules

Composition of the configuration file:

~] # grep "Section" / etc/httpd/conf/httpd.conf

# Section 1: Global Environment / / Global environment configuration

# Section 2: 'Main' server configuration / / Master server configuration

# Section 3: Virtual Hosts / / Virtual Host configuration

Configuration format: the value corresponding to the directive value instruction to complete, such as listen 80

Directive: insensitive to character case

Value: when path, depends on file system

Common configuration: modify the configuration file httpd-t or service httpd configtest test syntax

1. Restart is required to modify the listening IP and Port / / to modify the listening port

Listen [IP:] PORT

Omitting ip means monitoring all local IP; Listen can be repeated multiple times.

2. Persistent connection

Persistent Connection (persistent connection): a connection is established. After each resource is acquired, it will not be disconnected, but will continue to wait for other requests to be completed.

How to disconnect?

Quantity limit: 100 resource requests default to 100

Time limit: configurable

Side effects: for servers with large concurrent visits, persistent connections will use some requests to get no response

Compromise: use a short persistent connection time; 5 seconds

Httpd-2.4 supports millisecond persistence

Non-persistent connection

KeepAlive On | whether Off / / is enabled or not

MaxKeepAliveRequests # / / maximum number of connection requests

KeepAliveTimeout # / / connection timeout

Test:

Telnet HOST PORT

GET / URL HTTP/1.1

Host: HOSTNAME or IP

3 、 MPM

Multipath Process Module: multichannel processing module

Prefork, worker, event

Httpd-2.2 does not support compiling multiple modules at the same time, so you can only select one at the time of compilation. The package installed by rpm provides three binary program files, which are used to support different MPM mechanisms. Confirmation method:

# ps aux | grep httpd

The default is / usr/sbin/httpd, which uses prefork

View the list of modules:

View statically compiled modules

# httpd-lamp / httpd.worker-l///httpd.event-l

Compiled in modules:

Core.c / / core module

Prefork.c / already used

Http_core.c / / functional core module

Mod_so.c / / supports dynamic loading and unloading of modules

View all statically compiled and dynamically loaded modules

# httpd-M

Change the httpd program you are using:

/ etc/sysconfig/httpd

HTTPD=

Restart service takes effect

Configuration of prefork:

StartServers 8 / / how many service child processes are started when the service starts

MinSpareServers 5 / / minimum number of idle processes

MaxSpareServers 20 / / maximum number of idle processes

ServerLimit 256 / / generate quantity for MaxClients during the server declaration cycle

MaxClients 256 / maximum concurrency

MaxRequestsPerChild 4000 / / how many times can a process respond to requests and then die

Configuration of worker:

StartServers 4 / / how many service child processes are started when the service starts

MaxClients 300 / / maximum number of concurrency, maximum number of threads started

MinSpareThreads 25 / / minimum idle thread

MaxSpareThreads 75 / / maximum idle thread

ThreadsPerChild 25 / / number of threads that can be started per process

MaxRequestsPerChild 0 / / number of requests for maximum response per thread

PV, UV

PV: the number of Page View page visits

UV: number of User View users

Effective visits to independent IP

4 、 DSO

Configuration instruction to realize module loading

LoadModule

Module paths can use relative addresses

Relative to the path pointed to by ServerRoot (/ etc/httpd)

/ etc/httpd/modules/

5. Define the document page path of 'Main' server

DocumentRoot

Document path mapping:

The path DocumentRoot points to is the starting position of the URL path.

DocumentRoot "/ var/www/html"

Test/index.html-- > http://HOST:PORT/test/index.html

6. Site access control

You can indicate which resources are controlled based on two types of paths

File system path:

Access control mechanism:

Based on source address

Based on account number

7. Access control based on source address in Directory

(1) Options recommends closing (if the download site is open and the default page is not specified, the resources on the site can be opened, and the non-download site is not safe. In the case where no default page is defined, the colleague http's client does not do the test page. Opening the website will get all the website resources on the server side.)

For example, the main page without default definition (index.html) stored in the default root directory will be indexed to other documents in the root directory.

All available features: Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews

None, All

Indexes: index; / / when visiting the default main page under the website directory

FollowSymlinks: allows tracking of symbolic link files; allows access to linked files in the root directory of the website

SymLinksifOwnerMatch: allow tracking link files and source files to be the same owner (use less)

MultiViews: whether to allow multiple views (do not open) (use less)

(2) access control mechanism based on source address.

Order: check order

Order allow,deny

Order deny,allow

Allow from

Deny from

Source address:

IP

NetAddr:

172.16

172.16.0.0

172.16.0.0/16

172.16.0.0/255.255.0.0

8. Defining default access to the main page does not specify access from left to right

DirecotryIndex index.html index.html.var index.php

9. Log settings

Error log:

ErrorLog logs/error_log

LogLevel warn

Debug, info, notice, warn, error, crit, alert, emerg

Debugging information, notification, warning, error, critical strike, alarm, emergency

Defining warn is recorded at a higher level than warn.

Access Log:

CustomLog logs/access_log combined

LogFormat "% h% l% u% t\"% r\ "% > s% b\"% {Referer} I\ "\"% {User-Agent} I\ "" combined

% h: client IP address

% l: Remote logname (from identd, if supplied). -indicates empty; Telnet names are mostly empty

% u: Remote user, (from auth; may be bogus if return status (% s) is 401); most user names for authentication do not need authentication-indicated as empty

% t:Time the request was received (standard english format), the time the server received the request; default is English format

% r:First line of request, the first line of the request message (URL of the request, protocol version, etc.) (method url version)

% > s: response status code

% b: the size of the response message in bytes, excluding the header of the response message

% {Referer} I: the value of the header "referer" in the request message; the access entry of the current resource, that is, from which page the hyperlink jumps

% {User-Agent} I: the value of the header "User-Agent" in the request message; that is, the application used to make the request

Details: http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#formats

10. Path alias

DocumentRoot "/ www/htocs"

Http://www.magedu.com/download/bash-4.4.2-3.el6.x86_64.rpm

-- > / www/htdocs/download/bash-4.4.2-3.el6.x86_64.rpm

Alias / URL/ "/ PATH/TO/SOMEDIR/"

Alias / 3 / "/ 1Compact 2"

Http://www.magedu.com/3/index.html

-- > / 1Compact 2Compact 3 /

Alias / bbs/ "/ forum/htdocs"

Http://www.magedu.com/bbs/index.html

-- > / forum/htdocs/bbs/

11. Set the default character set

AddDefaultCharset UTF-8

GBK, GB2312, GB18030 Chinese characters

12. User-based access control

Authentication Challenge:

WWW-Authenticate: the response code is 401, rejecting the client request, and indicating that the customer is required to provide the account number and password

Certification:

Authorization: the client user enters the account and password and sends the request message again. If the authentication is passed, the server sends the response resource.

Type of certification:

Basic: plaintext

Digest: message digest

Security domain: a path that requires user authentication before it can be accessed

It should be identified by name and used to inform the user of the reason for authentication.

Where is the user's account number and password stored?

Virtual account: the authentication ID used only when accessing a service

Storage:

Text file

SQL database

Ldap

Nis

Basic Certification:

(1) define a security domain

/ / specify the path

Options None

AllowOverride None

AuthType Basic

AuthName "STRING" / secure domain name

Account file used in AuthUserFile "/ PATH/TO/HTTPD_USER_PASSWD_FILE" / / authentication

Require user username1 username2. / / all legitimate (allowed) accounts are fine

Allow all users in the account file to log in and access:

Require valid-user

(2) provide account number and password storage (text file)

Use the htpasswd command for management

Htpasswd [options] passwordfile username

-c: passwordfile is created automatically, so you should use it only when adding the first user

-m: md5 encrypted user password

-s: sha1 encrypted user password

-D: delete the specified user

(3) implement group-based authentication

Options None

AllowOverride None

AuthType Basic

AuthName "STRING"

AuthUserFile "/ PATH/TO/HTTPD_USER_PASSWD_FILE"

AuthGroupFile "/ PATH/TO/HTTPD_GROUP_FILE"

Require group GROUP1 GROUP2...

To provide: user account files and group files

Group files: define a group per line

GRP_NAME:user1 user2 user3... Group names colon user names are separated by spaces

Example:

Options None

AllowOverride None

AuthType Basic

AuthName "Administator private"

AuthUserFile "/ etc/httpd/conf.d/.htpasswd"

AuthGroupFile "/ etc/httpd/conf.d/.htgroup"

Require group webadmin

13. Virtual host

There are three implementation scenarios:

Based on ip:

Prepare at least one ip address for each virtual host

Based on port:

Preparing at least one dedicated port; for each virtual host is rarely used in practice

Based on hostname:

Prepare at least one dedicated hostname for each virtual host

Any of the above three methods can be mixed.

Note: the general virtual host should not be mixed with the central host, so to use the virtual host, first disable the central host

Disable central host: comment DocumentRoot

Each virtual host has a dedicated configuration:

SeverName

DocumentRoot ""

ServerAlias: the alias of the virtual host; the following can be defined in the virtual host

ErrorLog

CustomLog

Ip addr add [ip/mask] dev eth0 / add a new IP

Ip addr list / / View

Example 1: based on ip

ServerName web1.magedu.com

DocumentRoot "/ vhosts/web1/htdocs"

ServerName web2.magedu.com

DocumentRoot "/ vhosts/web2/htdocs"

Example 2: based on port

ServerName web2.magedu.com

DocumentRoot "/ vhosts/web2/htdocs"

ServerName web3.magedu.com

DocumentRoot "/ vhosts/web3/htdocs"

Example 3: based on hostname

ServerName web1.magedu.com

DocumentRoot "/ vhosts/web1/htdocs"

ServerName web2.magedu.com

DocumentRoot "/ vhosts/web2/htdocs"

ServerName web3.magedu.com

DocumentRoot "/ vhosts/web3/htdocs"

14. Built-in status page / / built-in status page

SetHandler server-status

Order deny,allow

Deny from all

Allow from 172.16

Implementation: access control based on account

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.