Shulou(Shulou.com)06/02 Report--

This article mainly introduces how to use the wget command in Linux, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let Xiaobian take you to understand.

The Linux common command wget command is used to download files from the specified URL. Wget is very stable, it has strong adaptability in the case of narrow bandwidth and unstable network. If the download fails due to network reasons, wget will keep trying until the whole file is downloaded. If the server interrupts the download process, it will connect to the server again and continue the download from where it stopped. This is useful for downloading large files from servers that limit link time.

WgetLinux system download file tool

Add that wget supports HTTP,HTTPS and FTP protocols, and you can use HTTP agents. The so-called automatic download means that wget can be executed in the background after the user exits the system. This means that you can log in to the system, start a wget download task, and then exit the system, and wget will be executed in the background until the task is completed, compared with most other browsers that require the user to participate in downloading large amounts of data all the time, which saves a lot of trouble.

Used to download resources from the network, no specified directory, download resources back to the current directory by default. Although wget is powerful, it is relatively easy to use:

Support for breakpoint downloading function was also the biggest selling point of network ants and FlashGet at that time. Now, Wget can also use this function, and those users whose network is not very good can rest assured.

Support both FTP and HTTP downloads. Although most software can be downloaded using HTTP, sometimes it is still necessary to use FTP to download software.

Supporting proxy servers generally does not expose their systems directly to the Internet for systems with high security intensity, so supporting proxies is a necessary function for downloading software.

The setting is convenient and simple, and users who are accustomed to the graphical interface are no longer used to the command line. However, the command line actually has more advantages in setting. At least, the mouse can be clicked many times less, and don't worry about whether it is wrong to click the mouse.

Small programs, completely free programs can be ignored, because now the hard drive is too big; completely free has to be considered, even though there are many so-called free software on the Internet, but the advertisements of these software are not what we like.

Syntax wget [parameter] [URL address] option startup parameter:-V,-- version displays the version of wget and exits-h,-- help prints syntax help-b,-- background starts and goes to the background to execute-e,-- execute=COMMAND executes commands in the format of `.wgetrc'. Wgetrc format see / etc/wgetrc or ~ / .wgetrc records and input file parameters:-o,-- output-file=FILE writes records to FILE files-a,-- append-output=FILE appends records to FILE files-d,-- debug prints debug output-Q,-- quiet quiet mode (no output)-v,-- verbose verbose mode (this is the default)-nv -- non-verbose turns off verbose mode But not in quiet mode-I,-- input-file=FILE downloads URLs-F that appears in the FILE file,-- force-html treats the input file as a HTML format file-B -- base=URL prefixes URL as the relative link that appears in the file specified by the-F-I parameter-- sslcertfile=FILE optional client certificate-- KEYFILE of sslcertkey=KEYFILE optional client certificate-- egd-file=FILE specifies the file name download parameter of EGD socket:-- bind-address=ADDRESS specifies the local use address (hostname or IP) Use when there is more than one IP or name locally)-t,-tries=NUMBER sets the maximum number of attempts to link (0 means unlimited). -O-- output-document=FILE writes the document to the FILE file-nc,-- no-clobber do not overwrite the existing file or use the prefix-c,-- continue then downloads the undownloaded file-- progress=TYPE sets the process bar tag-N,-- timestamping do not re-download the file unless it is newer than the local file-- server-response print server response-- spider does not download anything-T -- timeout=SECONDS sets the number of seconds of response timeout-w,-- SECONDS seconds between wait=SECONDS attempts-- waitretry=SECONDS waits for 1 between relinks. SECONDS seconds-random-wait waits for 0 between downloads. 2*WAIT seconds-Y,-- proxy=on/off turns on or off agent-Q,-- quota=NUMBER sets download capacity limit-- limit-rate=RATE limits download output rate directory parameters:-nd-- no-directories does not create directories-- x,-- force-directories forces directory creation-- nH,-- no-host-directories does not create host directories-P,-- directory-prefix=PREFIX saves files to the directory PREFIX/... -cut-dirs=NUMBER ignores the NUMBER layer remote directory HTTP option parameter:-- http-user=USER sets the HTTP user name to USER. -- http-passwd=PASS sets the http password to PASS-C,-- cache=on/off allows / disallows server-side data caching (generally allowed)-E -- html-extension saves all text/html documents with the .html extension-- ignore-length ignores the `Content-Length' header domain-- header=STRING inserts the string STRING in headers-- proxy-user=USER sets the agent's user name to USER-- proxy-passwd=PASS sets the agent's password to PASS-- referer=URL contains `Referer: URL' header-s in the HTTP request,-- save-headers saves the HTTP header to the file-U -- user-agent=AGENT sets the name of the agent to AGENT instead of Wget/VERSION-- no-http-keep-alive closes the HTTP active link (permanent link)-- cookies=off does not use cookies-- load-cookies=FILE loads cookie from the file FILE before starting the session-- save-cookies=FILE saves cookies to the FILE file after the session ends. FTP option parameter:-nr,-- dont-remove-listing does not remove the `.upload 'file-g -- glob=on/off turns on or off the globbing mechanism for file names-- passive-ftp uses passive transfer mode (the default). -- active-ftp uses active transfer mode-- retr-symlinks when recursive Recursive download parameters for linking to a file (instead of a directory):-r,-- recursive recursive download-- use with caution!-- level=NUMBER maximum recursive depth (inf or 0 for infinity)-- delete-after partially deletes files after now-k,-- convert-links converts non-relative links to relative links-backup-converted before converting file X. Back it up as X.orig-m,-- mirror is equivalent to-r-N-l inf-nr-p,-- page-requisites download shows the inclusion and exclusion of all images in the recursive download of the HTML file (accept/reject):-A,-accept=LIST semicolon-delimited list of accepted extensions-- reject=LIST semicolon-delimited list of unacceptable extensions-D -- domains=LIST semicolon delimited list of accepted domains-- exclude-domains=LIST semicolon delimited list of unacceptable domains-- follow-ftp tracking FTP links in HTML documents-- follow-tags=LIST semicolon delimited list of tracked HTML tags-G,-- ignore-tags=LIST semicolon delimited list of ignored HTML tags-H,-- span-hosts goes to external host-L when recursive -- relative only tracks relative links-- I,-- include-directories=LIST allows the list of directories-X,-- exclude-directories=LIST is not included in the list of directories-- np,-- no-parent does not trace back to the parent directory wget-S-- spider url does not download only the process parameter URL: download the specified URL address.

An instance uses wget to download a single file.

The following example of wget http://www.jsdig.com/testfile.zip downloads a file from the network and saves it in the current directory, and a progress bar is displayed during the download process, including (percentage of download completed, bytes already downloaded, current download speed, remaining download time).

Download and save under a different file name

Wget-O wordpress.zip http://www.jsdig.com/download.aspx?id=1080wget commands by default with the last character that matches /, and the file name is usually incorrect for dynamically linked downloads.

Error: the following example downloads a file and saves it with the name download.aspx?id=1080:

Wget http://www.jsdig.com/download?id=1 is still named after download.php?id=1080, even if the downloaded file is in zip format.

Correct: to solve this problem, we can use the parameter-O to specify a file name:

Wget-O wordpress.zip http://www.jsdig.com/download.aspx?id=1080wget speed limit download

Wget-limit-rate=300k http://www.jsdig.com/testfile.zip when you execute wget, it takes up all possible broadband downloads by default. But the speed limit is necessary when you are ready to download a large file and you need to download other files.

Use wget breakpoint to resume transmission

Wget-c http://www.jsdig.com/testfile.zip uses wget-c to restart interrupted files, which is very helpful for us to download large files when we are suddenly interrupted due to network and other reasons, we can continue to download instead of downloading a file. You can use the-c parameter when you need to continue interrupting downloads.

Download using wget background

Wget-b http://www.jsdig.com/testfile.zip Continuing in background, pid 1840. Output will be written to `wget-log'. When downloading very large files, we can use the parameter-b to download in the background. You can use the following command to check the download progress:

Tail-f wget-log masquerade agent name download

Wget-user-agent= "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16" http://www.jsdig.com/testfile.zip some websites can reject your download request by determining that the proxy name is not a browser. However, you can use the-- user-agent parameter to disguise.

Test download link

When you plan to download regularly, you should test whether the download link is valid at the scheduled time. We can add the-- spider parameter to check.

Wget-- spider URL if the download link is correct, it will display:

Spider mode enabled. Check if remote file exists. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Remote file exists and could contain further links, but recursion is disabled-- not retrieving. This ensures that the download will take place at the scheduled time, but when you give the wrong link, the following error will be displayed:

Wget-- spider url Spider mode enabled. Check if remote file exists. HTTP request sent, awaiting response... 404 Not Found Remote file does not exist-- broken Linklink! You can use the-spider parameter in the following situations:

Check before regular download

Interval to check whether a website is available

Check the dead links on the website page

Increase the number of retries

Wget-tries=40 URL may fail if there is a problem with the network or downloading a large file. By default, wget retries the connection to download the file 20 times. If necessary, you can use-- tries to increase the number of retries.

Download multiple files

Wget-I filelist.txt first, save a download link file:

Cat > filelist.txt url1 url2 url3 url4 then uses this file and the parameter-I to download.

Mirror website

Wget-- mirror-p-- convert-links-P. / LOCAL URL download the entire website locally.

-- download miror account opening image.

-p download all files that are displayed normally for the html page.

-- after convert-links is downloaded, it is converted to a local link.

-P. / LOCAL saves all files and directories to the local specified directory.

Filter specified format downloads

Wget-reject=gif ur downloads a website, but you don't want to download images, you can use this command.

Save the download information in the log file

Wget-o download.log URL does not want the download information to be displayed directly on the terminal but in a log file that can be used.

Limit the total size of downloaded files

Wget-Q5m-I filelist.txt when you want to download more than 5m files and exit the download, you can use. Note: this parameter does not work for a single file download and is only valid for recursive downloads.

Download files in the specified format

Wget-r-A.pdf url can use this feature in the following situations:

Download all the pictures on a website.

Download all the videos from a website.

Download all the PDF files for a website.

FTP download

Wget ftp-url wget-- ftp-user=USERNAME-- ftp-password=PASSWORD url can use wget to download the ftp link.

Download using wget anonymous ftp:

Wget ftp-url ftp downloads authenticated with wget username and password:

Wget-- ftp-user=USERNAME-- ftp-password=PASSWORD url Thank you for reading this article carefully. I hope the article "how to use the wget Command in Linux" shared by the editor will be helpful to you. At the same time, I also hope you will support us and follow the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.