Shulou(Shulou.com)06/02 Report--

This article mainly introduces "what are the basic knowledge of python crawler". In daily operation, I believe that many people have doubts about the basic knowledge of python crawler. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "what is the basic knowledge of python crawler?" Next, please follow the editor to study!

1. What is a reptile?

Crawlers, that is, web crawlers, can be understood as spiders crawling on the web. The Internet is compared to a big web, and a crawler is a spider crawling around on the web. If it encounters resources, it will grab it. What do you want to catch? You control it.

For example, it is crawling a web page, and in this network he finds a way, which is actually a hyperlink to the web page, so it can climb to another web page to get data. In this way, the whole web is within reach of the spider, and it is not a problem to climb down in minutes.

two。 The process of browsing the web

In the process of browsing the web, we may see many good-looking pictures, such as http://zhimaruanjian.com/, through the DNS server, find the server host, send a request to the server, after the server parses, sends to the user the browser HTML, JS, CSS and other files, the browser parses out, the user can see all kinds of pictures.

Therefore, the web page that users see is essentially made up of HTML code, and the crawler crawls these contents. By analyzing and filtering these HTML codes, we can achieve access to pictures, text and other resources.

The meaning of 3.URL

URL, that is, uniform resource locator, that is, what we call URL, uniform resource locator is a concise expression of the location and access method of resources that can be obtained from the Internet. It is the address of standard resources on the Internet. Every file on the Internet has a unique URL that contains information indicating the location of the file and what the browser should do with it.

The format of URL consists of three parts:

The first part of ① is the protocol (or service mode).

The second part of the ② is the IP address (and sometimes the port number) of the host where the resource is stored.

The third part of the ③ is the specific address of the host resource, such as directory and file name.

When crawling data, a crawler must have a target URL in order to obtain data. Therefore, it is the basic basis for a crawler to obtain data, and an accurate understanding of its meaning is of great help to crawler learning.

4. Configuration of the environment

Learning Python, of course, the configuration of the environment, at first I used Notepad++, but found that its prompt function is really too weak, so, in Windows I used PyCharm, in Linux I used Eclipse for Python, and there are several more excellent IDE, you can refer to this article to learn the IDE recommended by Python. A good development tool is a thruster for moving forward. I hope you can find an IDE that suits you.

At this point, the study of "what are the basic knowledge of python crawler" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.