Shulou(Shulou.com)06/02 Report--

As programmers or software testers, I'm sure you've all heard of python.

The Python language has become more and more popular in the past two years, and there is a reason for its gradual rise.

Such as market demand, easy to learn, support for multiple languages. Of course, these are very official.

To put it bluntly, that is

Write a web service, you can use python

Write a server script, you can use python

Write a desktop client, you can use python

To do machine learning data mining, you can use python

You can still use python to write test tool automation scripts.

The Python language is supported for free!

Since it's so good, how can you use python to do meaningful zhuan (money)?

Today, the editor is learning python crawler technology with you.

1. Platitudes-study preparation

Learning to prepare in advance is a good start, especially for learning a language. Interest is the best teacher. If you learn crawler skills, you can set a goal for yourself, such as crawling for a girl, crawling data from a fashion website and packing it for her.

Basic knowledge must be mastered.

What is a reptile? Where did the data come from? For these basic knowledge points, please search by yourself! You also have to master:

HTML, understand the structure of the web page, content, etc., to help the subsequent data crawling.

Python

Because it is relatively simple, you can listen to some Daniel's blog articles or what others say on a zero basis.

Python play automated testing, this point has the foundation of students, you can skip ha ~

TCP/IP protocol, HTTP protocol

Understand the basic principles of network requests and network transmission to help understand the logic of crawlers when writing crawlers in the future.

Second, the idea of crawling the whole website

When users are browsing the web, they will look at the pictures.

Click on the URL to see the picture, the user enters the URL-DNS server-server host-server request-server parsing-send browser HTML, JS, CSS- browser parsing-parsing image

Crawlers need to crawl, a web page made up of HTML code, and then get pictures and text!

III. Environmental configuration

Environment configuration is always the most important environment, as anyone who has tested it knows. It's the same with python. You need to master several IDE that are easy to use. Let's take a look at some commonly used ones:

1. Notepad++, is simple, but the prompt function is not strong.

2. PyCharm, for general IDE functions, such as debugging, syntax highlighting, code jumping, etc., can also be used for Django development, support Google App Engine, and even cooler, PyCharm supports IronPython!

Good development tools are a prerequisite for all work to be done.

Crawling so much data, isn't it every minute to make money? learn the skill quickly!

Original link: http://www.magedu.com/71963.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.