Shulou(Shulou.com)06/02 Report--

This article mainly explains "Python how to crawler blog article", interested friends might as well take a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "Python how to crawler blog" bar!

one。 Big data and the Foundation of data Mining

The * * part briefly introduces three questions:

1. What is big data?

2. What is data mining?

3. The difference between big data and data mining?

1. Big data (Big Data)

Big data (big data) refers to the data set that can not be captured, managed and processed with conventional software tools within a certain time range. New processing models are needed to have stronger decision-making power, insight and process optimization ability to adapt to massive, high growth rate and diversified information assets.

The following picture shows big data's classic 4V features.

IBM large database framework and visualization technology, big data commonly used: Hadoop, Spark, now more real-time data analysis, including Taobao, JD.com, nearby cuisine and so on.

Below are some of big data's applications.

When it comes to big data, we have to mention Hadoop, and when it comes to Hadoop, we have to mention Map-Reduce.

MapReduce is a software framework composed of thousands of business machines on a large cluster, and in a reliable, fault-tolerant way to deal with massive data sets at the TB level in parallel. MapReduce's idea is "divide and rule". Mapper is responsible for the score, and Reducer is responsible for summarizing the results of the map phase.

2. data mining (Data Mining)

Data mining (Data Mining): a cross-discipline of database, machine learning, artificial intelligence and statistics.

Data mining needs to find valuable knowledge, at the same time, the top is the wisdom to discover knowledge and valuable information.

Because it is mainly for big data analysis of web page data, it needs to be classified as follows:

Web mining is mainly divided into three categories: Web log mining, Web content mining and Web structure mining.

3. Machine learning

When it comes to the knowledge related to machine learning and data mining, two pictures are usually added. It vividly expresses the analogy between computer intelligence and human traditional knowledge.

two。 Install Python and basic knowledge

1. Install Python

Before you start programming with Python, you need to introduce the installation process of Python. The python interpreter can be installed with built-in use in Linux

The steps are as follows:

* step: open a Web browser and visit the official website

Step 2:

Click the Download link on the home page of the official website, enter the download interface, select the version of the Python software, the author chooses to download python 2.7.8, and click on the "Download" link.

Download address of Python:

Step 3: select the file download address and download the file.

Step 4: double-click the downloaded "python-2.7.8.msi" software and install the software.

Step 5: select the default settings in the Python installation wizard, click "Next", select the installation path, here set to the default installation path "C:Python27", and click the "Next" button, as shown in the figure.

Note 1: it is recommended to install Python under C disk, usually the path is C:Python27, there is no Chinese path.

In the Python installation wizard, select the default settings, click "Next", select the installation path, here set to the default installation path "C:Python27", and click the "Next" button.

After successful installation, as shown in the following figure:

Step 6: assuming that everything is installed normally, click "start", select "programs", and find the successfully installed Python software, as shown in the figure:

Select the third icon in the figure above, that is, click "Python (command line) command line mode", and run the program to enter the following code:

Print 'hello world'

The python command-line mode interpreter prints out the "hello world" string, as shown in the following figure.

Select the * * images in the figure, and click "IDLE (Python GUI)", that is, the integrated development environment (Python Integrated Development Environment,IDLE) running Python. The running result is as follows.

Note 2: it is recommended that you use IDLE to write scripts, complete code rather than command line mode.

2. Basic knowledge of Python

Here is a brief introduction, mainly introduces the basic knowledge of conditional statements, loop statements, functions and so on.

A, function and operation

Here's a simple example. Open the IDLE tool-> Click bar "File"-> New File New File-> name it test.py file, and add the following code to the test file:

Save the file. And click Run- > Run Module in the test.py file, and the output is shown in the following figure.

B, conditional statement

Including single-branch, double-branch and multi-branch statements, if-elif-else.

(1)。 Single branch statement

Its basic format is:

If condition: statement statement

It should be noted that the condition of the if conditional statement in Ptthon does not need parentheses (), and the condition needs to be followed by a colon, which does not have curly braces {} but uses TAB to distinguish. Among them, condition condition judgment usually has Boolean expression (True | False 0-false | 1-true or non-0 is true) and relational expression (> =

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.