Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about the principle analysis of Python simulation landing on various websites by grabbing packages. Many people may not know much about it. In order to let everyone know more, Xiaobian summarizes the following contents for everyone. I hope you can gain something according to this article.

I. Tutorial

1.1 basic introduction

By analyzing the login process and using Python to simulate landing on a website provided by an experiment, we will learn and practice Python network programming during the experiment, Python to achieve simulated landing methods, and use Firefox to capture packet analysis plug-ins to analyze network packets.

Simulated login can help users automate a lot of operations, and it has different uses in different occasions. Whether it is automating some daily tedious operations or being used for crawlers, it is a very practical skill. This course is implemented using Firefox and Python, and the environment requirements are as follows:

Python libraries: urllib, http.cookiejar, Django

Firefox Requirements: Live http header plug-in (available)

1.3 material

1.4 ready

1)Install Live Http Headers

2)Launching a web application

Use ps -aux| grep codebox query to get the codebox process number, and then use kill -9 process number to stop the codebox process. The implementation process is shown in the following figure:

First install the demo dependent web framework django, and test whether the installation is successful:

After successful startup, enter http://localhost:8000/polls in the browser to see the login page indicating successful startup

II. Analysis of login process

To login programmatically, you first need to understand the login process for a typical Web application.

2.1 grab requesting

2.2 principle analysis

Why two requests instead of one? By analyzing the login request, it was found that after the login was successful, the server sent a 302 redirect response, and the server asked the browser to re-request the home page, which generated a second request. Analyzing the second request, you can see that it has one more request header than the login request:

The entire login process is shown below:

How does the server differentiate between different user sessions? Why do I need to write back cookies after successful login?

2.3 summary

For servers, login = authentication + write session. For browsers, login = send login info + get cookie with sessionid. It can be said that as long as the sessionid is obtained, the simulated login is realized. It allows us to drift away from the system.

3. Use Python to implement login (simple example)

Once you understand the principles and details of the login process, start writing login simulations in Python.

3.1 import module

Don't forget to write file headers and import necessary dependency modules

3.2 construct login request

The entity part of the login request is as follows:

All Python code:

additional parameters

hotlink protection

Web application resources are URL, as long as you get the URL can be referenced anywhere. Sounds convenient, but it could lead to your resources being stolen by someone else.

3.3 Send request and save cookie

Python code is as follows:

If the landing is successful,

You can see the sessionid in the specified file my_cookies.txt.

3.4 Use cookies to access system services

After we successfully simulated login, we can open this url directly through opener to use this system service. The code is implemented as follows:

If you are using this cookie in another python file, you need to load the cookie before opening the url:

After reading the above content, do you have a further understanding of the principle analysis of Python simulation landing on various websites by grabbing packages? If you still want to know more knowledge or related content, please pay attention to the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.