Shulou(Shulou.com)06/01 Report--

This article will explain in detail what cookies means in crawlers. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Introduction to cookies what is cookie?

Cookie, sometimes in its plural form Cookies, refers to the data (usually encrypted) stored on the user's local terminal by some websites in order to identify the user and perform session tracking. The definitions in RFC2109 and 2965 are obsolete, and the newly replaced specification is RFC6265 [1]. Cookie is actually browser caching.

Life cycle of cookie

Session cookie: if expires (a timestamp) is not set, the browser (session) will expire automatically when it is closed.

Persistent cookie: set expires, depending on the set expiration time (expires time can be modified, but many websites will make settings to prevent modification)

Properties of cookie

Name: is the name of a cookie.

Value: the value of a cookie.

Domain: a domain name that can access this cookie. For example, www.baidu.com:baidu.com is a top-level domain name.

Domain name division: http://aaa.bbb.secode.first.com, each point is divided into a domain name, the domain name from back to front is top-level, first-level, second-level, and so on.

For non-top-level domain names, such as second-level domain name or third-level domain name, the domain of cookie can only be top-level domain name or second-level domain name or third-level domain name itself, and the cookie of other second-level domain names cannot be set, otherwise cookie cannot be generated.

The top-level domain name can only be set to domain as the top-level domain name, not as the second-level domain name or the third-level domain name, otherwise cookie cannot be generated.

The second-level domain name can read the cookie that sets domain as the top-level domain or its own, but cannot read the cookie of other second-level domain names domain. So if you want cookie to be shared among multiple second-level domain names, you need to set domain as the top-level domain name, so that it can be in all second-level domain names or to the value of this cookie.

The top-level domain name can only get the cookie whose domain is set as the top-level domain name, and cannot be obtained if the other domain is set to the second-level domain name.

To sum up: the domain name is inherited and can only be used by itself or its parents, not its child domain name or sibling domain name. For example:

A.b.c.d.com

A.b.x.d.com

C you can use the domain name cookie whose domain is d.com or c.d.com, but not the domain name of x.d.com or b.c.d.com. Just understand it here. In this case, just know that there is such a situation. Generally, we need to pay little attention to it.

Path: the page path where you can access this cookie. For example, www.jianshu.com/u/680e0e38ddc9,path is / u/680e0e38ddc9, so only pages under the / u/680e0e38ddc9 path can read this cookie.

Expires/Max-Age: timeout for this cookie. If its value is set to a time (a timestamp), the cookie becomes invalid when this time is reached.

If it is not set, the default value is Session, which means that cookie will fail together with session.

This cookie becomes invalid when the browser is closed (not the browser tab, but the entire browser).

Size: this cookie size.

Http: the httponly property of cookie. If this property is true, the information for this cookie will only be included in the http request header

You cannot access this cookie through [xss_clean].

Secure: Boolean. Default is false. Set whether this cookie can only be passed through https.

We focus on name and value as crawlers.

Cookie application

Get the corresponding cookies in any way, such as browser, selenium, packet, etc.

Save cookies in memory, file, database, etc.

In the project where you want to apply the existing cookie, you can get the corresponding cookie in various ways: file, database, network, etc., then set it, and then access the corresponding resources.

Practical application:

There are several special servers for login operation, and all accounts are saved in the database. These specially logged-in servers log in. After the login is successful, save cookies to the database.

There is a special application server that reads cookies from the database and performs corresponding business operations. This kind of server does not handle login operations.

This is the end of this article on "what is the meaning of cookies in reptiles". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.