Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about Google optimization spider crawling and indexing skills, many people may not understand, in order to make you better understand, the editor summed up the following, I hope you can get something according to this article.

Since google controls most of the search market, you should always pay attention to and listen to some of its developments. "Google published an article called optimizing spider crawling and indexing techniques," should it attract our attention?

The content of the article highly emphasizes some skills that can improve the crawling ability of your site. Here are some specific examples of URL on display.

"the Internet is a big world, and new content is published and created all the time," said Susan Moskwa, an analyst at Google. "Google has unlimited resources, so when it faces almost unlimited resources available online, google spiders only have the ability to find and crawl part of the content, while in the face of already crawled content, we can only index part of it."

Moskwa said: "URL is like a bridge between your site and search engine spiders. Spiders need to be able to find and crawl through this bridge (IE, find and crawl your URL) to your website content page. If your URL address is too complex or too redundant, spiders will spend a lot of time tracking and anti-tracking their crawled footprints, if the URL address is well organized and points directly to the content of the relevant area." So you don't crawl to blank or repetitive content pages. "

If you want to be better indexed by Google, you must delete the user details from the URL address. You can browse the slides for details. Basically, the parameters of URL do not change the content of the page, so these parameters should be deleted or put in the cookie. This will reduce the number of URL pointing to the same content and speed up retrieval. (this should refer to the use of static addresses as much as possible to reduce the number of duplicate URL of the parameters provided by users.)

Google said that aimless and unlimited space and breadth is actually a waste of time, so for those url addresses that are unique addresses, whether past or future, it is better to have a time flag. For example, example/2009/08//11/title

Tell Google which pages can be ignored and that there is no need to crawl. Including some login pages, contact information, shopping carts and other pages, similar to the behavior that requires users to perform, spiders can not recognize the operation. This can be done by using a robots.txt file.

Finally, avoid repetition as much as possible. Google wants each content page to correspond to a unique address. Of course, they can't identify all of them, so there are some typical link factors that allow you to identify which URL of specific content pages are more popular.

After reading the above, do you have any further understanding of Google's techniques for optimizing spider crawling and indexing? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.