In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-04-05 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Development >
Share
Shulou(Shulou.com)06/03 Report--
This article introduces the knowledge of "how to use shell script to find out the empty pages and error pages of the website". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!
The code is as follows:
Time cat sitemap.txt | while read line;do curl-l $line-m 5-connect-timeout 5-o / dev/null-s-w "$line"% {http_code} "% {size_download}"\ n "; done
I added a time to see how long it takes to execute the code.
% {http_code} means to return the HTTP status code, from which we can know whether the link is a normal 20000 link or a 404error link.
% {size_download} means to return the size of the current page. If the value is too small, it means that these pages are likely to be low-quality empty pages, and you have to find a way to eliminate them.
This is the end of the content of "how to use the shell script to find out the empty pages and 404 error pages of the site". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.