Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

What are the solutions to grabbing Gzip pages by php file_get_contents?

2025-01-17 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Development >

Share

Shulou(Shulou.com)06/03 Report--

This article mainly explains "php file_get_contents grab Gzip web page garbled what are the solutions", the explanation content in the article is simple and clear, easy to learn and understand, please follow the small series of ideas slowly in-depth, together to study and learn "php file_get_contents grab Gzip web page garbled what are the solutions"!

The copy code is as follows:

Request header information Original header information

Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Encoding gzip, deflate

Accept-Language zh-cn,zh;q=0.8,en-us;q=0.5,en;q=0.3

Connection keep-alive

Cookie __utma=225240837.787252530.1317310581.1335406161.1335411401.1537; __utmz=225240837.1326850415.887.3.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=%E4%BB%BB%E4%BD%95%E9%A1%B9%E7%9B%AE%E9%83%BD%E4%B8%8D%E4%BC%9A%E9%82%A3%E4%B9%88%E7%AE%80%E5%8D%95%20site%3Awww.nowamagic.net; PHPSESSID=888mj4425p8s0m7s0frre3ovc7; __utmc=225240837; __utmb=225240837.1.10.1335411401

Host www.nowamagic.net

User-Agent Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20100101 Firefox/12.0

Here are some solutions:

1. Use the zlib library that comes with it

If the zlib library is already installed on the server, the garbled code problem can be easily solved with the following code:

The copy code is as follows:

$data = file_get_contents("compress.zlib://".$ url);

2. Use CURL instead of file_get_contents

The copy code is as follows:

function curl_get($url, $gzip=false){

$curl = curl_init($url);

curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 10);

if($gzip) curl_setopt($curl, CURLOPT_ENCODING, "gzip"); //key here

$content = curl_exec($curl);

curl_close($curl);

return $content;

}

3. Use the gzip decompression function

The copy code is as follows:

function gzdecode($data) {

$len = strlen($data);

if ($len

< 18 || strcmp(substr($data,0,2),"\x1f\x8b")) { return null; // Not GZIP format (See RFC 1952) } $method = ord(substr($data,2,1)); // Compression method $flags = ord(substr($data,3,1)); // Flags if ($flags & 31 != $flags) { // Reserved bits are set -- NOT ALLOWED by RFC 1952 return null; } // NOTE: $mtime may be negative (PHP integer limitations) $mtime = unpack("V", substr($data,4,4)); $mtime = $mtime[1]; $xfl = substr($data,8,1); $os = substr($data,8,1); $headerlen = 10; $extralen = 0; $extra = ""; if ($flags & 4) { // 2-byte length prefixed EXTRA data in header if ($len - $headerlen - 2 < 8) { return false; // Invalid format } $extralen = unpack("v",substr($data,8,2)); $extralen = $extralen[1]; if ($len - $headerlen - 2 - $extralen < 8) { return false; // Invalid format } $extra = substr($data,10,$extralen); $headerlen += 2 + $extralen; } $filenamelen = 0; $filename = ""; if ($flags & 8) { // C-style string file NAME data in header if ($len - $headerlen - 1 < 8) { return false; // Invalid format } $filenamelen = strpos(substr($data,8+$extralen),chr(0)); if ($filenamelen === false || $len - $headerlen - $filenamelen - 1 < 8) { return false; // Invalid format } $filename = substr($data,$headerlen,$filenamelen); $headerlen += $filenamelen + 1; } $commentlen = 0; $comment = ""; if ($flags & 16) { // C-style string COMMENT data in header if ($len - $headerlen - 1 < 8) { return false; // Invalid format } $commentlen = strpos(substr($data,8+$extralen+$filenamelen),chr(0)); if ($commentlen === false || $len - $headerlen - $commentlen - 1 < 8) { return false; // Invalid header format } $comment = substr($data,$headerlen,$commentlen); $headerlen += $commentlen + 1; } $headercrc = ""; if ($flags & 1) { // 2-bytes (lowest order) of CRC32 on header present if ($len - $headerlen - 2 < 8) { return false; // Invalid format } $calccrc = crc32(substr($data,0,$headerlen)) & 0xffff; $headercrc = unpack("v", substr($data,$headerlen,2)); $headercrc = $headercrc[1]; if ($headercrc != $calccrc) { return false; // Bad header CRC } $headerlen += 2; } // GZIP FOOTER - These be negative due to PHP's limitations $datacrc = unpack("V",substr($data,-8,4)); $datacrc = $datacrc[1]; $isize = unpack("V",substr($data,-4)); $isize = $isize[1]; // Perform the decompression: $bodylen = $len-$headerlen-8; if ($bodylen < 1) { // This should never happen - IMPLEMENTATION BUG! return null; } $body = substr($data,$headerlen,$bodylen); $data = ""; if ($bodylen >

0) {

switch ($method) {

case 8:

// Currently the only supported compression method:

$data = gzinflate($body);

break;

default:

// Unknown compression method

return false;

}

} else {

// I'm not sure if zero-byte body content is allowed.

// Allow it for now... Do nothing...

}

// Verifiy decompressed size and CRC32:

// NOTE: This may fail with large data sizes depending on how

// PHP's integer limitations affect strlen() since $isize

// may be negative for large sizes.

if ($isize != strlen($data) || crc32($data) != $datacrc) {

// Bad format! Length or CRC doesn't match!

return false;

}

return $data;

}

Use:

The copy code is as follows:

$html=file_get_contents('https://www.jb51.net/');

$html=gzdecode($html);

Thank you for reading, the above is "php file_get_contents grab Gzip web page garbled what are the solutions" content, after learning this article, I believe that we have a deeper understanding of php file_get_contents grab Gzip web page garbled what are the solutions to this problem, the specific use of the situation also needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Development

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report