file_get_contents returning corrupted image from URL

Question

I'm using this code to download image files from the Mapquest Traffic API.

<?php
$files = glob("/root/TrafficHistory/*.gif");
$lastFileName = $files[count($files)-1];

$newFile = time() . ".gif";

$imgData = file_get_contents("http://www.mapquestapi.com/traffic/v2/flow?key=**REMOVED**&mapLat=34.05396382838673&mapLng=-118.24529104634557&mapHeight=1000&mapWidth=1000&mapScale=433342");
file_put_contents("/root/TrafficHistory/" . $newFile, $imgData);

$md5Old = md5_file($lastFileName);

if ($md5Old == md5_file("./" . $newFile)) {
    unlink("./" . $newFile);
    echo "\033[31mTraffic data same at " . time() . " aborting.\033[0m\n";
} else {
    echo "\033[32mNew traffic data downloaded to /root/TrafficHistory/" . $newFile  . " at " . time() . "\033[0m\n";
}
?>

The code works flawlessly though my error occurs while saving the image. As seen here you can see that the traffic overlay downloaded through file_get_contents does not seem to fully download the GIF image. Here's a link to the GIF in question (it display's correctly in the browser though will not open in PhotoShop stating unexpected end of file). Note that all of the images downloaded through file_get_contents display this way. Though when viewed directly through the API link they display perfectly. Here's a link to the Mapquest Traffic API as well, if it's needed. Thanks for your help.

Answer 1

The file you provided is definitely truncated. It's at least 8 bytes short. There's a GIF image data block at the end with a declared length of 168 bytes, but only 162 bytes remain in the file at that point. Even after that there should be, at a minimum, a 0 terminator and then a GIF "trailer" block (the byte 0x3B).

Browsers are designed to render partially downloaded images, so they don't complain about this, although you can see the corruption: on the road that touches the very bottom of the image, there is a pattern of vertical lines (Firefox) or horizontal lines (Chrome) that shouldn't be there.

If you execute the following code to affix eight bytes to the file, it clears up that spot of corruption and also becomes possible to open the file in other programs:

$data = file_get_contents('map.gif');
$data .= "\x00\x00\x00\x00\x00\x00" . "\x00" . "\x3B";
file_put_contents('map-repaired.gif', $data);

That's six (null) bytes to finish the block, the terminating data block's length 0 byte, then the trailer byte. Such a crude fix won't work as a general solution for all the other images though.

There's nothing wrong with the code you've provided. I don't have a key for the API so I can't actually test it, but I can only imagine there must be a bug at MapQuest's end. Perhaps its GIF compressor is broken, or perhaps the server is sending an incorrect Content-Length HTTP header? I don't really know.

Since MapQuest can return the images in PNG format, and since that apparently works, go for it. PNG compression is always superior to GIF compression (except in the case when the images are extremely tiny -- around 100 bytes; then the headers of the more complex PNG format put it at a disadvantage) so there is no reason to prefer GIF in this case.

Answer 2

I've ran into other instances of file_get_contents() truncating data from an HTTP request. I solved the problem by using CURL instead:

$url = "http://www.mapquestapi.com/traffic/v2/flow?key=**REMOVED**&mapLat=34.05396382838673&mapLng=-118.24529104634557&mapHeight=1000&mapWidth=1000&mapScale=433342";

$curl = curl_init();

curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_TIMEOUT, 300);
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 8.0; Trident/4.0)');
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);

$imgData = curl_exec($curl);
$http_status = curl_getinfo($curl, CURLINFO_HTTP_CODE); // You may not need this
curl_close($curl);

For more information or to set your own CURL options, see the PHP Manual.