OpenGraph Debugger reporting bad HTTP response codes

Question

For a number of sites that are functioning normally, when I run them through the OpenGraph debugger at developers facebook com/tools/debug, Facebook reports that the server returned a 502 or 503 response code.

These sites are clearly working fine on servers that are not under heavy load. URLs I've tried include but are not limited to:

http://ac.mediatemple.net http://freespeechforpeople.org

These are in fact all sites hosted by MediaTemple. After talking to people at MediaTemple, though, they've insisted that it must be a bug in the API and is not an issue on their end. Anyone else getting unexpected 500/502/503 HTTP response codes from the Facebook Debug tool, with sites hosted by MediaTemple or anyone else? Is there a fix?

Note that I've reviewed the Apache logs on one of these and could find no evidence of Apache receiving the request from Facebook, or of a 502 response etc.

Answer 1

Got this response of them:

At this time, it would appear that (mt) Media Temple servers are returning 200 response codes to all requests from Facebook, including the debugger. This can be confirmed by searching your access logs for hits from the debugger. For additional information regarding viewing access logs, please review the following KnowledgeBase article:

Where are the access_log and error_log files for my server? http://kb.mediatemple.net/questions/732/Where+are+the+access_log+and+error_log+files+for+my+server%3F#gs

You can check your access logs for hits from Facebook by using the following command:

cat <name of access log> | grep 'facebook' 

This will return all hits from Facebook. In general, the debugger will specify the user-agent 'facebookplatform/1.0 (+http://developers.facebook.com),' while general hits from Facebook will specify 'facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php).'

Using this information, you can perform even further testing by using 'curl' to emulate a request from Facebook, like so:

curl -Iv -A "facebookplatform/1.0 (+http://developers.facebook.com)" http://domain.com 

This should return a 200 or 206 response code.

In summary, all indications are that our servers are returning 200 response codes, so it would seem that the issue is with the way that the debugger is interpreting this response code. Bug reports have been filed with Facebook, and we are still working to obtain more information regarding this issue. We will be sure to update you as more information becomes available.

So good news, is that they are busy with it solving it. Bad news, it's out of our control.

There's a forum post here of the matter:

https://forum.mediatemple.net/topic/6759-facebook-503-502-same-html-different-servers-different-results/

With more than 800 views, and recent activity, it states that they are working hard on it.

I noticed that https MT sites don't even give a return code:

Error parsing input URL, no data was scraped.

---

RESOLUTION

MT admitted it was their fault and fixed it:

During our investigation of the Facebook debugger issue, we have found that multiple IPs used by this tool were being filtered by our firewall due to malformed requests. We have whitelisted the range of IP addresses used by the Facebook debugger tool at this time, as listed on their website, which should prevent this from occurring again.

We believe our auto-banning system has been blocking several Facebook IP addresses. This was not immediately clear upon our initial investigation and we apologize this was not caught earlier.

The reason API requests may intermittently fail is because only a handful of the many Facebook IP addresses were blocked. The API is load-balanced across several IP ranges. When our system picks up abusive patterns, like HTTP requests resulting in 404 responses or invalid PUT requests, a global firewall rule is added to mitigate the behavior. More often than not, this system works wonderfully and protects our customers from constant threats.

So, that being said, we've been in the process of whitelisting the Facebook API ranges today and confirming our system is no longer blocking these requests. We'd still like those affected to confirm if the issue persists. If for any reason you're still having problems, please open up or respond to your existing support request