What Steps do you Take to Troubleshoot Problems with PHP cURL?

Question

Almost any working PHP programmer has faced having to use CURL to send raw HTTP requests, whether it's for credit card payment processing, nefarious screen scraping, or something in-between.

Almost any forum where PHP programmers congregate has a large number of people who can't get the cURL functions to do what they want.

When cURL isn't working for you, what troubleshooting techniques do you use to figure out why it isn't working? What weird gotchas with PHP's curl implementation have you run into? If someone asks a "HALP MY CURL IZ BROKEN" question on a forum, what are the steps you take to figure out why their request isn't working?

Answer 1

I find the CURLINFO_HEADER_OUT option to be very useful.

<?php
$curl = curl_init('http://www.php.net');

curl_setopt($curl, CURLOPT_HEADERFUNCTION, 'dbg_curl_data');
curl_setopt($curl, CURLOPT_WRITEFUNCTION, 'dbg_curl_data');
curl_setopt($curl, CURLINFO_HEADER_OUT, true);

curl_exec($curl);

echo '<fieldset><legend>request headers</legend>
  <pre>', htmlspecialchars(curl_getinfo($curl, CURLINFO_HEADER_OUT)), '</pre>
</fieldset>';

echo '<fieldset><legend>response</legend>
  <pre>', htmlspecialchars(dbg_curl_data(null)), '</pre>
</fieldset>';

function dbg_curl_data($curl, $data=null) {
  static $buffer = '';

  if ( is_null($curl) ) {
    $r = $buffer;
    $buffer = '';
    return $r;
  }
  else {
    $buffer .= $data;
    return strlen($data);
  }
}

Answer 2

Actually, I never use CURL (in php). The PHP stream api is much neater, and can be used to POST data as well. Wez Furlong has an article about this.

If I were to use it? I'd start with turning on all messages (setting error reporting to E_ALL). If I find that PHP doesn't tell me what I need in the error messages, I'd probably use a proxy approach to see what's actually going on. Changing the target url to a local php page containing something like

<?php
var_dump($_POST);
var_dump($_GET);
var_dump($_SERVER);

is one way. Another way is to use a utility like netcat to listen on port 80 and send the request there:

netcat -l -p 80

This won't return anything to curl, but it will allow you to see exactly what is sent the server, which might be enough to diagnose the problem.

You can also retrieve the headers from PHP using the apache_request_headers() function. In most cases I prefer the netcat approach, though, since it guarantees that I see the unmodified truth, and also display the raw post data.