Replicating HTTP requests with PHP & cURL

Question 1

here is the magic soup you are missing, a $cookie file in curl_setopt.

curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie);

then you would fist curl post to the login form, save the cookie file, and then check for the filetime on the cookie ( to see if its out of date ) and create new cookie or send the $cookie file in your subsequent requests.

note i dont have this line

curl_setopt($ch, CURLOPT_COOKIESESSION, true);

also note http://curl.haxx.se/libcurl/c/CURLOPT_COOKIESESSION.html

Pass a long set to 1 to mark this as a new cookie "session". It will force libcurl to ignore all cookies it is about to load that are "session cookies" from the previous session. By default, libcurl always stores and loads all cookies, independent if they are session cookies or not. Session cookies are cookies without expiry date and they are meant to be alive and existing for this "session" only.

I think you are telling it to start a new session every time.

p.s. - I use pacer as well.

public function Login(){
        $cookie_file = __DIR__."/cookie.txt";
        $cookie_file = str_replace("\\", "/", $cookie_file);
        $this->_cookie_file = $cookie_file;
        $new_file = false;
        if(!is_file($cookie_file)){
            $h = fopen($cookie_file, "w");
            fclose($h);
            $file_time = time();
            $new_file = true;
        }else{
            $file_time = filemtime($cookie_file);
        }

        //login
        if($file_time < (time() - 1800) || $new_file){
            $url = "https://pacer.login.uscourts.gov/cgi-bin/check-pacer-passwd.pl";
            $post = array(
                    "loginid"=>"loginID",
                    "passwd"=>"password",
                    "client"=> "client",
                    "faction"=>"Login",
                    "appurl"=>"https://pcl.uscourts.gov/search"
            );


            $res = $this->_cUrl->cPost($url, $post, $cookie_file);
            $this->Log("LOGGING IN AT ".date("Y-m-d H:i:s"));
            sleep(2);
            $this->Log("SLEEPING 2 ..",E_USER_DEPRECATED);
        }

    }

from my curl library class.

public function cPost($url, $post, $cookie_file="cookie.txt"){
        if(is_array($post)){
            $post_string = $this->encodePost($post);
        }else{
            $post_string = $post;
        }

        $cookie = str_replace("\\", "/", $cookie_file);
        $fc = fopen($cookie, "r");
        fclose($fc);
        $ch = curl_init();

        curl_setopt($ch, CURLOPT_VERBOSE, 1);
        curl_setopt($ch, CURLOPT_STDERR, $this->_error_handle); 
        fwrite($this->_error_handle,"Starting debug file ".date('Y-m-d H:i:s')."\n");

        curl_setopt ($ch, CURLOPT_URL, $url); 
        curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
        curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"); 
        curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
        curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
        curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
        curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
        curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie);
        curl_setopt ($ch, CURLOPT_REFERER, $url); 
        curl_setopt($ch, CURLINFO_HEADER_OUT, true); // enable tracking
        curl_setopt ($ch, CURLOPT_POSTFIELDS, $post_string); 
        curl_setopt ($ch, CURLOPT_POST, 1); 
        $result = curl_exec ($ch); 
        if ( curl_errno($ch) ) {
            $response = 'ERROR -> ' . curl_errno($ch) . ': ' . curl_error($ch);
            throw new CurlException($response);
        } else {
            $returnCode = (int)curl_getinfo($ch, CURLINFO_HTTP_CODE);
            switch($returnCode){
                case 404:
                    $response = 'ERROR -> 404 Not Found';
                    throw new CurlException($response, CurlException::ER_RETURN_CODE);
                break;
                default:

                break;
            }
        }
        curl_close($ch);
        return $result;
    }

to access there search form.

$url = "https://pcl.uscourts.gov/dquery";
        $post = array(
            "case_no"=>$case_no,
            "mdl_id"=>"",
            "stitle"=>"",
            "nos"=> array(
                    "370",
                    "371",
                    "440",
                    "470",
                    "480",
                    "890"
                ),
            "date_filed_start"=>$date_filed_start,
            "date_filed_end"=>$date_filed_end,
            "date_term_start"=>"",
            "date_term_end"=>"",
            "date_dismiss_start"=>"",
            "date_dismiss_end"=>"",
            "date_discharge_start"=>"",
            "date_discharge_end"=>"",
            "party"=>$party,
            "ssn4"=>"",
            "ssn"=>"",
            "court_type"=>"cv",
            "default_form"=>"cvb"
        );

        print_r($post);

        $html = $this->_cUrl->cPost($url, $post, $this->_cookie_file);

I have this code in production environment for over a year now - here are the keys to the kingdom lol.

Question 2

In Chromes network tab you can find the "Copy as cURL" functionality. It will the command line to the clipboard that will replicate that request with cURL. From there on it should be trivial to convert it into PHP code.

enter image description here