groovy/grails - Can't make HTTPBuilder work with a URL (provided by Paypal)

Question

PLEASE NOTE: With the code/info below, this problem can be reproduced in 2-3 minutes.

BACKGROUND: Firstly I haven't found a plugin to help with the Paypal Advanced Interface (their mid-level solution between "Standard" and "Pro"), but if if I don't need to roll my own HTTPBuilder and other interface code I'm definitely for that.

Now, I'm able to narrow down the HTTPBuilder problem to making the curl command work from the DOS prompt using the correct URL; I just can't get it to work with groovy's HTTPBuilder.

So, what works that one can easily try is:

 c:\groovy\ex>curl https://pilot-payflowpro.paypal.com
 curl: (52) Empty reply from server

But still a reply. Or the real URL to get a SECURETOKEN back from Paypal is more like:

c:\groovy\ex>curl https://pilot-payflowpro.paypal.com -kd "PARTNER=PayPal&VENDOR=ROAdvanced&USER=ROAdvanced&PWD=joespizza1&TRXTYPE=S&MODE=TEST&AMT=40&CREATESECURETOKEN=Y&SECURETOKENID=12528208de1413abc3d60c86233"
RESULT=0&SECURETOKEN=15XTWEZtFlkeNqtWCBZHSTgcs&SECURETOKENID=12528208de1413abc3d60c86233&RESPMSG=Approved

OR you'll get a result like the following, but either result is good, since Paypal is sending a response in both cases!

RESULT=7&SECURETOKENID=12528208de1413abc3d60c86233&RESPMSG=Field format error: Secure Token Id already been used

---

Ok, my code is as follows. Please note my code errors out with an immediate failure even though (I think) I'm using a delay:

 Class: org.apache.http.NoHttpResponseException
 Message: The target server failed to respond

This happens around the

  http.request(GET, ContentType.ANY) {

Please note even with setting the delays on HTTPBuilder, this failure happens IMMEDIATELY. I'll put the whole code/stack trace in at the end. Also note if one takes out adding the SSL security as described in this post, the result is the same error, i.e. the server failed to respond.

---

So the code is:

 package apps

 import grails.converters.*
 import org.codehaus.groovy.grails.web.json.*; // package containing JSONObject,  JSONArray,...
 import groovyx.net.http.*
 import static groovyx.net.http.ContentType.*
 import static groovyx.net.http.Method.*

 import javax.net.ssl.X509TrustManager
 import javax.net.ssl.SSLContext
 import java.security.cert.X509Certificate
 import javax.net.ssl.TrustManager
 import java.security.SecureRandom
 import org.apache.http.conn.ssl.SSLSocketFactory
 import org.apache.http.conn.scheme.Scheme
 import org.apache.http.conn.scheme.SchemeRegistry
 import org.apache.http.conn.ssl.X509HostnameVerifier


class PaypalController {

   def index() {
    def paypalUrl = "https://pilot-payflowpro.paypal.com?PARTNER=PayPal&VENDOR=ROAdvanced&USER=ROAdvanced&PWD=joespizza1&TRXTYPE=S&MODE=TEST&AMT=40&CREATESECURETOKEN=Y&SECURETOKENID=12528208de1413abc3d60c86cdr87"
    //def paypalUrl = "https://pilot-payflowpro.paypal.com"
    println "Making new http Builder with paypalURL ..."
    def http = new HTTPBuilder(paypalUrl)

    println "Now setting timeouts ..."
    http.getClient().getParams().setParameter("http.connection.timeout", new Integer(12000))
    http.getClient().getParams().setParameter("http.socket.timeout", new Integer(30000))

    //=== SSL UNSECURE CERTIFICATE ===

    println "Don't know if needed, but since timeouts don't work to get a response, setting up SSL bypass"

    def sslContext = SSLContext.getInstance("SSL")
    sslContext.init(null, [new X509TrustManager() {
        public X509Certificate[] getAcceptedIssuers() {null }

        public void checkClientTrusted(X509Certificate[] certs, String authType) { }

        public void checkServerTrusted(X509Certificate[] certs, String authType) { }
    }] as TrustManager[], new SecureRandom())

    //SSLSocketFactory sf = new org.apache.http.conn.ssl.SSLSocketFactory(sslContext, org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)

    SSLSocketFactory sf = new org.apache.http.conn.ssl.SSLSocketFactory(sslContext)
    sf.setHostnameVerifier(SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)
    def httpsScheme = new Scheme("https", sf, 443)
    http.client.connectionManager.schemeRegistry.register(httpsScheme)


    println "Now doing the get"
    // perform a GET request, expecting JSON response data
    try {
        http.request(GET, ContentType.ANY) {
            println "Issued the get waiting for the response"
            // Sleeping doesn't help
            //Thread.sleep(2000)
            // println "Done sleeping, looking to process success"

            response.success = { resp, any ->
                println "in success code"
                println "My response handler got response: ${resp.statusLine}"
                println "Response length: ${resp.headers.'Content-Length'}"
                assert resp.status == 200

                def result = any.results;

                render(view: "index", model: [message: "Request sent", result: result]);
            }
            println "past the success code"
        }//end of request
    } catch (groovyx.net.http.HttpResponseException ex) {
        println "Had response exception ...."
        ex.printStackTrace()
        return null
    } catch (java.net.ConnectException ex) {
        println "Had connection exception ...."
        ex.printStackTrace()
        return null
    }

    finally {
        http.shutdown()
    }
}//end of method


def fail() {}

def success() {}

}

------------------- Running this code I get (with no delays anywhere) -------------------------

  Making new http Builder with paypalURL ...
  Now setting timeouts ...
  Don't know if needed, but since timeouts don't work to get a response, setting up SSL bypass
  Now doing the get
  Issued the get waiting for the response
  past the success code
  | Error 2013-07-19 13:27:32,301 [http-bio-8080-exec-10] ERROR errors.GrailsExceptionResolver  - NoHttpResponseException occurred when processing request: [GET] /apps/paypal/index
  The target server failed to respond. Stacktrace follows:
  Message: The target server failed to respond
  Line | Method
  ->> 101 | parseHead             in org.apache.http.impl.conn.DefaultResponseParser
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
  |   210 | parse                 in org.apache.http.impl.io.AbstractMessageParser
  |   271 | receiveResponseHeader in org.apache.http.impl.AbstractHttpClientConnection
  |   233 | receiveResponseHeader in org.apache.http.impl.conn.DefaultClientConnection
  |   209 | receiveResponseHeader in org.apache.http.impl.conn.AbstractClientConnAdapter
  |   292 | doReceiveResponse     in org.apache.http.protocol.HttpRequestExecutor
  |   126 | execute . . . . . . . in     ''
  |   483 | execute               in org.apache.http.impl.client.DefaultRequestDirector
  |   641 | execute . . . . . . . in org.apache.http.impl.client.AbstractHttpClient
  |   576 | execute               in     ''
  |   477 | doRequest . . . . . . in groovyx.net.http.HTTPBuilder
  |   441 | doRequest             in     ''
  |   390 | request . . . . . . . in     ''
  |    56 | index                 in apps.PaypalController$$EOC4TiIi
  |   195 | doFilter . . . . . .  in grails.plugin.cache.web.filter.PageFragmentCachingFilter
  |    63 | doFilter              in grails.plugin.cache.web.filter.AbstractFilter
  |   886 | runTask . . . . . . . in java.util.concurrent.ThreadPoolExecutor$Worker
  |   908 | run                   in     ''
  ^   619 | run . . . . . . . . . in java.lang.Thread

----------------- I had to add the following to BuildConfig.groovy to get this to this work ----

...

dependencies {
    // specify dependencies here under either 'build', 'compile', 'runtime', 'test' or 'provided' scopes e.g.
    compile('org.codehaus.groovy.modules.http-builder:http-builder:0.5.2') {
            excludes "commons-logging", "xml-apis", "groovy"
    }

---

---------------------------- Code update per Ralf's response -------------

Code hopefully matching Groovy HTTPBuilder documentation:

NOTE THIS PRODUCES THE SAME ERROR AT THE SAME LINE, I.E.:

org.apache.http.NoHttpResponseException, Message: The target server failed to respond

at the line: http.request(POST) {

class PaypalController {

def index() {
    def paypalUrl = "https://pilot-payflowpro.paypal.com?PARTNER=PayPal&VENDOR=ROAdvanced&USER=ROAdvanced&PWD=joespizza1&TRXTYPE=S&MODE=TEST&AMT=40&CREATESECURETOKEN=Y&SECURETOKENID=12528208de1413abc3d60c86cdr87"

    println "Making new http Builder with paypalURL ..."
    def http = new HTTPBuilder(paypalUrl)

    println "Now doing the post as a request per example"
    // perform a GET request, expecting JSON response data
    try {
        http.request(POST) {
            body =  [ status : 'update!' , source : 'httpbuilder' ]
            requestContentType = ContentType.URLENC
            println "Issued the post waiting for the response"

            response.success = { resp ->
                println "in success code"
                println "My response handler got response: ${resp.statusLine}"
                println "Response length: ${resp.headers.'Content-Length'}"
                assert resp.status == 200

                //def result = any.results;

                render(view: "index", model: [message: "Request sent", result: result]);
            }
            println "past the success code"
        }//end of request
    } catch (groovyx.net.http.HttpResponseException ex) {
        println "Had response exception ...."
        ex.printStackTrace()
        return null
    } catch (java.net.ConnectException ex) {
        println "Had connection exception ...."
        ex.printStackTrace()
        return null
    }

    finally {
        http.shutdown()
    }
}//end of method

def fail() {}
def success() {}
}

Answer 1

found your problem: curl -d is not a GET but a POST request. If you leave out the -d in your curl request, you'll run in to the same problem as with your groovy...

Try this groovy code:

def url = new URL("https://pilot-payflowpro.paypal.com")
def body= "PARTNER=PayPal&VENDOR=ROAdvanced&USER=ROAdvanced&PWD=..."
def connection = url.openConnection()
connection.setRequestMethod("POST")
connection.doOutput = true

def writer = new OutputStreamWriter(connection.outputStream)
writer.write(body)
writer.flush()
writer.close()
connection.connect()
def response = connection.content.text

Answer 2

Here is the http-builder code with a proper body:

    @Grab(group='org.codehaus.groovy.modules.http-builder',module='http-builder', version='0.5.0')
     import grails.converters.*
     import org.codehaus.groovy.grails.web.json.*; // package containing JSONObject,  JSONArray,...
     import groovyx.net.http.*
     import static groovyx.net.http.ContentType.*
     import static groovyx.net.http.Method.*

    class PaypalController {

    def index() {
        def paypalUrl = "https://pilot-payflowpro.paypal.com"

        println "Making new http Builder with paypalURL ..."
        def http = new HTTPBuilder(paypalUrl)

        println "Now doing the post as a request per example"
        // perform a GET request, expecting JSON response data
        try {
            http.request(POST,JSON) {
                body =  [ 
                    PARTNER:'PayPal',
                    VENDOR:'yyy',
                    USER:'yyy',
                    PWD:'xxx',
                    TRXTYPE:'S',
                    MODE:'TEST',
                    AMT:'40',
                    CREATESECURETOKEN:'Y',
                    SECURETOKENID:'xxx'
                ]
                requestContentType = ContentType.URLENC
                println "Issued the post waiting for the response"

                response.success = { resp ->
                    println "in success code"
                    println "My response handler got response: ${resp.statusLine}"
                    println "Response length: ${resp.headers.'Content-Length'}"
                    assert resp.status == 200

                    //def result = any.results;

                    render(view: "index", model: [message: "Request sent", result: result]);
                }
                println "past the success code"
            }//end of request
        } catch (groovyx.net.http.HttpResponseException ex) {
            println "Had response exception ...."
            ex.printStackTrace()
            return null
        } catch (java.net.ConnectException ex) {
            println "Had connection exception ...."
            ex.printStackTrace()
            return null
        }

        finally {
            http.shutdown()
        }
    }//end of method
    }
    def controller = new PaypalController()
    controller.index()

the @Grab statement lets you execute this in the groovyConsole...