How to use httpClient to filer parts of an xml file in Java

Question

I am currently doing a project in which I have to request data from the metabolite database PubChem. I am using Apache's HttpClient. I am doing the following:

HttpClient httpclient = new DefaultHttpClient();
HttpGet pubChemRequest = new HttpGet("http://pubchem.ncbi.nlm.nih.gov/summary/summary.cgi?cid="
    + cid + "&disopt=SaveXML");
pubChemRequest.getAllHeaders();
System.out.println(pubChemRequest);
HttpResponse response = null;
response = httpclient.execute(pubChemRequest);
HttpEntity entity = response.getEntity();
pubChemInchi = EntityUtils.toString(entity);

The problem is that this code streams the entire XML file:

<?xml version="1.0"?>
<PC-Compound
xmlns="http://www.ncbi.nlm.nih.gov"
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xs:schemaLocation="http://www.ncbi.nlm.nih.gov ftp://ftp.ncbi.nlm.nih.gov/pubchem/specifications/pubchem.xsd">

etc.

What I want is that I can request, for example, the PubChem ID and it will paste the value that corresponds to that ID. I have found that this can be done with the native Java method, but I need to use the HttpClient for this. With the native Java, it would be done like this:

cid = 5282253
reader = new PCCompoundXMLReader(
new URL("http://pubchem.ncbi.nlm.nih.gov/summary/summary.cgi?cid=" + cid + "&disopt=SaveXML").newInputStream())
mol = reader.read(new NNMolecule())
println "CID: " + mol.getProperty("PubChem CID")

(Note: This piece of code was written in Groovy, but it also works in Java after some adjustments)

So, if anyone can help me out, that would be great:)

Answer 1

There are multiple ways to do this.

If you want to turn the response into a bean and dont expect the the structure of the response to change I'd look at using XStream. Another option is using the SAX parser directly.

Ofcourse the quick and dirty approach is to turn your responses content into a bufferedReader. Then feed that reader into the XMLReader you are using.

An example using you code from above would be:

HttpClient httpclient = new DefaultHttpClient();
HttpGet pubChemRequest = new HttpGet("http://pubchem.ncbi.nlm.nih.gov/summary/summary.cgi?cid="
    + cid + "&disopt=SaveXML");
pubChemRequest.getAllHeaders();
System.out.println(pubChemRequest);
HttpResponse response = null;
response = httpclient.execute(pubChemRequest);
BufferedReader br = new BufferedReader(new InputStreamReader((response.getEntity().getContent())));
cid = 5282253
reader = new PCCompoundXMLReader(br)
mol = reader.read(new NNMolecule())
println "CID: " + mol.getProperty("PubChem CID")

Googling for RESTful webservice clients or XMLReaders should give you plenty more information on this subject

Answer 2

Try using NameValuePair

For eg:

List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);

nameValuePairs.add(new BasicNameValuePair("username", user123));

nameValuePairs.add(new BasicNameValuePair("password", pass123));

httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));

HttpResponse response = httpclient.execute(httppost);