java https login and browsing

Question

I want to login to an https site on the internet with java and then read some information. I already looked with firebug to the headers, however I couldn't manage to make it ...

Firebug tells:

https://service.example.net/xxx/unternehmer/login.html?login=Anmelden&loginname=xxx&password=xxx&sessionid=&sprache=de

And then I want to browse to this site:

https://service.example.net/xxx/unternehmer/ausgabe.html?code=PORTAL;sessionid=03112010150442

how can I do this with java? I already tried something like:

import java.net.*;
import java.io.*;
import java.security.*;
import javax.net.ssl.*;

public class HTTPSClient {

  public static void main(String[] args) {
    int port = 443; // default https port
    String host = "service.example.net";
    try {
      SSLSocketFactory factory = (SSLSocketFactory) SSLSocketFactory.getDefault();

      SSLSocket socket = (SSLSocket) factory.createSocket(host, port);

      // enable all the suites
      String[] supported = socket.getSupportedCipherSuites();
      socket.setEnabledCipherSuites(supported);


      Writer out = new OutputStreamWriter(socket.getOutputStream());
      // https requires the full URL in the GET line
      out.write("POST https://" + host + "//xxx/unternehmer/login.html?login=Anmelden&loginname=xxx&password=xxx&sessionid=&sprache=de HTTP/1.1\r\n");
      out.write("Host: " + host + "\r\n");
      out.write("\r\n");
      out.flush();

      // read response
      BufferedReader in = new BufferedReader(
        new InputStreamReader(socket.getInputStream()));

      // read the header
      String s;
      while (!(s = in.readLine()).equals("")) {
          System.out.println(s);
      }
      System.out.println();

      // read the length
      String contentLength = in.readLine();
      int length = Integer.MAX_VALUE;
      try {
        length = Integer.parseInt(contentLength.trim(), 16);
      }
      catch (NumberFormatException ex) {
        // This server doesn't send the content-length
        // in the first line of the response body
      }
      System.out.println(contentLength);

      int c;
      int i = 0;
      while ((c = in.read()) != -1 && i++ < length) {
        System.out.write(c);
      }

      System.out.println("1.part done");

      out.close();
      in.close();
      socket.close();

    }
    catch (IOException ex) {
      System.err.println(ex);
    }

  }

}

unfortunately that doesnt work for the login .... and i also dont know where to get this sessionid...everytime it is a different one. i hope you can help me. ps: i replaced some relevant information with xxx

Answer 1

Problem solved :)

First I added the libraries from apache:

1. httpclient
2. commons-httpclient
3. commons-codec
4. commons-logging

then I combined several tutorials.

my code:

import java.io.BufferedWriter;
import java.io.FileWriter;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.http.client.params.CookiePolicy;

  public class Test {

     public static final String TARGET_HTTPS_SERVER = "www.example.net"; 
     public static final int    TARGET_HTTPS_PORT   = 443; 

     public static void main(String[] args) throws Exception {

         HttpClient httpClient = new HttpClient();
         httpClient.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);

         PostMethod post = new PostMethod("https://www.example.com/login.html");
         post.setRequestHeader(new Header(
                 "User-Agent", "Mozilla/5.0 /Windows; U; Windows NT 4.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.0"));

         post.addParameter("login", "true");
         post.addParameter("username", "xxx");
         post.addParameter("password", "xxx");
         post.addParameter("language", "de");
         httpClient.executeMethod(post);


         System.out.println(post.getResponseBodyAsString());
         String body=post.getResponseBodyAsString();
//Get the session id by parsing the code, i know this is not pretty
             String sessionid=body.substring(body.indexOf("session")+10,body.indexOf("session")+10+14);
             System.out.print(sessionid);


             GetMethod get=new GetMethod("https://www.example.com/thesiteyouwannabrowse?sessionid="+sessionid);

         get.setRequestHeader(new Header(
             "User-Agent", "Mozilla/5.0 /Windows; U; Windows NT 4.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.0"));
         httpClient.executeMethod(get);

         System.out.println(get.getResponseBodyAsString());
         //write it into a file
         try{
                // Create file 
                FileWriter fstream = new FileWriter("file.html");
                    BufferedWriter out = new BufferedWriter(fstream);
                out.write(get.getResponseBodyAsString());
                //Close the output stream
                out.close();
                }catch (Exception e){//Catch exception if any
                  System.err.println("Error: " + e.getMessage());
             }     
         post.releaseConnection();
     }
  }

Answer 2

I myself have done similar things. I got it working using this "manual" approach, but it was quite a hassle, especially with the cookie management.

I would recommend you to have a look at Apache HttpClient library. (I threw away the code I had when I realized how easy it was to use this library.)

As org.life.java points out, here http://hc.apache.org/httpclient-3.x/sslguide.html is a good howto on how to get started with SSL using this library.