Java URL Connection blank input stream

Question

The application I am working on has a terribly slow loading web page. If I run the following code it works fine but only because of the call to sleep. If I don't sleep then the InputStream is just a bunch of spaces, probably due to the application it is calling from. Is there any non-hack way around this?

public class PublishTool extends Thread {

  private URL publishUrl;

  private String filerLocation;

  public PublishTool() {
  }

  public PublishTool(String publishUrl, String filerLocation) throws NibException {

    try {
      this.publishUrl = new URL(publishUrl);
    } catch (MalformedURLException e) {
      throw new NibException("Publish Url :" + publishUrl + " is not valid. ");
    }

    this.filerLocation = filerLocation;

  }

  public void run() {

    File filerFile = new File(filerLocation);
    BufferedWriter writer = null;


    try {
      URLConnection conn = publishUrl.openConnection();
      BufferedReader reader = new BufferedReader(new InputStreamReader(new BufferedInputStream(conn.getInputStream())));

      writer = new BufferedWriter(new FileWriter(filerLocation));

      Thread.sleep(1000l);

      while (reader.ready()) {
        writer.write(reader.readLine() + "\n");
      }

    } catch (MalformedURLException e) {
      throw new IllegalStateException("Malformed URL for : " + publishUrl + " " + filerLocation, e);
    } catch (IOException e) {
      throw new IllegalStateException("IO Exception for  : " + publishUrl + " " + filerLocation, e);
    } catch (InterruptedException e) {
      throw new IllegalStateException("Thread was interrupted early... publishing might have failed.");
    } catch (NibException e) {
      throw new IllegalStateException("Publishing File Copy failed : " + filerLocation + ".bak" + " to " + filerLocation);
    } finally {
      try {
        writer.flush();
        writer.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

Answer 1

Don't use reader.ready(). Just call readLine() and let readLine() block until the data's ready. The end of the data will generally be signalled with a null line.

In case its helpful, a couple of code examples on my web site: reading from a URL.

Answer 2

Firstly, it would be helpful if you posted your actual code.

My guess the problem is calling Reader.ready. Similar to InputStream.available, that returns true if there is already buffered input. If it needs to wait for, say, a socket the it will return false. Generally you don't need ready. Use readLine, and break out of the loop if it returns null (for end of stream).