Web Crawler Java

Question

...

public static void download() {
             try {
                 URL oracle = new URL("http://api.wunderground.com/api/54f05b23fd8fd4b0/geolookup/conditions/forecast/q/US/CO/Denver.json");
                 BufferedReader in = new BufferedReader(new InputStreamReader(oracle.openStream()));

                 File file = new File("C:\\Users\\User\\Desktop\\test2.json");
                 if (!file.exists()) {
                     file.createNewFile();
                 }
                 FileWriter fw = new FileWriter(file.getAbsoluteFile());
             BufferedWriter bw = new BufferedWriter(fw);

                 String inputLine;
                 while ((inputLine = in.readLine()) != null) {
                     bw.write(inputLine+"\n");
                 }
                 fw.close();
                 in.close();
                 System.out.println("Finished...");
             }
             catch(MalformedURLException e){e.printStackTrace();}
             catch(IOException e){e.printStackTrace();}
         }

I'm making a web-crawler to retrieve weather updates from Wunderground. I did get this to work, but it doesn't parse the entire document (cuts of the last bit). What did I do wrong?

Answer 1

You are wrapping your FileWriter with a BufferedWriter

FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);

but only closing the FileWriter

fw.close();

Since the FileWriter doesn't have access and doesn't know about the BufferedWriter, it won't flush any possible remaining buffer. You could either call flush() on bw or close bw instead of fw. Calling bw.close() will take care of closing the wrapped FileWriter.

bw.close();
in.close();