File I/O bottleneck found via VisualVM

Question

I've found a bottleneck in my app that keeps growing as data in my files grow (see attached screenshot of VisualVM below).

Below is the getFileContentsAsList code. How can this be made better performance-wise? I've read several posts on efficient File I/O and some have suggested Scanner as a way to efficiently read from a file. I've also tried Apache Commons readFileToString but that's not running fast as well.

The data file that's causing the app to run slower is 8 KB...that doesn't seem too big to me.

I could convert to an embedded database like Apache Derby if that seems like a better route. Ultimately looking for what will help the application run faster (It's a Java 1.7 Swing app BTW).

Here's the code for getFileContentsAsList:

public static List<String> getFileContentsAsList(String filePath) throws IOException {
    if (ReceiptPrinterStringUtils.isNullOrEmpty(filePath)) throw new IllegalArgumentException("File path must not be null or empty");

    Scanner s = null;
    List<String> records = new ArrayList<String>();

    try {
        s = new Scanner(new BufferedReader(new FileReader(filePath)));
        s.useDelimiter(FileDelimiters.RECORD);

        while (s.hasNext()) {
           records.add(s.next());
        }
    } finally {
        if (s != null) {
            s.close();
        }
    }

    return records;
}

Answer 1

So, file.io gets to be REAL expensive if you do it a lot...as seen in my screen shot and original code, getFileContentsAsList, which contains file.io calls, gets invoked quite a bit (18.425 times). VisualVM is a real gem of a tool to point out bottlenecks like these!

After contemplating over various ways to improve performance, it dawned on me that possibly the best way is to do file.io calls as little as possible. So, I decided to use private static variables to hold the file contents and to only do file.io in the static initializer and when a file is written to. As my application is (fortunately) not doing excessive writing (but excessive reading), this makes for a much better performing application.

Here's the source for the entire class that contains the getFileContentsAsList method. I took a snapshot of that method and it now runs in 57.2 ms (down from 3116 ms). Also, it was my longest running method and is now my 4th longest running method. The top 5 longest running methods run for a total of 498.8 ms now as opposed to the ones in the original screenshot that ran for a total of 3812.9 ms. That's a percentage decrease of about 85% [100 * (498.8 - 3812.9) / 3812.9].

package com.mbc.receiptprinter.util;

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.logging.Level;

import org.apache.commons.io.FileUtils;

import com.mbc.receiptprinter.constant.FileDelimiters;
import com.mbc.receiptprinter.constant.FilePaths;

/*
 * Various File utility functions.  This class uses the Apache Commons FileUtils class.
 */
public class ReceiptPrinterFileUtils {

    private static Map<String, String> fileContents = new HashMap<String, String>();

    private static Map<String, Boolean> fileHasBeenUpdated = new HashMap<String, Boolean>();

    static {
        for (FilePaths fp : FilePaths.values()) {
            File f = new File(fp.getPath());
            try {
                FileUtils.touch(f);
                fileHasBeenUpdated.put(fp.getPath(), false);
                fileContents.put(fp.getPath(), FileUtils.readFileToString(f));
            } catch (IOException e) {
                ReceiptPrinterLogger.logMessage(ReceiptPrinterFileUtils.class, 
                                                Level.SEVERE, 
                                                "IOException while performing FileUtils.touch in static block of ReceiptPrinterFileUtils", e);
            }
        }
    }

    public static String getFileContents(String filePath) throws IOException {
        if (ReceiptPrinterStringUtils.isNullOrEmpty(filePath)) throw new IllegalArgumentException("File path must not be null or empty");
        File f = new File(filePath);
        if (fileHasBeenUpdated.get(filePath)) {
            fileContents.put(filePath, FileUtils.readFileToString(f));
            fileHasBeenUpdated.put(filePath, false);
        }
        return fileContents.get(filePath);
    }

    public static List<String> convertFileContentsToList(String fileContents) {
        List<String> records = new ArrayList<String>();
        if (fileContents.contains(FileDelimiters.RECORD)) {
            records = Arrays.asList(fileContents.split(FileDelimiters.RECORD));
        }
        return records;
    }

    public static void writeStringToFile(String filePath, String data) throws IOException {
        fileHasBeenUpdated.put(filePath, true);
        FileUtils.writeStringToFile(new File(filePath), data);
    }

    public static void writeStringToFile(String filePath, String data, boolean append) throws IOException {
        fileHasBeenUpdated.put(filePath, true);
        FileUtils.writeStringToFile(new File(filePath), data, append);
    }
}

Answer 2

The size of an ArrayList is multiplied by 1.5 when necessary. This is O(log(N)). (Doubling was used in Vector.) I would certainly use an O(1) LinkedList here, and BufferedReader.readLine() rather than a Scanner if I was trying to speed it up. It's hard to believe that the time to read one 8k file is seriously a concern. You can read millions of lines in a second.

Answer 3

ArrayLists have a good performance at reading and also on writing IF the lenth does not change very often. In your application the length changes very often (size is doubled, when it is full and an element is added) and your application needs to copy your array into an new, longer array.

You could use a LinkedList, where new elements are appended and no copy actions are needed. List<String> records = new LinkedList<String>();

Or you could initialize the ArrayList with the approximated finished Number of Words. This will reduce the number of copy actions. List<String> records = new ArrayList<String>(2000);