Using opencsv - java.lang.OutOfMemoryError: Java heap space

Question

I am using opencsv to parse two csv files. I only copy some values from the two files.

I have a seperate function which processes the CDax.csv. Which looks like that:

public HashMap<String,String> readCDax() throws Exception {
    String csvDaxFile = "C:\\Users\\CDAX.csv";

    CSVReader reader = new CSVReader(new FileReader(csvDaxFile), ';');
    String [] line;
    HashMap<String, String> cdaxMap = new HashMap<String, String>();

    while ((line = reader.readNext()) != null) {            
        cdaxMap.put(line[0], line[7]);
    }

    System.out.println("Process CDax File!");

    reader.close();
    return cdaxMap;
}

My main method is run() which I execute in my main method:

public void run() throws Exception {

while ((firstLine = reader.readNext()) != null && (secondLine = reader.readNext()) != null && i<10) {            

    //fileName of the String
    fileName = firstLine[0];

    writerPath = "C:\\Users\\" + fileName + ".csv";
    //write csv file
    CSVWriter writer = new CSVWriter(new FileWriter(writerPath), ';');

    //write Header
    //String[] entries = "Name;Date;TotalReturn;Currency".split(";");
    String [] entries = {"Name","Date", "TotalReturn", "Currency"};

    writer.writeNext(entries);

    //create Content

    //companyName of the String
    companyName = secondLine[1];

    //currency
    currency = secondLine[2];

    //dates
    dateList = new ArrayList<String>();
    for(int p = 3; p < firstLine.length; p++) {
        dateList.add(firstLine[p]);
    }

    //total returns
    returnList = new ArrayList<String>();
    for(int j = 3; j < secondLine.length; j++) {
        returnList.add(secondLine[j]);
    }

    // cDaxList
    cDaxList = new ArrayList<String>();
    for(int j = 1; j <dateList.size(); j++) {
        if(cDaxMethodValuesMap.containsKey(dateList.get(j))){
            cDaxList.add(cDaxMethodValuesMap.get(dateList.get(j)));
        } else{
            dateList.add("na"); // I get the error here!
        }
    }

    if(dateList.size()!=returnList.size()) {
        System.out.println("Dates and Returns do not have the same length!");
    }

    int minSize = Math.min(dateList.size(), returnList.size());

    //"Name;Date;TotalReturn;Currency"
    List<String[]> data = new ArrayList<String[]>();
    for(int m = 0; m < minSize; m++) {
        data.add(new String[] {companyName, dateList.get(m), returnList.get(m), currency, cDaxList.get(m)});
    }

    writer.writeAll(data);

    //close Writer
    writer.close();

    i++;
    System.out.println(fileName + " parsed successfully!");


}
System.out.println("Done");

}

However when I run my program I get:

Process CDax File!
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2760)
    at java.util.Arrays.copyOf(Arrays.java:2734)
    at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
    at java.util.ArrayList.add(ArrayList.java:351)
    at com.TransformCSV.main.ParseCSV.run(ParseCSV.java:109)
    at com.TransformCSV.main.ParseCSV.main(ParseCSV.java:21)

I am getting the error in this method:

cDaxList = new ArrayList<String>();
for(int j = 1; j <dateList.size(); j++) {
    if(cDaxMethodValuesMap.containsKey(dateList.get(j))){
        cDaxList.add(cDaxMethodValuesMap.get(dateList.get(j)));
    } else{
        dateList.add("na"); //I get the error here!!!
    }
}

I tried to put up the heapsize via the vm settings, however I do not think that this should be done because I only read in both csv files only 3000 values.

I appreciate your reply!

Answer 1

Your loop:

for(int j = 1; j <dateList.size(); j++) {

is looping through dateList but in that loop you are adding to dateList:

dateList.add("na"); //I get the error here!!!

so dateList will get bigger and bigger until you run out of memory. dateList.size() is evaluated every time through the loop, not once at the beginning.