Java - Improving iteration through large String List

Question

I have to parse files which has around 50000 lines and has to iterate through each line, parse, create a List and save to database. Initially I thought the time taken is because of reading the file. But the file is actually read within a second. But the parsing of data takes long time.

public static final String  record      =   "dlrCode,partNumber,5,0.00,5000.00,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0";
    public static final String COMMA        =   ",";
    public static final String QUOTES       =   "\"";
    public static final String EMPTY_STRING =   "";

    public static void main(String[] args){


        List<String> recordsList            =   new ArrayList<String>();
        Date time                           =   new Date();
        Part partVO                         =   null;
        PartHistory partHistoryVO           =   null;
        List<PartHistory> partHistoryList   =   null;
        List<Part> partsList                =   new ArrayList<Part>();
        int splitLength                     =   0;
        Calendar cal                        =   Calendar.getInstance();
        int historySplitCount               =   0;
        int monthCountReverse               =   0; 

        //add 20000 records to list
        for(int i=0; i<20000; i++){
            recordsList.add(record);
        }
        System.out.println("Added in "+((new Date()).getTime() - time.getTime()) +" ms");

        //reset time
        time                            =   new Date();

        //parse records
        for(String sCurrentLine :  recordsList){
            partVO = new Part();
            partHistoryList =   new ArrayList<PartHistory>();

            //Parsing inventory information
            partVO.setDealerCode(sCurrentLine.split(COMMA)[0]);
            partVO.setPartNumber(sCurrentLine.split(COMMA)[1]);
            partVO.setDmsMfId(sCurrentLine.split(COMMA)[2]);
            partVO.setQtyOnHand(Math.round(Float.parseFloat(sCurrentLine.split(COMMA)[3])));
            partVO.setDealerNet(Float.parseFloat(sCurrentLine.split(COMMA)[4]));

            //Parsing history information
            //starting from the 6th record as the first 5 records are used above
            historySplitCount   =   5;

            //to subtract one month from current date
            monthCountReverse   =   -1; 

            splitLength =   sCurrentLine.split(COMMA).length;

            while(splitLength>=(historySplitCount+1)){

                partHistoryVO   =   new PartHistory();
                //subtract one month from current date
                cal.add(Calendar.MONTH, monthCountReverse);
                partHistoryVO.setMonth(cal.get(Calendar.MONTH)+1);
                partHistoryVO.setYear(cal.get(Calendar.YEAR));

                partHistoryVO.setLineHitsMonthly(Math.round(Float.parseFloat(sCurrentLine.split(COMMA)[historySplitCount])));
                historySplitCount++;

                partHistoryVO.setQuantity(Math.round(Float.parseFloat(sCurrentLine.split(COMMA)[historySplitCount])));
                historySplitCount++;
                partHistoryList.add(partHistoryVO);
            }
            partVO.setHistoryList(partHistoryList);

            partsList.add(partVO);
        }
        System.out.println("Iterated in "+((new Date()).getTime() - time.getTime()) +" ms");
    }

Output

Added in 15 ms  
Iterated in 12823 ms

Can the iteration time be improved and brought under atleast 5 seconds?

Answer 1

You're calling sCurrentLine.split(COMMA) several times in your code. Make a final String[] variable the first time you call it in the loop and use that instead thereafter and it'll get that many times faster.

Answer 2

For each line, you call the split() function multiple times, sCurrentLine.split(COMMA)[0],

a better way is to split it once and store into an array

String[] elements = sCurrentLine.split(COMMA);
dealerCode = elements[0];
partNumber = elements[1];

FYI, to count how much time spent, you can also use System.currentTimeMillis(), this does not need to create a new Date instance :)

long timeStarts = System.currentTimeMillis();
//loop goes here
long timeTook = System.currentTimeMillis() - timeStarts;