You're calling
sCurrentLine.split(COMMA)
several times in your code. Make a
final String[]
variable the first time you call it in the loop and use that instead thereafter and it'll get that many times faster.
Java - Improving iteration through large String List
-
19-07-2023 - |
Frage
I have to parse files which has around 50000 lines and has to iterate through each line, parse, create a List and save to database. Initially I thought the time taken is because of reading the file. But the file is actually read within a second. But the parsing of data takes long time.
public static final String record = "dlrCode,partNumber,5,0.00,5000.00,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0,0.00,0";
public static final String COMMA = ",";
public static final String QUOTES = "\"";
public static final String EMPTY_STRING = "";
public static void main(String[] args){
List<String> recordsList = new ArrayList<String>();
Date time = new Date();
Part partVO = null;
PartHistory partHistoryVO = null;
List<PartHistory> partHistoryList = null;
List<Part> partsList = new ArrayList<Part>();
int splitLength = 0;
Calendar cal = Calendar.getInstance();
int historySplitCount = 0;
int monthCountReverse = 0;
//add 20000 records to list
for(int i=0; i<20000; i++){
recordsList.add(record);
}
System.out.println("Added in "+((new Date()).getTime() - time.getTime()) +" ms");
//reset time
time = new Date();
//parse records
for(String sCurrentLine : recordsList){
partVO = new Part();
partHistoryList = new ArrayList<PartHistory>();
//Parsing inventory information
partVO.setDealerCode(sCurrentLine.split(COMMA)[0]);
partVO.setPartNumber(sCurrentLine.split(COMMA)[1]);
partVO.setDmsMfId(sCurrentLine.split(COMMA)[2]);
partVO.setQtyOnHand(Math.round(Float.parseFloat(sCurrentLine.split(COMMA)[3])));
partVO.setDealerNet(Float.parseFloat(sCurrentLine.split(COMMA)[4]));
//Parsing history information
//starting from the 6th record as the first 5 records are used above
historySplitCount = 5;
//to subtract one month from current date
monthCountReverse = -1;
splitLength = sCurrentLine.split(COMMA).length;
while(splitLength>=(historySplitCount+1)){
partHistoryVO = new PartHistory();
//subtract one month from current date
cal.add(Calendar.MONTH, monthCountReverse);
partHistoryVO.setMonth(cal.get(Calendar.MONTH)+1);
partHistoryVO.setYear(cal.get(Calendar.YEAR));
partHistoryVO.setLineHitsMonthly(Math.round(Float.parseFloat(sCurrentLine.split(COMMA)[historySplitCount])));
historySplitCount++;
partHistoryVO.setQuantity(Math.round(Float.parseFloat(sCurrentLine.split(COMMA)[historySplitCount])));
historySplitCount++;
partHistoryList.add(partHistoryVO);
}
partVO.setHistoryList(partHistoryList);
partsList.add(partVO);
}
System.out.println("Iterated in "+((new Date()).getTime() - time.getTime()) +" ms");
}
Output
Added in 15 ms
Iterated in 12823 ms
Can the iteration time be improved and brought under atleast 5 seconds?
Lösung
Andere Tipps
For each line, you call the split() function multiple times, sCurrentLine.split(COMMA)[0],
a better way is to split it once and store into an array
String[] elements = sCurrentLine.split(COMMA);
dealerCode = elements[0];
partNumber = elements[1];
FYI, to count how much time spent, you can also use System.currentTimeMillis()
, this does not need to create a new Date instance :)
long timeStarts = System.currentTimeMillis();
//loop goes here
long timeTook = System.currentTimeMillis() - timeStarts;
Lizenziert unter: CC-BY-SA mit Zuschreibung
Nicht verbunden mit StackOverflow