MyBatis - Pulling out 44000 rows into 3000 objects

Question

I am using mybatis 3.1.0 jar along with spring mybatis jar.

It is taking me 16 seconds to pullout the 44000 rows into 3000 entity objects.

Time Taken: Normal query execution time : 11 seconds

Any suggestions to increase the performance?

Thanks.

Answer 1

Certain measures on mybatis configuration to handle situations like these,

1)Set fetchSize to a considerable amount.I set my fetchsize as 1000

2)Not to retrieve and map all columns(mapping only necessary columns reduced the time)

3)Make use of nested joins instead of nested selects.

Answer 2

First off, do some analysis to see how much of your time is spent (a) pulling out data, e.g. run the query in your standalone DB tool and benchmark that, and then (b) how much is spent marshalling the data into your objects.

Subsequent steps will depend on which of (a) or (b) appears least performant.

If (a), then spend time tuning the query - indexes on tables, denormalise underlying structure.

If (b) consider a flatter or less heavily populated model.

Edit:

extra thoughts on DB side to reduce 11s duration:

• check your query is hitting indexes at every point, not doing any table scans
• check your query pulls back the minimum fields, e.g. if you just need 8 fields, don't pull back 20
• check your query isn't doing inefficient subqueries if you're on MySql (are you?)
• check your query isn't calling any unnecessary functions or subroutines (probably unlikely but worth mentioning...)

extra thoughts on object side:

• avoid setting object fields you don't need

extra general thoughts:

• add some logging for, e.g. a single case (or for, say, 100) to see where time's spent in the application, you need general code optimisation techniques here - anything that's a loop, or isn't 100% necessary, look at changing/removing

• if performance is most important consideration, consider changing your requirement - structure your page, or screen, differently, for example to allow you to make a faster DB retrieval realistic

2nd edit - do you really need to use all 44,000 rows and the 3,000 objects they feed? Could you get by with fewer, e.g. break them into 10 groups and paginate with them? (wild guess, your app might do nothing of that sort)