Bulk insertion of CollectionTable elements in Hibernate / JPA

Question

We are using Hibernate 4.2 as the backing library for JPA 2.0 entities. We have an entity like the following:

@Entity
public class MyEntity {
    ....
    @ElementCollection
    @MapKeyColumn(name = "key")
    @Column(name = "value")
    @CollectionTable("MyEntityMap")
    private Map<String, Integer> theMap;
    ...
}

The map potentially has thousands of entries. I have set hibernate.jdbc.batch_size=50, but Hibernate still generates an insert statement for each entry in the map, when I say entityManager.persist(myEntity). Is there a way to make Hibernate insert the values in a bulk insert like INSERT INTO MyEntityMap () VALUES (), (), (), (), ()?

Answer 1

I had the exact same scenario, this is what i ended up doing ...

Use the below configurations within the persistence.xml/equivalent file

<property name="hibernate.jdbc.batch_size" value="10"/> 
<property name="hibernate.order_inserts" value="true"/> 
<property name="hibernate.order_updates" value="true"/>

The example you gave is bulk insert, what the above configuration does is only send the individual insert statements as a single batch to the DB, it does not actually rewrite the inserts into

INSERT INTO MyEntityMap () VALUES (), (), (), (), ()

and admittedly the batch insert only gives a slight performance improvement, the real performance improvement can be derived from bulk inserts.

Bulk inserts are not a ANSI SQL feature but rather a feature of individual DB providers, the MySQL DB connector driver provides a configuration to rewrite batch inserts into bulk inserts

to enable it use a connection string like the below

jdbc.url=jdbc:mysql://localhost:3306/macula?useUnicode=true&characterEncoding=utf8&characterSetResults=utf8&rewriteBatchedStatements=true

Now once your done with this , if you try to run the application, and enable general query logging in mysql, you will see that your inserts are being rewritten as bulk inserts ... but you also will get is a BatchedTooManyRowsAffectedException:)

This is thrown by the org.hibernate.jdbc.Expectations.BasicExpectation.verifyOutcome() method (line 67 )

As hibernate know nothing about the rewrite, it is not able to make sense of the update count, i could not find a reasonable way to hook in my implementation of the Expectations class ...

This is kind of a dead end, we cannot use the rewriteBatchedStatements feature with hibernate !!

Well unless you are willing to do a evil hack that is ... just copy paste the class into your project code under the exact same package hierarchy this would mean that hibernate will pick up your copy pasted class rather than the one packaged within hibernate jar, and then just comment out that if statement ... and yes then everything works smoothly !!

Consider the above hack with care ;)