One-to-Many relationship and Serialized Object field

Question

I have following Persistable classes with 1-to-N relationship.

@PersistenceCapable
public class Pet {

  @Persistent(primaryKey = "true", valueStrategy = IdGeneratorStrategy.IDENTITY)
  Long id;

  @Persistent
  String name;

  @Element(column = "PET_ID")
  List<Photo> photos;

  // getters and setters

and

@PersistenceCapable
public class Photo {

  @Persistent(primaryKey = "true", valueStrategy = IdGeneratorStrategy.IDENTITY)
  Long id;

  @Persistent
  String desc;

  @Persistent(serialized="true")
  Object image;

  // getters and setters

  // hash and equal using field id

Field List photos uses FK to establish 1-N relationship between Pet (1) and Photo (N). Field Object image in Photo is a serialized to hold the image object.

For datastore operations I use PetDao, which has following methods

public final static PersistenceManagerFactory pmf = JDOHelper
            .getPersistenceManagerFactory("datastore");

public void storePet(Pet pet) {
    // get PM and current tx
    try {
       tx.begin();
       pm.makePersistent(pet);
       tx.commit();
    } catch (Exception e) {
       // rollback and close pm
    }           
}

public void storePhoto(Long petId, Photo photo) {
    // get PM and current tx
    try {
       tx.begin();
       Pet pet = pm.getObjectById(Pet.class,petId);
       pet.addPhoto(photo);
       tx.commit();
    } catch (Exception e) {
       // rollback and close pm
    }
}

I create and persist objects as

Pet pet = new Pet();
pet.setName("Nicky");

Photo photo = new Photo();
photo.setDesc("Photo 1");
photo.setImage(new Image("image 1"));
pet.addPhoto(photo);

.... add photo 2 and photo 3

PetDao petDao = new PetDao();       
petDao.storePet(pet);

// i have one more photo so add it directly
photo = new Photo();
photo.setDesc("Photo 4");
photo.setImage(new Image ("image 4"));      

petDao.storePhoto((long)0, photo);

Everything persists as required and datastore ends up with 1 pet in PET table and 4 photo in PHOTO table.

But when I analyze the DataNucleus log for petDao.storePhoto((long)0, photo) code, I see that DataNucleus retrieves all the image objects from datastore.

Native          [DEBUG] INSERT INTO PHOTO ("DESC",IMAGE,PET_ID,PHOTOS_INTEGER_IDX) VALUES (<'Photo 4'>,<UNPRINTABLE>,<0>,<3>)
Persist         [DEBUG] Execution Time = 70 ms (number of rows = 1) on PreparedStatement "org.datanucleus.store.rdbms.ParamLoggingPreparedStatement@190a0d6"
Persist         [DEBUG] Object "in.m.pet.Photo@10deb5f" was inserted in the datastore and was given strategy value of "3"
Native          [DEBUG] SELECT A0.IMAGE FROM PHOTO A0 WHERE A0.ID = <1>
Retrieve        [DEBUG] Execution Time = 1 ms
Native          [DEBUG] SELECT A0.IMAGE FROM PHOTO A0 WHERE A0.ID = <0>
Retrieve        [DEBUG] Execution Time = 0 ms
Native          [DEBUG] SELECT A0.IMAGE FROM PHOTO A0 WHERE A0.ID = <2>
Retrieve        [DEBUG] Execution Time = 0 ms

After adding the "Photo 4" using INSERT INTO PHOTO... statement, DataNucleus retrieves the earlier three image Objects by firing 3 SELECT IMAGE FROM PHOTO statements. These retrieves may be quite large as number of image objects increases, resulting in unnecessary load on datastore impacting the performance.

Same thing happens if I select the pet using pm.getObjectById() and detach the Pet object and add photo to detached object and then attach it back to object graph with pm.makePersistent(pet). FetchGroup is as follows

@PersistenceCapable(detachable="true")
@FetchGroup(name="detachPhotos", members={@Persistent(name="photos")})
public class Pet {
   ....
}

and detach pet with fetchgroup

public Pet getPet(Long id){
    PersistenceManager pm = pmf.getPersistenceManager();
    pm.getFetchPlan().addGroup("detachPhotos");
    Pet pet = pm.getObjectById(Pet.class, id);      
    return pm.detachCopy(pet);  
}

My question is how to avoid these unnecessary retrials of Object image from datastore.

One more observation: if i call petDao.storePhoto((long)0, photo) from another app or use a separate instance of PMF in PetDao.storePhoto method, then DataNucleus will not fire the SELECT to retrieve image objects.

Answer 1

Got answer in DataNucleus Performance Tuning, which recommends to set datanucleus.persistenceByReachabilityAtCommit=false if reach-ability is not needed by the app. Setting this as false, resolves the image retrieval issue without any other side effect with Pet/Photo.

To quote from DN doc

DataNucleus verifies if newly persisted objects are memory reachable on commit, if they are not, they are removed from the database. This process mirrors the garbage collection, where objects not referenced are garbage collected or removed from memory. Reachability is expensive because it traverses the whole object tree and may require reloading data from database. If reachability is not needed by your application, you should disable it.

Check whether reach-ability is required by your app before setting it to false.

Answer 2

If a 1-N relationship can get large, you may want to consider mapping it relationally, i.e. instead of mapping Pet.photos, map Photo.pet. This will prevent you from navigating from Pet to Photo in a OO fashion without a query, but will prevent the SQL statements you are concerned about.

You storePhoto would then look like below and the 1-N would not be fetched.

public void storePhoto(Photo photo) {
    // get PM and current tx
    try {
       tx.begin();
       pm.makePersistent(photo); // assuming pet was already set
       tx.commit();
    } catch (Exception e) {
       // rollback and close pm
    }
}