Datanucleus JDO map String to MySQL-type bigint in App Engine

Question

I am trying to access Datastore and Cloud SQL from my App Engine application via JDO. If I set primary key type as long, everything works well with Cloud SQL but Datastore doesn't work and if I set up primary key as String and want to convert it to JDBC-Type "Bigint" I get following error when trying to access Cloud SQL:

javax.jdo.JDOUserException: Exception thrown while loading remaining rows of query
at org.datanucleus.api.jdo.JDOAdapter.getUserExceptionForException(JDOAdapter.java:1085)
at org.datanucleus.store.rdbms.query.ForwardQueryResult.closingConnection(ForwardQueryResult.java:273)
at org.datanucleus.store.query.AbstractQueryResult.disconnect(AbstractQueryResult.java:108)
at org.datanucleus.store.rdbms.query.AbstractRDBMSQueryResult.disconnect(AbstractRDBMSQueryResult.java:69)
at org.datanucleus.store.rdbms.query.JDOQLQuery$2.transactionPreClose(JDOQLQuery.java:698)
at org.datanucleus.store.connection.AbstractManagedConnection.transactionPreClose(AbstractManagedConnection.java:92)
at org.datanucleus.store.connection.ConnectionManagerImpl$2.transactionPreRollBack(ConnectionManagerImpl.java:333)
at org.datanucleus.TransactionImpl.internalPreRollback(TransactionImpl.java:523)
at org.datanucleus.TransactionImpl.rollback(TransactionImpl.java:446)
at org.datanucleus.api.jdo.JDOTransaction.rollback(JDOTransaction.java:180)
at de.studienarbeit3.dokumente.DokumentListServlet.doPost(DokumentListServlet.java:82)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:123)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:63)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:125)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectRequest(DevAppServerModulesFilter.java:366)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectModuleRequest(DevAppServerModulesFilter.java:349)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doFilter(DevAppServerModulesFilter.java:116)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at com.google.appengine.tools.development.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:97)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:487)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
NestedThrowablesStackTrace:
Somehow org.datanucleus.store.rdbms.mapping.BigIntRDBMSMapping.getLong() was called on the mapping for field="de.studienarbeit3.dokumente.Dokument.did", which should have been impossible!
org.datanucleus.exceptions.NucleusException: Somehow         org.datanucleus.store.rdbms.mapping.BigIntRDBMSMapping.getLong() was called on the mapping for field="de.studienarbeit3.dokumente.Dokument.did", which should have been impossible!
at org.datanucleus.store.mapped.mapping.AbstractDatastoreMapping.getLong(AbstractDatastoreMapping.java:110)
at org.datanucleus.store.rdbms.mapping.BigIntRDBMSMapping.getString(BigIntRDBMSMapping.java:182)
at org.datanucleus.store.mapped.mapping.SingleFieldMapping.getString(SingleFieldMapping.java:202)
at org.datanucleus.store.rdbms.fieldmanager.ResultSetGetter.fetchStringField(ResultSetGetter.java:137)
at org.datanucleus.identity.IdentityUtils.getApplicationIdentityForResultSetRow(IdentityUtils.java:125)
at org.datanucleus.store.rdbms.query.PersistentClassROF.getIdentityForResultSetRow(PersistentClassROF.java:542)
at org.datanucleus.store.rdbms.query.PersistentClassROF.getObjectForApplicationId(PersistentClassROF.java:473)
at org.datanucleus.store.rdbms.query.PersistentClassROF.getObject(PersistentClassROF.java:371)
at org.datanucleus.store.rdbms.query.ForwardQueryResult.nextResultSetElement(ForwardQueryResult.java:176)
at org.datanucleus.store.rdbms.query.ForwardQueryResult$QueryResultIterator.next(ForwardQueryResult.java:380)
at org.datanucleus.store.rdbms.query.ForwardQueryResult.processNumberOfResults(ForwardQueryResult.java:138)
at org.datanucleus.store.rdbms.query.ForwardQueryResult.advanceToEndOfResultSet(ForwardQueryResult.java:159)
at org.datanucleus.store.rdbms.query.ForwardQueryResult.closingConnection(ForwardQueryResult.java:261)
at org.datanucleus.store.query.AbstractQueryResult.disconnect(AbstractQueryResult.java:108)
at org.datanucleus.store.rdbms.query.AbstractRDBMSQueryResult.disconnect(AbstractRDBMSQueryResult.java:69)
at org.datanucleus.store.rdbms.query.JDOQLQuery$2.transactionPreClose(JDOQLQuery.java:698)
at org.datanucleus.store.connection.AbstractManagedConnection.transactionPreClose(AbstractManagedConnection.java:92)
at org.datanucleus.store.connection.ConnectionManagerImpl$2.transactionPreRollBack(ConnectionManagerImpl.java:333)
at org.datanucleus.TransactionImpl.internalPreRollback(TransactionImpl.java:523)
at org.datanucleus.TransactionImpl.rollback(TransactionImpl.java:446)
at org.datanucleus.api.jdo.JDOTransaction.rollback(JDOTransaction.java:180)
at de.studienarbeit3.dokumente.DokumentListServlet.doPost(DokumentListServlet.java:82)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:123)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:63)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:125)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectRequest(DevAppServerModulesFilter.java:366)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectModuleRequest(DevAppServerModulesFilter.java:349)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doFilter(DevAppServerModulesFilter.java:116)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at com.google.appengine.tools.development.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:97)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:487)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

My package.jdo is as followed:

  <jdo>
    <package name="de.studienarbeit3.dokumente">
        <class name="Autor" detachable="true" identity-type="application">
            <datastore-identity value-strategy="identity"/>
            <field name="aid" primary-key="true" persistence-modifier="persistent" value-strategy="identity"/>
            <field name="vorname"/>
            <field name="nachname" />
            <field name="dokumente" />

        </class>
        <class name="Dokument" detachable="true" identity-type="application">
            <datastore-identity value-strategy="identity"/>
            <field name="did" primary-key="true" persistence-modifier="persistent" value-strategy="identity">
            </field>
            <field name="autor"/>
            <field name="titel"/>
            <field name="text"/>
            <field name="datum"/>

        </class>
    </package>
</jdo>

and package-sql.orm is this:

<orm>
    <package name="de.studienarbeit3.dokumente">
        <class name="Autor" detachable="true" persistence-modifier="persistence-capable" table="Autor">
            <datastore-identity value-strategy="identity"/>
            <field name="aid" primary-key="true" persistence-modifier="persistent" value-strategy="identity">
                <column name="aid" jdbc-type="bigint" length="20"/>
            </field>
            <field name="vorname" persistence-modifier="persistent">
                <column name="vorname"/>
            </field>    
            <field name="nachname" persistence-modifier="persistent">
                <column name="nachname"/>
            </field>
            <field name="dokumente" persistence-modifier="persistent" mapped-by="autor">
                <collection element-type="de.studienarbeit3.dokumente.Dokument"/>
            </field>
        </class>
        <class name="Dokument"  detachable="true" persistence-modifier="persistence-capable" table="Dokument">
            <datastore-identity />
            <field name="did" primary-key="true" persistence-modifier="persistent" value-strategy="identity">
                <column name="did" jdbc-type="bigint" length="20" />
            </field>
            <field name="autor" persistence-modifier="persistent" default-fetch-group="true">
                <column name="autor" jdbc-type="bigint" length="20"/>
                <foreign-key name="DOKUMENTAUTOR_FK" delete-action="restrict"/>
            </field>
            <field name="titel" persistence-modifier="persistent">
                <column name="titel"/>
            </field>
            <field name="text" persistence-modifier="persistent">
                <column name="text"/>
            </field>
            <field name="datum" persistence-modifier="persistent">
                <column name="datum"/>
            </field>
        </class>
    </package>
</orm>

Is there any possible way to fix this problem? I have tried this plugin without success: http://blog.wp.weightpoint.se/2012/03/01/portable-owned-jdo-relations-appengine/

According to this http://www.datanucleus.org/servlet/jira/browse/NUCRDBMS-597 it was fixed in Datanucleus 3.1.0.m4. GAE has 3.1.3. and it doesn't work. Any help would be very helpful, thank you

Answer 1

Firstly you're using old unsupported versions there. Secondly if you check JIRA you also see http://www.datanucleus.org/servlet/jira/browse/NUCRDBMS-705

Solution : use a recent release (yes, "GAE CloudSQL" may be packaging some ancient version because they need that for their GAE/Datastore plugin, but you can just as easily use a recent release with MySQL where you don't need their plugin)