GWT / JSNI - "DataCloneError - An object could not be cloned" - how do I debug?

Question

I am attempting to call out to parallels.js via JSNI. Parallels provides a nice API around web workers, and I wrote some lightweight wrapper code which provides a more convenient interface to workers from GWT than Elemental. However I'm getting an error which has me stumped:

com.google.gwt.core.client.JavaScriptException: (DataCloneError) @io.mywrapper.workers.Parallel::runParallel([Ljava/lang/String;Lcom/google/gwt/core/client/JavaScriptObject;Lcom/google/gwt/core/client/JavaScriptObject;)([Java object: [Ljava.lang.String;@1922352522, JavaScript object(3006), JavaScript object(3008)]): An object could not be cloned.

This comes from, in hosted mode:

at com.google.gwt.dev.shell.BrowserChannelServer.invokeJavascript(BrowserChannelServer.java:249) at com.google.gwt.dev.shell.ModuleSpaceOOPHM.doInvoke(ModuleSpaceOOPHM.java:136) at com.google.gwt.dev.shell.ModuleSpace.invokeNative(ModuleSpace.java:571) at com.google.gwt.dev.shell.ModuleSpace.invokeNativeVoid(ModuleSpace.java:299) at com.google.gwt.dev.shell.JavaScriptHost.invokeNativeVoid(JavaScriptHost.java:107) at io.mywrapper.workers.Parallel.runParallel(Parallel.java)

Here's my code:

Example client call to create a worker:

    Workers.spawnWorker(new String[]{"hello"}, new Worker() {
        @Override
        public String[] work(String[] data) {
            return data;
        }

        @Override
        public void done(String[] data) {
            int i = data.length;
        }
    });

The API that provides a general interface:

public class Workers {

    public static void spawnWorker(String[] data, Worker worker) {
        Parallel.runParallel(data, workFunction(worker), callbackFunction(worker));
    }

    /**
     * Create a reference to the work function.
     */
    public static native JavaScriptObject workFunction(Worker worker) /*-{
        return worker == null ? null : $entry(function(x) {
            worker.@io.mywrapper.workers.Worker::work([Ljava/lang/String;)(x);
        });
    }-*/;

    /**
     * Create a reference to the done function.
     */
    public static native JavaScriptObject callbackFunction(Worker worker) /*-{
        return worker == null ? null : $entry(function(x) {
            worker.@io.mywrapper.workers.Worker::done([Ljava/lang/String;)(x);
        });
    }-*/;        
}

Worker:

public interface Worker extends Serializable {

    /**
     * Called to perform the work.
     * @param data
     * @return
     */
    public String[] work(String[] data);

    /**
     * Called with the result of the work.
     * @param data
     */
    public void done(String[] data);

}

And finally the Parallels wrapper:

public class Parallel {

    /**
     * @param data Data to be passed to the function
     * @param work Function to perform the work, given the data
     * @param callback Function to be called with result
     * @return
     */
    public static native void runParallel(String[] data, JavaScriptObject work, JavaScriptObject callback) /*-{
        var p =  new $wnd.Parallel(data);
        p.spawn(work).then(callback);
    }-*/;
}

What's causing this?

The JSNI docs say, regarding arrays:

opaque value that can only be passed back into Java code

This is quite terse, but ultimately my arrays are passed back into Java code, so I assume these are OK.

EDIT - ok, bad assumption. The arrays, despite only ostensibly being passed back to Java code, are causing the error (which is strange, because there's very little googleability on DataCloneError.) Changing them to String works; however, String isn't sufficient for my needs here. Looks like objects face the same kinds of issues as arrays do; I saw Thomas' reference to JSArrayUtils in another StackOverflow thread, but I can't figure out how to call it with an array of strings (it wants an array of JavaScriptObjects as input for non-primitive types, which does me no good.) Is there a neat way out of this?

EDIT 2 - Changed to use JSArrayString wherever I was using String[]. New issue; no stacktrace this time, but in the console I get the error: Uncaught ReferenceError: __gwt_makeJavaInvoke is not defined. When I click on the url to the generated script in developer tools, I get this snippet:

self.onmessage = function(e) {self.postMessage((function (){
    try {
      return __gwt_makeJavaInvoke(3)(null, 65626, jsFunction, this, arguments);
    }
     catch (e) {
      throw e;
    }
  })(e.data))}

I see that _gwt_makeJavaInvoke is part of the JSNI class; so why would it not be found?

Answer 1

If you never use dev mode it is currently safe to pretend that a Java String[] is a JS array with strings in it. This will break in dev mode since arrays have to be usable in Java and Strings are treated specially, and may break in the future if the compiler optimizes arrays differently.

Cases where this could go wrong in the future:

The semantics of Java arrays and JavaScript arrays are different - Java arrays cannot be resized, and are initialized with specific values based on the component type (the data in the array). Since you are writing Java code, the compiler could conceivable make assumptions based on details about how you create and use that array that could be broken by JS code that doesn't know to never modify the array.

Some arrays of primitive types could be optimized into TypedArrays in JavaScript, more closely following Java semantics in terms of resizing and Java behavior in terms of allocation. This would be a performance boost as well, but could break any use of int[], double[], etc.

---

Instead, you should copy your data into a JsArrayString, or just use the js array to hold the data rather than going back and forth, depending on your use case. The various JsArray types can be resized and already exist as JavaScript objects that outside JS can understand and work with.

---

Reply to EDIT 2:

At a guess, the parallel.js script is trying to run your code from another scope such a in the webworker (that's the point of the code, right) where your GWT code isn't present. As such, it can't call the makeJavaInvoke which is the bridge back into dev mode (would be a different failure with compiled JS). According to http://adambom.github.io/parallel.js/ there are specific requirements that a passed callback must meet to be passed in to spawn and perhaps then - your anonymous functions definitely do not meet them, and it may not be possible to maintain java semantics.

Before I get much deeper, check out this answer I did a while ago addressing the basic issues with webworkers and gwt/java: https://stackoverflow.com/a/11376059/860630

As noted there, WebWorkers are effectively new processes, with no shared code or shared state with the original process. The Parallel.js code attempts to paper over this with a little bit of trickery - shared state is only available in the form of the contents passed in to the original Parallel constructor, but you are attempting to pass in instances of 'java' objects and calling methods on them. Those Java instances come with their own state, and potentially can link back to the rest of the Java app by fields in the Worker instance. If I were implementing Worker and doing something that referenced other data than what was passed in, then I would be seeing further bizarre failures.

So the functions you pass in must be completely standalone - they must not refer to external code in any way, since then the function can't be passed off to the webworker, or to several webworkers, each unaware of each other's existence. See https://github.com/adambom/parallel.js/issues/32 for example:

That's not possible since it would

• require a shared state across workers
• require us to transmit all scope variables (I don't think there's even a possibility to read the available scopes)

The only thing which might be possible would be cache variables, but these can already be defined in the function itself with spawn() and don't make any sense in map (because there's no shared state).

Without being actually familiar with how parallel.js is implemented (all of this answer so far is reading the docs and a quick google search for "parallel.js shared state", plus having experiemented with WebWorkers for a day or so and deciding that my present problem wasn't yet worth the bother), I would guess that then is unrestricted, and you can you pass it whatever you like, but spawn, map, and reduce must be written in such a way that their JS can be passed off to the new JS process and completely stand alone there.

This may be possible from your normal Java code when compiled, provided you have just one implementation of Worker and that impl never uses state other than what is directly passed in. In that case the compiler should rewrite your methods to be static so that they are safe to use in this context. However, that doesn't make for a very useful library, as it seems you are trying to achieve. With that in mind, you could keep your worker code in JSNI to ensure that you follow the parallel.js rules.

Finally, and against the normal GWT rules, avoid $entry for calls you expect to happen in other contexts, since those workers have no access to the normal exception handling and scheduling that $entry enables.

(and finally finally, this is probably still possible if you are very careful at writing Worker implementations and write a Generator that invokes each worker implementation in very specific ways to make sure that com.google.gwt.dev.jjs.impl.MakeCallsStatic and com.google.gwt.dev.jjs.impl.Pruner can correctly act to knock out the this in those instance methods once they've been rewritten as JS functions. I think the cleanest way to do this is to emit the JSNI in the generator itself, call a static method written in real Java, and from that static method call the specific instance method that does the heavy lifting for spawn, etc.)

Answer 2

You can find working example of GWT and WebWorkers here: https://github.com/tomekziel/gwtwwlinker/

This is a preliminary work, but using this pattern I was able to pass GWT objects to and from webworker using serialization provided by AutoBeanFactory.