How to provide js-ctypes in a spidermonkey embedding?

Question

Summary

I have looked over the code the SpiderMonkey 'shell' application uses to create the ctypes JavaScript object, but I'm a less-than novice C programmer. Due to the varying levels of insanity emitted by modern build systems, I can't seem to track down the code or command that actually links a program with the desired functionality.

---

method.madness

This js-ctypes implementation by The Mozilla Devs is an awesome addition. Since its conception, scripting has been primarily used to exert control over more rigorous and robust applications. The advent of js-ctypes to the SpiderMonkey project, enables JavaScript to stand up and be counted as a full fledged object oriented rapid application development language flying high above 'the bar' set by various venerable application development languages such as Microsoft's VB6.

---

Shall we begin?

I built SpiderMonkey with this config: ./configure --enable-ctypes --with-system-nspr

followed by successful execution of: make && make install

The js shell works fine and a global ctypes javascript object was verified operational in that shell.

Working with code taken from the first source listing at How to embed the JavaScript Engine -MDN, I made an attempt to instantiate the JavaScript ctypes object by inserting the following code at line 66:

    /* Populate the global object with the ctypes object. */
    if (!JS_InitCTypesClass(cx, global))
        return NULL;
    /*

I compiled with: g++ $(./js-config --cflags --libs) hello.cpp -o hello

It compiles with a few warnings:

hello.cpp: In function ‘int main(int, const char**)’:
hello.cpp:69:16: warning: converting to non-pointer type ‘int’ from NULL [-Wconversion-null]
hello.cpp:80:20: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
hello.cpp:89:17: warning: NULL used in arithmetic [-Wpointer-arith]

But when you run the application:

./hello: symbol lookup error: ./hello: undefined symbol: JS_InitCTypesClass

---

Moreover

JS_InitCTypesClass is declared extern in 'dist/include/jsapi.h', but the function resides in 'ctypes/CTypes.cpp' which includes its own header 'CTypes.h' and is compiled at some point by some command during 'make' to yeild './CTypes.o'. As I stated earlier, I am less than a novice with the C code, and I really have no idea what to do here.

Please give or give direction to a generic example of making the js-ctypes object functional in an embedding.

Answer 1

The problem: Most platform distributors are ignorant to recommendations given by code developers. Therefore js-ctypes are not enabled for mozjs185 on most if not all systems hosting the library. This leaves some issues for you to sort out.

After you have configured with --enable-ctypes and --with-sytem-nspr followed by make and then make install (might need to be root for last command)

You will likely have two versions of libmozjs185.so on your system. One with ctypes enabled (located in /usr/local/lib) and one without ctypes enabled (located in /usr/lib).

You, as per the question, want to link against the library with ctypes. So that's what you do by specifying: -L/usr/local/lib -lnspr4 -lmozjs185 to your compiler. It will compile ok. But when the application runs, the os library loader will load the first instance of the library that it finds. Unfortunately this will likely be the library located in /usr/lib and this version likely does not have ctypes enabled. That's where you'll run into this [solved] problem: g++ Undefined Symbol Error using shared library

The bottom line is this: multiple versions of the same library is creating one hell of an issue. The best way to provide js-ctypes to a spidermonkey embedding is therefore linking in the ctypes enabled static version of mozjs185 to your program unless you want to write a library for dealing with multiple platform libary loaders, or create your own (rebranded version of mozjs185) which I have covered in great detail over @ SpiderMonkey Build Documentation - MDN

To perform the static linking you will need to use these parameters with g++: -lnspr4 -lpthread -lmozjs185-1.0 provided that you have built and installed the development package correctly. This is the 'best' (platform independant) way to provide js-ctypes to a spidermonkey embedding. Although this does increase the size of your application by at least 3.5 MB. If you have built the debug version it could be more than 15 times larger.

Answer 2

The Hack

It had already occurred to me that linkage was failing because of conditional defines in the header files as well as scattered lib and header locations. Well enough... I tried to define the JS_HAS_CTYPES on the command line but if it worked at all it certainly was not enough.

I decided that since the SpiderMonkey shell has its own unique makefile, and already has working access to the functionality I am trying to capture, simply renaming js.cpp to js.cpp.tmp and allowing my code to stand in its place, almost worked.

The file compiled well and no runtime linking errors were thrown on application execution, but the code ('JSNativeObject' ctypes) almost completely failed JS_InitCTypesClass. Seeing that my linking error had long been forgotten, I immediately went looking through the output of make to see if I could 'swipe' the compilation code and... We Have a BINGO!

---

The Compilation

After restoring the shell/js.cpp to its original target, I moved the hello.cpp to the root source directory of spidermonkey and began to correct the relative paths that were created by the makefile as well as performing removal of constructs that obviously bore no presence or relavence to my application.

While the following commands appear to render an operational binary, the author can give no affinity as to the correctness or completeness of this listing.

c++ -o hello.o -c  -Idist/system_wrappers_js -include config/gcc_hidden.h \
-DEXPORT_JS_API -DOSTYPE=\"Linux3.2\" -DOSARCH=Linux -I. -Idist/include \
-Idist/include/nsprpub  -I/usr/include/nspr -fPIC  -fno-rtti \
-fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth \
-Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align \
-Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type -pedantic \
-Wno-long-long -fno-strict-aliasing -pthread -pipe  -DNDEBUG -DTRIMMED -Os \
-freorder-blocks -fomit-frame-pointer -DJS_HAS_CTYPES -DMOZILLA_CLIENT \
-include js-confdefs.h -MD -MF .deps/hello.pp hello.cpp;

c++ -o hello -fno-rtti -fno-exceptions -Wall -Wpointer-arith \
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy \
-Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof \
-Wno-variadic-macros -Werror=return-type -pedantic \
-Wno-long-long -fno-strict-aliasing -pthread -pipe  -DNDEBUG \
-DTRIMMED -Os -freorder-blocks -fomit-frame-pointer hello.o \
-lpthread -Wl,-rpath-link,/bin -Wl,-rpath-link,/usr/local/lib \
-Ldist/bin -Ldist/lib -L/usr/lib -lplds4 -lplc4 -lnspr4 \
-lpthread -ldl editline/libeditline.a libjs_static.a -ldl;

The two commands listed above were placed into an executable shell script named 'mkhello' which was saved to the root source directory.

From what I can gather it is a two stage compilation method. For what reason I am unsure but the explanation seems very educational. Thoughts?

EDIT see comment below for explanation on 'two stage compilation method'.

---

The Code: hello.cpp

/*
 * This define is for Windows only, it is a work-around for bug 661663.
 */
#ifdef _MSC_VER
# define XP_WIN
#endif

/* Include the JSAPI header file to get access to SpiderMonkey. */
#include "jsapi.h"


/* The class of the global object. */
static JSClass global_class = {
    "global", JSCLASS_GLOBAL_FLAGS,
    JS_PropertyStub, JS_PropertyStub, JS_PropertyStub, JS_StrictPropertyStub,
    JS_EnumerateStub, JS_ResolveStub, JS_ConvertStub, JS_FinalizeStub,
    JSCLASS_NO_OPTIONAL_MEMBERS
};

/* The error reporter callback. */
void reportError(JSContext *cx, const char *message, JSErrorReport *report)
{
    fprintf(stderr, "%s:%u:%s\n",
            report->filename ? report->filename : "<no filename=\"filename\">",
            (unsigned int) report->lineno,
            message);
}

int main(int argc, const char *argv[])
{
    /* JSAPI variables. */
    JSRuntime *rt;
    JSContext *cx;
    JSObject  *global;

    /* Create a JS runtime. You always need at least one runtime per process. */
    rt = JS_NewRuntime(8 * 1024 * 1024);
    if (rt == NULL)
        return 1;

    /* 
     * Create a context. You always need a context per thread.
     * Note that this program is not multi-threaded.
     */
    cx = JS_NewContext(rt, 8192);
    if (cx == NULL)
        return 1;
    JS_SetOptions(cx, JSOPTION_VAROBJFIX | JSOPTION_JIT | JSOPTION_METHODJIT);
    JS_SetVersion(cx, JSVERSION_LATEST);
    JS_SetErrorReporter(cx, reportError);

    /*
     * Create the global object in a new compartment.
     * You always need a global object per context.
     */
    global = JS_NewCompartmentAndGlobalObject(cx, &global_class, NULL);
    if (global == NULL)
        return 1;

    /*
     * Populate the global object with the standard JavaScript
     * function and object classes, such as Object, Array, Date.
     */
    if (!JS_InitStandardClasses(cx, global))
        return 1;

    /* Populate the global object with the ctypes object. */
    if (!JS_InitCTypesClass(cx, global))
        return NULL;
    /*

    /* Your application code here. This may include JSAPI calls
     * to create your own custom JavaScript objects and to run scripts.
     *
     * The following example code creates a literal JavaScript script,
     * evaluates it, and prints the result to stdout.
     *
     * Errors are conventionally saved in a JSBool variable named ok.
     */
    char *script = "ctypes.open";
    jsval rval;
    JSString *str;
    JSBool ok;
    const char *filename = "noname";
    uintN lineno = 0;

    ok = JS_EvaluateScript(cx, global, script, strlen(script),
                           filename, lineno, &rval);
    if (rval == NULL | rval == JS_FALSE)
        return 1;

    str = JS_ValueToString(cx, rval);
    printf("%s\n", JS_EncodeString(cx, str));

    /* End of your application code */

    /* Clean things up and shut down SpiderMonkey. */
    JS_DestroyContext(cx);
    JS_DestroyRuntime(rt);
    JS_ShutDown();
    return 0;
}

---

Conclusion

$ ./mkhello
# ...
# error free garbage scrolls....
$ ./hello
function open() {
    [native code]
}

Follow this example to provide js-ctypes to a SpiderMonkey embedding. You may or may not need to re-create these steps in order but it is highly recommended from my current perspective.