C++ binary to Java gets "java.lang.Error: Invalid memory access"

Question

I have a Java library interfacing (via JNA) with a native C++ DLL. This DLL provides functions to set callbacks that are invoked when certain hardware events occur.

All of these callbacks work except one, even though it has near-identical definition to another one.

These callback signatures and enums are defined in C++ code:

typedef enum _KEYSTATE
{
  KEYSTATE_NONE = 0, 
  KEYSTATE_UP, 
  KEYSTATE_DOWN, 
  KEYSTATE_HOLD, 
  KEYSTATE_INVALID, 
} KEYSTATETYPE, *P_KEYSTATETYPE;

typedef enum _DKTYPE
{
  DK_NONE = 0,
  DK_1,
  DK_2,
  DK_3,
  DK_4,
  DK_5,
  DK_6,
  DK_7,
  DK_8,
  DK_9,
  DK_10,
  DK_INVALID,
  DK_COUNT = 10
} DKTYPE, *P_DKTYPE;

typedef enum _GESTURETYPE
{
  GESTURE_NONE      = 0x00000000, 
  GESTURE_PRESS     = 0x00000001,
  GESTURE_TAP       = 0x00000002,
  GESTURE_FLICK     = 0x00000004,
  GESTURE_ZOOM      = 0x00000008,
  GESTURE_ROTATE    = 0x00000010,
  GESTURE_MOVE      = 0x00000020,
  GESTURE_HOLD      = 0x00000040,
  GESTURE_RELEASE   = 0x00000080,
  GESTURE_SCROLL    = 0x00000100,
  GESTURE_ALL       = 0xFFFF
} GESTURETYPE, *P_GESTURETYPE;

typedef enum _EVENTTYPE
{
  EVENT_NONE = 0,
  EVENT_ACTIVATED,
  EVENT_DEACTIVATED,
  EVENT_CLOSE,
  EVENT_EXIT,
  EVENT_INVALID,
} EVENTTYPETYPE, *P_EVENTTYPETYPE;

typedef HRESULT (CALLBACK *DynamicKeyCallbackFunctionType)(DKTYPE, KEYSTATETYPE); 
typedef HRESULT (CALLBACK *AppEventCallbackType) (EVENTTYPETYPE, DWORD, DWORD);
typedef HRESULT (CALLBACK *TouchpadGestureCallbackFunctionType)(GESTURETYPE, DWORD, WORD, WORD, WORD);
typedef HRESULT (CALLBACK *KeyboardCallbackFunctionType)(UINT uMsg, WPARAM wParam, LPARAM lParam);

The following interfaces are defined in Java for callbacks:

interface DynamicKeyCallbackFunction extends StdCallLibrary.StdCallCallback {
  int callback(int rawDynamicKeyType, int rawDynamicKeyState);
}

interface AppEventCallbackFunction extends StdCallLibrary.StdCallCallback {
  int callback(int appEventType, WinDef.UINT dwAppMode, WinDef.UINT dwProcessID);
}

interface TouchpadGestureCallbackFunction extends StdCallLibrary.StdCallCallback {
  int callback(int gestureType, WinDef.UINT dwParameters,
               WinDef.USHORT wXPos, WinDef.USHORT wYPos, WinDef.USHORT wZPos);
}

interface KeyboardCallbackFunction extends StdCallLibrary.StdCallCallback {
  int callback(WinDef.UINT uMsg, WinDef.UINT_PTR wParam, WinDef.INT_PTR lParam);
}

With these functions in the API/Library class/interface to set them:

// These are defined in an RazerAPI.java file as wrappers to simplify usage
// lib is an instance of a RazerLibrary from JNA

public Hresult RzSBAppEventSetCallback(AppEventCallbackFunction callback) {
  return Hresult.getFromApiValue(lib.RzSBAppEventSetCallback(callback));
}

public Hresult RzSBDynamicKeySetCallback(DynamicKeyCallbackFunction callback) {
  return Hresult.getFromApiValue(lib.RzSBDynamicKeySetCallback(callback));
}

public Hresult RzSBKeyboardCaptureSetCallback(KeyboardCallbackFunction callback) {
  return Hresult.getFromApiValue(lib.RzSBKeyboardCaptureSetCallback(callback));
}

public Hresult RzSBGestureSetCallback(TouchpadGestureCallbackFunction callback) {
  return Hresult.getFromApiValue(lib.RzSBGestureSetCallback(callback));
}

// These are the methods in the interface RazerLibrary.java file passed to JNA

int RzSBAppEventSetCallback(RazerAPI.AppEventCallbackFunction callback);
int RzSBDynamicKeySetCallback(RazerAPI.DynamicKeyCallbackFunction callback);
int RzSBKeyboardCaptureSetCallback(RazerAPI.KeyboardCallbackFunction callback);
int RzSBGestureSetCallback(RazerAPI.TouchpadGestureCallbackFunction callback);

When registering callbacks, something similar to this is used (simplified to not clutter up with hundreds of lines, links to full code files at end of post).

public class RazerManager implements DynamicKeyCallbackFunction {
  private static DynamicKeyCallbackFunction dkCallback;

  private RazerManager() {
    dkCallback = this;

    RazerAPI api = RazerAPI.INSTANCE;

    api.RzSBDynamicKeySetCallback(dkCallback);
  }

  @Override
  public int callback(int type, int state) {
    System.out.printf("DK Callback: %s %s", type, state);
    return 0; // S_OK
  }
}

public class Touchpad implements TouchpadGestureCallbackFunction {
  private static TouchpadGestureCallbackFunction gestureCallback;

  private Touchpad() {
    gestureCallback = this;

    RazerAPI api = RazerAPI.INSTANCE;

    api.RzSBGestureSetCallback(gestureCallback);
  }

  @Override
  public int callback(int gestureType, UINT param, USHORT x, USHORT y, USHORT z) {
    System.out.printf("Gesture: %s", gestureType);
  }
}

An instance of RazerManager is created in the main loop of a simple swing app used for testing, RazerManager creates an instance of Touchpad in its constructor. A while loop is used to process messages with GetMessage followed by Translate and DispatchMessage if valid, like the following:

public class Main {
  public static void main(String[] args) throws IOException {
    // call javax's invokeLater to run app
  }

  public Main() {
    /* JFrame created here and then shown with setVisible(true); */ 

    RazerManager manager = RazerManager.getInstance();

    // Yes this is horrible, blocking everything but it's simply
    // used to get messages to go through and trigger the callbacks
    WinUser.MSG msg = new WinUser.MSG();
    while (true) {
      int hasMessage = User32.INSTANCE.GetMessage(msg, null, 0, 0);
      if (hasMessage != 0) {
        User32.INSTANCE.TranslateMessage(msg);
        User32.INSTANCE.DispatchMessage(msg);
      }
    }
  }
}

Now, gesture events are handled just fine, the callback in Touchpad class gets called and returns properly. When a dynamic key event is generated, an exception is thrown inside the while loop processing windows messages:

Exception in thread "AWT-EventQueue-0" java.lang.Error: Invalid memory access
  at com.sun.jna.Native.invokeInt(Native Method)
  at com.sun.jna.Function.invoke(Function.java:383)
  at com.sun.jna.Function.invoke(Function.java:315)
  at com.sun.jna.Library$Handler.invoke(Library.java:212)
  at com.sun.proxy.$Proxy11.DispatchMessage(Unknown Source)
  at com.sharparam.jblade.tester.Main.<init>(Main.java:113)
  at com.sharparam.jblade.tester.Main$1.run(Main.java:25)
  at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
  at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:733)
  at java.awt.EventQueue.access$200(EventQueue.java:103)
  at java.awt.EventQueue$3.run(EventQueue.java:694)
  at java.awt.EventQueue$3.run(EventQueue.java:692)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
  at java.awt.EventQueue.dispatchEvent(EventQueue.java:703)
  at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
  at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
  at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
  at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
  at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
  at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

This only happens with dynamic key events, not with gesture or appevent. I can't for the life of me understand why, as dynamic key and gesture callbacks are very similar on the C++ side.

EDIT: Full code can be found in the GitHub repo, specifically RazerLibrary.java, RazerAPI.java, RazerManager.java and Touchpad.java. The swing test can be found as a gist.

Answer 1

So it turns out that you can't (or at least you have to do it in a different way than I did) have one class implement multiple callback interfaces. Creating explicit implementations of the different callback interfaces and assigning them to the callback fields in RazerManager solved it.

This explains why the callback in Touchpad was working but not the ones in RazerManager (Touchpad implemented one interface while RazerManager did three).

To demonstrate:

public class MyClass {
  private static MyCallbackInterface myCallback;

  private MyClass() {
    myCallback = new CallbackInterface() {
      @Override
      public int callback(/* parameters */) {
        // Do stuff with data here
        return 0;
      }
    }

    nativeLib.SetCallback(myCallback);
  }

It seems the native library or JNA gets confused somewhere when a single class implements more than one callback interface, and doesn't know which one to call. So it calls one at random (or the first one defined?).