Ok I think I figured this out:
XIDeviceEvent *d_ev = (XIDeviceEvent*) ev.xcookie.data;
KeyCode keycode = d_ev->detail;
int keysyms_per_keycode;
KeySym *keysym = XGetKeyboardMapping (dpy, keycode, 1, &keysyms_per_keycode);
char * result = XKeysymToString (keysym[0]);
Where the 0 in keysym[0] probably means no modifiers. keysym[1] is with shift (-> capital letters) and up to keysyms_per_keycode-1 there are all the other possible modifiers.