Currently (by default in openFrameworks 0.8.0+), openFrameworks is using GLFW for its windowing system on OS X.
GLFW doesn't load xibs/nibs, but sets up menus in the source. You can see where GLFW sets up the menu bar here:
https://github.com/glfw/glfw/blob/76afd4172727892286fe5728580021a26d198c49/src/cocoa_window.m#L738
If you would like to modify any aspect of the the Cocoa Window, GLFW offers a window handle, which is accessible in openFrameworks in the following way (this will only work on OS X of course, but native window handles are available for X11 and Windows as well):
#include <Cocoa/Cocoa.h>
// ...
NSWindow * appWindow = (NSWindow *)ofGetCocoaWindow();
if(appWindow) {
// make your obj-c calls here
}
Also, you may need to make sure you source files are named with the .mm suffix or set up to be interpreted as objective-c++ in the project settings.
Good luck!