You can change the behaviour that the underline only appears when pressing the Alt key by setting a property with the UIManager
UIManager.getDefaults().put("Button.showMnemonics", Boolean.TRUE);
I think you need to do this before you display your first Swing component, but I'm not sure.