From the idea given by @Lord Zsolt, the same can be implemented in OS X as:
Add the following code when you want to darken your window.
NSView *transparentBlackView = [[NSView alloc] initWithFrame:[[yourwindow contentView] frame]];
CALayer *viewLayer = [CALayer layer];
[viewLayer setBackgroundColor:CGColorCreateGenericRGB(0.0, 0.0, 0.0, 0.4)]; //RGB plus Alpha Channel
[transparentBlackView setWantsLayer:YES];
[transparentBlackView setLayer:viewLayer];
[[yourwindow contentView] addSubview:transparentBlackView];
Also don't forget to add QuartzCore.framework
to your project. And import it in your class using #import<QuartzCore/QuartzCore.h>