DWM and painting unresponsive apps

Question

In Vista and later, if an app becomes unresponsive, the Desktop Window Manager is able to handle redrawing it when necessary (move a window over it, drag it around, etc.) because it has kept a pixel buffer for it. Windows also tries to detect when an app has become unresponsive after some timeout, and tries to make the best of the situation -- I believe it dims out the window, adds "Not Responding" to its title bar, and perhaps some other effects.

Now, we have a skinned app that uses window regions and layered windows, and it doesn't play well with these effects. We've been developing on XP, but have noticed a strange effect when testing on Vista. At some points the app may spend a few moments on some calculation or callback, and "if a message has not been retrieved within 5 seconds, the DWM declares the window to be hung" (MSDN - Preventing Hangs in Windows Applications). It seems that when this kicks in, a strange graphical problem occurs: any pixels that would be 100% transparent due to the window regions turn black, which effectively makes the window rectangular again, with a black background. There seem to be other anomalies, with the original window's pixels being shifted a bit in some child dialogs.

I am working on reducing such delays (ideally Windows will never need to step in like this), and trying to maintain responsiveness while it's busy, but I'd still like to figure out what is causing it to render like that, as I can't guarantee I can eliminate all delays.

Basically, I just would like to know what Windows is doing when this happens, and how I can make my app behave properly with it. Skinned apps have to still work on Vista and later, so I need to figure out what I'm doing that's non-standard. I don't even know exactly how to look for information into how Windows now handles unresponsive apps, as my searches only return people having issues with apps that are unresponsive, or very rudimentary explanations of what the DWM does with such apps. Heck I'm not even 100% sure it's the DWM responsible, but it seems likely. Any potential leads?

Photo of problem; screen shots won't capture the effect (note that the white dialog's buffer is shifted -- it is shifted exactly by the distance it has been offset from the main (blue) window):

Answer 1

I have found that it is called 'ghosting' when it does this, and it was introduced in XP (though it seems DWM takes it a step further since it keeps track of every app's pixels). It is possible to disable it for the duration of your app's life with DisableProcessWindowsGhosting(). Some folks say that it is not recommended, but considering how badly it plays with layered windows, the downsides of turning it off are negligible. I've tested it and it does precisely what it claims; now if the server takes a little longer than expected to return, it won't mangle the entire interface.

If there is a better way to get my layered windows to work with Vista/7's handling of ghosting without disabling it outright, I'd certainly still be interested, but I'll probably just accept this as the answer otherwise.

Answer 2

This is standard behavior; you can also notice it in Office 2007 / 2010 when they freeze.

Answer 3

I think there are basically two things that will help with this:

First, like you've been doing, try not to hang in the first place: do as little heavy lifting in UI threads as possible.

Second, don't use multiple top-level windows to implement what the user is meant to see as just one; instead, do the compositing yourself. That will prevent the layers from being moved out-of-sync from one another.