How can I detect the text where the mouse pointer is, using C#?

Question

I'm looking for a way able to detect the text where the mouse pointer is, for whatever standard Windows app (IE, Firefox, Word, Notepad etc) the user is using at present. Is this possible ? (It's not for evil purposes, honest!). I've tried Googling and searching SO, but haven't got anything useful so far.

Answer 1

A few ideas:

1. use win32 api to find the control under the curser. Then use control dependent code to get the part of the text under the cursor. In simple cases a WM_GetText might be enough
2. I vaguely recall that there is an accessibility api doing exactly what you want. But I don't remember what is was called. http://msdn.microsoft.com/en-us/library/ms697707.aspx might be a starting point.
3. OCR, should not be too hard with known font

I'd try pursuing the accessibility idea.

Answer 2

Depending on what you point at, it can work in some cases.

eg. you could get the underlying window handle (hwnd) and send wm_gettext to it and you might get what you want back. (e.g. text on buttons, textboxes, some labels etc)

however, it will not do what you want if you point somehwere on a browser page or so..

Meh, CodeInChaos beat me to it..

Answer 3

No, it's not possible -- at least not in a general way. The only possibility is to query the application that put the text on the screen, in essence asking it "what did you put at this location?"

Answer 4

You could take a screen shot, then use OCR to translate the image to text. I think you can query the OS for the mouse cursor position, then correlate where it is in the image.

Not a good solution, but it gives you something.