Question

I've seen an application used to help the blind, it's called JAWS (it acts as a screen reader) it detects string and reads it in a lot of applications like MS Office applications, notepad, internet explorer.....etc. Is it possible to detect text selection in another application ?? how?i think accessibility is used but i don't know how to to it! i can replace selection detection with hotekey press. _ i tried to find a solution as the following

  1. get the top and activated window or from mouse location.
  2. get its child from mouse location.
  3. get the selected text or set it.

In ms word i used spy++ to detect the control that contains the text i get "Microsoft Word Document" enter image description here

Was it helpful?

Solution

There's no easy way to do this, because there's no single consistent way to get at text (selected or otherwise) from an arbitrary applications. Apps such as JAWS usually have a whole battery of techniques they use, depending on the app or control:

  • For EDIT and RichEdit controls, the various EM_ messages work.

  • For Internet Explorer, the HTML DOM can be used.

  • For Word, the Text Object Model interfaces can be used to access text and formatting. Other apps may support similar app-specific models.

  • Some (but not all) apps and app frameworks support Accessibility APIs such as UIAutomation or IAccessible2, which allows access to information about the controls in the app, and also information about text and text selection.

  • For apps that don't support any of the above, Screenreaders often use a technique called an Off-screen model, a complex technique that involves intercepting all graphical output calls, and maintaining an in-memory database of what was drawn where, so they can look up the text that should be at any point on the screen.

Since none of these covers everything in its own right, screenreaders typically try all of them as appropriate for the current app: you can almost think of a screenreader as being a library of special-case code to extract information from various apps.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top