Is it possible to see remaining valid choices during speech recognition?

Question

I am working with System.Speech.Recognition.Choices class, and I would like to have a way to present to the user the possible choices remaining for the given speech session (I'm not sure of the correct term here).

For example, if the choices are "thermal", "thermite", and "off", then I would like to show a list with all three. If the user starts the "th" sound, I would like to show only the first two, and as the user continues have the other choices fall off. Basically, I want to display the remaining possible choices.

Is this possible?

Thanks! Erick

Answer 1

Depending on your grammars the System.Speech.Recognition.RecognitionResult should have an Alternate property that displays a list of ordered recognitions that are alternatives to the recognition with the highest confidence. Many speech enabled applications use this to disambiguate possible mis-recognitions.

// Handle the SpeechRecognized event. 
void SpeechRecognizedHandler(object sender, SpeechRecognizedEventArgs e)
{
  if (e.Result == null) return;

  // Display the recognition alternates for the result.
  foreach (RecognizedPhrase phrase in e.Result.Alternates)
  {
    Console.WriteLine(" alt({0}) {1}", phrase.Confidence, phrase.Text);
  }
}

Another possible solution (that works well in some scenarios) is to use the garbage rule in your grammars to act as a wild card (this would not work well with your th example above). The following would recognize something like "open my message please" because of the wildcards (aka GARBAGE rules).

<rule id="WildcardExample" scope="private">
    <item>open</item>
    <ruleref special="GARBAGE"/>
    <item>message</item>
    <ruleref special="GARBAGE"/>    
</rule>

Yet another option is to specify choices (using the one-of tag) in your grammar and use alternates. For instance:

<rule id="ListExample" scope="private">
    <item>I'd like a</item>
    <one-of>
        <item>chocolate</item>
        <item>coco</item>
        <item>vanilla</item>
    </one-of>
    <item>please</item>
</rule>

If the user said "I'd like a Chocolate please" you can use the confidence values for the alternates to check and see if you need to display a dialog box asking "did you want chocolate or coco?"

// Handle the SpeechRecognized event. 
    void SpeechRecognizedHandler(object sender, SpeechRecognizedEventArgs e)
    {
      if (e.Result == null) return;

      // Display the recognition alternates for the result.
      foreach (RecognizedPhrase phrase in e.Result.Alternates)
      {
        if (.2 > Math.Abs(e.Result.Confidence-phrase.Confidence && phrase.Text.Equals("coco"))
        {
           //Set chocolate or coco grammar/rule
           Console.WriteLine(" Did you mean chocolate or coco?");
           break;
        }
      }
    }

• Decent MSDN link for Grammar elements

Your final options (although not well documented at all) is to use Hypothesis events. The Speech Engine basically calls this callback while it is in the process for generating a recognition result. This would fit your TH example above. However in practice it is hard to design a UI that keeps up with these events and still provides a good (aka non sluggish) user experience.