How robust and reliable can leapmotion gestures be and how to configure them to the maximum?

Question

i'm developing a simple video player (in WPF and c# as it has to be for windows) that will have to be positioned in a public place (therefore with plenty of random untrained users). The controls will be extremely basic, swipe gestures to scroll among the videos, and once one of them is centered on the screen, a tap to play and pause it.

I finished the player with all its fancy stuff and i managed to attach the gestures to it, but hey, definitely not what i need yet. The gestures have plenty of misses (negative falses), specially the tap (but even if many less, also the swipe quite a lot i'd say).

Now, what i need is not at all a high precision (just left and right swipe and tap as i said, so no precise interaction with anything), but i need to get the gestures as reliable as possible, so that let's say if once in a while i get a miss it's no problem, but it doesn't have to be frustrating for the user as it is right now.

I tried playing around with the gesture config (lengths and speeds), but i don't really see a big difference, the default values looked almost to be the best ones. Is there anything else i can do? If that could help, is there any hint on how to develop new gestures, not based on the finger tips as they seem to be right now, but on the full hand movement? Being pretty new to it i'm not sure if that's feasible (i suppose it is) and if so how shall i start and proceed.

Also, do you think it's possible to reach the level of reliability i am looking for?

Answer 1

Swipes seem fairly reliable to me. Compare what you are seeing in your app with what you see in the diagnostic visualizer with "gesture drawing" turned on (press the o key). If the gestures work better in the visualizer, then you might be skipping frames and thereby missing some gesture objects. This will be the most apparent with tap gestures and if you rely on the STATE_START gesture state to detect the beginning of a swipe gesture.

I pretended I was using your app with the visualizer open and swipes worked fine for me; I rarely saw a miss. The only thing that might be problematic is a false return swipe when I move my hand back to swipe in the same direction again. Setting a delay between swipes might help that, though.

Still, if the main problem with your swipes are misses, then lowering the minimum distance and/or speed should improve things. If you aren't seeing much difference, then verify that these values are really getting changed; setting config values is asynchronous, so you have to wait a frame before reading them back. I would expect the problem to become false positives when these are set too low.

If swipes still don't work well enough for you, you could try simply tracking the x position of a hand with an extended finger (or two fingers, etc) and scroll when the position is past some boundary to the left or right. Clear affordances on the screen will help the user here.

Taps are harder to make reliable, especially when you are using more than one type of gesture. If you change the settings to make the tap recognition more permissive, you can easily get a lot of false positives. There should be ways to tune it to work, though. For example, ignore taps while a swipe is happening, or ignore taps that occur outside a predefined box centered on the y-axis, etc.

Alternatives to taps include using the Pointable class TouchDistance property. Many developers use a timed hover to activate buttons.

For other ideas, please see this article in the Leap Motion documentation: https://developer.leapmotion.com/documentation/GetStarted/Leap_Menu_Design_Guidelines.html

It discusses menus, but the general principles also apply in your case.