Droid TalkBack - Setting tab order for accessibility users

Question

In my droid application I have many views that have a TextView label that sits above a Button. My problem is that when a user is using TalkBack, they 'flick' the screen either right or left to navigate the screen.

For example, I need the focus to switch from the TextView label to the button underneath it when the user flicks to the right. Instead, it is navigating to the next element to the right of there label.

I have tried using "android:nextFocusDown="@+id/myButton", but this does not seem to work.

Any help is appreciated. Thank You!