I found that the best was to accomplish this was to abandon my own code and use Android's Expandable List View adapter. Very convenient. I followed the example given at this link: http://www.androidhive.info/2013/07/android-expandable-list-view-tutorial/
There is too much code to post here, but it's similar to a standard Android adapter implementation. The docs are here: http://developer.android.com/reference/android/widget/ExpandableListView.html