I don't get your questin. Writing a custom adapter is the way to go in most cases. So you are on the right way. This adapter should use a datastructure what does what you want.
edit:
this should work right? (Item is my baseclass for dynamic data)
public class MyBaseListAdapter implements ListAdapter {
List<? extends Item> items;
@Override
public int getCount() {
return items.size();
}
edit II:
this line may also be improtant for you:
adapter.notifyDataSetChanged();
and you should try to sync or block access while you change the data