Fragment doesn't call onSaveInstanceState when swiping ViewPager?

Question

I'm having an issue with the ViewPager where my ListView is loosing it's scroll position.

The state of the ListView can easily be stored and restored using:

@Override
public View onCreateView (LayoutInflater inflater, ViewGroup container,
        Bundle savedInstanceState) 
{
    View v = inflater.inflate(R.layout.frag_cool_things, container, false);

    AdvListView listView = (AdvListView) v.findViewById(R.id.lv0);
    listView.setOnItemClickListener( mOnListItemClicked );

    if (null != savedInstanceState)
    {
        listView.onRestoreListViewInstanceState(savedInstanceState.getParcelable("list_state"));
    }

    mListView = listView;

    return v;
}

@Override
public void onSaveInstanceState (Bundle outState) 
{
    super.onSaveInstanceState(outState);

    outState.putParcelable("list_state", mListView.onSaveListViewInstanceState());
}

However the problem is that when fragments are being swiped onDestroyView() gets called but never calls onSaveInstanceState (Bundle outState).

Rotating the screen and such restores the ListView state just fine but swiping I can't figure out how to restore the list properly.