I've just had this same issue and managed to fix it for myself so here's what I did.
I am inflating android.R.layout.simple_spinner_dropdown_item in my spinner adapter for each item layout. When I looked at this layout I noticed it had no margins but just a set height which was coming from android:attr/dropdownListPreferredItemHeight.
So I thought I'd get this list preferred item height value programatically and set it as my spinners minimum height. Using the answer to this stackoverflow question I was able to get the correct value and set it to my spinner.
android.util.TypedValue value = new android.util.TypedValue();
getActivity().getTheme().resolveAttribute(android.R.attr.listPreferredItemHeight, value, true);
android.util.DisplayMetrics metrics = new android.util.DisplayMetrics();
getActivity().getWindowManager().getDefaultDisplay().getMetrics(metrics);
float ret = value.getDimension(metrics);
mSpinner.setMinimumHeight((int) (ret - 1 * metrics.density));
I've removed 1dip because it seems that the returned value has been rounded up which makes it 1 pixel too high.
Now all my spinner are the same height, whether they have items or are empty.