What kind of layout are you using, could it be RelativeLayout? that might be your problem in this particular instance.
Anyway, in android there are many screen sizes, each of them you should handle with different layouts depending on orientation, size and density. There is no magic formula as far as I know, but 9-patch images can help a lot. Another thing you should do is make sure that you understand how match_parent and wrap_content values work for height and width.
Check this out: Suppoting different screens