Finally I was able to remove the titlebar space. As it turns out, including this tag
android:theme="@android:style/Theme.Light.NoTitleBar.Fullscreen"
on application in manifest made the app fullscreen and corrected the issue. But I wanted the status bar to be visible in my app. So I had to compromise the status bar on the android versions in which the titlebar space was not reduced and let it be same on other versions. So here is how I resolved it.
in Manifest, I kept the code as it is
android:theme="@android:style/Theme.Light.NoTitleBar"
to remove the titlebar in the app
and in the activity which used the custom linear layout I checked the android version and kept the app fullscreen on versions less than and equal to Gingerbread.
if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.GINGERBREAD)
{
requestWindowFeature(Window.FEATURE_NO_TITLE);
getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN,
WindowManager.LayoutParams.FLAG_FULLSCREEN);
}