I just checked the HTML code that was being generated for the LI item that I was adding and I saw that the LI did not have the class ui-screen-hidden class set for it. All the other LI items had this set.
So I modified the code as shown here
$(document).ready(function(){
$('ul').append('<li class="ui-screen-hidden"><a href="#">Sample</a></li>');
$('ul').listview('refresh');
});