The issue is that the images change the height of the div. You can keep them from doing that by floating them. Also, if you want them to always fit within the container, you'll need to adjust the width of the images. So, something like this might work:
#cool img {
float:left;
width:25%;
height: auto;
}
In the interest of creating semantic HTML, you may want to change your markup to something more like the following:
<nav>
<ul>
<li><a href="#"><img src="/my/cool/image.png" alt="Menu item 1"/></a></li>
<li><a href="#"><img src="/my/cool/image.png" alt="Menu item 2"/></a></li>
<li><a href="#"><img src="/my/cool/image.png" alt="Menu item 3"/></a></li>
</ul>
</nav>