I figured it out!
It turned out to be a conflict with Bootstrap/FontAwesome between my application and rails_admin.
I was using Bootstrap 3 along with FontAwesome 4 whereas the version of rails_admin I'm using (0.5.0) was using Bootstrap 2 with FontAwesome 3.
Since rails_admin was meant to use the old FontAwesome, it was using the old-style syntax (e.g. 'icon icon-pencil') instead of the new-style (e.g. 'fa fa-pencil'). Because of this, the icons were not showing up.
I resolved this by removing FontAwesome 4 from my app (I wasn't using it much anyway) and letting rails_admin use the version of FontAwesome is was bundled with.