Well, I think I solved this thanks to you guys leading me in the right direction:
In the head, I put this:
<noscript>
<style>
[class*="fa-"]:before
{
content:"+";
}
</style>
</noscript>
This selects each fa-icon class item, which are all defined in the pattern:
.fa-heart:before {
content: "\f004";
}
And replaces the content with a generic character, or indeed anything else we might care to insert (including nothing).
Does this seem reasonable or is there some terrible side-effect I've missed here?
I realise there is a remote possibility that someone has font-loading blocked but javascript enabled, but for now I'm happy to ignore that customer demographic ;)
Edit to add:
After all this time this answer is still getting votes (thanks!), clearly it's still an issue, so I thought I'd post this link about greatly improving (and trimming) FontAwesome in a variety of ways: FontAwesome Fixed
My favourite is actually embedding selected icons as inline Base64 coded SVG, either in the HTML or your CSS. This seems pretty elegant and efficient, especially if you only need a couple of icons out of the set. FA still feels like a fudge to me.