Oh hey, I figured it out almost immediately afterward. I need to add the following lines to web.config:
<httpHandlers>
<add verb="GET" path="*.woff" type="System.Web.StaticFileHandler" />
<add verb="GET" path="*.ttf" type="System.Web.StaticFileHandler" />
<add verb="GET" path="*.svg" type="System.Web.StaticFileHandler" />
...