I put all the glue together thanks to a little testing and these two links:
http://support.appharbor.com/kb/getting-started/managing-environments http://scottonwriting.net/sowblog/archive/2009/05/21/163350.aspx
AppHarbor (I assume) is adding the SendGrid variables to <system.net> mailSettings in the web.config and Elmah will honor those settings. For a more secure configuration, you can add the useSsl="true" flag to <errorMail>
All I needed to add was the following:
<elmah>
<!-- AppHarbor clears App_Data on each deploy -->
<errorLog type="Elmah.XmlFileErrorLog, Elmah" logPath="~/App_Data" />
<errorMail from="error@a.com"
to="admin@a.com"
subject="Application Exception"
async="false"
smtpPort="0"
useSsl="true"
>
</errorMail>
<security allowRemoteAccess="true" />
</elmah>
AppHarbor and Elmah took care of the rest.