Premature optimization. You'll load the model exactly one time. And atof is not a very complex operation to begin with (it's basically just a loop of char const *c = instring; while(isdigit(*c)){ v *= 10; v += *c - '0'; c++;}
I wonder, would it be preferable to store normals in this custom binary object format, or to generate them upon loading?
Load them. Normals are something you want the modeling artist being able to fine tune.