On i5 x64 even imul seems to about 40% faster [than double multiplication]. Integer addition should also happen in fewer cycles / better throughput. About the "inexactness" problem you should be aware that doubles can be more inexact than integers.
Calculate which numbers cause problems when converting decimal to floating point?
If you have access to the original data (e.g. decimal representation of the weights, multiplying them with a large power of ten should produce exact integers without any rounding artifacts. With long longs the only concern will be that of an overflow.
How to address possible rounding instability depends on the dynamic range of your weights, and the maximum number of iterations. E.g. if your weights are all less than 1.0 and larger than 2^-52, then multiplying with 2^52 gives exact integers with no rounding errors. Then the "instability" is determined by the possibility of an overflow. (2^12 * 2^52) >= 2^64.