I have two thoughts regarding this.
First: the rectilinear grid you are using presents a difficulty. That is, without looking too deeply at your code, growth along an x-, y-, or z-axis is likely proceed at a different rate than growth along a diagonal axis.
This problem is difficult to work around. If you allow d-26 connectivity (central cell connects to the cube of all cells around it), then how do you account for the (slightly) longer distance to diagonal cells? If you only allow growth along x-, y-, and z-axes, how do you account for the fact that cell in a diagonal direction now takes longer to get to than it should?
This may account for some of the fuzziness.
Second: Right now we are using the metric of "it looks irregular". But is it really? What we need is a good statistical test of how irregular it is versus how irregular we would expect it to be. Certainly it is the case that you wouldn't expect the edge of your blob to be completely spherical, since this is, after all, a random process.