Rewriting code / better initial outline

Question

Single-programmer project. I'm constantly having to rewrite my own code on an on-going project. It's not buggy, but every time I expand my program (which previously-planned features), I keep having to expand previous classes, which cause me to re-write code that worked fine before.

I have unit-tests, so bugs aren't introduced but it still seems counter-productive and seems to be wasting time. I'm constantly adding features that weren't originally needed, but become needed as project grows in complexity.

Should I continue with this iterative development? It seems with every iteration I keep having to re-write more and more of old code as new features are added.

Is there a better way to manage my project? I know (approximately) what I want my program to do, and how to do it.
Also, what tools / development-philosophies should I use to manage one-man projects? I have a running to-do list (and that's all) has been doing a fine job so far. Maybe some kind of abstract class-outlining program?