Getting a working product quickly can often be the most important thing so taking a concious decision to skip over some engineering practices may be the right one.
However, you need to ensure you are making the right trade-offs. Refactoring is not free and dealing with technical debt needs to be planned for.
The path of least resistance once the end users see your initial version is usually to keep adding functionality over revisiting initial design decisions.
To put it another way, once version 1.0 is in the wild you will have a hard job persuading management that you need to spend a large number of man days reworking things under the hood for no perceivable change or benefit to the customer.
Without knowing the details of your app or requirements it's impossible to give concrete advice. In general though spending some time up front thinking about the design is orders of magnitude quicker and simpler than trying to do the same thing some way into development.