When to design for scalability?

Question

I am about to start a project that I hope will some day require the capacity for scaling. The key question for me is: should I invest the effort to design for this eventuality now, or should I cross that bridge when I get there?

I know how these things go: everyone thinks their project might scale, but most projects never do.

Is there a set of best practices that will allow you to scale more easily at a later stage, while not investing too much in something that may never be needed?

Obviously I have no experience with scalable web sites. Currently I am leaning towards Scala / Play! / Akka. From what I can glean from the Akka website, it is very suitable for this kind of project (in that it provides a toolset that allows development on a single machine and scaling out to an arbitrary number of machines).

The project is a consumer-facing web site that involves some user interaction (comments, messages, voting etc.). The main focus is editorial, though. It's no Facebook :)

Not being experienced in any of these technologies (my experience mainly coming from PHP, AS3, Objective-C), I probably have a little too much on my plate right now. But then I am not going to go at it right away. I am aware that I need to get some experience using Scala and Play! first.

Any advice is greatly appreciated.

Answer 1

Start out by simply designing you model, build unit tests for that and then set your presentation layer on top of it. As long as your model is sensible, it will be easy to scale Play out to any number of machines. If you go for the build in JPA support, you can always deal with the question of which DB to go for.

You have larger things on your plate for the time being. So just make sure your design is consistent and sensible, then scaling will not be a problem.

Answer 2

You've got a head start by using a stateless web framework like Play! that won't get in the way when you need to scale. As ExxKA says, keep your model clean and sensible. This will help you keep the advantage of complexity.

Don't consider it a failure when you need to refactor your code - or even rewrite important parts of it. This is a natural part of a growing project, like a snake shedding its skin.

There are inevitably things that you'll learn in the process of writing the project, so don't try to anticipate them all right now.