Best hosting option for migrating off Google App Engine (GAE) in my case? [closed]

Question

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.

Closed 8 years ago.

I have a small Google App Engine site which seems to be outgrowing it, and I want to migrate somewhere else.

It is based on Java / Stripes Framework / Objectify, and only uses URLFetch from Google services. It uses ~60 front-end hours and ~4 GB datastore at the moment, with ~5k visitors doing ~25k page views per day.

Reasons why I believe I should migrate:

• I have made some assumptions early on which no longer are valid and am running into 1MB memcache/datastore limits. While I could refactor this it would likely increase the number of datastore operations / overall worsen performance characteristics, and involve a database conversion step (which I may as well use to migrate somewhere else)
• I want to add some features which would involve a significant increase in stored data (to ca 100 GB) and front-end hours
• As I'm using resources past the free quota, the increase in costs seem to rise quite fast. While they are quite manageable now, if the application becomes more popular, I'm afraid it may no longer be affordable.
• Some stability issues (getting some OutOfMemoryErrors and other errors that I cannot explain, and cannot replicate very well on my local environment)

I'm evaluating the following options:

1. Staying on GAE, optimizing the application, living with the growing costs (Cons: still will be having high costs and reliability issues)

2. Moving to AWS EC2 / EBS with MongoDB as a datastore (Pros: probably the most mature solution, Cons: appears difficult to set up, easy to make architecture/design mistakes).

3. Using Appscale to hopefuly largely leave my application as-is, but host it on AWS EC2 (Pros: seems easy on paper, Cons: seems to presume a largely Unix development environment, no idea if production ready or what is happening behind the scenes)

4. Use CloudFoundry.com with MongoDB as a datastore (Cons: no idea if production ready, post-beta costs are not known)

5. Get a VPS or a dedicated server with some hosting provider, deploy using MongoDB as a datastore (Cons: probably teaches me less of the things I want to learn than the other options, plenty of sysadmining to do)

It is a hobby site, so part of the goal is to also learn some new technologies in practice, I'd just want to invest my time in learning the right ones.

Notes - I have some, but quite limited system administration skills, especially on Linux, and I don't enjoy doing it. I have done a small project in MongoDB before (never put in production though). I've never used any of the AWS infrastructure.

My questions:

a. Is AppScale mature enough for the purpose of running a small website without much hassles (bugs, lack of documentation, etc.)? Is the learning curve very steep? How much system administration would using it in scenario #3 require? And most importantly - do I understand correctly that the Google 1MB and so forth limits are all there on AppScale?

b. Is CloudFoundry mature enough for the purpose of running a small website without much hassles (bugs, lack of documentation, etc.)? Is the learning curve very steep? How much system administration would using it in scenario #4 require? I presume moving off from CloudFoundry.com to anther CloudFoundry should be fairly easy if needed.

c. How much sysadmining does deploying on AWS EC2 / EBS involve for the described application? Assuming I don't care that much for temporary outages, but care about permanent data loss, do I need to mirror the EBS on my own, or can I just leave AWS to do it?

d. Which of the new options (AppScale, CloudFoundry, EC2/EBS) would work fine with a Windows / Eclipse based development approach? Which has the best Eclipse plug-ins?

I'm asking because upon quick review of AppScale docs, it seems they assume the development VM will be hosted by a Unix host, which is another hassle for me.

e. Which of my options 1.-5. would you recommend in my case?

I'm split between #2 and #4 at the moment.

Answer 1

Just some observation:

a. AppScale is a thin wrapper around other technologies (runtimes, datastores), so in general it's as reliable as those underlying parts are. For a small non-mission-critical website is IMO reliable enough. Btw, the memcache 1MB limit is per-object, not per-memcache. So I suppose you could break it up into multiple smaller objects.

b. I don't have experience with CloudFoundry, but they do say they are "beta" and they do not have SLA: http://support.cloudfoundry.com/entries/20971351-cloud-foundry-sla\

c. I'd guess a few hours a week. ESB is a disk-based service so you should not be loosing data with it. But you can still do incremental backups of ECB to S3. There are many solutions that do it automatically, for example: http://www.stardothosting.com/blog/2012/05/automated-amazon-ebs-snapshot-backup-script-with-7-day-retention/

d. IMO EC2 is the most mature with the most tools available. Note that AppScale is just a wrapper - you can deploy it to EC2. Dev environment (Eclipse+Windows) has nothing to do with deployment environment (Usually Linux, can also be Windows on EC2).

e. Personally I'd recommend staying with GAE (= option 1). IMHO anything else would be less reliable and more costly (due to setup/support costs, not even comparing base service costs).

Btw, if you are getting OutOfMemoryErrors you should really review your code. Where do you keep massive amounts of data in memory?