Import data into Google App Engine in a way that is “easy” for the user of the application

Question

Building an application using Python in GAE that handles a lot of user data such as contacts, appointments, etc...

Would like to allow users to import their old data from other applications. For example an appointment might look like:

Start time          Duration  Service     Customer Id
2012-08-02 09:50AM, 01:00:00, Hair cut,   94782910,    
2012-08-02 10:50AM, 00:30:00, Dye job,    42548910, 
...

I'm unfamiliar with accepted practices with handling this type of situation. I also see issues with handling this on google app engine specifically because requests can not take longer than 30 seconds.

Ideally, it seems like users should be able to upload CSV files of their data via a web page, but I don't really know of a good way to do this with app engine.

Another way I can think of would be to let users cut and paste text directly into and HTML text area. Then javascript could be used to iterate the data and POST it to the server one row at a time or in small chunks. This sounds really sketchy to me though.

Any ideas on what a "good" way to handle this would be?

Thanks so much!

Answer 1

It all depends on how many rows your going to import, if its a moderate amount (up to 50 rows) you can do the import in the handler. If its a larger amount you will need to use a pipeline (which supports working with csv) and maybe send the user an email once the import has completed.

Answer 2

Have the user upload the file, then start a task that runs the import. Email results/errors to the user at the end. The other way I have done, is get the user to create the spreadsheet in google docs and have them supply the sheet key or link if it's published and then start a task that processes the spreadsheet directly from google docs.