real time number crunching and storage on cloud

Question

I have some hardware devices that send some data that need to be stored on the cloud server and also I need to do some real time processing on them.

The data they send need to be preserved for months in some custom binary files. These files related to each device can grow in size up to 10GB over time.

There will client programs (mobile / web) that will be looking at the processed data at real time.

My prefered choice of language is C/C++/C#, since there is time sensitive number crunching involved.

Goal is write scalable application that can have thousands of such devices monitored on the cloud.

1. Do I have to upfront write the code for running on the cloud ( undestand Azure / amazon EC2) ? Can I write multi threaded desktop application and later migrate to cloud ?

2. I have used Message passing interface (MPI) in the past for clusters. Can I still use MPI ?

3. If I use microsoft azure API can I still host my software on Amazon cloud ?

4. For mobile devices to talk to the server, I understand that I need to have a webservice running. how can I convert a desktop program writeen in C++ / C# to act as a web service talking to client?

5. Are there any 3rd part frame works or tools taht can help me with my work ?

Answer 1

1. With most cloud compute services you can deploy an off-the-shelf server and install your own software on it. So, yes, you can write and test you application locally then migrate to the cloud once you get all the bugs worked out. Here are the available EC2 server configurations.
2. I have not tried MPI but you should be able to run just about anything you want on the servers in the cloud. However, Amazon does offer the Simple Queue Service which provides message passing in the cloud. Your software does not need to run in the cloud to use this service.
3. I have not used Azure. I doubt there are any restrictions regarding which external servers you use for storage and/or compute. However, keeping your cloud storage and compute resources within a single provider will reduce costs, improve performance and provide you with a unified management interface and billing system.
4. Web servers are fairly simple things. See this post. That took me about 10 seconds to find.
5. There is plenty of third party software out there. Figure out what you need in more detail and ask more specific questions