What is the best data access paradigm for scalability?

Question

There are so many different options coming out of microsoft for data access. Which one is the best for scalable apps?

Linq

Should we be using Linq? It certainly seems easy but if you know your SQL does it really help. Also I hear that you can't run Async queries in ASP.NET using Linq. Therefore I wonder if it is really scalable? Are there any really big sites using Linq (With the possible exception of stackoverflow).

Entity Framework

Don't hear so much razzmatazz about the Entity Framework. Seems closer to the Object model I'm familure with.

Astoria/Dynamic Data

Should we be exposing our data as a service?

I'm pretty confused and thats before I get into the other ORM products like NHibernate. Any ideas or wisdom on which is better?

Answer 1

I would recommend either NHibernate or Entity Framework. For large sites, I'd use ADO.NET Data Services. I wouldn't do anything large with LINQ to SQL. I think Stack Overflow might end up with some interesting scale problems being 2-tier rather than 3-tier, and they'll also have some trouble refactoring as the physical aspects of the database change and those changes ripple throughout the code. Just a thought.

Answer 2

I think ADO.Net Data Services (formerly called Astoria) has a huge role to play. It fits nicely with the REST style architecture of the web.

Since web is scalable, I guess anything which follows its architecure is scalable too.. Also, you might want to keep a lookout for SQL Server Data Services..

Answer 3

If you are talking about relational databases, then my vote is for encapsulating all of your data operations in stored procedures, regardless of how you access them from the other layers.

If you disable all read/write access to the database, except via stored procedures, you can hide your data model behind well defined contracts. The data model is free to change, just so the stored procedures still honor their inputs and outputs.

This gives DBAs total freedom to tune your application and make it scale. This is a very, very difficult task when SQL is being generated by a tool outside of the database.

Answer 4

Locking into stored procedures seems to be a waning way of thought these days, at least that have been my current observations. That way of thinking does lend itself to the ORM world since they are typically more affective going against tables directly but any ORM worth their salt will also allow the use of procs – sometime you have no choice.

There is plenty of opinions around EF and regardless of what anyone says, good or bad, it is a V1 product and with the rule of thumb that MS takes about 3 revs to get it right it might be prudent to wait for the next rev at least.

It seems that the biggest player out there in this space is NHibernate and there is plenty of support for this in the community. Linq, the language feature, shouldn’t be too far off in making its way to the NHibernate stack.

Answer 5

Use whatever works for you. These are all easiest to set up if you already have a fairly normalized database (ie, good definition of primary keys and foreign keys). However, if you've got data that doesn't easily normalize, the Entity Framework is more flexible than LINQ to SQL, but it does take more work to configure.

Answer 6

We've been experimenting with LINQ in a clustered environment and it appears to be scaling well on the individual machines and across the cluster. Of the 3 options that you've provided I would say that LINQ is the better choice although each option has a slightly different target audience so you should define what you will be doing with the data before deciding on the acesss paradigm.

Answer 7

I would suggest linq. It scales well on our site and is simple enough to use.

Answer 8

use stored procedures with LINQ...but don't let the sprocs turn into a data access layer!

Answer 9

This post is from 2008 before the cloud really took off. It seems like an update to the answer is required. I will just provide some links and an overview. I am sure that there are more up-to-date posts at this site on this topic, and if I find them, then I will add the links here.

When it comes to data scalability and transaction processing scalability, in 2017 we need to be talk about the Cloud and Cloud Service Providers.

I think the top three Cloud Providers these days are:

• Amazon Web Services (AWS)
• Google's Cloud Platform (GCP)
• Microsoft Azure

Cost

One of the great thing about using cloud services is that there are no upfront costs, no termination fees, and you pay only for what you use. (Quoting Mr.Alba's 2016 article "A Side-by-Side Comparison of AWS, Google Cloud and Azure")

We use AWS ourselves. We pay only while we have VMs installed and running, so it can be a cheap way to start up. Typically, service providers charge by the minute or by the hour but you are guaranteed to have it for that entire time.

A cheaper way to go is best-effort spot pricing. The Spot price represents the price above which you have to bid to guarantee that a single Spot request is fulfilled. When your bid price is above the Spot price, Amazon EC2 launches your Spot instance, and when the Spot price rises above your bid price, Amazon EC2 terminates your Spot instance. (Shamelessly quoting Amazon's User Guide here)

A Side-by-Side Comparison of AWS, Google Cloud and Azure is a good article doing a side-by-side comparison of these three service providers available here.

For a more academic look at cloud services, read the 2010 paper by Yu, Wang, Ren, and Lou "Achieving Secure, Scalable, and Fine-grained Data Access Control in Cloud Computing" in the INFOCOM 2010 Proceedings available here, but you may need to be an IEEE member to gain access to it. While it is somewhat dated, it is excellent and you can use it as a jumping off point.

Scaling in the cloud has been exploding, and until recently that scaling was done by starting up new Virtual Machines which tooks seconds, but with Containers one can spin up new instances in milliseconds. For more information on this, check out Docker and Docker Containers here.

I apologize for this answer being just a bunch of links for more information, but I thought the answer to this question should have an update. I hope this inspires someone to provide more first hand details. If you have already posted some related information, please consider providing links to your own posts. Thanks!