ORM vs Handcoded Data Access Layer

Question

I'm a bit scared to ask this question as it may start a religous war so I want to be really clear on what I'm looking for. I'm looking for a reason(s) why you would or have jumped one way or the other and also for items to add to my lists. I'm looking for the big ticket, big bang items. Also, items specific to a product, maybe, if they are really relevant. At this point I'm trying to evaluate ORM vs Manual not product A vs product B.

ORM Advantages

 - Quick to code and low maintenance (in some/most scenarios) 
 - Additional features for "free" (no developer effort)

Hand Coded Advantages

 - More Efficient (at runtime, maybe not at dev time?)
 - Less layers of complexity
 - Most ORMS seem to struggle with being retricted to sprocs only

In the interests of full disclosure, I really don't like the idea of "something" executing code against my database that I can't directly modify, if I see fit but I can see the potentially massive development time advatages of an ORM.

Its probably also worth noting I'm in a .Net world

[edit] (the question at Using an ORM or plain SQL? seems to answer many of the questions and reinforce the point about performance)

So, to alter my question slightly

Has any built an app using an ORM in the early stages and then gradually replaced with with a handcoded DAL? What were the pitfalls of this approach?

[Further Edit - getting to the heart of the problem now] Having a website be able to execute any SQL against my database is scary. If all access is through sprocs my database lives in nice, safe, comfortable isolation. Using exclusively sprocs removes a lot of, if not all, SQL injection attack vectors. Any comments on that?

Answer 1

We initially wrote an app using JPA ever since the day it went into production we have regretted it. The amount of database calls being made by the ORM were astronomical, so we have now started the process of piece-meal rewriting the application using good ol' fashioned JDBC utilizing Spring's JDBC helper classes. The pitfalls of starting with an ORM is that we spent a good deal of time learning JPA and at the end of the day we have to replace it with JDBC so our application can be more scalable without adding 3 other nodes to our Oracle RAC. So if you balance it out, the control and precision of JDBC was worth the extra lines of code you have to write. Also, most people do not understand that SQL is not something that can be generated and expected to perform. It is something that has to be written and tweaked to gain maximium performance.

Answer 2

I've used Subsonic for a couple of largish projects. I'm not advocating the use of one ORM over another, but I wouldnt do another database-related project without one. The ability to regenerate the entire db access layer whenever I change the database structure, the ability to add features in one place that affect the entire DB layer.

The thing is that you have to understand how it interacts with the database, otherwise you run the risk of writing (severely) under-performing code. Your ORM tool might give you a nice object-oriented view of your data, but you might find you're streaming whole tables to the client in order to do simple processing. The database is good at relating and filtering data, so make sure it still has that job. Ironically, Subsonic's lack of joins (in the version I used), helped with that, because it forced me to create DB Views to combine data where needed.

A friend of mine worked with a guy who developed an in-house ORM tool. It had the nice feature of locally caching everything that you could possibly want from the database, done by walking through the foreign keys. The downside was that reading one column from one row in the DB resulted in an excess of 11,000 select statements.

Answer 3

I've been writing and maintaining my own ORM for 8 years now. It started in Java, then translated to C#. I can't imagine writing any database backed system without it. It is way simpler than NHibernate, and doesn't have all its features but it gets the job done and it is quite fast even though it uses reflection extensively, since it replaces XML configuration with reflection over the DAO class definitions.

I'm quite happy with it, and would not use any other approach.

EDIT: regarding SQL injection attacks: a recent system I developed using this ORM was extensively audited and absolutely no injection was permitted. The reason is simple: it generates SQL on the fly and always uses parameters, no string concatenation.

Answer 4

A few weeks ago I started development on a new project. I had complete control over the tools I could use. I started with db40 because I wanted to eliminate this question altogether; I just wanted to persist my objects directly, forget about ADO.NET or OR/M. But db40 had problems so I ha to abandon it.

My next choice was ADO.NET because I thought it would be fast and easy. But I had to write way too much code, use too much "string sql" and it was just a chore altogether. I mean, it was a royal PITA and after coding two repositories with it I wanted to cut my wrists.

My third choice was NHibernate. I have had previous problems with NHibernate in a situation where I needed to use disconnected objects (such was the case this time as well, hence why it took me three tries to get to it). But that was NHibernate 1.2. This time I got the 2.0 binaries and had zero problems with updating disconnected objects. I also had to write way fewer lines of code and it was super-simple when I needed to refactor things, which was probably most important since my design changed rapidly.

I'm sold on NHibernate now. It appears highly optimized as well. Honestly, I can't find a negative to it. And there's no stored procedures.

I know it's my OR/M of choice going forward and I'll never write straight ADO.NET again.

Answer 5

I have struggled with this for a few years. Looked at a lot of things. And have over the last 3 months realized "why am I wasting all this time on this". I kind of view the ORM as a solved problem. I would rather trust some team that has complete focus on ORM to write an ORM layer than me. Mentally, I'm just ready for new challenges.

My choice is NHibernate. I'm really a newb to this right now. But I like the potential to use Fluent NHibernate or Castle ACtiveRecord also. This seems to be where the mindshare is.

But not sure what I would do in an "everything is a sproc" world.

Answer 6

Have you tried redbean? In development phase it's fluid and easy to use while in production mode (frozen) its fast. http://redbeanphp.com

Answer 7

I would argue that ORMs are only "quick to code" in the very beginning, particularly if you do not have an existing installed schema. Once you need to start squeezing more performance out of your system, you will need an ORM can be "quickly disposed of".

I find them to be a can of worms.

Answer 8

Stored Procs don't remove the risk of sql injection. The person that is likely to concat queries in code is just as likely to do so in a stored proc. And any good ORM doesn't do string concats anyway so there is no issue.

Answer 9

I have inherited a web app which was created with NHib and MyGeneration, unfortunately didn't get the svn repo, and no longer have the initial templates (arrggg).

Have kept nhibernate for the Create/Update/Delete back end stuff, but the front end (read only) was somewhat witlessly implemented, and runs like a 2 legged dog, and is now being rewritten in plain old ADO.NET, and goes up to 10 times faster.

Am not saying that this is due to NHibernate, it is due to developers not knowing how much crap they were sending along the wire, and assuming that blindly using a tool would mean they didn't have to think about it.

For the purpose of readonly queries, you can often make them much more efficient writing it by hand.

For things that have to write to the DB, is generally easier, and not that much slower, to let a decent ORM do it.

My personal preference is for SubSonic, which I have found performs pretty well for medium sized projects, and if you find a bottleneck, you can work around it by hand.

Tools are great, and anything that gives me more weekend is doubly great, but there is no substitute for the tool between your ears.

Answer 10

Comment on your Stored Proc question. No matter how the functionality gets written, DAL, ORM, stored procs, there will be SQL involved. If you use stored procedures you write the SQL inside, and it's relatively easy to build in useful abstractions. If you use DAL, you write the SQL, but in practice it's more likely to be unabstracted SQL with increased exposure to injection. If you use ORM, the tool writes the SQL (which you are nonetheless ultimately responsible for - including injectability). IMHO, the fewest moving parts (especially ones you know and understand) the better.

Answer 11

i had to rewrite a DAL of a production app which used an object database. We wanted to support both the object DB and sql databases. Some features i really didn't want to implement myself which where features present already were

• transparent lazyloading with querywise overrides
• Entity identity (always get the same reference for equal entities)
• Insert/Update batching
• Select batching (Futures in NHibernate)
• Id generation
• translating the query API to sql (criteria was a perfect due to the modular nature of the query API)

thats why i chose NHibernate as my ORM which i'm happy with. And i got some nice features for free.

• Automapping of classes (altogether 3 pages of code to generate the whole mappings)
• LINQ support (ever tried to implement LINQ-provider on your own?)
• extensive logging for free
• easier unit testing with inmemory sqlite
• support for different db-providers
• schema generation super easy (dont have to maintain any generation scripts for different db providers)
• optimistic concurrency
• documentation for N/Hibernate is a great learning resource for relational techniques

i can't quite imagine how to write better performing sql by hand than what NHibernate is generating. Some arguments against:

• load whole object when only few properties needed --> there is always Projecting into DTOs or in C# 3.0 into anonymous types (SetProjection() / Select())
• many selects instead of joins --> using eager loading where needed (SetFetchMode() / Fetch())
• tweaked sql is always more performant than generated --> architecture optimisation >> micro optimisation, with ORM it was much easier to refactor architecture because of fewer code and less repeated logic (try changing something like one-to-many to many-to-many in handwritten dal)

Answer 12

You don't need to abandon your SPROCs or hand crafted SQL with a good ORM like nHibernate.

I've replaced my ORM data access methods after the fact, it's not hard at all, it's the best of both worlds.

Answer 13

I have used Castle Project's Active Record implementation with NHibernate for a personal project. The project was never deployed, partly because of my inability to work with the ORM choice that I made.

The reason that I decided to use ORM was because I wanted the ability to switch from SQL Server to MYSQL without changing my code. In real projects the decision to use SQL server is already made for me, and it is easier to just write datalayers in the traditional 3-tier way when you know that the database won't change. But for personal projects I use shared hosting, and MySQL is a cheaper solution.

The reason I went with Castle was because it was very easy to use. With the activewriter plugin for visual studio, I was able to generate my classes and database using a designer (kind of like the linq to SQL designer), and it seemed to fit well with my mental model of how the ORM should dictate your application architecture.

The problem that I had with that ORM was that I wasn't able to find information on how to properly optimize it. It was making extremely large calls to the database. (The SQL query generated to get a user was 1mb of text when I logged it.)

So, I using the ORM, I saved myself a lot of work upfront, but I ran into a wall when I tried to fix the problem with the framework generating too much SQL. (On which I am still slowly working).

Even though I didn't find the solution perfect, I would still try ORM for other personal projects, because it's just me, and I want to spend my project time on fun code, not datalayers.

At work, however, I would probably not be too quick to suggest that we use an ORM, before I was an expert (Maybe after I get several working personal projects under my belt).

Answer 14

nice question and nice answers. I've just got involved with these issues too. I prefer an ORM like SubSonic to hand coded DAL.

Answer 15

Okay, this was for a PHP application, not a .net one, but I think the same qualities are important.

I re-wrote a PHP intranet app some years ago that was heavily database-based. As is typical of many PHP apps that have "just grown", it didn't use any sort of ORM and had a lot of duplicated code, much of it assembling similar queries in different ways.

I had a cursory look at some ORM frameworks, but they all had a big problem: they would have required customising them to work in our structure. The big missing component was database scalability. So I wrote my own handler. Then it was a simple matter to keep going and build our own object-based data layer. We got to implement features we wanted, too, that did what we needed.

In retrospect, this was a hugely useful exercise for me. It means I now know how ORM layers work and how they have to work. It was also bucking the trend of a lot of database scaling, because it raised the problem up into the application, where the knowledge about the data was.

Answer 16

I have been VERY successful simply using a DataSets in .Net. We have a SOAP endpoint that will first validate the input with an .XSD. Then, you can use a rules engine to do some validation. Then...

If the request is to insert data:

• Use GetXml() on the DataSet to convert to XML.
• Send the XML to a sproc that uses OPENXML in SQL Server.

If the request is to select data:

• Write a sproc to spit out the data with multiple SELECT statements.
• Load that data into a DataSet (being sure to build "nested" relationships where needed).
• Use GetXml() on the DataSet to covert to XML used in SOAP.

I've used this process with a great deal of success in several production environments. It's fast and simple if you know you'll only use SQL Server.