Engineering approach for Metadata Driven Architecture

Question

Recently came across MDDA, I think the term was first coined by Salesforce (pg3) and others have followed it like Metadata Driven Architecture at benefitfocus.com (less dense and well written).

The design sounds useful for enterprise products where the platform is flexible enough to help customers tailor (by end user or a solution engineer) their experience but keeping core system behavior intact.

1. This might be a common pattern in enterprise SaaS apps, how do they let customers get the flexibility they want without adding a series of if-else logic?
2. What are some specific engineering design patterns on MDDA?
3. Are there any open source alternatives where I can look at the code and talk more concretely about it?
4. It should great in theory but in practice when a system is built with so much flexibility, what are some of the issues with this?

Answer 1

Oof, big topic. Yes, this can be a typical pattern in enterprise-class software, but I don't necessarily think it's a good one. Consider avoiding it if you can. The core issue here is that it exists at a super-high level of abstraction. When a tool doesn't know what the user wants to do, it inevitably abdicates the decision and gives you a framework for building your own tool. This starts off as a cool sales pitch about customizability, but in some ways MDA is the ultimate scope creep; if you become that flexible, really there's nothing your software can't do (in theory).

This is such a classic mistake that there are all kinds of jokes about it.

Zawinski's Law -- "all truly useful programs experience pressure to evolve into toolkits and application platforms"

Greenspun's Tenth Rule -- and others too.

Bottom line for me, MDA exists at a very high level of abstraction. Systems should avoid over-abstraction. I think it's too much to say categorically that you should never do this, sometimes it might work, but you better have a damn airtight case why it's necessary before you do it, otherwise I'd argue it's over-abstraction.

Specific engineering design patterns - really it's about creating database entities and relationships in a very abstract way, and allowing the user with a UI to specify and relate those things. On the UI side, same concept except instead of using database entities/relationships, you're using panels, inputs, forms, etc.

Open source alternatives - not sure on this one, but I would be wary of them if there are, because usually "super flexible" solutions end up getting customized to the problem domain (CRM, or whatever). Maybe some of the interface and data builders that come with Eclipse?

The issues that come with these approaches are many.

Testing the Beast

Because it's so abstract, testing becomes very hard. It's not possible to test every combination, so designing a representative covering suite is a very hard thing to do. That new feature you developed, does it interact poorly with Customer X's bizarre customization that was really working against how the tool was supposed to function in the first place?

Understanding Your Own System

Second, it allows the user to encode knowledge into the system that the system itself doesn't have. Imagine a back-end engineer looking at a customization of his own system and having a hard time recognizing where to start, because knowing your way around depends on knowledge outside of the system's specs. An earlier commenter pointed out that for salesforce, you have to hire special consultants to help you implement it. Yep, sounds right.

Inappropriate User Expectations

There's a general tradeoff between system power and simplicity; these systems are very powerful, and tend to be the opposite of simple. So basically all of the downsides of complexity come in the bargain for more flexibility. This also creates in the users and product managers a sense of "we can do anything" which makes it very difficult to set and understand proper system scope boundaries.