How do you decide if data should be stored persistently or calculated at runtime?

Question

I'm designing a service that contains different plans and features. I want to freely assign and remove features from plans. The service needs to tell the client both an ID's plan and the features it should have as a part of its plan. I came up with two ideas for how to accomplish this.

---

The first idea is to store the plans and features in a database. There would be one table for the plans, and there would be one table using the plans as foreign keys and an associated feature column.

| id | Plan |     | Plan | Feat |   
-------------     ---------------
| 1  | A    |     | A    | F1   |
| 2  | B    |     | B    | F1   |
| 3  | C    |     | A    | F2   |
                  | C    | F2   |

The advantage I see here is that as new features are added or removed, I only have to add code to support the new feature. The database will be able to store new plans and features without any changes.

---

The second idea is to only store the plan in the database and determine the features for each plan in the controller.

plans = {
   "A": {
       "F1": True,
       "F2": True
   }   
   "B": {
       "F1": True,
       "F2": False
   }
   "C": {
       "F1": False,
       "F2": True
   }
   "default": {
       "F1": False,
       "F2": False
   }
}

def determineFeatures(plan):
    return json.dumps(plans[plan]) if plans.get(plan) != None else json.dumps(plans["default"])

The advantage I see here is reducing the number of calls to the database. It also seems more transparent since the any dev can see the feature assignments, making it easier for debugging and adjusting.

---

In this case, which design (if either), is ideal for my needs? And what criteria should I use for determining if data should be stored or calculated at runtime?

Answer 1

what criteria should I use for determining if data should be stored or calculated at runtime?

• How frequently does the data change?
• Will changes in the data require changes in the code?
• How resource constrained is your environment?

Change Frequency

If the data changes minute to minute, or is expected to turn on a dime (could change at any time, even if it usually doesn't) then option 1 is obviously the only contender.

If the frequency of change is much slower, and is amenable to coinciding with a code release then Option 2 has some merit.

Implementation

While it would obviously be ideal to allow any feature to be combined with any feature, there is just no guarantee that a given set of features will work as desired.

Such a change may require an in depth verification of behaviour. To which hard coding the exact combinations of features, but allowing the window dressing to vary (option 2) is perfectly fine. It allows defects to be identified, fixed, and delivered with the new plan type.

Alternately if may pay to soft code the feature sets (option 1), and lock down access to that configuration. Testers can then verify each combination and release the plan type to the business if it is valid on the current code base without needing to ship it with a deployment, a simple db update script is all that is required. Any defects found can be feed into the next release which hopefully opens up the new plan type.

Constraints

Whether you like it or not, your software operates within a few budgets.

• Execution Time: How much CPU is available, and when
• Memory Space: How much memory is available, and when
• Memory Speed: How quickly does memory operate
• Storage Space: How much space is there for storing data
• Storage Speed: How quickly can data be stored/retrieved
• Network Bandwidth: How much data can be sent across the network in a period of time
• Network Speed : How long till the other end receives the data
• Parallel Space: How much distribution of behaviour is possible that can act simultaneously
• Turn Around Time: How much time is there to get the job done in
• Process Competition: How many other process are there out there that also desire to control the same budget
• Operation Time: How much and for how long can support personnel handle the system
• Maintenance Time: How much and how long does it take to keep the system current
• Development Time: How much and how long does it take to offer a substantive change/addition/removal of functionality from the system

To name a few. Without specifically knowing which is your tightest budget, whether one option is clearly superior to the other is hard to tell.

Often neither option is clearly superior. In which case you are best to pick the design that allows you to change your mind later, often when you are running near the limit of a budget.

When none of your options are suitable its time to go back and take a look at the whole system. Something will have to give. Either the feature cannot be supported currently, or something else will need to change.

Otherwise there will be a clearly superior option, go with it.

Answer 2

I would say it's both simpler and more complex that the technical reasons you are focusing on.

If you implement a cache the two options will perform virtually identically. The real question you need to ask yourself is : Is the hardcoded data part of the application, or something the application works on.

If you have requirements like "As a user I want to add a new plan..." then obviously you can't hardcode the plans. A user can't edit the code and redeploy

If you have one like "As a user I want to select the country from a drop down" its less clear. You don't have a requirement to add/remove countries... yet. But one might come up just around the corner.

As a developer you usually have to understand the business and think ahead to many of these problems, even if agile methodologies would have you just implement the requirements in the quickest way possible. No one wants to hear "Oh you didn't say you wanted to be able to open the doors." or similar.

So if your application is an ecommerce store, you probably know that maintaining an up to date database of geopolitical regions is out of the scope and you can just hardcode the list. Where as if you are making a Mapping application, adding and removing countries could well be a critical part of the app.

Answer 3

How do you decide when to persist data instead of calculating it?

A simple answer to that question is that you persist the calculation when it's detected as a bottleneck for your application.