Should default configuration for deployed services be set as per production usage?

Question

Today, I and my colleague had a difference of opinion regarding the usage of default values in software configuration. We both agreed that for a consumer software, the default configuration should be determined by the most common usage and user experience.

However, the contention is in the usage of default values for enterprise-like deployed services where there are multiple deployment environments (production, dev, stage, etc). Think of default API hosts, endpoints, paths, constants, etc. as configuration variables. My opinion is that default values should be something which makes the most sense for production customers. Please note that this doesn't include authentication/sensitive credentials, certificates, etc as developers shouldn't have direct access to such production parameters.

Pros:

• Makes the job of TechOps easier. They rely on convention over configuration paradigm. TechOps can strictly focus on infrastructure and not application specific settings.
• A developer's environment tries to mimic a production environment. A developer easily knows what the values of specific settings will be, for most (if not all) production customers. He/she needs to look up the source and not have to ssh customer setups.

Cons:

• More often than not, developer's environment is not compatible with production because not every component is set up by each developer nor is every developer concerned about every component. It becomes developer's responsibility to set their configuration accordingly.
• It leads to writing defensive code which would otherwise never be needed for production scenarios. For example, in our case for a dev setup, an HTTP connection is retried because of default configuration and then given up after receiving a 404. In production setup, TechOps would have ensured the availability and correct endpoint.

A fair compromise, IMO, is to setup different default configurations for each environment and use scripts to deploy them. However, this isn't how our software is designed at the moment.

In today's age of CICD, (using docker, for example) how relevant is the convention over configuration paradigm. What are some preferred approaches to select such defaults?

EDIT: Interesting read is Default values - are they good or evil? which is similar but questions the existence of defaults, which isn't exactly the same.

EDIT2: For those considering downvoting, please consider leaving a comment on what should be improved or is wrong with the question. It'll be helpful for newcomers like me on this SE.

Answer 1

There should be no default configuration.

Using the production configuration as default is problematic because

• this config is necessarily incomplete (e.g. credentials)
• this config might accidentally lead to devs accessing production resources
• production config changes would require changes to the default config

Similarly, using a development configuration as default may lead the problems because

• this config might be used accidentally in production

Default configurations in general also have the issue that this configuration is maintained together with your code. At first this sounds great, but to a certain degree config and code are orthogonal. I should be able to experiment with config changes without having to commit a new version of the code.

The solution is not to get rid of any configuration, but to get rid of a default configuration: running the system then involves a conscious choice. Ideally, the configuration is provided through environment variables or comparable techniques . These variables should not refer to a mode like “development” or “production”, but either contain the configuration data directly (like a database connection string) or point to a config file.

Recommended reading: The Twelve-Factor App, especially the pages on config, treating backing services as attached resources, and build, release, run phases. It is perfectly fine to disagree with that website, but it represents a proven approach.

Note that the 12-factor concept of “config” is fairly narrow. The config describes the environment, not every tweakable option of the software. E.g. for a typical web app the database server is config, the directory containing HTML templates is not.

You will not get around of some kind of scripting during deployment of your software. Docker doesn't help or hinder with configuration, but the relevant part is that environment-configuration should not be part of the image, so that you can use the same image in development and production.

Answer 2

I would approach this from a risk-based stance.

Some development options can be detrimental or downright dangerous if set in a production environment. For those options, your default should be to set the production settings. These options are things like sidestepping caches, or enabling debug mode.

Conversely, some production settings should be avoided or barred from developers (especially if there is a division between development and operations). Things like production database credentials.

It leads to writing defensive code which would otherwise never be needed for production scenarios. For example, in our case for a dev setup, an HTTP connection is retried because of default configuration and then given up after receiving a 404. In production setup, TechOps would have ensured the availability and correct endpoint.

While in an ideal world that endpoint would be up 100%, there is no guarantee that it will be in the real world (in fact, it's almost certain to be unavailable at some point, from time to time). Network (resolving and routing) issues, power issues, hardware issues, maintenance time. They might not happen often, but they do happen. For example, I've seen stuff set up in development to have a very short timeout - but once the application is out in production, the response times vastly increased and the application fell over.

So - have sensible, safe, defaults and documentation on what needs to be configured for various environments.