In SemVer, am I allowed to change the in-practice behaviour of undefined behaviour usages

Question

I maintain a small (tiny!) .NET library.

It has a few ill-defined edge cases, which I call out explicitly in the docs. Places, where the "correct" behaviour is not self-evidently well-defined, so I explicitly said

For invocations like this, the method may return any of the viable outputs.

Note that in all of these 'may return any' cases, I suspect that it will always return the top-most value, but that will be dependant on .NET's implementation of various methods and is not in anyway guaranteed by this library! Frankly, any of these would be a mis-use of the library and could arguably throw instead.

---

I want to make a change, to fix a bug, which will still satisfy the first sentence above, but which will happen to switch to return the bottom-most value. (Probably).

I think that I could release this as a patch, and still be honouring SemVer. Because even though the in-practice functionality has changed, it is functionality which the docs explicitly instructed users not to rely upon.

Have I correctly interpretted SemVer?

Answer 1

Non-contractual behavior is non-contractual.

Strictly speaking, it thus isn't something to be concerned about at all, and should only bump the patch-number in SemVer.

Unfortunately, some people always think they are clever when they exploit happenstance, instead of coding to the interface, and far more people don't actually know the contract as well as they think, or simply don't care. The documentation-writers might even have "helpfully" documented it for you!

So, it will probably still break some users, even though you are doing The Right Thing™.

In the end, you have to decide how much back-compat you will give to happenstance and common law features, thus blocking bug-fixes and improvements.

Answer 2

SemVer isn't a rule about what you are allowed to change, it is a signal to your consumers about how safe it is to consume an update. If code breaks on a patch update, your customers aren't going to care that technically you were "allowed" to make that change, they are going to stop trusting your patch updates.

If you think consumers are going to want to intentionally depend on undefined behavior, then define it. If you think consumers are going to want to spend extra time to ensure they or a transitive dependency doesn't accidentally depend on undefined behavior, then version it accordingly.

Answer 3

This type of unspecified behavior seems to be beyond what is defined in Semantic Versioning:

For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise.

The phrase "this system" refers to Semantic Versioning. The public API that your library provides is not only the implementation, but also includes the documentation of the API. Since this behavior is consistent with the documentation, on the surface it feels like this would be a backwards compatible change that would cause the PATCH version to be incremented.

However, note the last sentence in the description of what it takes SemVer to work - the API needs to be "clear and precise". Your API is not precise if the same version of your library may return different values in different environments. It seems like some change to the .NET framework libraries that are used will cause your library to behave very differently. As a user, I may see what I would think is a backwards incompatible change if I disregard your documentation. And this difference in perspective is why SemVer requires the precise API.

I'd recommend considering marstato's comment. Look at the design and implementation decisions that you made in the API. Is there a way to make this behavior consistent across environments that is still meaningful and then create a more clear and precise API?