Use empty string, null or remove empty property in API request/response

Question

When transferring object through an API, as in schemaless JSON format, what is the ideal way to return non-existent string property? I know that there are different ways of doing this as in examples in the listed links below.

• Avoid null
• Return null
• Remove empty property

I'm sure I have used null in the past but don't have a good reason to give for doing that. It seems straight forward to use null when dealing with the database. But database seems like an implementation detail that shouldn't concern the party on the other side of the API. E.g. they probably use a schemaless datastore that only store properties with values (non-null).

From a code point of view, restricting string functions to work only with one type, i.e. string (not null), makes them easier to prove; avoiding null is also a reason for having Option object. So, if the code that produces request/response doesn't use null, my guess is the code on the other side of the API won't be forced to use null too.

I rather like the idea of using an empty string as an easy way to avoid using null. One argument that I heard for using null and against the empty string is that empty string means the property exists. Although I understand the difference, I also wonder if it's the just implementation detail and if using either null or empty string makes any real life difference. I also wonder if an empty string is analogous to an empty array.

So, which is the best way of doing it that addresses those concerns? Does it depend on the format of the object being transferred (schema/schemaless)?

Answer 1

TLDR; Remove null properties

The first thing to bear in mind is that applications at their edges are not object-oriented (nor functional if programming in that paradigm). The JSON that you receive is not an object and should not be treated as such. It's just structured data which may (or may not) convert into an object. In general, no incoming JSON should be trusted as a business object until it is validated as such. Just the fact that it deserialized does not make it valid. Since JSON also has limited primitives compared to back-end languages, it is often worth it to make a JSON-aligned DTO for the incoming data. Then use the DTO to construct a business object (or error trying) for running the API operation.

When you look at JSON as just a transmission format, it makes more sense to omit properties that are not set. It's less to send across the wire. If your back-end language does not use nulls by default, you could probably configure your deserializer to give an error. For example, my common setup for Newtonsoft.Json translates null/missing properties to/from F# option types only and will otherwise error. This gives a natural representation of which fields are optional (those with option type).

As always, generalizations only get you so far. There are probably cases where a default or null property fits better. But the key is not to look at data structures at the edge of your system as business objects. Business objects should carry business guarantees (e.g. name at least 3 characters) when successfully created. But data structures pulled off the wire have no real guarantees.

Answer 2

Going with an empty string is a definitive no. Empty string still is a value, it is just empty. No value should be indicated using a construct which represents nothing, null.

From API developer's point of view, there exist only two types of properties:

• required (these MUST have a value of their specific type and MUST NOT ever be empty),
• optional (these MAY contain a value of their specific type but MAY also contain null.

This makes it quite clear that when a property is mandatory, ie. required, it can never be null.

On the other hand, should an optional property of an object not be set and left empty, I prefer to keep them in the response anyway with the null value. From my experience it makes it easier for the API clients to implement parsing, as they're not required to check whether a property actually exists or not, because it's always there, and they can simply convert the response to their custom DTO, treating null values as optional.

Dynamically including/removing fields from the response forces including additional conditions on the clients.

---

Either way, whichever way you choose, make sure you keep it consistent and well documented. That way it really does not matter what you use for your API, as long as the behaviour is predictable.

Answer 3

null Usage is Application/Language Dependent

Ultimately the choice of whether or not to use null as a valid application value is largely determined by your application and programming language/interface/edge.

At a fundamental level, I'd recommend trying to use distinct types if there are distinct classes of values. null may be an option if your interface allows it and there are only two classes of a property you're trying to represent. Omitting a property may be an option if your interface or format allows it. A new aggregate type (class, object, message type) may be another option.

For your string example, if this is in the programming language, I'd ask myself a couple questions.

1. Do I plan on adding future types of values? If so, an Option will probably be better for your interface design.
2. When do I need to validate consumers calls? Statically? Dynamically? Before? After? At all? If your programming language supports it, use the benefits of static typing as it avoids the amount of code you have to create for validation. Option probably fills this case best if your string was non-nullable. However, you're probably going to have to check user input for a null string value anyways though, so I'd probably defer back to the first line of questioning: how many types of values do/will I want to represent.
3. Is null indicative of a programmer error in my programming language? Unfortunately, null is often the default value for uninitialized (or non-explicitly initialized) pointers or references in some languages. Is null as a value acceptable as the default value? Is it safe as a default value? Sometimes null is indicative of deallocated values. Should I provide consumers of my interface with an indication of these potential memory management or initialization problems in their program? What's the failure mode of such a call in the face of such problems? Is the caller in the same process or thread as mine so that such errors are a high risk to my application?

Depending on your answers to these questions, you'll probably be able to hone in on whether or not null is right for your interface.

Example 1

1. Your application is safety critical
2. You're using some type of heap initialization on startup and null is a possible string value passed back upon failure to allocate space for a string.
3. There's a a potential such a string hits your interface

Answer: null is probably not appropriate

Rationale: null in this case is actually used to indicate two different types of values. The first may be a default value which the user of your interface may want to set. Unfortunately, the second value is a flag to indicate that your system is not functioning correctly. In such cases, you probably want to fail as safely as possible (whatever that means for your system).

Example 2

1. You're using a C struct which has a char * member.
2. Your system does not use heap allocation and you're using MISRA checking.
3. Your interface accepts this struct as a pointer and checks to make sure that struct does not point to NULL
4. The default and safe value of the char * member for your API can be indicated by a single value of NULL
5. Upon your user's struct initialization, you would like to provide the user the possibility of not explicitly initializing the char * member.

Answer: NULL may be appropriate

Rationale: There is a little chance that your struct passes the NULL check but is uninitialized. However, your API may not be able to account for this unless you have some sort of checksum on the struct value and/or range checking of the address of the struct. MISRA-C linters may help users of your API by flagging usage of structs before their initialization. However, as for the char * member, if the pointer to struct points to an initialized struct, NULL is the default value of an unspecified member in a struct initializer. Therefore, NULL may serve as a safe, default value for the char * struct member in your application.

If it's on a serialization interface, I'd ask myself the following questions about whether or not to use null on a string.

1. Is null indicative of a potential client side error? For JSON in JavaScript, this is probably a no as null isn't necessarily used as an indication of allocation failure. In JavaScript, it is used as an explicit indication of object absence from a reference to be set problematically. However, there are non-javascript parsers and serializers out there that map JSON null to the native null type. If this is the case, this begs the discussion of whether or not native null usage is okay for your particular language, parser and serializer combination.
2. Does explicit absence of a property value impact more than a single property value? Sometimes a null is actually indicating that you have a new message type entirely. It may be cleaner for your consumers of the serialization format to just specify a completely different message type. This ensures that their validation and application logic can have clean separation between the two distinctions of messages that your web interface provides.

General Advice

null can not be a value on an edge or interface that does not support it. If you're using something that's extremely loose in nature on the typing of values of properties (i.e. JSON), try to push some form of schema or validation on the consumers edge software (e.g. JSON Schema) if you can. If it's a programming language API, validate the user input statically if possible (via typing) or as loud as is sensible in runtime (a.k.a. practice defensive programming on consumer facing interfaces). As importantly, document or define the edge so there's no question as to:

• What type(s) of value a given property accepts
• What ranges of value are valid for a given property.
• How an aggregate type should be structured. What properties must/should/can be present in an aggregate type?
• If it's some type of container, how many items can or should the container hold, and what are the types of values the container holds?
• What order, if any, are properties or instances of a container or aggregates type returned?
• What side-effects are there of setting particular values and what are the side effects of reading those values?

Answer 4

Here my personal analysis of these questions. It is not backed by any book, paper, study or whatever, just my personal experience.

Empty strings as null

This is a no-go for me. Do not mix the semantics of empty string with that of non-defined. In many cases they may be perfectly interchangeable, but you can run into cases where non-defined and defined-but-empty do mean something different.

A kind of stupid example: let's say there is an attribute that stores a foreign key, and that attribute is not defined or is null, that would mean that there is no relation defined, whereas an empty string "" could be understood as a defined relation and the ID of the foreign record is that empty string.

Not defined vs null

This is not a black or white topic. There are pros and cons to both approaches.

In favour of explicitly defining null values, there are these pros:

• The messages are more self-descriptive in that you get to know all the keys just by looking at any message
• Related to the previous point, it is easier to code and detect errors in the consumer of the data: it is easier to detect errors if you fetch the wrong keys (maybe misspelling, maybe the API changed, etc).

In favour of assuming a non existing key equals the semantics of null:

• Some changes are easier to accomodate. For instance, if a new version of the message schema includes a new key, you can code the consumer of the information to work with this future key, even when the producer of the message has not been updated and does not serve this information yet.
• Messages can be less verbose or shorter

In case the API is somehow stable, and you thoroughly document it, I think it is perfectly OK to state that a non-existent key equals the meaning of null. But if it is more messy and chaotic (as it pretty often is), I think you can avoid headaches if you explicitly define every value in every message. I.e. if in doubt, I tend to follow the verbose approach.

All that said, the foremost important thing: state your intentions clearly and be consistent. Don't do one thing here and the other one there. Predictable software is better software.

Answer 5

I would supply an empty string in a situation where a string is present, and it happens to be an empty string. I would supply null in a situation where I want to explicitly say "no, this data is not there". And omit the key to say "no data there, don't bother".

You judge which of these situations can happen. Does it make sense for your application to have empty strings? Do you want to distinguish between saying explicitly "no data" by using null and implicitly by having no value? You should only have both possibilities (null and no key present) if both need to be distinguished by the client.

Now keep in mind that this is all about transmitting data. What the receiver does with the data is their business, and they will do what is most convenient to them. The receiver should be able to handle anything you throw at it (possibly by rejecting the data) without crashing.

If there are no other considerations, then I would transmit what is most convenient for the sender, and document it. I'd prefer not to send absent values at all because this is likely to improve the speed of encoding, transmitting and parsing JSON.

Answer 6

Although I cannot say what is best it is almost certainly not a simple implementation detail, it changes the structure of how you can interact with that variable.

If something can be null you should always treat it as if it will be null at some point, so you will always have two workflows, one for null, one for a valid string. A split workflow isn't necessarily a bad thing, as there's quite a bit of error handling and special-case-uses which you could utilise, but it does obfuscate your code.

If you always interact with the string in the same way it will probably be easier for the functionality to stay in your head.

So as with any "what is best" question I'm left with the answer: it depends. If you want to split your workflow and more explicitly capture when something is not set, use null. If you would rather the program just keep doing as it does, use an empty string. The important thing is that you are consistent, pick a common return and stick with that.

Considering you are creating an API, I would recommend sticking to an empty string as there's less for the user to compensate for, because as a user of an API I won't know every reason your API could give me a null value unless you're very well documented, which some users won't read anyway.

Answer 7

Document!

TL;DR>

Do as you see fit - sometimes the context in which it is used matters. Example, Bind Variables to an Oracle SQL: Empty String is interpreted as a NULL.

Simply I would say - Make sure you document each scenario mentioned

• NULL
• Blank (empty)
• Missing (removed)

Your code might act in different ways to it - document how your code reacts to it:

• Fail (Exception etc), maybe even fails validation (possibly a checked exception) vs can't handle the situation correctly (NullPointerException).
• Provide reasonable defaults
• Code behaves differently

Then in addition to that - it is up to you to behave consistently, and possibly adopt some of your own best practices. Document that consistent behavior. Examples:

• Treat Null and Missing the same
• Treat an empty string exactly as such. Only in the case of a SQL bind, it might be considered a blank. Make sure your SQL behaves consistently and in an expected way.

Answer 8

tl;dr - if you use it: be consistent in what it means.

If you included null, what would it mean? There is a universe of things what it could mean. One value is simply not enough to represent a missing or unknown value (and these are only two of the myriad of possibilities: e.g. Missing - it was measured, but we don't know it yet. Unknown - we didn't try to measure it.)

In an example I came across recently, a field could be empty because it was not reported to protect someone's privacy, but it was known on side of sender, not known at side of sender, but know to the original reporter or unknown to both. And all of these mattered to the receiver. So usually one value is not enough.

With an open world assumption (you simply don't know about things not stated), you would just leave it out and it could be anything. With closed world assumption (things not stated are false, for example in SQL) you better make clear what null means and be as consistent with that definition as you can...