Are null references really a bad thing?

softwareengineering.stackexchange https://softwareengineering.stackexchange.com/questions/12777

Question

I've heard it said that the inclusion of null references in programming languages is the "billion dollar mistake". But why? Sure, they can cause NullReferenceExceptions, but so what? Any element of the language can be a source of errors if used improperly.

And what's the alternative? I suppose instead of saying this:

Customer c = Customer.GetByLastName("Goodman"); // returns null if not found
if (c != null)
{
    Console.WriteLine(c.FirstName + " " + c.LastName + " is awesome!");
}
else { Console.WriteLine("There was no customer named Goodman.  How lame!"); }

You could say this:

if (Customer.ExistsWithLastName("Goodman"))
{
    Customer c = Customer.GetByLastName("Goodman") // throws error if not found
    Console.WriteLine(c.FirstName + " " + c.LastName + " is awesome!"); 
}
else { Console.WriteLine("There was no customer named Goodman.  How lame!"); }

But how is that better? Either way, if you forget to check that the customer exists, you get an exception.

I suppose that a CustomerNotFoundException is a bit easier to debug than a NullReferenceException by virtue of being more descriptive. Is that all there is to it?

Was it helpful?

Solution

null is evil

There is a presentation on InfoQ on this topic: Null References: The Billion Dollar Mistake by Tony Hoare

Option type

The alternative from functional programming is using an Option type, that can contain SOME value or NONE.

A good article The “Option” Pattern that discuss the Option type and provide an implementation of it for Java.

I have also found a bug-report for Java about this issue: Add Nice Option types to Java to prevent NullPointerExceptions. The requested feature was introduced in Java 8.

OTHER TIPS

The problem is that because in theory any object can be a null and toss an exception when you attempt to use it, your object-oriented code is basically a collection of unexploded bombs.

You're right that graceful error handling can be functionally identical to null-checking if statements. But what happens when something you convinced yourself couldn't possibly be a null is, in fact, a null? Kerboom. Whatever happens next, I'm willing to bet that 1) it won't be graceful and 2) you won't like it.

And do not dismiss the value of "easy to debug." Mature production code is a mad, sprawling creature; anything that gives you more insight into what went wrong and where may save you hours of digging.

There are several problems with using null references in code.

First, it's generally used to indicate a special state. Rather than defining a new class or constant for each state as specializations are normally done, using a null reference is using a lossy, massively generalized type/value.

Second, debugging code becomes more difficult when a null reference appears and you attempt to determine what generated it, which state is in effect and its cause even if you can trace its upstream execution path.

Third, null references introduces additional code paths to test.

Fourth, once null references are used as valid states for parameters as well as return values, defensive programming (for states caused by design) requires more null reference checking to be done in various places…just in case.

Fifth, the language's runtime is already performing type checks when it performs selector lookup on an object's method table. So you're duplicating effort by checking if the object's type is valid/invalid and then having the runtime check the valid object's type to invoke its method.

Why not use the NullObject pattern to take advantage of the runtime's check to have it invoke NOP methods specific to that state (conforming to the regular state's interface) while also eliminating all the extra checking for null references throughout your codebase?

It involves more work by creating a NullObject class for each interface with which you want to represent a special state. But at least the specialization is isolated to each special state, rather than the code in which the state might be present. IOW, the number of tests are reduced because you have fewer alternate execution paths in your methods.

Nulls aren't so bad, unless you're not expecting them. You should need to explicitly specify in code that you're expecting null, which is a language-design problem. Consider this:

Customer? c = Customer.GetByLastName("Goodman");
// note the question mark -- 'c' is either a Customer or null
if (c != null)
{
    // c is not null, therefore its type in this block is
    // now 'Customer' instead of 'Customer?'
    Console.WriteLine(c.FirstName + " " + c.LastName + " is awesome!");
}
else { Console.WriteLine("There was no customer named Goodman.  How lame!"); }

If you try to invoke methods on a Customer?, you should get a compile-time error. The reason more languages don't do this (IMO) is that they don't provide for the type of a variable to change depending on what scope it's in. If the language can handle that, then the problem can be solved entirely within the type system.

There's also the functional way to handle this problem, using option-types and Maybe, but I'm not as familiar with it. I prefer this way because correct code should theoretically only need one character added to compile correctly.

There's plenty of excellent answers that cover the unfortunate symptoms of null, so I'd like to present an alternative argument: Null is a flaw in the type system.

The purpose of a type system is to ensure that the different components of a program "fit together" properly; a well-typed program can't "off the rails" into undefined behavior.

Consider a hypothetical dialect of Java, or whatever your preferred statically-typed language is, where you can assign the string "Hello, world!" to any variable of any type:

Foo foo1 = new Foo();  // Legal
Foo foo2 = "Hello, world!"; // Also legal
Foo foo3 = "Bonjour!"; // Not legal - only "Hello, world!" is allowed

And you can check variables like so:

if (foo1 != "Hello, world!") {
    bar(foo1);
} else {
    baz();
}

There's nothing impossible about this - someone could design such a language if they wanted to. The special value need not be "Hello, world!" - it could've been the number 42, the tuple (1, 4, 9), or, say, null. But why would you do this? A variable of type Foo should only hold Foos - that's the whole point of the type system! null is not a Foo any more than "Hello, world!" is. Worse, null is not a value of any type, and there's nothing you can do with it!

The programmer can never be sure that a variable actually holds a Foo, and neither can the program; in order to avoid undefined behavior, it has to check variables for "Hello, world!" before using them as Foos. Note that doing the string check in the previous snippet doesn't propagate the fact that foo1 is really a Foo - bar will likely have its own check as well, just to be safe.

Compare that to using a Maybe/Option type with pattern matching:

case maybeFoo of
 |  Just foo => bar(foo)
 |  Nothing => baz()

Inside the Just foo clause, both you and the program know for sure that our Maybe Foo variable truly does contain a Foo value - that information is propagated down the call chain, and bar doesn't need to do any checks. Because Maybe Foo is a distinct type from Foo, you're forced to handle the possibility that it could contain Nothing, so you can never by blindsided by a NullPointerException. You can reason about your program much more easily and the compiler can omit null checks knowing that all variables of type Foo really do contain Foos. Everyone wins.

(Throwing my hat in the ring for an old question ;) )

The specific problem with null is that it breaks static typing.

If I have a Thing t, then the compiler can guarantee I can call t.doSomething(). Well, UNLESS t is null at runtime. Now all bets are off. The compiler said it was OK, but I find out much later that t does NOT in fact doSomething(). So rather than being able to trust the compiler to catch type errors, I have to wait until runtime to catch them. I might as well just use Python!

So, in a sense, null introduces dynamic typing into a statically typed system, with expected results.

The difference between that an divide by zero or log of negative, etc. is that when i = 0, it's still an int. The compiler can still guarantee its type. The problem is that the logic mis-applies that value in a way that isn't permitted by the logic ... but if the logic does that, that's pretty much the definition of a bug.

(The compiler can catch some of those problems, BTW. Like flagging expressions like i = 1 / 0 at compile time. But you can't really expect the compiler to follow into a function and ensure that the parameters are all consistent with the function's logic)

The practical problem is that you do a lot of extra work, and add null checks to protect yourself at runtime, but what if you forget one? The compiler stops you from assigning:

String s = new Integer(1234);

So why should it allow assignment to a value (null) that will break de-references to s?

By mixing "no value" with "typed" references in your code, you're putting an extra burden on the programmers. And when NullPointerExceptions happen, tracking them down can be even more time consuming. Rather than relying on static typing to say "this is a reference to something expected," you're letting the language say "this may well be a reference to something expected."

A null pointer is a tool

not an enemy. You just ought to use 'em right.

Think just minute about how long it takes to find and fix a typical invalid pointer bug, compared to locating and fixing a null pointer bug. It's easy to check a pointer against null. It is a PITA to verify whether or not a non-null pointer points to valid data.

If still not convinced, add a good dose of multithreading to your scenario, then think again.

Moral of the story: Don't throw the kids with the water. And don't blame the tools for the accident. The man who invented the number zero long time ago was quite clever, since that day you could name the "nothing". The null pointer is not so far away from that.

EDIT: Although the NullObject pattern seems to be a better solution than references that may be null, it introduces problems on its own:

  • A reference holding a NullObject should (according to the theory) do nothing when a method is called. Thus, subtle errors can be introduced due to errorneously unassigned references, which now are guaranteed to be non-null (yehaa!) but perform an unwanted action: nothing. With an NPE it is next to obvious where the problem lies. With a NullObject that behaves somehow (but wrong), we introduce the risk of errors being detected (too) late. This is not by accident similar to an invalid pointer problem and has similar consequences: The reference points to something, that looks like valid data, but isn't, introduces data errors and can be hard to track down. To be honest, in these cases I would without a blink prefer an NPE that fails immediately, now, over a logical/data error which suddenly hits me later down the road. Remember the IBM study about cost of errors being a function of at which stage they are detected?

  • The notion of doing nothing when a method is called on a NullObject does not hold, when a property getter or a function is called, or when the method returns values via out. Let's say, the return value is an int and we decide, that "do nothing" means "return 0". But how can we be sure, that 0 is the right "nothing" to be (not) returned? After all, the NullObject should not do anything, but when asked for a value, it has to react somehow. Failing is not an option: We use the NullObject to prevent the NPE, and we surely won't trade it against another exception (not implemented, divide by zero, ...), do we? So how do you correctly return nothing when you have to return something?

I can't help, but when someone tries to apply the NullObject pattern to each and every problem, it looks more like trying to repair one mistake by doing another one. It is without much doubt a good and useful solution in some cases, but it surely isn't the magic bullet for all cases.

Null references are a mistake because they allow non-sensical code:

foo = null
foo.bar()

There are alternatives, if you leverage the type system:

Maybe<Foo> foo = null
foo.bar() // error{Maybe<Foo> does not have any bar method}

The generally idea is to put the variable in a box, and the only thing you can do is unboxing it, preferably enlisting the compiler help like proposed for Eiffel.

Haskell has it from scratch (Maybe), in C++ you can leverage boost::optional<T> but you can still get undefined behaviour...

And what's the alternative?

Optional types and pattern matching. Since I don't know C#, here is a piece of code in a fictional language called Scala# :-)

Customer.GetByLastName("Goodman")    // returns Option[Customer]
match
{
    case Some(customer) =>
    Console.WriteLine(customer.FirstName + " " + customer.LastName + " is awesome!");

    case None =>
    Console.WriteLine("There was no customer named Goodman.  How lame!");
}

The problem with nulls is that languages that allow them pretty much force you into programming defensively against it. It takes a lot of effort (far more than trying to use defensive if-blocks) to make sure that

  1. the objects you expect them not to be null are indeed never null, and
  2. that your defensive mechanisms indeed deal with all potential NPEs effectively.

So, indeed, nulls end up being a costly thing to have.

The issue to what degree your programming language attempts to prove your program's correctness before it runs it. In a statically typed language you prove that you have the correct types. By moving to default of non-nullable references (with optional nullable references) you can eliminate many of the cases where null is passed and it shouldn't be. The question is whether the extra effort in handling non-nullable references is worth the benefit in terms of program correctness.

The problem isn't so much null, it's that you can't specify a non-null reference type in a lot of modern languages.

For example, your code might look like

public void MakeCake(Egg egg, Flour flour, Milk milk)
{
    if (egg == null) { throw ... }
    if (flour == null) { throw ... }
    if (milk == null) { throw ... }

    egg.Crack();
    MixingBowl.Mix(egg, flour, milk);
    // etc
}

// inside Mixing bowl class
public void Mix(Egg egg, Flour flour, Milk milk)
{
    if (egg == null) { throw ... }
    if (flour == null) { throw ... }
    if (milk == null) { throw ... }

    //... etc
}

When class references get passed around, defensive programming encourages you to check all parameters for null again, even if you just checked them for null right beforehand, especially when building reusable units under test. The same reference could easily be checked for null 10 times across the codebase!

Wouldn't it be better if you could have a normal nullable type when you get the thing, deal with null then and there, then pass it as a non-nullable type to all your little helper functions and classes without checking for null all the time?

Thats the point of the Option solution - not to allow you to switch on error states, but to allow design of functions which implicitly cannot accept null arguments. Whether or not the "default" implementation of types are nullable or non-nullable is less important than the flexibility of having both tools available.

http://twistedoakstudios.com/blog/Post330_non-nullable-types-vs-c-fixing-the-billion-dollar-mistake

This is also listed as the #2 most voted feature for C# -> https://visualstudio.uservoice.com/forums/121579-visual-studio/category/30931-languages-c

Null is evil. However, the lack of a null can be a greater evil.

The problem is that in the real world you often have a situation where you do not have data. Your example of the version without nulls can still blow up--either you made a logic mistake and forgot to check for Goodman or perhaps Goodman got married between when you checked and when you looked her up. (It helps in evaluating the logic if you figure Moriarty is watching every bit of your code and trying to trip you up.)

What does the lookup of Goodman do when she's not there? Without a null you have to return some sort of default customer--and now you're selling stuff to that default.

Fundamentally, it comes down to whether it's more important that the code work no matter what or that it work correctly. If improper behavior is preferable to no behavior then you don't want nulls. (For an example of how this might be the right choice consider the first launch of the Ariane V. An uncaught /0 error caused the rocket to make a hard turn and the self-destruct fired when it came apart due to this. The value it was trying to calculate actually no longer served any purpose the instant the booster was lit--it would have made orbit despite the routine returning garbage.)

At least 99 times out of 100 I would choose to use nulls. I would jump for joy at note to self's version of them, though.

Optimise for the most common case.

Having to check for null all the time is tedious - you want to be able to just get hold of the Customer object and work with it.

In the normal case, this should work just fine - you do the look up, get the object and use it.

In the exceptional case, where you're (randomly) looking up a customer by name, not knowing whether that record/object exists, you'd need some indication that this failed. In this situation, the answer is to throw a RecordNotFound exception (or let the SQL provider beneath do this for you).

If you're in a situation where you don't know whether you can trust the data coming in (the parameter), perhaps because it was entered by a user, then you could also provide the 'TryGetCustomer(name, out customer)' pattern. Cross-reference with int.Parse and int.TryParse.

Exceptions are not eval!

They are there for a reason. If your code is running bad, there is a genius pattern, called exception, and it tells you that some thing is wrong.

By avoiding using null objects you are hiding part of those exceptions. I am not spiking about OP example where he convert null pointer exception to well typed exception, this might actually be good thing as it increase readability. How ever when you take the Option type solution as @Jonas pointed out, you are hiding exceptions.

Than in your production application, instead of exception to be thrown when you are clicking on button to select empty option, nothing just happening. Instead of null pointer exception would be thrown and we would probably get a report of that exception(as in many production applications we have such mechanism), and than we could actually fix it.

Making your code bulletproof by avoiding exception is bad idea, making you code bulletproof by fixing exception, well this the path that I would choose.

Nulls are problematic because they must be explicitly checked for, yet the compiler is unable to warn you that you forgot to check for them. Only time-consuming static analysis can tell you that. Fortunately, there are several good alternatives.

Take the variable out of scope. Way too often, null is used as a place holder when a programmer declares a variable too early or keeps it around too long. The best approach is simply to get rid of the variable. Don't declare it until you have a valid value to put in there. This isn't as difficult a restriction as you might think, and it makes your code a lot cleaner.

Use the null object pattern. Sometimes, a missing value is a valid state for your system. The null object pattern is a good solution here. It avoids the need for constant explicit checks, but still allows you to represent a null state. It is not "papering over an error condition," as some people claim, because in this case a null state is a valid semantic state. That being said, you shouldn't use this pattern when a null state isn't a valid semantic state. You should just take your variable out of scope.

Use a Maybe/Option. First of all, this lets the compiler warn you that you need to check for a missing value, but it does more than replace one type of explicit check with another. Using Options, you can chain your code, and carry on as if your value exists, not needing to actually check until the absolute last minute. In Scala, your example code would look something like:

val customer = customerDb getByLastName "Goodman"
val awesomeMessage =
  customer map (c => s"${c.firstName} ${c.lastName} is awesome!")
val notFoundMessage = "There was no customer named Goodman.  How lame!"
println(awesomeMessage getOrElse notFoundMessage)

On the second line, where we're building the awesome message, we don't make an explicit check if the customer was found, and the beauty is we don't need to. It's not until the very last line, which might be many lines later, or even in a different module, where we explicitly concern ourselves with what happens if the Option is None. Not only that, if we had forgotten to do it, it wouldn't have type checked. Even then, it's done in a very natural way, without an explicit if statement.

Contrast that with a null, where it type checks just fine, but where you have to make an explicit check on every step of the way or your entire application blows up at runtime, if you're lucky during a use case your unit tests exercise. It's just not worth the hassle.

The central problem of NULL is that it makes system unreliable. In 1980 Tony Hoare in the paper dedicated to his Turing Award wrote:

And so, the best of my advice to the originators and designers of ADA has been ignored. …. Do not allow this language in its present state to be used in applications where reliability is critical, i.e., nuclear power stations, cruise missiles, early warning systems, antiballistic missile defense systems. The next rocket to go astray as a result of a programming language error may not be an exploratory space rocket on a harmless trip to Venus: It may be a nuclear warhead exploding over one of our own cities. An unreliable programming language generating unreliable programs constitutes a far greater risk to our environment and to our society than unsafe cars, toxic pesticides, or accidents at nuclear power stations. Be vigilant to reduce the risk, not to increase it.

ADA language has changed a lot since that, however such problems still exist in Java, C# and many other popular languages.

It is developer's duty to create contracts between a client and a supplier. For example, in C#, as in Java, you can use Generics to minimise the impact of Null reference by creating readonly NullableClass<T> (two Options):

class NullableClass<T>
{
     public HasValue {get;}
     public T Value {get;}
}

and then use it as

NullableClass<Customer> customer = dbRepository.GetCustomer('Mr. Smith');
if(customer.HasValue){
  // one logic with customer.Value
}else{
  // another logic
}

or use two options style with C# extension methods:

customer.Do(
      // code with normal behaviour
      ,
      // what to do in case of null
)

The difference is significant. As a client of a method you know what to expect. A team can have the rule:

If a class is not of type NullableClass then it's instance must be not null.

The team can strengthen this idea by using Design by Contract and static checking at compilation time, e.g with precondition:

function SaveCustomer([NotNullAttribute]Customer customer){
     // there is no need to check whether customer is null 
     // it is a client problem, not this supplier
}

or for a string 

function GetCustomer([NotNullAndNotEmptyAttribute]String customerName){
     // there is no need to check whether customerName is null or empty 
     // it is a client problem, not this supplier
}

These approach can drastically increase application reliability and software quality. Design by Contract is a case of Hoare logic, which was populated by Bertrand Meyer in his famous Object-Oriented Software Construction book and Eiffel language in 1988, but it is not in use invalidly in modern software crafting.

No.

A language's failure to account for a special value like null is the language's problem. If you look at languages like Kotlin or Swift, which force the writer to explicitly deal with the possibility of a null value at every encounter, then there is no danger.

In languages like the one in question, the language allows null to be a value of any-ish type, but does not provide any constructs for acknowledging that later, which leads to things being null unexpectedly (especially when passed to you from somewhere else), and thus you get crashes. That is the evil: letting you use null without making you deal with it.

Basically the central idea is that null pointer reference issues should get caught at compile time instead of run time. So if you are writing a function that takes some object and you don't want anyone to call it with null references then type system should allow you to specify that requirement so that compiler can use it to provide guarantee that call to your function would never compile if caller does not satisfy your requirement. This can be done in many ways, for example, decorating your types or using option types (also called nullable types in some languages) but obviously that's lot more work for compiler designers.

I think it is understandable why in 1960s and 70s, compiler designer didn't go all-in to implement this idea because machines were limited in resources, compilers needed to be kept relatively simple and no one really knew how bad this would turn out 40 years down the line.

I have one simple objection against null:

It breaks the semantics of your code completely by introducing ambiguity.

Oftentimes you expect the result to be of a certain type. This is semantically sound. Say, you asked the database for a user with a certain id, you expect the resut to be of a certain type (=user). But, what if there is no user of that id? One (bad) solution is: adding a null value. So the result is ambigous: either it is a user or it is null. But null is not the expected type. And this is where the code smell begins.

In avoiding null, you make your code semantically clear.

There are always ways around null: refactoring to collections, Null-objects, optionals etc.

If it will be semantically possible for a storage location of reference or pointer type to be accessed before code has been run to compute a value for it, there isn't any perfect choice as to what should happen. Having such locations default to null pointer references which may be freely read and copied, but which will fault if dereferenced or indexed, is one possible choice. Other choices include:

  1. Have locations default to a null pointer, but do not allow code to look at them (or even determine that they are null) without crashing. Possibly provide an explicit means of setting a pointer to null.
  2. Have locations default to a null pointer, and crash if an attempt is made to read one, but provide a non-crashing means of testing whether a pointer holds null. Also provide an explict means of setting a pointer to null.
  3. Have locations default to a pointer to some particular compiler-supplied default instance of the indicated type.
  4. Have locations default to a pointer to some particular program-supplied default instance of the indicated type.
  5. Have any null-pointer access (not just dereferencing operations) call some program-supplied routine to supply an instance.
  6. Design the language semantics such that collections of storage locations will not exist, except an an inaccessible compiler temporary, until such time as initialization routines have been run on all members (something like an array constructor would have to be supplied with either a default instance, a function that would return an instance, or possibly a pair of functions--one to return an instance and one that would be called on previously-constructed instances if an exception occurs during the construction of the array).

The last choice could have some appeal, especially if a language included both nullable and non-nullable types (one could call the special array constructors for any types, but one only be required to call them when creating arrays of non-nullable types), but would probably not have been feasible around the time null pointers were invented. Of the other choices, none seem more appealing than allowing null pointers to be copied but not dereferenced or indexed. Approach #4 might be convenient to have as an option, and should be fairly cheap to implement, but it should certainly not be the only option. Requiring that pointers must by default point to some particular valid object is far worse than having pointers default to a null value which can be read or copied but not dereferenced or indexed.

Yes, NULL is a terrible design, in object-oriented world. In a nutshell, NULL usage leads to:

  • ad-hoc error handling (instead of exceptions)
  • ambiguous semantic
  • slow instead of fast failing
  • computer thinking vs. object thinking
  • mutable and incomplete objects

Check this blog post for a detailed explanation: http://www.yegor256.com/2014/05/13/why-null-is-bad.html

Licensed under: CC-BY-SA with attribution
scroll top