NULL: The worst mistake of computer science? (2015)(lucidchart.com)
lucidchart.com
NULL: The worst mistake of computer science? (2015)
https://www.lucidchart.com/techblog/2015/08/31/the-worst-mistake-of-computer-science/
368 comments
You know who works on a platform with NULL but doesn't have quite so many problems with it? DBAs.
There's some need to draw a distinction between the basic idea of NULL, and the way that NULL has been implemented in most high-level programming languages.
In most RDBMSes, values can't be null unless you say they are. Sometimes explicitly, as in table definitions, sometimes implicitly, when you select a JOIN type. Either way, though, the fact that the developer is in control of when it can and cannot happen means that it always has a knowable meaning. (Or should, anyway.)
The problem with many programming languages is, you're given it whether you want it or not. In a low-level language like C, that's reasonable, because it takes a sensible approach to how it works: Only pointers can be null, and all pointers are nullable for obvious (especially in the 1970s) reasons.
More generally, I'm not going to fault languages from that era for trying it out, because this stuff was new, and things were still being felt out. So I don't really fault Tony Hoare for giving null references a try in ALGOL W.
What seems much more bothersome is high level languages like Java and C# cargo culting this behavior. They could have followed the lead from languages like SQL and let the programmer be in control. They should have. They already throw exceptions when a memory allocation fails, and they allow inline variable initialization, and declaring variables at the point of usage, and composite data types have constructors, so they lack all of (early) C's reasons why ubiquitous nulls were a good idea. They could have, I think quite easily, made nullability optional. At which point it'd have basically the same semantics as optional types from functional programming, so I doubt we'd be worrying about it anymore.
But they didn't.
There's some need to draw a distinction between the basic idea of NULL, and the way that NULL has been implemented in most high-level programming languages.
In most RDBMSes, values can't be null unless you say they are. Sometimes explicitly, as in table definitions, sometimes implicitly, when you select a JOIN type. Either way, though, the fact that the developer is in control of when it can and cannot happen means that it always has a knowable meaning. (Or should, anyway.)
The problem with many programming languages is, you're given it whether you want it or not. In a low-level language like C, that's reasonable, because it takes a sensible approach to how it works: Only pointers can be null, and all pointers are nullable for obvious (especially in the 1970s) reasons.
More generally, I'm not going to fault languages from that era for trying it out, because this stuff was new, and things were still being felt out. So I don't really fault Tony Hoare for giving null references a try in ALGOL W.
What seems much more bothersome is high level languages like Java and C# cargo culting this behavior. They could have followed the lead from languages like SQL and let the programmer be in control. They should have. They already throw exceptions when a memory allocation fails, and they allow inline variable initialization, and declaring variables at the point of usage, and composite data types have constructors, so they lack all of (early) C's reasons why ubiquitous nulls were a good idea. They could have, I think quite easily, made nullability optional. At which point it'd have basically the same semantics as optional types from functional programming, so I doubt we'd be worrying about it anymore.
But they didn't.
> NULL is a value that is not a value. And that’s a problem.
The problem isn't NULL, it's languages not enforcing the necessary checks for the "no data" condition. Option can still be NULL ("None" in rust), wrapping NULL in a struct doesn't provide any safety. The safety of Option wrapper types is from the other language features (like rust's "match") and a stricter compiler that forces the programmer to write the NULL check.
NULL would be fine if C required you to write this:
The problem isn't NULL, it's languages not enforcing the necessary checks for the "no data" condition. Option can still be NULL ("None" in rust), wrapping NULL in a struct doesn't provide any safety. The safety of Option wrapper types is from the other language features (like rust's "match") and a stricter compiler that forces the programmer to write the NULL check.
NULL would be fine if C required you to write this:
foo_t *maybe_get_foo(/*...*/) {
if (/*foo_is_available*/) {
return foo;
} else {
return NULL;
}
}
foo_t *f = maybe_get_foo();
if (!f) { /*...*/ } // REQUIRED or compile error
do_something(f->bar); // only allowed after NULL check
I've made my peace with null. Null is basically just an implicit
If you write your program with the "blow up early" mentality anway, or use static checking tools and a bit of discipline, I've found that null looses it's terror.
assert(valid(x))
If you write your program with the "blow up early" mentality anway, or use static checking tools and a bit of discipline, I've found that null looses it's terror.
Isn't there an inherent need in programming to express an explicit "nothing" value? Coming from Python and JS, I never found None/null to be much of a problem. I in fact like the distinction of null and undefined in JS. Using null allows you to distinguish from the accidental undefined.
Progress, it's now a solved problem in modern languages like Rust, Swift or Kotlin
See for example: https://kotlinlang.org/docs/reference/null-safety.html
It is not possible to have a NULL type that works for all situations and has stable semantics.
The issue is, NULL should be a concept, not a value. I see no problem with using sentinel values, so long as they are well designed, and such good design comes with skill and experience, just as with all other aspects of architecture. The quest to have a single value that can be used for all the various possible meanings of NULL, to me, is the root of the problem.
The issue is, NULL should be a concept, not a value. I see no problem with using sentinel values, so long as they are well designed, and such good design comes with skill and experience, just as with all other aspects of architecture. The quest to have a single value that can be used for all the various possible meanings of NULL, to me, is the root of the problem.
I agree with pretty much everything in the article. However, I would give Java a lower score because no one uses java.lang.Optional in practice, and there is too much legacy libraries and application code that cannot or will not be changed. Also, the @NotNull annotation isn't in Java SE; it is made available through various third-party libraries.
A language with a null value can dramatically simplify things for a language designer, though. In the case of Java, we know that every array of objects is initialized to null references. Thereafter, we can construct and assign objects to each slot of the array. Otherwise we run into issues that C++ faces - when we construct the array, the field of every object is uninitialized, so they are potentially dangerous if read or destructed, and need the special syntax of placement new to be initialized. The trick to avoiding null here is to avoid pre-allocating an array, and instead to grow a vector one element at a time. The C++ std::vector<E> is very accessible and performant, whereas Java java.util.List<E> is very clunky to use compared to native arrays.
Another case that gets simplified is object construction. When the memory for an object has been allocated but before the user's constructor code has run, what values should the fields have, assuming that they are observable? In a Java constructor, all fields are initially set to null/0, then you simply assign values to fields in the body code of the constructor. In C++ constructors however, you should initialize fields in the initializer list, and then you have still have the option to initialize fields in the body.
I still think pervasive null values are bad for the programmer (rather than the language designer). Now that I have preliminary experience in Rust, I see that its design is much safer and still practical, so I think this language shows the way forward.
A language with a null value can dramatically simplify things for a language designer, though. In the case of Java, we know that every array of objects is initialized to null references. Thereafter, we can construct and assign objects to each slot of the array. Otherwise we run into issues that C++ faces - when we construct the array, the field of every object is uninitialized, so they are potentially dangerous if read or destructed, and need the special syntax of placement new to be initialized. The trick to avoiding null here is to avoid pre-allocating an array, and instead to grow a vector one element at a time. The C++ std::vector<E> is very accessible and performant, whereas Java java.util.List<E> is very clunky to use compared to native arrays.
Another case that gets simplified is object construction. When the memory for an object has been allocated but before the user's constructor code has run, what values should the fields have, assuming that they are observable? In a Java constructor, all fields are initially set to null/0, then you simply assign values to fields in the body code of the constructor. In C++ constructors however, you should initialize fields in the initializer list, and then you have still have the option to initialize fields in the body.
I still think pervasive null values are bad for the programmer (rather than the language designer). Now that I have preliminary experience in Rust, I see that its design is much safer and still practical, so I think this language shows the way forward.
That 'nothing' is inconvenient applies the same in mathematics with zero. Why do we have to have a number we can't divide by? What is 0 to the power of 0? It's a special case we always need to worry about. But its inclusion in the number system is not in question.
And I remember my distress using a financial package being told that my unused zero value still MUST have a currency! My pocket is empty, how can it have a currency? If a farmers field is empty - must I say what it is empty of - cows, sheep, aardvarks?
I think worrying about inconsistency here is worrying about the inconsistency of the world we live in. 'Nothing' is a mysterious thing we need to accept and respect.
And I remember my distress using a financial package being told that my unused zero value still MUST have a currency! My pocket is empty, how can it have a currency? If a farmers field is empty - must I say what it is empty of - cows, sheep, aardvarks?
I think worrying about inconsistency here is worrying about the inconsistency of the world we live in. 'Nothing' is a mysterious thing we need to accept and respect.
Rich Hickey's "Maybe Not" should be watched by anyone who thinks nulls/nils/undefineds are okay. It should also be watched by anyone who think that Optional/Maybe/Nullable and good enough:
https://www.youtube.com/watch?v=YR5WdGrpoug
https://www.youtube.com/watch?v=YR5WdGrpoug
To someone who has been using Asm and C for decades, these arguments just make no sense. Reading this article reminds me of the arguments against pointers, another thing that's frequently criticised by those who don't actually understand how computers work and try to "solve" problems by merely slathering everything in thicker and thicker layers of leaky abstraction. It's not far from "goto considered harmful" either.
any reference can be null, and calling a method on null produces a NullPointerException.
...which immediately tells you to go fix the code.
There are many times when it doesn’t make sense to have a null. Unfortunately, if the language permits anything to be null, well, anything can be null.
That's not an argument. See above.
3. NULL is a special-case
...because it indicates the absence of a value, which is a special case.
though it throws a NullPointerException when run.
...and the cause is obvious. I'm not even a regular Java user (and don't much like the language myself, but for other reasons) and I know the difference between the Boxed types and the regular ones.
NULL is difficult to debug
Seriously? A "nullpo crash" is one of the more trivial things to debug, because it's very distinctive and makes it easy to trace the value back (0 stands out; other addresses, not so much.) What's actually hard to debug? Extraneous null checks that silently cause failures elsewhere.
The proposed "solution" is straightforward, but if you reserve the special null value to indicate absence then you can make do with just one value instead of a pair, of which half the time half of the value is completely useless. If you can check for absence/null, you will have no problems using Maybe/Optional. If you can't, Maybe/Optional won't help you anyway --- because it's ultimately the same thing, using a value without checking for its absence.
any reference can be null, and calling a method on null produces a NullPointerException.
...which immediately tells you to go fix the code.
There are many times when it doesn’t make sense to have a null. Unfortunately, if the language permits anything to be null, well, anything can be null.
That's not an argument. See above.
3. NULL is a special-case
...because it indicates the absence of a value, which is a special case.
though it throws a NullPointerException when run.
...and the cause is obvious. I'm not even a regular Java user (and don't much like the language myself, but for other reasons) and I know the difference between the Boxed types and the regular ones.
NULL is difficult to debug
Seriously? A "nullpo crash" is one of the more trivial things to debug, because it's very distinctive and makes it easy to trace the value back (0 stands out; other addresses, not so much.) What's actually hard to debug? Extraneous null checks that silently cause failures elsewhere.
The proposed "solution" is straightforward, but if you reserve the special null value to indicate absence then you can make do with just one value instead of a pair, of which half the time half of the value is completely useless. If you can check for absence/null, you will have no problems using Maybe/Optional. If you can't, Maybe/Optional won't help you anyway --- because it's ultimately the same thing, using a value without checking for its absence.
At least C# has the syntactic sugar to easily check for null references which lets you avoid the horrors of code like `(if s != null && s.length)`. Instead you can type `s?.length`. Never have I appreciated syntactic sugar as much.
Other candidates:
- null terminated strings
- machine dependent integer widths
- null terminated strings
- machine dependent integer widths
Nulls in strongly typed languages can get rather weird but from a C/C++ perspective it is the same as 0. nullptr is just a correctly casted 0.
NULL in 'relational' databases in particular is a disaster. Or at least according to the notorious Fabian Pascal.
http://www.dbdebunk.com/2017/04/null-value-is-contradiction-...
Codd never proposed it in his original relational model. For good reason.
http://www.dbdebunk.com/2017/04/null-value-is-contradiction-...
Codd never proposed it in his original relational model. For good reason.
Julia Missing Values
"Julia provides support for representing missing values in the statistical sense, that is for situations where no value is available for a variable in an observation, but a valid value theoretically exists. Missing values are represented via the missing object, which is the singleton instance of the type Missing. missing is equivalent to NULL in SQL and NA in R, and behaves like them in most situations."
https://docs.julialang.org/en/v1/manual/missing/index.html
+
"First-Class Statistical Missing Values Support in Julia 0.7"
https://julialang.org/blog/2018/06/missing
"Julia provides support for representing missing values in the statistical sense, that is for situations where no value is available for a variable in an observation, but a valid value theoretically exists. Missing values are represented via the missing object, which is the singleton instance of the type Missing. missing is equivalent to NULL in SQL and NA in R, and behaves like them in most situations."
https://docs.julialang.org/en/v1/manual/missing/index.html
+
"First-Class Statistical Missing Values Support in Julia 0.7"
https://julialang.org/blog/2018/06/missing
NULL is certainly a mistake, but even more of a mistake is not allowing distinct states in variables.
NULL is just another case of a state of a variable. Other states are 1, 15, 0xffffffff, etc.
That mainstream languages don't handle this is the worst mistake of the computer industry.
NULL is just another case of a state of a variable. Other states are 1, 15, 0xffffffff, etc.
That mainstream languages don't handle this is the worst mistake of the computer industry.
I've been using kotlin and swift. They've partly removed null with the 'maybe' feature.
So instead of calling methods on null objects the methods are just not called if the object is null.
This helps when there's a race condition, and you attempt to call a method on a null object, and then that solves itself by the same code being called again without the race condition.
But a lot of the time if the object is null and the method is not called you still have an error, but it's just not a null pointer error now.
This 'nullless' code is nice in some places, especially with UI lifecycles calling code repeatedly, but other times it just changes the type of error you debug.
So instead of calling methods on null objects the methods are just not called if the object is null.
This helps when there's a race condition, and you attempt to call a method on a null object, and then that solves itself by the same code being called again without the race condition.
But a lot of the time if the object is null and the method is not called you still have an error, but it's just not a null pointer error now.
This 'nullless' code is nice in some places, especially with UI lifecycles calling code repeatedly, but other times it just changes the type of error you debug.
Nah. There will always be missing values, no matter how many layers of safety measures we wrap around the fact. Hitting a NULL in C is very unforgiving; but that's just the spirit of C, there are plenty of ways to provide a less bumpy ride in higher level languages.
My own baby, Snigl [0], uses the type system to trap runaway missing values without wrapping. Which means that you get an error as soon as you pass a NULL, rather than way down the call stack when it's used.
https://gitlab.com/sifoo/snigl#types
My own baby, Snigl [0], uses the type system to trap runaway missing values without wrapping. Which means that you get an error as soon as you pass a NULL, rather than way down the call stack when it's used.
https://gitlab.com/sifoo/snigl#types
I don't write business software that needs many 9s of uptime so I find null to be fine. Yes, returning null is kind of throwing your hands up but I find that to be kind of the point. It allows my software to fail fast if it does and makes it incredibly obvious where things are going wrong. Generally if a reference has a value of null where it shouldn't, I can pinpoint the location of the bug within a few minutes or even seconds.
IMO it makes the program much easier to reason about compared to returning some sort of empty value and then failing much much later in the program.
IMO it makes the program much easier to reason about compared to returning some sort of empty value and then failing much much later in the program.
> it means that C-strings cannot be used for ASCII or extended ASCII. Instead, they can only be used for the unusual ASCIIZ.
This is a very pedantic quibble, and I'm not even sure it's correct. ASCII has NUL as well, and ASCIIZ isn't a character set AFAIK.
> C++ NULL boost::optional, from Boost.Optional
First of all, nullptr, second, std::optional.
> Objective C nil, Nil, NULL, NSNull Maybe, from SVMaybe
Nil is not a thing in Objective-C, to my knowledge.
> Swift Optional
You're looking for nil. So it should be four stars?
> Swift’s UnsafePointer must be used with unsafeUnwrap or !
You're confusing Optional and UnsafePointer.
This is a very pedantic quibble, and I'm not even sure it's correct. ASCII has NUL as well, and ASCIIZ isn't a character set AFAIK.
> C++ NULL boost::optional, from Boost.Optional
First of all, nullptr, second, std::optional.
> Objective C nil, Nil, NULL, NSNull Maybe, from SVMaybe
Nil is not a thing in Objective-C, to my knowledge.
> Swift Optional
You're looking for nil. So it should be four stars?
> Swift’s UnsafePointer must be used with unsafeUnwrap or !
You're confusing Optional and UnsafePointer.
This tidbit gets a ton of mileage but I think it's overrated. There are a lot of unsafe shortcuts we take to get better ergonomics and NULL is one of them.
I think it's a bit unlikely we'll fully get rid of null, but we can get rid of some of the pitfalls. TypeScript for example pretty much fixes the problem, by enforcing you check for null when needed, though TypeScript takes a handful of other soundness shortcuts. Go makes null less harmful by treating nil pointers like empty values by convention.
I think it's a bit unlikely we'll fully get rid of null, but we can get rid of some of the pitfalls. TypeScript for example pretty much fixes the problem, by enforcing you check for null when needed, though TypeScript takes a handful of other soundness shortcuts. Go makes null less harmful by treating nil pointers like empty values by convention.
Author forgot about Erlang, and it's wonderful lack of NULL
So how does NaN in Python and NA in R relate to NULL? I know Python has the None type, but it's not the same as NaN. One of the most annoying things in Numpy is that there is no way to indicate that an integer value is "missing", similar to NaN for floats. In R both integers and strings can be NA (if I remember correctly). So for numeric types at least, there is definitely the need to somehow indicate that a value is "missing".
Linux does well with some macros: IS_ERR, IS_ERR_VALUE, PTR_ERR, ERR_PTR, PTR_ERR_OR_ZERO, ERR_CAST, IS_ERR_OR_NULL
https://elixir.bootlin.com/linux/latest/source/include/linux...
No, it doesn't totally replace NULL, but it does solve some of the problems in a high-performance way.
https://elixir.bootlin.com/linux/latest/source/include/linux...
No, it doesn't totally replace NULL, but it does solve some of the problems in a high-performance way.
Honestly, I feel that the problem isn't null but that type systems (at least earlier on) tended to allow other types to be null, willy-nilly. Null is best considered a separate type to non-null values, and is basically not a problem if the type system handles that in some way. Be it option or union types - both solve it and it mostly stops being an issue.
Checking if the optional is present is very similar to checking for NULL values. Now if you have a nice match statement like rust and lambda functions for streams, that may make things a bit more readable.
You will still need analysis tools to check that all code paths check for none before accessing that value ..
You will still need analysis tools to check that all code paths check for none before accessing that value ..
Do people generally agree that Java Optional == Scala Maybe / Haskell's Maybe?
Java's Optional seems fundamentally flawed in that Java allows any reference to hold a null value and Optional can still throw a NPE when calling isPresent on it so it still gives people a footgun.
Java's Optional seems fundamentally flawed in that Java allows any reference to hold a null value and Optional can still throw a NPE when calling isPresent on it so it still gives people a footgun.
Conflating NULL with NUL terminated strings seems a stretch... both have problems, but separate problems. I suppose they're both related in that they provide a "special" value rather than separating that information though.
NULL is a convenient way to map singularities in your model of the problem.
I have on a few occasions tried to write NULL-less code, and it adds a good bit of work.
- model all possible states for a value
- determine appropriate default actions for all types
- meaningful place-holder values
It's a good exercise, and I think more code should be written this way, but - as an Engineer I'm trying to model just enough of the problem to solve it. I'm not trying to simulate every possible outcome in that domain.
Certain corners of your problem simply don't need to be modeled, and what's more the effort needed to model them can just be too much.
NULL is a great way to just throw up your hands and go "I don't know and I don't care". Much as when modelling a physical system singularities typically represent phenomena that the model doesn't take account of, so it goes with NULL. It simply says "Don't Go There".
I have on a few occasions tried to write NULL-less code, and it adds a good bit of work.
- model all possible states for a value
- determine appropriate default actions for all types
- meaningful place-holder values
It's a good exercise, and I think more code should be written this way, but - as an Engineer I'm trying to model just enough of the problem to solve it. I'm not trying to simulate every possible outcome in that domain.
Certain corners of your problem simply don't need to be modeled, and what's more the effort needed to model them can just be too much.
NULL is a great way to just throw up your hands and go "I don't know and I don't care". Much as when modelling a physical system singularities typically represent phenomena that the model doesn't take account of, so it goes with NULL. It simply says "Don't Go There".
In higher level programming, the consensus seems to be the less nulls the better. Which is why languages like C++, C#, etc are introducing Option-like syntax ( mostly to accommodate the database world and their NULLs ).
NULL exists to solve particular problems in computer science. It can also cause a lot of problems. You can argue it's the best solution and worst mistake depending on the situation.