Was there ever a need to have the stack be executable?

Question

Having the call stack allow code execution has been a source of much security concern over many years. Stack buffer overflows can be used to exploit badly written software where code stored in stack buffers can be executed.

I just wondered if there really was ever a reason that it wasn't just made non-executable? Why would anything on a call stack ever need to be executable?

Perhaps there is some historic reason

Answer 1

Creating a non-executable stack requires help from the hardware. Early Intel processors had no NX bit, and it wasn't until the 386 or Pentium that it was really useful. (There were earlier processors that did have no-execute protections. But it wasn't ubiquitous.)

At various times people have made use of executable stacks and even writable code segments to write self-modifying programs. I think we've all come to agree that self-modifying code is a bad idea, but I recall there being a lot of interest in it when I got started in the 80s. You can really write some amazingly tight programs if you throw away pesky restrictions like arbitrary separations between code and data.

Speaking of which, the concept of executing your data, and storing lots of that data on the stack, is an important part of Lisp. See Allocating a data page in linux with NX bit turned off for one discussion of how this impacts JIT compilers. If you want to JIT compile something very small and very often, allocating and running it on the stack can be convenient.

That said, as you say, it's been a big problem, which is why there's generally been moves away from it. This brings a bit of the Harvard architecture back into the Von Neumann architecture most of us have grown up with.

Answer 2

Since the 1940s, most computers have deliberately not distinguished between executable and non-executable memory. This single, unified code and data model is one of the key aspects of the Von Neumann architecture. At various times, some limited non-execution capabilities have been introduced in various memory-management designs, and in fact today most CPUs and OSes support marking individual memory pages as non-executable.