What are the advantages of build scripts?

Question

For most of my programming career, I've used the "build/compile/run" command in whatever IDE I'm working with to produce a runnable program. This is one button, pretty easy. As I learn more about different languages and frameworks, though, I see more and more talk of "build scripts" (ANT, Maven, Gradle, etc.) to get a project running. My understanding of these is that they are instructions to the compiler/linker/magical-program-maker that specify configuration details - minutiae.

I remember writing makefiles back in school, but I didn't see any special advantage then (we only used them when writing in a Unix terminal, where an IDE with it's handy "build" button wasn't present). Beyond that, I've seen other questions on here that discuss how build scripts can do more than just create your program - they can run unit tests as well as secure resources regardless of the host machine.

I can't shake the feeling that build scripts are important to understand as a developer, but I'd like a meaningful explanation; why should I use/write build scripts?

Responsibilities of Build Script and Build Server discusses the role it plays within a broader scope. I'm looking for specific advantages offered by a build script versus an IDE's "build/run" command or similarly simple methods.

Answer 1

Automation.

When you are developing, only in the most simple projects will the default "build" button do everything you need it to do; you may need to create WS out of APIs, generate docs, link with external resources, deploy the changes to a server, etc. Some IDEs allow you to customize the build process by adding extra steps or builders, but that only means that you are generating your build script through the IDE commodities.

But developing a system is not only writing code. There are multiple steps involved. An IDE independent script can be executed automatically, meaning that:

• When you commit a change to version control, a new build can be launched automatically by the server. It will ensure that you have not forgot to commit anything needed for the build.

• Similarly, after the build is done, tests can automatically be run to see if you broke something.

• Now the rest of the organization (QA, sysadmins) has a built product that

  a) is perfectly reproducible just from the control version.

  b) is common to all of them.

Even when I have been working as a one man team I have used scripts for that purpose; when I had developed the fix I would commit to SVN, export the SVN back in another directory and use the build script to generate the solution, which would go then to Preproduction systems and later to Production. If a few weeks later (with my local codebase already changed) someone complained of a bug, I would knew exactly which SVN revision I would have to checkout in order to properly debug the system.

Answer 2

Like code, a build script is executed by the computer. Computers are exceptionally good at following a set of instructions. In fact, (outside of self-modifying code), computers will execute the same sequence of instructions exactly the same way, given the same input. This delivers a level of consistency that, well, only a computer can match.

By contrast, us water filled fleshbags are just downright despicable when it comes to following steps. That pesky analytical brain has a tendency to question everything it comes across. "Oh...I don't need to that", or "Do I really to use this flag? Eh...I'll just ignore it." In addition, we have a tendency to get complacent. Once we've done something a few times, we start believing that we know the instructions, and don't have to look at the instruction sheet.

From "The Pragmatic Programmer":

In addition, we want to ensure consistency and repeatability on the project. Manual procedures leave consistency up to change; repeatability isn't guaranteed, especially if aspects of the procedure are open to interpretation by different people.

In addition, we are HILARIOUSLY sluggish in executing instructions (compared to a computer). On a large project, with hundreds of files and configurations, it would take years to manually execute all the steps in a build process.

I'll give you a real world example. I was working some embedded software, where most of the code was shared across a few different hardware platforms. Each platform had different hardware, but most of the software was the same. But there were little pieces that were specific to each hardware. Ideally, the common pieces would be placed in a library and linked into each version. However, the common pieces couldn't be compiled into a shared library. It had to be compiled with every different configuration.

At first, I manually compiled each configuration. It only took a few seconds to switch between configurations, and wasn't that big of a pain. Towards the end of the project, a critical defect was discovered in the shared portion of code, where a device would essentially take over the communication bus. This was BAD! Really bad. I found the bug in the code, fixed it, and recompiled every version. Except one. During the build process, I got distracted, and forgot one. The binaries got released, the machine was built, and a day later I get a phone call saying the machine stopped responding. I check it out, and discovered a device had locked the bus. "But I fixed that bug!".

I may have fixed it, but it never found its way onto that one board. Why? Because I didn't have an automated build process that built every single version with 1 click.

Answer 3

If all you ever want to do is <compiler> **/*.<extension>, build scripts serve little purpose (though one can argue that if you see a Makefile in the project you know you can build it with make). The thing is - non-trivial projects usually require more than that - at the very least, you'll usually need to add libraries and (as the project matures) configure build parameters.

IDEs are usually at least that configurable - but now the build process relies on IDE-specific options. If you are using Eclipse, Alice prefers NetBeans, and Bob wants to use IntelliJ IDEA, you can't share the configuration, and when one of you pushes a change to the source control, they need to either manually edit the IDE-created configuration files of the other developers, or notify the other developers so they'll do it themselves (which means there will be commits where the IDE configuration is wrong for some of the IDEs...).

You also need to figure out how to do that change in each and every one of the IDEs used by the team, and if one of them doesn't support that particular setting...

Now, this problem is culture-dependent - your developers might find it acceptable to not have the choice of IDEs. But developers that have experience with an IDE are usually both happier and more efficient when using it, and text-editor users tend to get religiously zealot about their favorite tools, so this is one of the places where you want to give freedom to the developers - and build systems allow you to do just that. Some people might have a build system preference, but it's not nearly as fanatic as IDE/editor preferences...

Even if you get all the developers to use the same IDE - good luck convincing the build server to use it...

Now, that's for the simple build process customizations, that IDEs tend to provide nice GUI for. If you want more complex stuff - like pre-processing/auto-generating source files before the compilation - you'll usually have to write a pre-build script in some basic text area inside the IDE's configuration. Which build systems you would still have to code that part, but you can do it in the actual editor you write the code in, and more importantly: the build system's framework itself usually provide some support for organizing these scripts.

Finally - build systems are good for more than just building the project - you can program them to do other tasks that everyone in the team might need performed. In Ruby on Rails, for example, there are build system tasks for running database migrations, for cleaning the temporary files, etc. Putting these tasks in the build system ensures that everyone on the team can do them consistently.

Answer 4

Many IDEs simply package up the commands used to build something and then generate a script and call it!

For example, in Visual Studio, you can see the command-line parameters for a C++ compile in the 'command line' box. If you look closely at the build output you'll see the temporary file that contains the build script that was used to run the compile.

Nowadays, it's all MSBuild, but that is still run directly by the IDE.

So the reason you use the command line is that you are going straight to the source and skipping the middle man, a middleman who might have been updated or require a heap of dependencies that you just don't want or need on a headless server that acts as your continuous integration (CI) server.

In addition, your scripts do more than the usual developer-oriented steps they're designed for. For example, after a build, you may want your binaries packaged and copied to a special location, or an installer package created, or a documentation tool run on them. Many different tasks are performed by a CI server that are pointless on a developer machine, so whilst you could create a IDE project that performed all these steps you'd then have to maintain two of them - a project for developers and another for builds. Some build tasks (static analysis for example) can take a long time that you wouldn't want for developer projects.

In short though, it's just easier - create a script to do all the things you want and it's quick and simple to kick that off on the command line or a build server configuration.

Answer 5

make

is a lot easier to remember and type than

gcc -o myapp -I/include/this/dir -I/include/here/as/well -I/dont/forget/this/one src/myapp.c src/myapp.h src/things/*.c src/things/*.h

And projects can have very complex compilation commands. A build script also has the ability to only recompile the things that changed. If you want to do a clean build,

make clean

is easier and more reliable once set up properly than trying to remember each and every place an intermediate file may have been produced.

Of course, if you use an IDE, clicking a build or clean button easy, too. However, it's much more difficult to automate moving the cursor to a specific place on the screen, especially when that place may move if the window moves, than it is to automate a simple text command.

Answer 6

How else would you do it? The only other way is to specify one long command line command.

Another reason is that makefiles allow incremental compilation, which speeds up compile time a lot.

Makefiles can also make a build process cross-platform. CMake generates different build scripts based on the platform.

Edit:

With an IDE, you are tied down to a particular way of doing things. Many people use vim or emacs even though they do not have many IDE-like features. They do it because they want the power these editors provide. Build scripts are necessary for those not using an IDE.

Even for those using an IDE, you might want to really know what is going on, so the build script offers you the low down details of implementation that a GUI approach doesn't have.

IDE's themselves use often build scripts internally too; the run button is just another way of running the make command.

Answer 7

The above answers cover a lot of good ground, but one real-world example that I'd like to add (that I can't add as a comment due to no karma), is from Android programming.

I'm a professional Android/iOS/Windows Phone developer, and I use Google services APIs (mostly Google Maps) a lot.

In Android, these services require that I add a keystore, or a type of developer ID file that tells Google that I am who I say I am, to a developer console. If my app is compiled with a different keystore, the Google Maps section of the app will not work.

Instead of adding and managing a dozen keystores to the developer console, only one of which can actually be used to update the app anyway, I include this keystore in our secure repo and use Gradle to tell Android Studio exactly what keystore to use when building for "debug" or "release". Now, I only have to add two keystores to my Google developer console, one for "debug" and one for "release", and all of my team members can clone the repo and get right to developing without having to go in to the dev console and add the SHA hash of their particular keystore, or worse, making me manage them. This has the added benefit of giving each team member a copy of the signing keystore, meaning that if I am out of the office and an update is scheduled, a team member only needs to follow a very short list of instructions in order to push an update.

Build automation like this keeps builds consistant, and reduces technical debt by cutting way down on set-up time when we get a new developer, a new machine, or when we have to re-image a machine.

Answer 8

Build script advantages:

• changes look like code (for example, in a git diff command), not like different checked options in a dialog

• creating more output than a simple build

In some of my previous projects, I've used the build scripts to:

• generate the project documentation (doxygen-based)
• build
• run unit tests
• generate unit test coverage reports
• pack binaries into a release archive
• generate internal release notes (based on "git log" messages)
• automated testing

Answer 9

Often you can call the "build" button in an automated way (Visual Studio accepts command line arguments, for example). People write build scripts as soon as they need something that the build button cannot provide.

For example, most IDEs will only let you build one platform at a time. Or only one language at a time. Then there's what you do with the built outputs: can your IDE roll them all up into an install package?