Unit Testing bash scripts

Question

We have a system that has some bash scripts running besides Java code. Since we are trying to Test Everything That Could Possibly Break, and those bash scripts may break, we want to test them.

The problem is it is hard to test bash scripts.

Is there a way or a best practice to test bash scripts? Or should we quit using bash scripts and look for alternative solutions that are testable?

Answer 1

I got the following answer from a discussion group:

it's possible to import (include, whatever) a procedure (function, whatever it's named) from an external file. That's the key to writing a testing script: you break up your script into independent procedures that can then be imported into both your running script and your testing script, and then you have your running script be as simple as possible.

This method is like dependency injection for scripts, and sounds reasonable. Avoiding bash scripts and using more testable and less obscure language is preferable.

Answer 2

There is actually a shunit2, an xUnit based unit test framework for Bourne based shell scripts. I haven't used it myself, but it might be worth checking out.

Similar questions have been asked before:

• Unit Testing for Shell Scripts
• Test Anything Protocol in Shell Scripts

Answer 3

TAP-compliant Bash testing: Bash Automated Testing System

TAP, the Test Anything Protocol, is a simple text-based interface between testing modules in a test harness. TAP started life as part of the test harness for Perl but now has implementations in C, C++, Python, PHP, Perl, Java, JavaScript, and others.

bats-core

Answer 4

Epoxy is a Bash test framework I designed mainly for testing other software, but I use it to test bash modules as well, including itself and Carton.

Main advantages are relatively low coding overhead, unlimited assertion nesting and flexible selection of assertions to verify.

I made a presentation comparing it to BeakerLib - a framework used by some at Red Hat.

Answer 5

Nikita Sobolev wrote an excellent blog post comparing a few different bash test frameworks: Testing Bash applications

For the impatient: Nikita's conclusion was to use Bats but it appears that Nikita missed the Bats-core project which appear to me to be the one to use going forward as the original Bats project has not been actively maintained since 2013.

Answer 6

Why do you say that it's "hard" to test bash scripts?

What's wrong with test wrappers like:

 #!/bin/bash
 set -e
 errors=0
 results=$($script_under_test $args<<ENDTSTDATA
 # inputs
 # go
 # here
 #
 ENDTSTDATA
 )
 [ "$?" -ne 0 ] || {
     echo "Test returned error code $?" 2>&1
     let errors+=1
     }

 echo "$results" | grep -q $expected1 || {
      echo "Test Failed.  Expected $expected1"
      let errors+=1
 }
 # and so on, et cetera, ad infinitum, ad nauseum
 [ "$errors" -gt 0 ] && {
      echo "There were $errors errors found"
      exit 1
 }

Answer 7

I quite like shell2junit, a utility to generate JUnit-like output from Bash script tests. This is useful because the report generated can then be read by continuous integration systems, such as the JUnit plug-ins for Jenkins and Bamboo.

While shell2junit doesn't provide the comprehensive Bash scripting framework like shunit2, it does allow you have nice reporting of the test results.

Answer 8

Try bashtest. It`s simple way to test your scripts. For example, you have do-some-work.sh which change some config files. For example, add new line PASSWORD = 'XXXXX' to config file /etc/my.cfg.

You write bash commands line by line and then check output.

Install:

pip3 install bashtest

Create tests is a just writing bash commands.

File test-do-some-work.bashtest:

# run the script  
$ ./do-some-work.sh > /dev/null

# testing that the line "PASSWORD = 'XXXXX'" is in the file /etc/my.cfg   
$ grep -Fxq "PASSWORD = 'XXXXX'" /etc/my.cfg && echo "YES"
YES

Run tests:

bashtest *.bashtest

You can find some examples here and here

Answer 9

Maybe this can be used, or contributed to

https://thorsteinssonh.github.io/bash_test_tools/

Intended to write results in TAP protocol which I imagine is good for CI, and good for those that want shell environments. I imagine some things run in shell environments so, some might argue should be tested in their shell environment.

Answer 10

Give a try to assert.sh

source "./assert.sh"

local expected actual
expected="Hello"
actual="World!"
assert_eq "$expected" "$actual" "not equivalent!"
# => x Hello == World :: not equivalent!

Hope it helps!

Answer 11

I can't believe no one talked about OSHT! It's compatible with both TAP and JUnit, it's pure shell (that is, no other languages involved), it works standalone too, and it's simple and direct.

Testing looks like this (snippets taken from the project page):

#!/bin/bash
. osht.sh

# Optionally, indicate number of tests to safeguard against abnormal exits
PLAN 13

# Comparing stuff
IS $(whoami) != root
var="foobar"
IS "$var" =~ foo
ISNT "$var" == foo

# test(1)-based tests
OK -f /etc/passwd
NOK -w /etc/passwd

# Running stuff
# Check exit code
RUNS true
NRUNS false

# Check stdio/stdout/stderr
RUNS echo -e 'foo\nbar\nbaz'
GREP bar
OGREP bar
NEGREP . # verify empty

# diff output
DIFF <<EOF
foo
bar
baz
EOF

# TODO and SKIP
TODO RUNS false
SKIP test $(uname -s) == Darwin

A simple run:

$ bash test.sh
1..13
ok 1 - IS $(whoami) != root
ok 2 - IS "$var" =~ foo
ok 3 - ISNT "$var" == foo
ok 4 - OK -f /etc/passwd
ok 5 - NOK -w /etc/passwd
ok 6 - RUNS true
ok 7 - NRUNS false
ok 8 - RUNS echo -e 'foo\nbar\nbaz'
ok 9 - GREP bar
ok 10 - OGREP bar
ok 11 - NEGREP . # verify empty
ok 12 - DIFF <<EOF
not ok 13 - TODO RUNS false # TODO Test Know to fail

The last test shows as "not ok", but the exit code is 0 because it's a TODO. One can set verbose as well:

$ OSHT_VERBOSE=1 bash test.sh # Or -v
1..13
# dcsobral \!= root
ok 1 - IS $(whoami) != root
# foobar =\~ foo
ok 2 - IS "$var" =~ foo
# \! foobar == foo
ok 3 - ISNT "$var" == foo
# test -f /etc/passwd
ok 4 - OK -f /etc/passwd
# test \! -w /etc/passwd
ok 5 - NOK -w /etc/passwd
# RUNNING: true
# STATUS: 0
# STDIO <<EOM
# EOM
ok 6 - RUNS true
# RUNNING: false
# STATUS: 1
# STDIO <<EOM
# EOM
ok 7 - NRUNS false
# RUNNING: echo -e foo\\nbar\\nbaz
# STATUS: 0
# STDIO <<EOM
# foo
# bar
# baz
# EOM
ok 8 - RUNS echo -e 'foo\nbar\nbaz'
# grep -q bar
ok 9 - GREP bar
# grep -q bar
ok 10 - OGREP bar
# \! grep -q .
ok 11 - NEGREP . # verify empty
ok 12 - DIFF <<EOF
# RUNNING: false
# STATUS: 1
# STDIO <<EOM
# EOM
not ok 13 - TODO RUNS false # TODO Test Know to fail

Rename it to use a .t extension and put it in a t subdirectory, and you can use prove(1) (part of Perl) to run it:

$ prove
t/test.t .. ok
All tests successful.
Files=1, Tests=13,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.11 cusr  0.16 csys =  0.31 CPU)
Result: PASS

Set OSHT_JUNIT or pass -j to produce JUnit output. JUnit can also be combined with prove(1).

I have used this library both testing functions by sourcing their files and then running assertions with IS/OK and their negatives, and scripts by using RUN/NRUN. For me, this framework provides the most gain for the least overhead.

Answer 12

I created shellspec because I wanted a easy-to-use and useful tool.

It written by pure POSIX shell script. It has tested with many shells more than shunit2. It has powerful features than bats/bats-core.

For example, support nested block, easy to mock/stub, easy to skip/pending, parameterized tests, assertion line number, execute by line number, parallel execution, random execution, TAP/JUnit formatter, coverage and CI integration, profiler and etc.

See the demo on the project page.

• https://github.com/shellspec/shellspec
• https://shellspec.info/

Answer 13

I’ve tried a lot of the solutions presented here, but found most of them to bulky and hard to use, so I built my own little testing framework: https://github.com/meonlol/t-bash

It’s just one file in the repo that you can simply run directly, with a basic set of JUnit style asserts.

I’ve used it professionally in several internal projects and were able to make our bash scripts super stable and regression resistant.

Answer 14

You might want to take a look at bash_unit:

https://github.com/pgrange/bash_unit

Answer 15

Take a look at Outthentic, it is simple, extensible by many languages ( Perl, Python, Ruby, Bash on choice ) and cross platform ( Linux, Windows ) framework to test any command line applications.

Answer 16

I have found it hard to justify using bash for larger scripts when Python has such huge advantages:

• Try/Except allows writing more robust scripts with the ability to undo changes in case of an error.
• You don't have to use obscure syntax such as 'if [ x"$foo" = x"$bar"]; then ...' which is prone to errors.
• Easy parsing of options and arguments using the getopt module (and there's an even easier module for parsing arguments, but the name escapes me).
• Python allows you to work with lists/dicts and objects instead of basic strings and arrays.
• Access to proper language tools such as regex, databases (sure you could pipe everything into the mysql command in bash, but it's not the nicest way to write code).
• No need to worry about using the correct form of $* or "$*" or "$@" or $1 or "$1", spaces in filenames isn't an issue, etc, etc, etc.

Now I only use bash for the simplest of scripts.