Fill a bash array from a NUL separated input

Question

I want to create a bash array from a NUL separated input (from stdin).

Here's an example:

## Let define this for clarity
$ hd() { hexdump -v -e '/1 "%02X "'; echo ;}
$ echo -en "A B\0C\nD\0E\0" | hd
41 20 42 00 43 0A 44 00 45 00

So this is my input.

Now, working with NUL works fine if not using the -a of read command:

$ while read -r -d '' v; do echo -n "$v" | hd; done < <(echo -en "A B\0C\nD\0E\0")
41 20 42 
43 0A 44 
45 

We get the correct values. But I can't store these values using -a:

$ read -r -d '' -a arr < <(echo -en "A B\0C\nD\0E\0")
$ declare -p arr
declare -a arr='([0]="A" [1]="B")'

Which is obviously not what I wanted. I would like to have:

$ declare -p arr
declare -a arr='([0]="A B" [1]="C
D" [2]="E")'

Is there a way to go with read -a, and if it doesn't work, why? Do you know a simple way to do this (avoiding the while loop) ?

Answer 1

read -a is the wrong tool for the job, as you've noticed; it only supports non-NUL delimiters. The appropriate technique is given in BashFAQ #1:

arr=()
while IFS= read -r -d '' entry; do
  arr+=( "$entry" )
done

---

In terms of why read -d '' -a is the wrong tool: -d gives read an argument to use to determine when to stop reading entirely, rather than when to stop reading a single element.

Consider:

while IFS=$'\t' read -d $'\n' words; do
  ...
done

...this will read words separated by tab characters, until it reaches a newline. Thus, even with read -a, using -d '' will read until it reaches a NUL.

What you want, to read until no more content is available and split by NULs, is not a '-d' of NUL, but no end-of-line character at all (and an empty IFS). This is not something read's usage currently makes available.

Answer 2

bash-4.4-alpha added a -d option to mapfile:

The `mapfile' builtin now has a -d option to use an arbitrary character as the record delimiter, and a -t option to strip the delimiter as supplied with -d.

— https://tiswww.case.edu/php/chet/bash/CHANGES

Using this, we can simply write:

mapfile -t -d '' arr < <(echo -en "A B\0C\nD\0E\0")

Answer 3

If anyone wonders, here's the function (using while) that I use to store values from a NUL-separated stdin:

read_array () {
    local i
    var="$1"
    i=0
    while read -r -d '' value; do
        printf -v "$var[$i]" "%s" "$value"
        i=$[$i + 1]
    done
}

It can then be used quite cleanly:

$ read_array arr < <(echo -en "A B\0C\nD\0E\0")
$ declare -p arr
declare -a arr='([0]="A B" [1]="C
D" [2]="E")'

Answer 4

Here's a simplification of @vaab's function. It uses bash 4.3's nameref feature:

read_array () {
  local -n a=$1
  while read -r -d '' value; do
    a+=("$value")
  done
}

Test:

test_it () {
  local -a arr
  read_array arr < <(echo -en "A B\0C\nD\0E\0")
  declare -p arr
}
test_it