Portable way to list all files in a directory without parsing ls or find

Question

I'm trying to get a list of all files in a given directory to provide a list of filenames for a simple dotfiles Makefile. The idea is that the list of filenames is put into a variable, and then a list of "meta" files such as the Makefile itself are removed from the list. Then, the files in the list are symlinked into $HOME.

I don't want to parse ls or find because I can't rely on some random program to name its configuration file without whitespace. And I also can't rely on programs to start their "dotfiles" filenames with a "dot".

I tried the for-loop solution at parse ls or find, which uses the * glob. The problem is that globs do not by default match files starting with a dot.

There are globbing options in each shell (bash, zsh, etc) that make the * glob expand to include .dotfiles, and even to exclude . and .. But that isn't portable.

Any suggestions?

Let me know if you see an XY problem.

Answer 1

Do the glob in two steps, and prune the . and .. entries manually, or with a test for "regular file". Of course the "echo" is just for testing. Add it to your list for non-trivial processing.

Edit: Silly me. Instead of actually looping twice, the list of all files (with and without dots) is just * .*. Also, the test command [ takes its argument as a string, so we need to quote it for spaces. Revised code:

# Process non-dotfiles and dotfiles.  (Only the regular files)
for f in * .*; do 
    if [ -f "$f" ]; then
        echo $f 
    fi; 
done

Or, use find to do the entire job at once, including the symlinking. For example:

find . -type f \( ! -iname Makefile \) -exec ln -s \{\} newdir/\{\} \;

Or, use a combination of the two, using find -print0 pattern as suggested in your source link: that will handle the spaces and the newlines correctly, letting your script get null-separated filenames to store and further process.

Answer 2

You say

I don't want to parse ls or find because I can't rely on some random program to name its configuration file without whitespace.

The whitespace problem is why find has the -print0 option, to print its output separated by null characters (ASCII 0) that cannot appear in filenames.