Is os.walk() missing symlinks to directories?

Question

I have a directory containing some files, some directories, some symlinks to files and some symlinks to directories.

When I do os.walk() with followlinks=false in the directory, I get the files and symlinks to files in the filenames list and the directories in the dirnames list. But the symlinks to directories does not show up anywhere. Is this a bug or a feature in Python, or am I doing something wrong?

I expect the symlinks to directories to show up in the filenames list, because they are not directories but symlinks, and the other symlinks (to other files) show up in the filenames list.

Example: The directory foo contains the following:

-rw-rw-r-- 4 rikno staff 136 Jan 14 11:10 firefox
lrwxr-xr-x 1 rikno staff   5 Jan 23 13:29 latex -> tetex
lrwxr-xr-x 2 rikno staff  68 Jan 14 11:10 mozilla -> firefox
drwxrwxr-x 3 rikno staff 102 Jan 23 13:29 tetex

I expect os.walk('foo') in the first iteration to return

('foo', ['tetex'], ['firefox', 'latex', 'mozilla'])

or at least

('foo', ['latex', 'tetex'], ['firefox', 'mozilla'])

but all I get is

('foo', ['tetex'], ['firefox', 'mozilla'])

and I never get anything about the symlink latex (which points to the directory tetex)

Solved:

Ok, the result is

('foo', ['latex', 'tetex'], ['firefox', 'mozilla'])

So the symlink to the directory shows up in the dirnames list.

I first expected the symlink to the directory to be in the filenames list and never looked in the dirnames list, and when experimenting with the code and the file system to find where the link was or why the link was "missing" I accidently mixed up my results.

Sorry for asking.

Answer 1

Running on my machine, os.walk() does show all the sym links:

>>> os.walk("foo").next()
('foo', ['tetex', 'latex'], ['mozilla', 'firefox'])
>>> os.walk("foo", followlinks=False).next()
('foo', ['tetex', 'latex'], ['mozilla', 'firefox'])

The only 'problem' I see here is that the sym link appears in the list of directories and not in the list of files.

In most use cases, this would be the expected behaviour since one would expect to be able to treat all entries in the file-list as files and not have to check if it is a sym-link or not.

This thread from python-dev briefly discusses the issue.

"... putting the symlinks-to-directories into the files list instead of the subdirectory list isn't really any better (it just moves the problem to different use cases, such as those that actually want to read the file contents)."

and from the linked issue page:

"For example to count the number of lines of all the files under a directory, a code could go like this:

for root, dirs, files in os.walk(top):
    for file in files:
        f = open(file)
        for n, l in enumerate(f, 1):
            pass
        print(file, n)

If, suddently, a symlink to a directory appeared in files, this will break. So I'm not convinced it's worth changing this. A symlink to a directory is not much closer to a file than to a directory, it really depends on the use case."

Answer 2

You wrote that you called os.walk() with followlinks set to False. Well, then that's the expected behavior:

By default, walk() will not walk down into symbolic links that resolve to directories. Set followlinks to True to visit directories pointed to by symlinks, on systems that support them.