NodeJS fs.watch on directory only fires when changed by editor, but not shell or fs module

Question

When the code below is ran, the watch is only triggered if I edit and save tmp.txt manually, using either my ide, TextEditor.app, or vim.

It doesn't by method of the write stream or manual shell output redirection (typing echo "test" > /path/to/tmp.txt").

Although if I watch the file itself, and not its dirname, then it works.

var fs, Path, file, watchPath, w;

fs = require('fs' );
Path = require('path');
file = __dirname + '/tmp.txt';
watchPath = Path.dirname(file); // changing this to just file makes it trigger

w = fs.watch ( watchPath, function (e,f) {
    console.log("will not get here by itself");
    w.close();
});
fs.writeFileSync(file,"test","utf-8");

fs.createWriteStream(file, {
    flags:'w',
    mode: 0777
} )
.end('the_date="'+new Date+'";' ); // another method fails as well

setTimeout (function () {
    fs.writeFileSync(file,"test","utf-8");
},500); // as does this one
// child_process exec and spawn fail the same way with or without timeout

So the questions are: why? and how to trigger this event programmatically from a node script?

Thanks!

Answer 1

It doesn't trigger because a change to the contents of a file isn't a change to the directory.

Under the covers, at least as of 0.6, fs.watch on Mac uses kqueue, and it's a pretty thin wrapper around kqueue file system notifications. So, if you really want to understand the details, you have to understand kqueue, and inodes and things like that.

But if you want a short "lie-to-children" explanation: What a user thinks of as a "file" is really two separate things—the actual file, and the directory entry that points to the actual file. This is what allows you to have things like hard links, and files that can still be read and written even after you've deleted them, and so on.

In general, when you write to an existing file, this doesn't make any change to the directory entry, so anyone watching the directory won't see any change. That's why echo >tmp.txt doesn't trigger you.

However, if you, e.g., write a new temporary file and then move it over the old file, that does change the directory entry (making it a pointer to the new file instead of the old one), so you will be notified. That's why TextEditor.app does trigger you.

Answer 2

The thing is, you've asked to watch the directory and not the file.

The directory isn't updated when the file is modified, such as via shell redirection; in this case, the file is opened, modified, and closed. The directory isn't changed -- only the file is.

When you use a text editor to modify a file, the usual set of system calls behind the scenes looks something like this:

fd = open("foo.new")
write(fd, new foo contents)
unlink("foo")
rename("foo.new", "foo")

This way, the foo file is either entirely the old file or entirely the new file, and there's no way for there to be a "partial file" with the new contents. The renaming operations do modify the directory, thus triggering the directory watch.

Answer 3

Although the above answers seems reasonable, they are not fully accurate. It is actually a very useful feature to be able to listen to a directory for file changes, not just "renames". I think this feature works as expected in Windows at least, and in node 0.9.2 is also working for mac since they changed to the FSEvents API that supports the feature:

Version 0.9.2 (Unstable)