You have to touch CMakeLists.txt
whenever you change the directory contents.
What you're doing goes counter to official advice, but I do it too. You could knock up a script which monitors a directory for changes, but CMake considers this to be not its business.
The reason, I suspect, is that directory modification timestamp attributes aren't very cross-platform.