What's the point of hidden files?

Question

What is the point of hidden files? In Microsoft Windows they exist, in Mac OS X they exist and in Linux they exist. It seems to me that it just makes detecting malware more difficult. The only upside I see is protection of necessary OS files.

Answer 1

Let me go type ls on my home directory for a moment.

~ $ ls
Desktop/      Downloads/    Movies/       Projects/
Development/  IdeaProjects/ Music/        Public/
Documents/    Library/      Pictures/     bin/
~ $ 

Yep, its a mac. And a relatively new one. I've got 12 directories there.

Now lets try ls -a

~ $ ls -a
./                   .gnome2/             Desktop/
../                  .hgignore_global     Development/
.CFUserTextEncoding  .inkscape-etc/       Documents/
.DS_Store            .local/              Downloads/
.Trash/              .m2/                 IdeaProjects/
.Xauthority          .mono/               Library/
.bash_history        .oracle_jre_usage/   Movies/
.bash_sessions/      .profile             Music/
.cache/              .profile.swp         Pictures/
.config/             .rnd                 Projects/
.gitconfig           .viminfo             Public/
.gitignore_global    .vimrc               bin/

And thats not actually too bad. But there's a lot more there. And things I don't care about seeing every single type I type ls.

Lets go for a moment to my old drive and its home directory...

/Vo/Ot/Us/Shagie $ ls -1 | wc -l
     129
/Vo/Ot/Us/Shagie $ ls -a1 | wc -l
     231

I've got a fair bit there. And there is a lot more that I don't care about seeing every time I type ls.

I know I have a .profile and a .DS_STORE and a .bash_history and a .Xcode/ and .adobe/ and .Trash/ and .android/ And whats more, that .org.eclipse.epp.usagedata.recording.userId has a really long name that throws off the columns. And there's .psql_history and .sqlite_history and... yea.

It is trivial to tweak one's own .bashrc (or whatever shell you prefer) to make it so that you always see them. I've got la aliased to ls -a so if I want to see them I can with just the slightest of different character placement.

But seeing them every time I do an ls? That's not something I want. It is too much information and makes it harder for me to do what I want - find a file in this directory.

Answer 2

On Unix they were an accident at least initially

https://plus.google.com/+RobPikeTheHuman/posts/R58WgWwN9jp

Second, and much worse, the idea of a "hidden" or "dot" file was created. As a consequence, more lazy programmers started dropping files into everyone's home directory. I don't have all that much stuff installed on the machine I'm using to type this, but my home directory has about a hundred dot files and I don't even know what most of them are or whether they're still needed. Every file name evaluation that goes through my home directory is slowed down by this accumulated sludge.

I'm pretty sure the concept of a hidden file was an unintended consequence. It was certainly a mistake.

Answer 3

No comment on the original creation of hidden files but now that they exist it's very tempting to keep using them.

Hidden files offer a convenient mechanism for associating arbitrary metadata with a directory location while remaining largely independent of file system or OS mechanics. There might be "better" ways to store folder icons or search indices for a directory as part of a file system but then I might find that those features only work on select partitions. Similarly there might be a "better" home for user configuration data but it's difficult to introduce one which wouldn't degrade the use of existing programs or be incompatible with existing file systems.

Hidden files are just hidden enough to discourage most users from accidentally invalidating that metadata by moving or removing them while remaining standard enough to be universally available and flexible enough to support a wide range of use cases.